Immunotherapy is used to treat almost all patients with advanced non-small cell lung cancer (NSCLC); however, identifying robust predictive biomarkers remains challenging. Here, the authors demonstrate the ability to integrate medical imaging, histopathology, and genomic features to predict immunotherapy response, using a cohort of 247 patients with advanced NSCLC who obtained multimodal baseline data during clinical diagnosis, including computed tomography images, digital programmed death ligand-1 immunohistochemical slices, and known outcomes of immunotherapy. Utilizing annotations from domain experts, the authors developed a computational workflow to extract patient-level features and integrated multimodal features into a risk prediction model using machine learning methods. The authors’ multimodal model (area under the curve (AUC) = 0.80, 95% confidence interval (CI) 0.74-0.86) outperformed unimodal measures, including tumor mutation burden (AUC = 0.61, 95% CI 0.52-0.70) and programmed death ligand-1 immunohistochemical score (AUC = 0.73, 95% CI 0.65-0.81). Therefore, the authors’ study provides a quantitative rationale for using multimodal features, leveraging expert-guided machine learning to enhance the prediction of immunotherapy responses in NSCLC patients.
Title：Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer
Publication Date：August 29, 2022
Corresponding Author Institution：Memorial Sloan Kettering Cancer Center
DOI：https://doi.org/10.1038/s43018-022-00416-8
Immunotherapy has shown great promise in treating patients with advanced non-small cell lung cancer. The authors indicate that combining data collected during routine clinical diagnosis with machine learning has the potential to improve predictions of immunotherapy responses and identify patients most likely to benefit.
Although immunotherapy is used to treat nearly all patients with non-small cell lung cancer (NSCLC), only a subset of patients derive any benefit. Significant efforts have been made to apply quantitative data analysis methods, including machine learning, to data collected during routine clinical work for NSCLC patients, particularly using radiology and histopathology data, to better identify those most likely to respond. However, due to the lack of multimodal datasets and integrated algorithms, these data patterns are often studied independently. The authors set out to establish a multimodal risk prediction model for advanced NSCLC patients based on deep learning and using annotations from domain experts. These patients had various types of baseline clinical data, including CT scan images, digital PD-1 or PD-L1 immunohistochemical slices, and clinical genomic sequencing data, along with known outcomes of immune checkpoint inhibitor treatment. The authors demonstrate that using routinely collected clinical data to inform treatment decisions has the potential to enhance the predictive biomarkers currently in use.
The authors developed a comprehensive deep learning model called Dynamic Attention and Masking (DyAM), which integrates annotated CT scans, digital PD-1 and PD-L1 immunohistochemical slices, and common genomic alterations in NSCLC. In each patient’s CT scan, radiologists segmented the tumors. Quantitative image features were extracted from the segmented regions to describe the spatial distribution of CT signal intensity for each tumor. In each patient’s digital PD-L1 immunohistochemical slice, the authors described the spatial organization of PD-L1 immune reactivity within the tumor. Utilizing these radiological, histopathological, and genomic features, the authors developed a deep learning model inspired by a paper that describes attention-based deep multiple-instance learning. The authors’ model employed “attention,” a machine learning technique that enables the model to consider each patient’s patterns with different weights, maximizing the predictive capability of each data type. In cases where patients had missing data, such as undivided disease in the CT scan or lack of CD274 expression, the authors’ model could mask the missing data patterns and still compute risk predictions.
The authors found that DyAM predicted immunotherapy responses more accurately than independent analyses of radiology, histopathology, and genomic alterations. DyAM was also able to distinguish between patients classified as immunotherapy responders and non-responders early in the treatment.
In summary, the authors demonstrate the potential of multimodal integration of routinely acquired clinical data to improve predictions of NSCLC immunotherapy responses. This approach is also applicable to other treatment settings and other cancers and diseases.
The authors’ method serves as a principled proof that integrating multiple data patterns obtained from routine clinical diagnostic work can yield better predictions of immunotherapy responses. Since there is no inherent limit to the number of patterns or data types that can be incorporated into DyAM, the authors’ model can be modified or extended to include other data patterns and applied to other cancer types, and even diseases outside of oncology.
The greatest limitation in the authors’ work stems from the size of the cohort (247 patients with both clinical and genomic data, of which 187 patients had radiological results and 105 patients had pathological results) and the restriction to a single center, both of which may limit the generalizability of the authors’ results. These factors prompted the authors to use annotations from domain experts to enhance the standardization of feature extraction, facilitating model convergence. However, starting from less processed data will allow the authors’ risk prediction model to learn in a data-driven manner which features are best associated with treatment responses.
Replicating the authors’ findings in independent patient cohorts would strengthen the case for multimodal integration. Ultimately, the authors’ method may lead to the generation of more robust predictive biomarkers and broadly improve precision oncology.
Original Link: https://www.nature.com/articles/s43018-022-00429-3
More High-End Job Information, Just in Doctoral Recruitment and Talent Circle
We Invite You to Join the Pharmaceutical and Scientific Research Circle, Participate in Hot Topic Discussions
Joining the Pharmaceutical and Scientific Research Circle Will Ensure You Don’t Miss Any Posts from Experts in the Circle
Many members have already joined the circle, and everyone’s questions can be answered in a timely manner.
