On August 26, 2024, Professor Shen Lin’s team from Peking University Cancer Hospital and Professor Dong Bin’s team from Peking University published a groundbreaking research article titled “Predicting gastric cancer response to anti-HER2 therapy or anti-HER2 combined immunotherapy based on multi-modal data” in the journal Signal Transduction and Targeted Therapy (Impact Factor: 40.8). This study developed a multimodal tumor treatment response prediction model, MuMo, using artificial intelligence technology, and gathered a multicenter data cohort of 429 HER2-positive gastric cancer patients, covering multimodal information such as imaging images, structured imaging reports, pathological images, structured pathological reports, and detailed clinical information. The MuMo model breaks the limitations of traditional single data modes, comprehensively capturing the disease characteristics of patients and effectively addressing the potential missing data issues in clinical multimodal data. The progress of this research has the potential to provide more precise treatment plans for HER2-positive gastric cancer patients, demonstrating the significant supporting role of multimodal analysis technology in clinical decision-making.
In the field of gastric cancer treatment, there are significant individual differences in the response of HER2-positive patients to anti-HER2 therapy, which presents many challenges for clinical decision-making. In recent years, the application of artificial intelligence (AI) technology in predicting cancer treatment responses has been increasing, with the core task being to use initial treatment data from patients to predict their response to specific treatment plans. This prediction can help doctors understand the possible outcomes of treatment early and choose the best treatment strategy, aiming to maximize treatment effectiveness and extend patient survival. However, most current studies still rely on single-modal data, such as using only imaging (e.g., CT) or pathological data (e.g., H&E stained scanned slices), which limits the model’s ability to capture the complex heterogeneity among patients and fails to comprehensively reflect disease characteristics, especially when dealing with HER2-positive gastric cancer patients. Therefore, there is an urgent need to develop an AI model that can integrate multimodal clinical data to comprehensively analyze and accurately predict patient responses to treatment, providing a scientific basis for personalized treatment strategies.
Figure 1: AI Technology-Based Multimodal Cancer Treatment Response Prediction Model (MuMo)
In this study, Professor Shen Lin’s team from Peking University Cancer Hospital and Professor Dong Bin’s team from Peking University jointly collected a large sample multicenter dataset of HER2-positive gastric cancer patients, covering clinical information, imaging images, structured imaging reports, pathological images, and structured pathological reports from 429 patients. Based on this dataset, this study developed a novel AI-driven multimodal cancer treatment response prediction model, MuMo (Figure 1). This model can effectively integrate data from different modalities, comprehensively depict patients’ disease characteristics, and address the issue of missing modality data in clinical scenarios, achieving precise predictions of HER2-positive gastric cancer patients’ responses to anti-HER2 therapy.
Figure 2: Performance Evaluation of the MuMo Model
Research shows that the MuMo model achieved significant results in predicting the responses of HER2-positive gastric cancer patients to anti-HER2 treatment and combined immunotherapy, with AUC values of 0.821 and 0.914 respectively (Figure 2). These results not only outperform the predictions made by individual doctors but are also comparable to the consultation results of six doctors. In addition, the predictions made by the MuMo model can effectively categorize patients into high-risk and low-risk groups, thus providing more targeted treatment recommendations. Furthermore, the AI-driven MuMo model demonstrates more stable and consistent prediction results compared to manual assessments, and further analysis reveals a high consistency between the predictions of the MuMo model and existing clinical knowledge. This series of results highlights the importance of multimodal data analysis in improving efficacy assessment and achieving personalized medicine, and showcases the potential value and practicality of AI models in clinical settings.
Figure 3: Cancer Treatment Response Prediction Analysis Method Based on Multimodal Data
With the continuous advancement of medical data analysis technology, integrating multimodal data has become key to improving the accuracy of treatment response predictions and achieving personalized treatment. This study provides a new perspective and strategy for predicting the responses of HER2-positive gastric cancer patients to anti-HER2 treatment, and in the future, this approach is expected to be applied in a wider range of cancer treatment fields, offering patients more precise treatment options and improving survival expectations (Figure 3). This research reflects the collaboration of the Big Data Science Research Center at Peking University, the Beijing International Center for Mathematical Research at Peking University, and Peking University Cancer Hospital, with the first authors being Chen Zifan, Chen Yang, Sun Yu, Tang Lei, and Zhang Li, and the corresponding authors being Professors Zhang Xiaotian, Dong Bin, and Shen Lin.
Team Introduction
Dong Bin, a distinguished professor at Peking University, serves at the Beijing International Center for Mathematical Research, and is also the deputy director of the International Machine Learning Research Center and a researcher at the National Biomedical Imaging Science Center. He is the deputy dean of the Changsha Institute of Computing and Digital Economy at Peking University. He graduated with a bachelor’s degree from the School of Mathematical Sciences at Peking University in 2003, obtained a master’s degree in mathematics from the National University of Singapore in 2005, and earned a Ph.D. in mathematics from the University of California, Los Angeles in 2009. After completing his Ph.D., he worked as a visiting assistant professor at the Department of Mathematics at the University of California, San Diego, and served as an assistant professor at the Department of Mathematics at the University of Arizona from 2011 to 2014. He joined Peking University at the end of 2014. His main research fields are machine learning, scientific computing, and computational imaging. Dong Bin received the Qiushi Outstanding Young Scholar Award in 2014, was invited to give a 45-minute presentation at the International Congress of Mathematicians (ICM) in 2022, and was selected for the New Cornerstone Research Fellow Program in 2023, and in the same year, he received the Wang Xuan Outstanding Young Scholar Award.
Shen Lin, a chief physician and professor at Peking University Cancer Hospital, is a Beijing Scholar, a Beijing Outstanding Contribution Expert, and the chief scientist of the National Key R&D Program for Chronic Disease. He has long been committed to precision treatment and translational research for digestive tumors and clinical research on new anti-tumor drugs. He has served as the deputy director of Peking University Cancer Hospital and the deputy director of the Beijing Institute of Cancer Prevention and Treatment. He currently serves as the director of the Department of Digestive Oncology, the director of the Phase I clinical trial ward, the chair of the Precision Oncology Committee of the Chinese Anti-Cancer Association, the first chair of the Clinical Research Committee of the Chinese Anti-Cancer Association, the chair of the Clinical Research Expert Committee of the Chinese Clinical Oncology Society, the incoming chair of the Gastric Cancer Expert Committee of the Chinese Clinical Oncology Society, the vice-chair of the Colorectal Cancer Committee of the Chinese Anti-Cancer Association, and the chair of the Clinical Oncology Committee of the Chinese Women Physicians Association.
