On August 26, 2024, Professor Shen Lin’s team from Peking University Cancer Hospital and Professor Dong Bin’s team from Peking University published a groundbreaking research article titled “Predicting gastric cancer response to anti-HER2 therapy or anti-HER2 combined immunotherapy based on multi-modal data” in Signal Transduction and Targeted Therapy (Impact Factor: 40.8). This study developed a multi-modal tumor treatment response prediction model named MuMo using artificial intelligence technology, aggregating a multi-center data cohort of 429 HER2-positive gastric cancer patients, covering multi-modal information such as imaging images, structured imaging reports, pathological images, structured pathological reports, and detailed clinical information. The MuMo model breaks the limitations of traditional single data modes, comprehensively capturing the disease characteristics of patients and effectively addressing the potential missing data issues in clinical modal data.
In the field of gastric cancer treatment, there are significant individual differences in the response to anti-HER2 treatment for HER2-positive patients, which poses many challenges for clinical decision-making. In recent years, the application of artificial intelligence (AI) technology in predicting cancer treatment response has been increasing, with its core task being to predict the extent of response to specific treatment plans using initial treatment data from patients. This prediction can help doctors understand potential treatment outcomes early and choose the best treatment strategy, aiming to maximize treatment efficacy and extend patient survival. However, most current studies still rely on single-modal data, such as using only imaging (like CT) or pathological data (like H&E stained scanned slices), which limits the model’s ability to capture the complex heterogeneity among patients and fails to fully reflect disease characteristics, especially when dealing with HER2-positive gastric cancer patients. Therefore, there is an urgent need to develop an AI model that can integrate multi-modal clinical data to comprehensively analyze and accurately predict patients’ responses to treatment, providing a scientific basis for personalized treatment strategies.
Figure 1: AI-based multi-modal cancer treatment response prediction model (MuMo)
In this study, Professor Shen Lin’s team from Peking University Cancer Hospital and Professor Dong Bin’s team from Peking University jointly collected a large sample multi-center dataset of HER2-positive gastric cancer patients, covering clinical information, imaging images, structured imaging reports, pathological images, and structured pathological reports of 429 patients. Based on this dataset, this study developed a novel AI-driven multi-modal cancer treatment response prediction model, MuMo (Figure 1). This model can effectively integrate data from different modalities, comprehensively characterize patients’ disease features, and address the issue of missing modal data in clinical scenarios, achieving precise predictions of HER2-positive gastric cancer patients’ responses to anti-HER2 treatment.
Figure 2: Performance evaluation of the MuMo model
The study showed that the MuMo model achieved significant results in predicting the responses of HER2-positive gastric cancer patients to anti-HER2 treatment and combined immunotherapy, with AUC values of 0.821 and 0.914 respectively (Figure 2). These results not only outperform predictions made by individual doctors but are also comparable to the consultation results of six doctors. Furthermore, the predictions made by the MuMo model can effectively classify patients into high-risk and low-risk groups, providing more targeted treatment recommendations. Additionally, the AI-based MuMo model demonstrated more stable and consistent prediction results than manual assessments, and further analysis revealed a high consistency between the predictions of the MuMo model and existing clinical knowledge. This series of findings highlights the importance of multi-modal data analysis in improving efficacy evaluation and achieving personalized medicine, showcasing the potential value and practicality of AI models in clinical settings.
Figure 3: Cancer treatment response prediction analysis method based on multi-modal data
With the continuous advancement of medical data analysis technology, integrating multi-modal data has become key to improving the accuracy of treatment response predictions and achieving personalized treatment. This study provides new perspectives and strategies for predicting the responses of HER2-positive gastric cancer patients to anti-HER2 treatment, and this method is expected to be applied in a broader range of cancer treatment fields in the future, offering patients more precise treatment options and improving survival expectations (Figure 3). This study reflects the collaboration of the Big Data Science Research Center, Beijing International Center for Mathematical Research, and Peking University Cancer Hospital, with the first authors being Chen Zifan, Chen Yang, Sun Yu, Tang Lei, and Zhang Li, and the corresponding authors being Professors Zhang Xiaotian, Dong Bin, and Shen Lin.
Team Introduction
Dong Bin, Peking University, tenured professor at the Beijing International Center for Mathematical Research, deputy director of the International Machine Learning Research Center, researcher at the National Biomedical Imaging Science Center, and deputy dean of the Peking University Changsha Institute of Computing and Digital Economy. He graduated with a bachelor’s degree from the School of Mathematical Sciences at Peking University in 2003, obtained a master’s degree from the Department of Mathematics at the National University of Singapore in 2005, and received a Ph.D. from the Department of Mathematics at the University of California, Los Angeles in 2009. After obtaining his Ph.D., he served as a visiting assistant professor in the Department of Mathematics at the University of California, San Diego, and an assistant professor in the Department of Mathematics at the University of Arizona from 2011 to 2014, before joining Peking University at the end of 2014. His main research areas include scientific computing, computational imaging, and machine learning. In 2014, he received the Qiusuo Outstanding Young Scholar Award and was invited to give a 45-minute presentation at the International Congress of Mathematicians (ICM) in 2022.
Shen Lin, Peking University Cancer Hospital, chief physician, professor, Beijing Scholar, expert with outstanding contributions in Beijing, and chief scientist of the National Key R&D Program on Chronic Diseases. He has long been committed to precision treatment and translational research of digestive tract tumors and clinical research of new anti-tumor drugs. He has served as the deputy dean of Peking University Cancer Hospital and the deputy director of the Beijing Cancer Prevention and Treatment Research Institute. He is currently the director of the Department of Gastrointestinal Oncology, director of the Phase I clinical trial ward, chairman of the Precision Medicine Committee of the Chinese Anti-Cancer Association, first chairman of the Clinical Research Committee of the Chinese Anti-Cancer Association, director of the Clinical Research Expert Committee of the Chinese Clinical Oncology Society, incoming chairman of the Gastric Cancer Expert Committee of the Chinese Clinical Oncology Society, vice chairman of the Colorectal Cancer Committee of the Chinese Anti-Cancer Association, and chairman of the Clinical Oncology Committee of the Chinese Women Physicians Association.