Interpretable and robust multimodal data integration for precise treatment response and survival prediction in gastric cancer

Fengtao Zhou; Yingxue Xu; Yanfen Cui; Shenyan Zhang; Yun Zhu; Weiyang He; Dafu Zhang; Jiguang Wang; Xin Wang; Ronald CK Chan; Louis Ho Shing Lau; Chu Han; Zhenhui Li; Hao Chen

doi:10.59717/j.xinn-inform.2025.100011

The Innovation Informatics > 2025 Vol. 1 > No. 1 > 100011

Next Article Previous Article

ARTICLE Open Access

Cite PDF

Interpretable and robust multimodal data integration for precise treatment response and survival prediction in gastric cancer

1.
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR 999077, China
2.
Department of Radiology, Shanxi Cancer Hospital/Shanxi Hospital Affiliated to Cancer Hospital, Taiyuan 030013, China
3.
Department of Pathology, The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou 510655, China
4.
Department of Radiology, The First Affiliated Hospital of Kunming Medical University, Kunming 650032, China
5.
Department of Gastrointestinal Surgery, Sichuan Province Cancer Hospital, University of Electronic Science and Technology of China, Chengdu 610054, China
6.
Department of Radiology, The Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Kunming 650118, China
7.
Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR 999077, China
8.
Department of Surgery, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong SAR 999077, China
9.
Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong SAR 999077, China
10.
Department of Medicine & Therapeutics, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong SAR 999077, China
11.
Department of Radiology, Guangdong Provincial People’s Hospital, Southern Medical University, Guangzhou 510000, China
12.
Division of Life Science, The Hong Kong University of Science and Technology, Hong Kong SAR 999077, China
13.
HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, Shenzhen 518000, China
14.
State Key Laboratory of Nervous System Disorders, The Hong Kong University of Science and Technology, Hong Kong SAR 999077, China

Show all affliationsShow less

More Information

Corresponding authors: lizhenhui@kmmu.edu.cn (Z.L.); jhc@cse.ust.hk (H.C.)
Received Date: September 16, 2025

Accepted Date: October 27, 2025

Published Online: November 06, 2025

The Innovation Informatics 1(1), https://doi.org/10.59717/j.xinn-inform.2025.100011 | Cite this article

GRAPHICAL ABSTRACT

GRAPHICAL ABSTRACT

DownLoad: Full size image

PUBLIC SUMMARY

PUBLIC SUMMARY
1. This study proposes an interpretable and robust multimodal fusion framework for gastric cancer analysis.
  
  This model robustly integrates multiple data sources to predict treatment response and patient survival.
  
  This model shows reliable performance across multiple datasets and can handle incomplete data scenarios.
  
  Its interpretability identifies key factors, providing trustworthy insights for treatment personalization.

ABSTRACT

ABSTRACT

Gastric cancer (GC) was the fifth most prevalent cancer in 2022. Integrating multimodal data enables significant improvement in precise GC analysis. However, the heterogeneity and incompleteness of multimodal data pose challenges for effective integration. The lack of interpretability in models hinders their trustworthiness in clinical practice. Therefore, this study presented an interpretable and robust multimodal data integration framework for GC analysis (iMD4GC), facilitating effective integration of diverse data sources to predict treatment response and overall survival. Three multimodal datasets were collected for model evaluation: GastricRes (n=698) for response prediction, GastricSur (n=801) for survival analysis, and TCGA-STAD (n=400) for further survival analysis. The iMD4GC achieved an area under the receiver operating characteristic curve (AUC) of 0.803 (95% CI 0.680-0.925) on GastricRes, a concordance index (c-index) of 0.715 (95% CI 0.646-0.784) and a time-dependent AUC of 0.739 (95% CI 0.644-0.833) on GastricSur, and a c-index of 0.661 (95% CI 0.504-0.818) and a time-dependent AUC of 0.690 (95% CI 0.500-0.881) on TCGA-STAD. The comparative analysis underscores the reliability of iMD4GC in integrating diverse multimodal data and addressing missing modalities. Further interpretability analysis identified pivotal factors across various data modalities, validating the alignment with clinical expertise and providing essential insights for informed decision-making in clinical contexts. The promising performance of iMD4GC highlights its potential in advancing clinical decision-making, personalizing treatment strategies, and improving patient prognosis in GC. Inherent interpretability fosters transparent decision analysis and provides valuable insights for clinicians, promoting its trustworthiness and acceptance in clinical settings.
- Gastric Cancer /
- Multimodal Data Integration /
- Neoadjuvant Chemotherapy /
- Overall Survival

FullText(HTML)

REFERENCES

[1]	Bray F., Laversanne M., Sung H., et al. (2024). Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA-Cancer J. Clin. 74:229−263. DOI:10.3322/caac.21834 View in Article CrossRef Google Scholar
[2]	Zurlo I.V., Schino M., Strippoli A., et al. (2022). Predictive value of NLR, TILs (CD4+/CD8+) and PD-L1 expression for prognosis and response to preoperative chemotherapy in gastric cancer. Cancer Immunol. Immunother. 71:45−55. DOI:10.1007/s00262-021-02960-1 View in Article CrossRef Google Scholar
[3]	Liu Z., Wang Y., Shan F., et al. (2021). Treatment switch in poor responders with locally advanced gastric cancer after neoadjuvant chemotherapy. Ann. Surg. Oncol. 28:8892−8907. DOI:10.1245/s10434-021-10087-x View in Article CrossRef Google Scholar
[4]	Yoshikawa T., Sasako M., Yamamoto S., et al. (2009). Phase II study of neoadjuvant chemotherapy and extended surgery for locally advanced gastric cancer. Br. J. Surg. 96:1015−1022. DOI:10.1002/bjs.6665 View in Article CrossRef Google Scholar
[5]	Lowy A.M., Mansfield P.F., Leach S.D., et al. (1999). Response to neoadjuvant chemotherapy best predicts survival after curative resection of gastric cancer. Ann. Surg. 229:303−308. DOI:10.1097/00000658-199903000-00001 View in Article CrossRef Google Scholar
[6]	Fu J., Tang L., Li Z.Y., et al. (2020). Diffusion kurtosis imaging in the prediction of poor responses of locally advanced gastric cancer to neoadjuvant chemotherapy. Eur. J. Radiol. 128:108974. DOI:10.1016/j.ejrad.2020.108974 View in Article CrossRef Google Scholar
[7]	Zhang J., Cui Y., Wei K., et al. (2022). Deep learning predicts resistance to neoadjuvant chemotherapy for locally advanced gastric cancer: A multicenter study. Gastric Cancer 25:1050−1059. DOI:10.1007/s10120-022-01328-3 View in Article CrossRef Google Scholar
[8]	Wang S., Zhu Y., Yu L., et al. (2019). RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification. Med. Image Anal. 58:101549. DOI:10.1016/j.media.2019.101549 View in Article CrossRef Google Scholar
[9]	Yu H., Zhang X., Song L., et al. (2021). Large-scale gastric cancer screening and localization using multi-task deep neural network. Neurocomputing 448:290−300. DOI:10.1016/j.neucom.2021.03.006 View in Article CrossRef Google Scholar
[10]	Meng L., Dong D., Chen X., et al. (2020). 2D and 3D CT radiomic features performance comparison in characterization of gastric cancer: A multi-center study. IEEE J. Biomed. Health Inform. 25:755−763. DOI:10.1109/jbhi.2020.3002805 View in Article CrossRef Google Scholar
[11]	Yuan M., Xia Y., Chen X., et al. (2023). Cluster-induced mask transformers for effective opportunistic gastric cancer screening on non-contrast CT scans. Med. Image Comput. Comput.-Assist. Interv. 14223:146-156. DOI:10.1007/978-3-031-439049_15 View in Article Google Scholar
[12]	Du G., Zeng Y., Chen D., et al. (2023). Application of radiomics in precision prediction of diagnosis and treatment of gastric cancer. Jpn. J. Radiol. 41:245−257. DOI:10.1007/s11604-022-01352-4 View in Article CrossRef Google Scholar
[13]	Jiang Y., Zhou K., Sun Z., et al. (2023). Non-invasive tumor microenvironment evaluation and treatment response prediction in gastric cancer using deep learning radiomics. Cell Rep. Med. 4:101146. DOI:10.1016/j.xcrm.2023.101146 View in Article CrossRef Google Scholar
[14]	Chen T., Li X., Mao Q., et al. (2022). An artificial intelligence method to assess the tumor microenvironment with treatment outcomes for gastric cancer patients after gastrectomy. J. Transl. Med. 20:100. DOI:10.1186/s12967-022-03298-7 View in Article CrossRef Google Scholar
[15]	Li X., Zhang Y., Zhang Y., et al. (2010). Survival prediction of gastric cancer by a seven-microRNA signature. Gut 59:579−585. DOI:10.1136/gut.2008.175497 View in Article CrossRef Google Scholar
[16]	Oh S.E., Seo S.W., Choi M.G., et al. (2018). Prediction of overall survival and novel classification of patients with gastric cancer using the survival recurrent network. Ann. Surg. Oncol. 25:1153−1159. DOI:10.1016/j.ejso.2018.10.287 View in Article CrossRef Google Scholar
[17]	Chen D., Fu M., Chi L., et al. (2022) Prognostic and predictive value of a pathomics signature in gastric cancer. Nat. Commun. 13:6903. DOI:10.1038/s41467-022-34703-w View in Article Google Scholar
[18]	Xu Q., Sun Z., Li X., et al. (2021). Advanced gastric cancer: CT radiomics prediction and early detection of downstaging with neoadjuvant chemotherapy. Eur. Radiol. 31:8765−8774. DOI:10.1007/s00330-021-07962-2 View in Article CrossRef Google Scholar
[19]	Sun J., Wang X., Zhang Z., et al. (2021). The sensitivity prediction of neoadjuvant chemotherapy for gastric cancer. Front. Oncol. 11:641304. DOI:10.3389/fonc.2021.641304 View in Article CrossRef Google Scholar
[20]	Cui Y., Zhang J., Li Z., et al. (2022). A CT-based deep learning radiomics nomogram for predicting the response to neoadjuvant chemotherapy in patients with locally advanced gastric cancer: A multicenter cohort study. EClinicalMedicine 46:101348. DOI:10.1016/j.eclinm.2022.101348 View in Article CrossRef Google Scholar
[21]	Gu J., Tong T., He C., et al. (2022). Deep learning radiomics of ultrasonography can predict response to neoadjuvant chemotherapy in breast cancer at an early stage of treatment: a prospective study. Eur. Radiol. 32:2099−2109. DOI:10.1007/s00330-021-08293-y View in Article CrossRef Google Scholar
[22]	Wang X., Zhang Y., Yang M., et al. (2024). Dynamic ultrasound-based modeling predictive of response to neoadjuvant chemotherapy in patients with early breast cancer. Sci. Rep. 14:31644. DOI:10.1038/s41598-024-80409-y View in Article CrossRef Google Scholar
[23]	Hottat N.A., Badr D.A., Lecomte S., et al. (2023). Assessment of diffusion-weighted MRI in predicting response to neoadjuvant chemotherapy in breast cancer patients. Sci. Rep. 13:614. DOI:10.1038/s41598-023-27787-x View in Article CrossRef Google Scholar
[24]	Ogier du Terrail J., Leopold A., Joly C., et al. (2023). Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat. Med. 29:135−146. DOI:10.1038/s41591-022-02155-w View in Article CrossRef Google Scholar
[25]	Zhang L., Dong D., Zhang W., et al. (2020). A deep learning risk prediction model for overall survival in patients with gastric cancer: A multicenter study. Radiother. Oncol. 150:73−80. DOI:10.2139/ssrn.3514655 View in Article CrossRef Google Scholar
[26]	Zhong H., Wang T., Hou M., et al. (2024). Deep learning radiomics nomogram based on enhanced CT to predict the response of metastatic lymph nodes to neoadjuvant chemotherapy in locally advanced gastric cancer. Ann. Surg. Oncol. 31:421−432. DOI:10.1245/s10434-023-14508-x View in Article CrossRef Google Scholar
[27]	Li J., Yin H., Wang Y., et al. (2023). Multiparametric MRI-based radiomics nomogram for early prediction of pathological response to neoadjuvant chemotherapy in locally advanced gastric cancer. Eur. Radiol. 33:2746−2756. DOI:10.1007/s00330-022-09219-y View in Article CrossRef Google Scholar
[28]	Zhang Y., Chen Q., Yang Z., et al. (2019). BioWordVec, improving biomedical word embeddings with subword information and MeSH. Sci. Data 6:52. DOI:10.1038/s41597-019-0055-0 View in Article CrossRef Google Scholar
[29]	Wasserthal J., Breit H.C., Meyer M.T., et al. (2023). TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images. Radiology: Artif. Intell. 5:e230024. DOI:10.1148/ryai.230024 View in Article CrossRef Google Scholar
[30]	He K., Zhang X., Ren S., et al. (2016). Deep residual learning for image recognition. CVPR 2016:770-778. DOI:10.1109/cvpr.2016.90 View in Article Google Scholar
[31]	Vaswani A., Shazeer N., Parmar N., et al. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst. arXiv preprint. DOI:10.48550/arXiv.1706.03762. View in Article Google Scholar
[32]	Wang X., Yang S., Zhang J., et al. (2022). Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81:102559. DOI:10.1016/j.media.2022.102559 View in Article CrossRef Google Scholar
[33]	Du J., Jia P., Dai Y., et al. (2019). Gene2vec: Distributed representation of genes based on co-expression. BMC Genomics 20:7−15. DOI:10.1186/s12864-018-5370-x View in Article CrossRef Google Scholar
[34]	Tarvainen A. andValpola H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Proc. 31st Int. Conf. Neural Inf. Process. Syst. pp:1195-1204 View in Article Google Scholar
[35]	Zadeh A., Liang P.P., Mazumder N., et al. (2018). Memory fusion network for multi-view sequential learning. Proc. AAAI Conf. Artif. Intell 2018:5634-5641. DOI:10.1609/aaai.v32i1.12021 View in Article Google Scholar
[36]	Tsai Y.H.H., Bai S., Liang P.P., et al. (2019). Multimodal transformer for unaligned multimodal language sequences. Proc. 57th Annu. Meet. Assoc. Comput. Linguist 2019:6558-6569. DOI:10.18653/v1/p19-1656 View in Article Google Scholar
[37]	Cui C., Liu H., Liu Q., et al. (2022) Survival prediction of brain cancer with incomplete radiology, pathology, genomic, and demographic data. Med. Image Comput. Comput. Assist. Interv 2022:626-635. DOI:10.1007/978-3-031-16443-9_60 View in Article Google Scholar
[38]	Choromanski K.M., Likhosherstov V., Dohan D., et al. (2020). Rethinking attention with performers. International Conference on Learning Representations. arXiv. 2009.14794 DOI:10.48550/arXiv.2009.14794 View in Article Google Scholar
[39]	Xiong Y., Zeng Z., Chakraborty R., et al. (2021). Nyströmformer: A nyström-based algorithm for approximating self-attention. Proc. AAAI Conf. Artif. Intell 2021:14138-14148. DOI:10.1609/aaai.v35i16.17664 View in Article Google Scholar
[40]	Hung H. and Chiang C.T. (2010). Estimation methods for time‐dependent AUC models with survival data. Can. J. Stat. 38:8−26. DOI:10.1002/cjs.10046 View in Article CrossRef Google Scholar
[41]	Lambert J. and Chevret S. (2016). Summary measure of discrimination in survival models based on cumulative/dynamic time-dependent ROC curves. Stat. Methods Med. Res. 25:2088−2102. DOI:10.1177/0962280213515571 View in Article CrossRef Google Scholar
[42]	Sundararajan M., Taly A., and Yan Q. (2017). Axiomatic attribution for deep networks. Int. Conf. Mach. Learn 70:3319-3328. DOI:10.48550/arXiv.1703.01365 View in Article Google Scholar
[43]	Gadermayr M. and Tschuchnig M. (2024). Multiple instance learning for digital pathology: A review of the state-of-the-art, limitations & future potential. Comput. Med. Imaging Graph. 112:102337−102337. DOI:10.1016/j.compmedimag.2024.102337 View in Article CrossRef Google Scholar
[44]	Tang W., Huang S., Zhang X., et al. (2023). Multiple instance learning framework with masked hard instance mining for whole slide image classification. Proc. IEEE/CVF Int. Conf. Comput. Vis 2023:4078-4087. DOI:10.1109/ICCV51070.2023.00377 View in Article Google Scholar
[45]	Graham S., Vu Q.D., Raza S.E.A., et al. (2019). Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58:101563. DOI:10.1016/j.media.2019.101563 View in Article CrossRef Google Scholar
[46]	Selvaraju R.R., Cogswell M., Das A., et al. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proc. IEEE Int. Conf. Comput. Vis. 2017:618-626. DOI:10.1109/iccv.2017.74 View in Article Google Scholar
[47]	Yang Y.C., Wang S.W., Wu I.C., et al. (2009). A tumorigenic homeobox (HOX) gene expressing human gastric cell line derived from putative gastric stem cell. Eur. J. Gastroenterol. Hepatol. 21:1016−1023. DOI:10.1097/MEG.0b013e32831d7b9c View in Article CrossRef Google Scholar
[48]	Ma Y.Y., Zhang Y., Mou X.Z., et al. (2017). High level of homeobox A9 and PBX homeobox 3 expression in gastric cancer correlates with poor prognosis. Oncol. Lett. 14:5883−5889. DOI:10.3892/ol.2017.6937 View in Article CrossRef Google Scholar
[49]	Shenoy U.S., Adiga D., Alhedyan F., et al. (2023). HOXA9 transcription factor is a double-edged sword: From development to cancer progression. Cancer Metastasis Rev. 43:709−728. DOI:10.1007/s10555-023-10159-2 View in Article CrossRef Google Scholar
[50]	Ge S., Xia X., Ding C., et al. (2018). A proteomic landscape of diffuse-type gastric cancer. Nat. Commun. 9:1012. DOI:10.1007/s10120-021-01226-0 View in Article CrossRef Google Scholar
[51]	Yan Q., Zeng P., Zhou X., et al. (2021). RBMX suppresses tumorigenicity and progression of bladder cancer by interacting with the hnRNP A1 protein to regulate PKM alternative splicing. Oncogene 40:2635−2650. DOI:10.1038/s41388-021-01666-z View in Article CrossRef Google Scholar
[52]	Wang X., Xu K., Liao X., et al. (2022). Construction of a survival nomogram for gastric cancer based on the cancer genome atlas of m6A-related genes. Front. Genet. 13:936658. DOI:10.3389/fgene.2022.936658 View in Article CrossRef Google Scholar
[53]	Mu G., Zhu Y., Dong Z., et al. (2021). Calmodulin 2 facilitates angiogenesis and metastasis of gastric cancer via STAT3/HIF-1A/VEGF-A mediated macrophage polarization. Front. Oncol. 11:727306. DOI:10.3389/fonc.2021.727306 View in Article CrossRef Google Scholar
[54]	Sun J., Cao Q., Lin S., et al. (2023). Knockdown of CALM2 increases the sensitivity to afatinib in HER2-amplified gastric cancer cells by regulating the Akt/FoxO3a/Puma axis. Toxicol. Vitro 87:105531. DOI:10.1016/j.tiv.2022.105531 View in Article CrossRef Google Scholar
[55]	Hu Y., Chen C., Tong X., et al. (2021). NSUN2 modified by SUMO-2/3 promotes gastric cancer progression and regulates mRNA m5C methylation. Cell Death Dis. 12:842. DOI:10.1038/s41419-021-04127-3 View in Article CrossRef Google Scholar
[56]	Chen R.J., Ding T., Lu M.Y., et al. (2024). Towards a general-purpose foundation model for computational pathology. Nat. Med. 30:850−862. DOI:10.1038/s41591-024-02857-3 View in Article CrossRef Google Scholar

Cited by

ABOUT THIS ARTICLE

Cite this article:

Zhou F., Xu Y., Cui Y., et al. (2025). Interpretable and robust multimodal data integration for precise treatment response and survival prediction in gastric cancer. The Innovation Informatics 1:100011. https://doi.org/10.59717/j.xinn-inform.2025.100011

Zhou F., Xu Y., Cui Y., et al. (2025). Interpretable and robust multimodal data integration for precise treatment response and survival prediction in gastric cancer. The Innovation Informatics 1:100011. https://doi.org/10.59717/j.xinn-inform.2025.100011

Welcome!

To request copyright permission to republish or share portions of our works, please visit Copyright Clearance Center's (CCC) Marketplace website at marketplace.copyright.com.

PDF

Figures(6) Tables(2)

Article Metrics

Article views(3101) PDF downloads(1748)

Interpretable and robust multimodal data integration for precise treatment response and survival prediction in gastric cancer

GRAPHICAL ABSTRACT

PUBLIC SUMMARY

ABSTRACT

REFERENCES

ABOUT THIS ARTICLE

Cite this article:

Welcome!

Share

Article Metrics

Relative Articles

Cited by

Catalog

{{newsColumn.name}}

Official Account

{{newsColumn.name}}

Stay connected

{{newsColumn.name}}

Interpretable and robust multimodal data integration for precise treatment response and survival prediction in gastric cancer

GRAPHICAL ABSTRACT

PUBLIC SUMMARY

ABSTRACT

REFERENCES

ABOUT THIS ARTICLE

Cite this article:

Welcome!

Share

Article Metrics

Relative Articles

Cited by

Catalog

Export File

Citation

Format

Content

{{newsColumn.name}}

Official Account

{{newsColumn.name}}

Stay connected

{{newsColumn.name}}