Article Contents
ARTICLE   Open Access     Cite

BC-SMART: Multimodal AI framework for breast cancer neoadjuvant chemotherapy response prediction and multidisciplinary evaluation

    Show all affliationsShow less
More Information
  • DownLoad: Full size image
    1. We developed BC-SMART to predict breast cancer response before neoadjuvant chemotherapy.

      BC-SMART integrates pretreatment magnetic resonance imaging, biopsy whole-slide images, and clinical data.

      The model performed well in both internal and external test cohorts.

      Heatmaps and feature attribution improved interpretation of multimodal predictions.

      BC-SMART may support pretreatment risk assessment and individualized treatment planning.

  • Accurate pretreatment prediction of pathologic complete response (pCR) after neoadjuvant chemotherapy (NAC) remains challenging because MRI, biopsy pathology, and clinicopathologic factors capture complementary but incomplete aspects of treatment sensitivity. We developed BC-SMART, a lightweight and interpretable multimodal artificial intelligence framework integrating pretreatment dynamic contrast-enhanced MRI, hematoxylin-eosin-stained biopsy whole-slide images, and routine clinicopathologic variables for pCR prediction in breast cancer. This multicenter retrospective study included 705 women from four institutions, divided into training, internal test, and external test cohorts. The MRI branch used a slice-wise attention network, the pathology branch used an enhanced multiple-instance learning model, and modality-specific scores were combined with clinical variables through transparent decision-level logistic-regression fusion. BC-SMART achieved areas under the receiver operating characteristic curve of 0.903 and 0.901, with accuracies of 0.872 and 0.829, in the internal and external test cohorts, respectively, outperforming clinical, MRI-only, and pathology-only models. Heatmap visualization and Shapley value analysis showed clinically plausible tumor, peritumoral, and tumor-stroma interface regions and quantified the relative contributions of each modality. In a multidisciplinary reader study supported by a prototype web-based interface, BC-SMART improved the performance and confidence of both junior and senior teams. These findings suggest that BC-SMART may provide a practical and interpretable tool for pretreatment pCR prediction and AI-assisted multidisciplinary decision-making in breast cancer.
  • 加载中
  • [1] Bray F., Laversanne M., Sung H., et al. (2024). Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 74:229−263. DOI:10.3322/caac.21834

    View in Article CrossRef Google Scholar

    [2] Tamirisa N. and Hunt K.K. (2022). Neoadjuvant chemotherapy, endocrine therapy, and targeted therapy for breast cancer: ASCO guideline. Ann. Surg. Oncol. 29:1489−1492. DOI:10.1245/s10434-021-11223-3

    View in Article CrossRef Google Scholar

    [3] Derks M.G.M. and Van de Velde C.J.H. (2018). Neoadjuvant chemotherapy in breast cancer: More than just downsizing. Lancet Oncol. 19:2−3. DOI:10.1016/S1470-2045(17)30914-2

    View in Article CrossRef Google Scholar

    [4] Cortazar P., Zhang L., Untch M., et al. (2014). Pathological complete response and long-term clinical benefit in breast cancer: The CTNeoBC pooled analysis. Lancet 384:164−172. DOI:10.1016/S0140-6736(13)62422-8

    View in Article CrossRef Google Scholar

    [5] Boman C., Tranchell C., Liu X., et al. (2025). Prognosis after pathologic complete response to neoadjuvant therapy in early-stage breast cancer: A population-based study. J. Natl. Compr. Canc. Netw. 23. DOI:10.6004/jnccn.2024.7093

    View in Article Google Scholar

    [6] Fayanju O.M., Ren Y., Thomas S.M., et al. (2018). The clinical significance of breast-only and node-only pathologic complete response (pCR) after neoadjuvant chemotherapy (NACT): A review of 20,000 breast cancer patients in the National Cancer Data Base (NCDB). Ann. Surg. 268:591−601. DOI:10.1097/SLA.0000000000002953

    View in Article CrossRef Google Scholar

    [7] Dong M., Chen J., Lu N., et al. (2025). Unraveling breast cancer response to neoadjuvant chemotherapy through integrated genomic, transcriptomic, and circulating tumor DNA analysis. Breast Cancer Res. 27:64. DOI:10.1186/s13058-025-02026-5

    View in Article CrossRef Google Scholar

    [8] Sammut S.J., Crispin-Ortuzar M., Chin S.-F., et al. (2022). Multi-omic machine learning predictor of breast cancer therapy response. Nature 601:623−629. DOI:10.1038/s41586-021-04278-5

    View in Article CrossRef Google Scholar

    [9] Kim J., Yu D., Kwon Y., et al. (2020). Genomic characteristics of triple-negative breast cancer nominate molecular subtypes that predict chemotherapy response. Mol. Cancer Res. 18:253−263. DOI:10.1158/1541-7786.MCR-19-0453

    View in Article CrossRef Google Scholar

    [10] Valenza C., Saldanha E.F., Gong Y., et al. (2025). Circulating tumor DNA clearance as a predictive biomarker of pathologic complete response in patients with solid tumors treated with neoadjuvant immune checkpoint inhibitors: A systematic review and meta-analysis. Ann. Oncol. 36:726−736. DOI:10.1016/j.annonc.2025.03.019

    View in Article CrossRef Google Scholar

    [11] Wei W., Ma M., and Liu Z. (2024). Research progress of artificial intelligence in evaluating the efficacy of neoadjuvant chemotherapy for breast cancer. EngMedicine 1:100024. DOI:10.1016/j.engmed.2024.100024

    View in Article CrossRef Google Scholar

    [12] Gradishar W.J., Moran M.S., Abraham J., et al. (2025). NCCN Guidelines® Insights: Breast Cancer, Version 5.2025. J. Natl. Compr. Canc. Netw. 23:426-436. DOI:10.6004/jnccn.2025.0053

    View in Article Google Scholar

    [13] Mann R.M., Kuhl C.K., Kinkel K., et al. (2008). Breast MRI: Guidelines from the European society of breast imaging. Eur. Radiol. 18:1307−1318. DOI:10.1007/s00330-008-0863-7

    View in Article CrossRef Google Scholar

    [14] Huang Y., Wang X., Cao Y., et al. (2025). Nomogram for predicting neoadjuvant chemotherapy response in breast cancer using MRI-based intratumoral heterogeneity quantification. Radiology 315:e241805. DOI:10.1148/radiol.241805

    View in Article CrossRef Google Scholar

    [15] Tang W., Jin C., Kong Q., et al. (2025). Development and validation of an MRI spatiotemporal interaction model for early noninvasive prediction of neoadjuvant chemotherapy response in breast cancer: A multicentre study. EClinicalMedicine 85:103298. DOI:10.1016/j.eclinm.2025.103298

    View in Article CrossRef Google Scholar

    [16] Liang Y., Tang W., Kong Q., et al. (2025). Localized intra- and inter-tumoral heterogeneity for predicting treatment response to neoadjuvant chemotherapy in breast cancer. IEEE J. Biomed. Health Inform. 30:6174−6185. DOI:10.1109/JBHI.2025.3647872

    View in Article CrossRef Google Scholar

    [17] Kashyap A., Rapsomaniki M.A., Barros V., et al. (2022). Quantification of tumor heterogeneity: From data acquisition to metric generation. Trends Biotechnol. 40:647−676. DOI:10.1016/j.tibtech.2021.11.006

    View in Article CrossRef Google Scholar

    [18] Ogier du Terrail J., Leopold A., Joly C., et al. (2023). Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat. Med. 29:135−146. DOI:10.1038/s41591-022-02155-w

    View in Article CrossRef Google Scholar

    [19] Yu J.-G., Wu Z., Ming Y., et al. (2023). Prototypical multiple instance learning for predicting lymph node metastasis of breast cancer from whole-slide pathological images. Med. Image Anal. 85:102748. DOI:10.1016/j.media.2023.102748

    View in Article CrossRef Google Scholar

    [20] Qu H., Zhou M., Yan Z., et al. (2021). Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precis. Oncol. 5:87. DOI:10.1038/s41698-021-00225-9

    View in Article CrossRef Google Scholar

    [21] Feng L., Liu Z., Li C., et al. (2022). Development and validation of a radiopathomics model to predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: A multicentre observational study. Lancet Digit. Health 4:e8−e17. DOI:10.1016/S2589-7500(21)00215-6

    View in Article CrossRef Google Scholar

    [22] Wang J., Dai J., Cheng Y., et al. (2025). Advances in the use of radiomics and pathomics for predicting the efficacy of neoadjuvant therapy in tumors. Transl. Oncol. 58:102435. DOI:10.1016/j.tranon.2025.102435

    View in Article CrossRef Google Scholar

    [23] Zhang W., Zhang S., You J., et al. (2025). Attention-based multimodal fusion transformer for predicting the efficacy of neoadjuvant therapy in breast cancer: a cross-institutional retrospective study. Breast Cancer Res. 28:4. DOI:10.1186/s13058-025-02181-9

    View in Article Google Scholar

    [24] Mao N., Dai Y., Zhou H., et al. (2025). A multimodal and fully automated system for prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer. Sci. Adv. 11:eadr1576. DOI:10.1126/sciadv.adr1576

    View in Article CrossRef Google Scholar

    [25] Zhao Z., Chen D., Wei X., et al. (2025). A multimodal synergistic model for personalized neoadjuvant immunochemotherapy in esophageal cancer. Cell Rep. Med. 6:102479. DOI:10.1016/j.xcrm.2025.102479

    View in Article CrossRef Google Scholar

    [26] Ghasemi A., Hashtarkhani S., Schwartz D.L., et al. (2024). Explainable artificial intelligence in breast cancer detection and risk prediction: A systematic scoping review. Cancer Innov. 3:e136. DOI:10.1002/cai2.136

    View in Article CrossRef Google Scholar

    [27] Li J. and Jiang Z. (2024). Chinese society of clinical oncology breast cancer (CSCO BC) guidelines in 2024: International contributions from China. Cancer Biol. Med. 21:838−843. DOI:10.20892/j.issn.2095-3941.2024.0374

    View in Article CrossRef Google Scholar

    [28] Guo J., Chen B., Cao H., et al. (2024). Cross-modal deep learning model for predicting pathologic complete response to neoadjuvant chemotherapy in breast cancer. NPJ Precis. Oncol. 8:189. DOI:10.1038/s41698-024-00678-8

    View in Article CrossRef Google Scholar

    [29] Lin H., Hua J., Gong Z., et al. (2025). Multimodal radiopathological integration for prognosis and prediction of adjuvant chemotherapy benefit in resectable lung adenocarcinoma: a multicentre study. Cancer Lett. 616:217557. DOI:10.1016/j.canlet.2025.217557

    View in Article CrossRef Google Scholar

    [30] Hu C., Qiao X., Huang R., et al. (2024). Development and validation of a multimodality model based on whole-slide imaging and biparametric MRI for predicting postoperative biochemical recurrence in prostate cancer. Radiol. Imaging Cancer 6:e230143. DOI:10.1148/rycan.230143

    View in Article CrossRef Google Scholar

    [31] He J., Xu J., Chen W., et al. (2025). Development of a deep learning model for T1N0 gastric cancer diagnosis using 2.5D radiomic data in preoperative CT images. NPJ Precis. Oncol. 9:249. DOI:10.1038/s41698-025-01055-9

    View in Article Google Scholar

    [32] Cen Y., Nong H., Du D., et al. (2025). CT-based 2.5D deep learning-multi-instance learning for predicting early recurrence of hepatocellular carcinoma and correlating with recurrence-related pathological indicators. J. Hepatocell. Carcinoma 12:2095-2108. DOI:10.2147/JHC.S541402

    View in Article Google Scholar

    [33] Mai W., Fan X., Zhang L., et al. (2025). Deep learning for differential diagnosis of parotid tumors based on 2.5D magnetic resonance imaging. Ann. Med. 57:2520401. DOI:10.1080/07853890.2025.2520401

    View in Article Google Scholar

    [34] Lu M.Y., Williamson D.F.K., Chen T.Y., et al. (2021). Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5:555−570. DOI:10.1038/s41551-020-00682-w

    View in Article CrossRef Google Scholar

    [35] Shao Z., Bian H., Chen Y., et al. (2021). TransMIL: Transformer based correlated multiple instance learning for whole slide image classification. Adv. Neural Inf. Process. Syst. 34:2136−2147. DOI:10.48550/arXiv.2106.00908

    View in Article CrossRef Google Scholar

    [36] Ilse M., Tomczak J.M. and Welling M. (2018). Attention-based deep multiple instance learning. Proceedings of the 35th International Conference on Machine Learning 80:2127-2136. https://proceedings.mlr.press/v80/ilse18a.html

    View in Article Google Scholar

    [37] Houssami N., Macaskill P., von Minckwitz G., et al. (2012). Meta-analysis of the association of breast cancer subtype and pathologic complete response to neoadjuvant chemotherapy. Eur. J. Cancer 48:3342−3354. DOI:10.1016/j.ejca.2012.05.023

    View in Article CrossRef Google Scholar

    [38] Asaoka M., Narui K., Suganuma N., et al. (2019). Clinical and pathological predictors of recurrence in breast cancer patients achieving pathological complete response to neoadjuvant chemotherapy. Eur. J. Surg. Oncol. 45:2289−2294. DOI:10.1016/j.ejso.2019.08.001

    View in Article CrossRef Google Scholar

    [39] Rudin C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1:206−215. DOI:10.1038/s42256-019-0048-x

    View in Article CrossRef Google Scholar

    [40] Eke C. and Shuib L. (2025). The role of explainability and transparency in fostering trust in AI healthcare systems: A systematic literature review, open issues and potential solutions. Neural Comput. Appl. 37:1999−2034. DOI:10.1007/s00521-024-10868-x

    View in Article CrossRef Google Scholar

    [41] Tonekaboni S., Joshi S., McCradden M.D., et al. (2019). What clinicians want: Contextualizing explainable machine learning for clinical end use. Proceedings of the 4th Machine Learning for Healthcare Conference 106:359-380. https://proceedings.mlr.press/v106/tonekaboni19a.html

    View in Article Google Scholar

    [42] Gaube S., Suresh H., Raue M., et al. (2021). Do as AI say: Susceptibility in deployment of clinical decision-aids. npj Digit. Med. 4:31. DOI:10.1038/s41746-021-00385-9

    View in Article Google Scholar

    [43] Topol E.J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nat. Med. 25:44−56. DOI:10.1038/s41591-018-0300-7

    View in Article CrossRef Google Scholar

    [44] Jha S. and Topol E.J. (2016). Adapting to artificial intelligence: Radiologists and pathologists as information specialists. JAMA 316:2353−2354. DOI:10.1001/jama.2016.17438

    View in Article CrossRef Google Scholar

  • Cite this article:

    Tang W., Weng Z., Chen Y., et al. (2026). BC-SMART: Multimodal AI framework for breast cancer neoadjuvant chemotherapy response prediction and multidisciplinary evaluation. The Innovation Oncology 1:100042. https://doi.org/10.59717/j.xinn-oncol.2026.100042
    Tang W., Weng Z., Chen Y., et al. (2026). BC-SMART: Multimodal AI framework for breast cancer neoadjuvant chemotherapy response prediction and multidisciplinary evaluation. The Innovation Oncology 1:100042. https://doi.org/10.59717/j.xinn-oncol.2026.100042

Welcome!

To request copyright permission to republish or share portions of our works, please visit Copyright Clearance Center's (CCC) Marketplace website at marketplace.copyright.com.

Figures(7)     Tables(3)

Supplementary Information

Share

  • Share the QR code with wechat scanning code to friends and circle of friends.

Article Metrics

Article views(566) PDF downloads(216)

Relative Articles

Cited by

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint