Improving prediction of treatment response and prognosis in colorectal cancer with AI-based medical image analysis
-
GRAPHICAL ABSTRACT
DownLoad:
Full size image
-
PUBLIC SUMMARY
-
Patients with colorectal cancer respond heterogeneously to the same treatment.
Accurately predict patient response and prognosis to treatment helps to select the optimal treatment plan.
Predictions of treatment response and prognosis can be used to tailor treatment plans at different stages.
Artificial intelligence-based medical image analysis can be used for prediction tasks.
-
-
ABSTRACT
The heterogeneous response and prognosis of patients with colorectal cancer (CRC) to standard treatment regimens remains a challenge for clinical management. Individually weak prognostic markers, defined by gene mutations and protein expression, are difficult to apply in routine clinical practice because of their high acquisition cost and mediocre prediction accuracy. Visual evaluation of medical images, including radiology and digital pathology images, is an important part of CRC management. With the rapid development of artificial intelligence (AI), high-dimensional imaging features other than visual information are increasingly being used to develop imaging markers. At different stages of treatment, accurate predictions of treatment response and prognosis may help in selecting patients and tailoring their treatment. Here, we review the current state of AI applied to the medical imaging of CRC and describe its recent progress in short-term response and long-term survival prediction. In addition, we illustrate how these AI-based approaches may affect clinical decision-making. Although few approaches have been applied in routine clinical practice, their results are promising. Finally, we discuss the challenges in applying AI in clinical practice and possible future solutions from three perspectives: model interpretability, model generalizability, and patient privacy protection. This comprehensive assessment underscores the transformative potential of AI in CRC management and emphasizes the need for further exploration and integration into routine clinical workflows. -
-
REFERENCES
[1] Sung, H., Ferlay, J., Siegel, R.L., et al. (2021). Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71: 209−249. DOI: 10.3322/caac.21660. [2] Benson, A.B., Venook, A.P., Al-Hawary, M.M., et al. (2022). Rectal cancer, version 2.2022, NCCN clinical practice guidelines in oncology. J. Natl. Compr. Cancer Netw. Network 20 : 1139–1167. DOI: 10.6004/jnccn.2022.0051. [3] Keller, D.S., Berho, M., Perez, R.O., et al. (2020). The multidisciplinary management of rectal cancer. Nat. Rev. Gastroenterol. Hepatol. 17: 414−429. DOI: 10.1038/s41575-020-0275-y. [4] Bosset, J.-F., Collette, L., Calais, G., et al. (2006). Chemotherapy with preoperative radiotherapy in rectal cancer. N. Engl. J. Med. 355: 1114−1123. DOI: 10.1056/NEJMoa060829. [5] Roh, M.S., Colangelo, L.H., O’Connell, M.J., et al. (2009). Preoperative multimodality therapy improves disease-free survival in patients with carcinoma of the rectum: NSABP R-03. J. Clin. Oncol. 27: 5124. DOI: 10.1200/JCO.2009.22.0467. [6] Rödel, C., Liersch, T., Becker, H., et al. (2012). Preoperative chemoradiotherapy and postoperative chemotherapy with fluorouracil and oxaliplatin versus fluorouracil alone in locally advanced rectal cancer: initial results of the German CAO/ARO/AIO-04 randomised phase 3 trial. Lancet Oncol. 13: 679−687. DOI: 10.1016/S1470-2045(12)70187-0. [7] Park, I.J., You, Y.N., Agarwal, A., et al. (2012). Neoadjuvant treatment response as an early response indicator for patients with rectal cancer. J. Clin. Oncol. 30: 1770−1776. DOI: 10.1200/JCO.2011.39.7901. [8] Smith, J.J., Strombom, P., Chow, O.S., et al. (2019). Assessment of a watch-and-wait strategy for rectal cancer in patients with a complete response after neoadjuvant therapy. JAMA Oncol. 5: e185896. DOI: 10.1001/jamaoncol.2018.5896. [9] Glynne-Jones, R., Wyrwicz, L., Tiret, E., et al. (2017). Rectal cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann. Oncol. 28: iv22−iv40. DOI: 10.1093/annonc/mdx224. [10] Maas, M., Nelemans, P.J., Valentini, V., et al. (2015). Adjuvant chemotherapy in rectal cancer: defining subgroups who may benefit after neoadjuvant chemoradiation and resection: A pooled analysis of 3,313 patients. Int. J. Cancer 137: 212−220. DOI: 10.1002/ijc.29355. [11] Dossa, F., Acuna, S.A., Rickles, A.S., et al. (2018). Association between adjuvant chemotherapy and overall survival in patients with rectal cancer and pathological complete response after neoadjuvant chemotherapy and resection. JAMA Oncol. 4: 930−937. DOI: 10.1001/jamaoncol.2017.5597. [12] Singh, M.P., Rai, S., Pandey, A., et al. (2021). Molecular subtypes of colorectal cancer: An emerging therapeutic opportunity for personalized medicine. Genes Dis. 8: 133−145. DOI: 10.1016/j.gendis.2019.10.013. [13] Shia, J., Schultz, N., Kuk, D., et al. (2017). Morphological characterization of colorectal cancers in The Cancer Genome Atlas reveals distinct morphology–molecular associations: Clinical and biological implications. Mod. Pathol. 30: 599−609. DOI: 10.1038/modpathol.2016.198. [14] Kawakami, H., Zaanan, A., and Sinicrope, F.A. (2015). Microsatellite instability testing and its role in the management of colorectal cancer. Curr. Treat Options. Oncol. 16: 1−15. DOI: 10.1007/s11864-015-0348-2. [15] Boland, C.R., and Goel, A. (2010). Microsatellite instability in colorectal cancer. Gastroenterology 138: 2073−2087. DOI: 10.1053/j.gastro.2009.12.064. [16] Lord, A.C., D’Souza, N., Shaw, A., et al. (2022). MRI-diagnosed tumor deposits and EMVI status have superior prognostic accuracy to current clinical TNM staging in rectal cancer. Ann. Surg. 276: 334−344. DOI: 10.1097/SLA.0000000000004499. [17] Snead, D.R.J., Tsang, Y., Meskiri, A., et al. (2016). Validation of digital pathology imaging for primary histopathological diagnosis. Histopathology 68: 1063−1072. DOI: 10.1111/his.12879. [18] Bi, W.L., Hosny, A., Schabath, M.B., et al. (2019). Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J. Clin. 69: 127−157. DOI: 10.3322/caac.21552. [19] Esteva, A., Kuprel, B., Novoa, R.A., et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature 542: 115−118. DOI: 10.1038/nature21056. [20] Gulshan, V., Peng, L., Coram, M., et al. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316: 2402−2410. DOI: 10.1001/jama.2016.17216. [21] Sun, R., Limkin, E.J., Vakalopoulou, M., et al. (2018). A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: An imaging biomarker, retrospective multicohort study. Lancet Oncol. 19: 1180−1191. DOI: 10.1016/S1470-2045(18)30413-3. [22] Skrede, O.-J., De Raedt, S., Kleppe, A., et al. (2020). Deep learning for prediction of colorectal cancer outcome: A discovery and validation study. Lancet 395: 350−360. DOI: 10.1016/S0140-6736(19)32998-8. [23] Lambin, P., Rios-Velazquez, E., Leijenaar, R., et al. (2012). Radiomics: Extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 48: 441−446. DOI: 10.1016/j.ejca.2011.11.036. [24] LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature 521: 436−444. DOI: 10.1038/nature14539. [25] Zhu, B., Liu, J.Z., Cauley, S.F., et al. (2018). Image reconstruction by domain-transform manifold learning. Nature 555: 487−492. DOI: 10.1038/nature25988. [26] Moeskops, P., Wolterink, J.M., Van Der Velden, B.H.M., et al. (2016). Deep learning for multi-task medical image segmentation in multiple modalities. Proc. Med. Image Comput. Comput. Interv. 9901: 478−486. DOI: 10.1007/978-3-319-46723-8_55. [27] Biller, L.H., and Schrag, D. (2021). Diagnosis and treatment of metastatic colorectal cancer: a review. JAMA 325: 669−685. DOI: 10.1001/jama.2021.0106. [28] Harrison, K., Pullen, H., Welsh, C., et al. (2022). Machine learning for auto-segmentation in radiotherapy planning. Clin. Oncol. 34: 74−88. DOI: 10.1016/j.clon.2021.12.003. [29] Grau, V., Mewes, A.U.J., Alcaniz, M., et al. (2004). Improved watershed transform for medical image segmentation using prior information. IEEE Trans. Med. Imaging 23: 447−458. DOI: 10.1109/TMI.2004.824224. [30] Seo, H., Badiei Khuzani, M., Vasudevan, V., et al. (2020). Machine learning techniques for biomedical image segmentation: An overview of technical aspects and introduction to state-of-art applications. Med. Phys. 47: e148−e167. DOI: 10.1002/mp.13649. [31] Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Proc. Med. Image Comput. Comput. Interv. 9351: 234−241. DOI: 10.1007/978-3-319-24574-4_28. [32] Jian, J., Xiong, F., Xia, W., et al. (2018). Fully convolutional networks (FCNs)-based segmentation method for colorectal tumors on T2-weighted magnetic resonance images. Phys. Eng. Sci. Med. 41: 393−401. DOI: 10.1007/s13246-018-0636-9. [33] Zhu, H., Zhang, X., Shi, Y., et al. (2021). Automatic segmentation of rectal tumor on diffusion‐weighted images by deep learning with U‐Net. J. Appl. Clin. Med. Phys. 22: 324−331. DOI: 10.1002/acm2.13381. [34] Huang, Y., Liang, C., He, L., et al. (2016). Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J. Clin. Oncol. 34: 2157−2164. DOI: 10.1200/JCO.2015.65.9128. [35] Nie, K., Shi, L., Chen, Q., et al. (2016). Rectal cancer: Assessment of neoadjuvant chemoradiation outcome based on radiomics of multiparametric MRI. Clin. Cancer Res. 22: 5256−5264. DOI: 10.1158/1078-0432.CCR-15-2997. [36] Van Griethuysen, J.J.M., Fedorov, A., Parmar, C., et al. (2017). Computational radiomics system to decode the radiographic phenotype. Cancer Res. 77: e104−e107. DOI: 10.1158/0008-5472.CAN-17-0339. [37] Zwanenburg, A., Vallières, M., Abdalah, M.A., et al. (2020). The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295: 328−338. DOI: 10.1148/radiol.2020191145. [38] Shin, J., Seo, N., Baek, S.-E., et al. (2022). MRI radiomics model predicts pathologic complete response of rectal cancer following chemoradiotherapy. Radiology 303: 351−358. DOI: 10.1148/radiol.211986. [39] Feng, L., Liu, Z., Li, C., et al. (2022). Development and validation of a radiopathomics model to predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: A multicentre observational study. Lancet Digit. Health 4: e8−e17. DOI: 10.1016/S2589-7500(21)00215-6. [40] Delli Pizzi, A., Chiarelli, A.M., Chiacchiaretta, P., et al. (2021). MRI-based clinical-radiomics model predicts tumor response before treatment in locally advanced rectal cancer. Sci. Rep. 11: 5379. DOI: 10.1038/s41598-021-84816-3. [41] Shaish, H., Aukerman, A., Vanguri, R., et al. (2020). Radiomics of MRI for pretreatment prediction of pathologic complete response, tumor regression grade, and neoadjuvant rectal score in patients with locally advanced rectal cancer undergoing neoadjuvant chemoradiation: An international multicenter study. Eur. Radiol. 30: 6263−6273. DOI: 10.1007/s00330-020-06968-6. [42] Liu, Z., Zhang, X.-Y., Shi, Y.-J., et al. (2017). Radiomics analysis for evaluation of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Clin. Cancer Res. 23: 7253−7262. DOI: 10.1158/1078-0432.CCR-17-1038. [43] Zhou, X., Yi, Y., Liu, Z., et al. (2019). Radiomics-based pretherapeutic prediction of non-response to neoadjuvant therapy in locally advanced rectal cancer. Ann. Surg. Oncol. 26: 1676−1684. DOI: 10.1245/s10434-019-07300-3. [44] Zhou, X., Yi, Y., Liu, Z., et al. (2020). Radiomics-based preoperative prediction of lymph node status following neoadjuvant therapy in locally advanced rectal cancer. Front. Oncol. 10: 604. DOI: 10.3389/fonc.2020.00604. [45] Shao, L., Liu, Z., Feng, L., et al. (2020). Multiparametric MRI and whole slide image-based pretreatment prediction of pathological response to neoadjuvant chemoradiotherapy in rectal cancer: A multicenter radiopathomic study. Ann. Surg. Oncol. 27: 4296−4306. DOI: 10.1245/s10434-020-08659-4. [46] Cui, Y., Yang, W., Ren, J., et al. (2021). Prognostic value of multiparametric MRI-based radiomics model: Potential role for chemotherapeutic benefits in locally advanced rectal cancer. Radiother. Oncol. 154: 161−169. DOI: 10.1016/j.radonc.2020.09.039. [47] Dai, W., Mo, S., Han, L., et al. (2020). Prognostic and predictive value of radiomics signatures in stage I‐III colon cancer. Clin. Transl. Med. 10: 288−293. DOI: 10.1002/ctm2.31. [48] Liu, Z., Meng, X., Zhang, H., et al. (2020). Predicting distant metastasis and chemotherapy benefit in locally advanced rectal cancer. Nat. Commun. 11: 4308. DOI: 10.1038/s41467-020-18162-9. [49] Zhang, X., Wang, L., Zhu, H., et al. (2020). Predicting rectal cancer response to neoadjuvant chemoradiotherapy using deep learning of diffusion kurtosis MRI. Radiology 296: 56−64. DOI: 10.1148/radiol.2020190936. [50] Kather, J.N., Krisam, J., Charoentong, P., et al. (2019). Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS Med. 16: e1002730. DOI: 10.1371/journal.pmed.1002730. [51] Liu, Z., Wang, S., Dong, D., et al. (2019). The applications of radiomics in precision diagnosis and treatment of oncology: Opportunities and challenges. Theranostics 9: 1303−1322. DOI: 10.7150/thno.30309. [52] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Proc. Adv. Neural Inf. Process. Syst. 30: 5998−6008. [53] Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.cs.AI. DOI: arxiv-2010.11929. [54] Shamshad, F., Khan, S., Zamir, S.W., et al. (2023). Transformers in medical imaging: A survey. Med. Image Anal. 88: 102802. DOI: 10.1016/j.media.2023.102802. [55] Jiang, X., Zhao, H., Saldanha, O.L., et al. (2023). An MRI deep learning model predicts outcome in rectal cancer. Radiology 307: e222223. DOI: 10.1148/radiol.222223. [56] Kaissis, G.A., Makowski, M.R., Rückert, D., et al. (2020). Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2: 305−311. DOI: 10.1038/s42256-020-0186-1. [57] Candemir, S., Nguyen, X. V, Folio, L.R., et al. (2021). Training strategies for radiology deep learning models in data-limited scenarios. Radiol. Artif. Intell. 3: e210014. DOI: 10.1148/ryai.2021210014. [58] Russakovsky, O., Deng, J., Su, H., et al. (2015). Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115: 211−252. DOI: 10.1007/s11263-015-0816-y. [59] Shin, H.-C., Roth, H.R., Gao, M., et al. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35: 1285−1298. DOI: 10.1109/TMI.2016.2528162. [60] Guan, H., and Liu, M. (2021). Domain adaptation for medical image analysis: A survey. IEEE Trans. Biomed. Eng. 69: 1173−1185. DOI: 10.1109/TBME.2021.3117407. [61] Li, X., Gao, H., Zhu, J., et al. (2021). 3D deep learning model for the pretreatment evaluation of treatment response in esophageal carcinoma: A prospective study (ChiCTR2000039279). Int. J. Radiat. Oncol. Biol. Phys. 111: 926−935. DOI: 10.1016/j.ijrobp.2021.06.033. [62] Shao, L., Liu, Z., Liu, J., et al. (2022). Patient-level grading prediction of prostate cancer from mp-MRI via GMINet. Comput. Biol. Med. 150: 106168. DOI: 10.1016/j.compbiomed.2022.106168. [63] Shao, L., Liu, Z., Yan, Y., et al. (2021). Patient-level prediction of multi-classification task at prostate MRI based on end-to-end framework learning from diagnostic logic of radiologists. IEEE Trans. Biomed. Eng. 68: 3690−3700. DOI: 10.1109/TBME.2021.3082176. [64] Yamashita, R., Long, J., Longacre, T., et al. (2021). Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. Lancet Oncol. 22: 132−141. DOI: 10.1016/S1470-2045(20)30535-0. [65] Vahadane, A., Peng, T., Sethi, A., et al. (2016). Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans. Med. Imaging 35: 1962−1971. DOI: 10.1109/TMI.2016.2529665. [66] Macenko, M., Niethammer, M., Marron, J.S., et al. (2009). A method for normalizing histology slides for quantitative analysis. IEEE Int. Symp. Biomed. imaging 2009 : 1107–1110. DOI: 10.1109/ISBI.2009.5193250. [67] Campanella, G., Hanna, M.G., Geneslaw, L., et al. (2019). Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25: 1301−1309. DOI: 10.1038/s41591-019-0508-1. [68] Lu, M.Y., Williamson, D.F.K., Chen, T.Y., et al. (2021). Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5: 555−570. DOI: 10.1038/s41551-020-00682-w. [69] Lee, Y., Park, J.H., Oh, S., et al. (2022). Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning. Nat. Biomed. Eng. Advance online publication DOI: 10.1038/s41551-022-00923-0. [70] Dabass, M., Vashisth, S., and Vig, R. (2022). A convolution neural network with multi-level convolutional and attention learning for classification of cancer grades and tissue structures in colon histopathological images. Comput. Biol. Med. 147: 105680. DOI: 10.1016/j.compbiomed.2022.105680. [71] Ding, K., Zhou, M., Wang, H., et al. (2022). Spatially aware graph neural networks and cross-level molecular profile prediction in colon cancer histopathology: A retrospective multi-cohort study. Lancet Digit. Health 4: e787−e795. DOI: 10.1016/S2589-7500(22)00168-6. [72] Cui, Y., Yang, X., Shi, Z., et al. (2019). Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur. Radiol. 29: 1211−1220. DOI: 10.1007/s00330-018-5683-9. [73] Nakanishi, R., Akiyoshi, T., Toda, S., et al. (2020). Radiomics approach outperforms diameter criteria for predicting pathological lateral lymph node metastasis after neoadjuvant (chemo) radiotherapy in advanced low rectal cancer. Ann. Surg. Oncol. 27: 4273−4283. DOI: 10.1245/s10434-020-08974-w. [74] Lovinfosse, P., Polus, M., Van Daele, D., et al. (2018). FDG PET/CT radiomics for predicting the outcome of locally advanced rectal cancer. Eur. J. Nucl. Med. Mol. Imaging 45: 365−375. DOI: 10.1007/s00259-017-3855-5. [75] Bang, J.-I., Ha, S., Kang, S.-B., et al. (2016). Prediction of neoadjuvant radiation chemotherapy response and survival using pretreatment [18 F] FDG PET/CT scans in locally advanced rectal cancer. Eur. J. Nucl. Med. Mol. Imaging 43: 422−431. DOI: 10.1007/s00259-015-3180-9. [76] Maffione, A.M., Marzola, M.C., Capirci, C., et al. (2015). Value of (18)F-FDG PET for predicting response to neoadjuvant therapy in rectal cancer: Systematic review and meta-analysis. Am. J. Roentgenol. 204: 1261−1268. DOI: 10.2214/AJR.14.13210. [77] Giannini, V., Mazzetti, S., Bertotto, I., et al. (2019). Predicting locally advanced rectal cancer response to neoadjuvant therapy with 18 F-FDG PET and MRI radiomics features. Eur. J. Nucl. Med. Mol. Imaging 46: 878−888. DOI: 10.1007/s00259-018-4250-6. [78] Schurink, N.W., van Kranen, S.R., Berbee, M., et al. (2021). Studying local tumour heterogeneity on MRI and FDG-PET/CT to predict response to neoadjuvant chemoradiotherapy in rectal cancer. Eur. Radiol. 31: 7031−7038. DOI: 10.1007/s00330-021-07724-0. [79] Patel, U.B., Taylor, F., Blomqvist, L., et al. (2011). Magnetic resonance imaging–detected tumor response for locally advanced rectal cancer predicts survival outcomes: MERCURY experience. J. Clin. Oncol. 29: 3753−3760. DOI: 10.1200/JCO.2011.34.9068. [80] Patel, U.B., Brown, G., Rutten, H., et al. (2012). Comparison of magnetic resonance imaging and histopathological response to chemoradiotherapy in locally advanced rectal cancer. Ann. Surg. Oncol. 19: 2842−2852. DOI: 10.1245/s10434-012-2309-3. [81] Horvat, N., Veeraraghavan, H., Khan, M., et al. (2018). MR imaging of rectal cancer: Radiomics analysis to assess treatment response after neoadjuvant therapy. Radiology 287: 833−843. DOI: 10.1148/radiol.2018172300. [82] Bulens, P., Couwenberg, A., Intven, M., et al. (2020). Predicting the tumor response to chemoradiotherapy for rectal cancer: Model development and external validation using MRI radiomics. Radiother. Oncol. 142: 246−252. DOI: 10.1016/j.radonc.2019.07.033. [83] Shayesteh, S., Nazari, M., Salahshour, A., et al. (2021). Treatment response prediction using MRI-based pre-, post-, and delta-radiomic features and machine learning algorithms in colorectal cancer. Med. Phys. 48: 3691−3701. DOI: 10.1002/mp.14896. [84] Chen, H., Shi, L., Nguyen, K.N.B., et al. (2020). MRI radiomics for prediction of tumor response and downstaging in rectal cancer patients after preoperative chemoradiation. Adv. Radiat. Oncol. 5: 1286−1295. DOI: 10.1016/j.adro.2020.04.016. [85] Wan, L., Peng, W., Zou, S., et al. (2021). MRI-based delta-radiomics are predictive of pathological complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Acad. Radiol. 28: S95−S104. DOI: 10.1016/j.acra.2020.10.026. [86] Peng, J., Wang, W., Jin, H., et al. (2023). Develop and validate a radiomics space-time model to predict the pathological complete response in patients undergoing neoadjuvant treatment of rectal cancer: An artificial intelligence model study based on machine learning. BMC Cancer 23: 365. DOI: 10.1186/s12885-023-10855-w. [87] Jin, C., Yu, H., Ke, J., et al. (2021). Predicting treatment response from longitudinal images using multi-task deep learning. Nat. Commun. 12: 1−11. DOI: 10.1038/s41467-021-22188-y. [88] Shi, L., Zhang, Y., Nie, K.E., et al. (2019). Machine learning for prediction of chemoradiation therapy response in rectal cancer using pre-treatment and mid-radiation multi-parametric MRI. Magn. Reson. Imaging 61: 33−40. DOI: 10.1016/j.mri.2019.05.003. [89] Armaghany, T., Wilson, J.D., Chu, Q., et al. (2012). Genetic alterations in colorectal cancer. Gastrointest Cancer Res. 5: 19−27. [90] Ghaffari Laleh, N., Ligero, M., Perez-Lopez, R., et al. (2023). Facts and hopes on the use of artificial intelligence for predictive immunotherapy biomarkers in cancer. Clin. Cancer Res. 29: 316−323. DOI: 10.1158/1078-0432.CCR-22-0390. [91] Cohen, R., Hain, E., Buhard, O., et al. (2019). Association of primary resistance to immune checkpoint inhibitors in metastatic colorectal cancer with misdiagnosis of microsatellite instability or mismatch repair deficiency status. JAMA Oncol. 5: 551−555. DOI: 10.1001/jamaoncol.2018.4942. [92] Merok, M.A., Ahlquist, T., Røyrvik, E.C., et al. (2013). Microsatellite instability has a positive prognostic impact on stage II colorectal cancer after complete resection: Results from a large, consecutive Norwegian series. Ann. Oncol. 24: 1274−1282. DOI: 10.1093/annonc/mds614. [93] Li, L.S., Morales, J.C., Veigl, M., et al. (2009). DNA mismatch repair (MMR)-dependent 5-fluorouracil cytotoxicity and the potential for new therapeutic targets. Br. J. Pharmacol. 158: 679−692. DOI: 10.1111/j.1476-5381.2009.00423.x. [94] Mandal, R., Samstein, R.M., Lee, K.-W., et al. (2019). Genetic diversity of tumors with mismatch repair deficiency influences anti-PD-1 immunotherapy response. Science 364: 485−491. DOI: 10.1126/science.aau0447. [95] Pei, Q., Yi, X., Chen, C., et al. (2022). Pre-treatment CT-based radiomics nomogram for predicting microsatellite instability status in colorectal cancer. Eur. Radiol. 32: 714−724. DOI: 10.1007/s00330-021-08167-3. [96] Ying, M., Pan, J., Lu, G., et al. (2022). Development and validation of a radiomics-based nomogram for the preoperative prediction of microsatellite instability in colorectal cancer. BMC Cancer 22: 1−13. DOI: 10.1186/s12885-022-09584-3. [97] Zhang, W., Huang, Z., Zhao, J., et al. (2021). Development and validation of magnetic resonance imaging-based radiomics models for preoperative prediction of microsatellite instability in rectal cancer. Ann. Transl. Med. 9: 134. DOI: 10.21037/atm-20-7673. [98] Zhang, W., Yin, H., Huang, Z., et al. (2021). Development and validation of MR-based deep learning models for prediction of microsatellite instability in rectal cancer. Cancer Med. 10(12): 4164−4173. DOI: 10.1002/cam4.3957. [99] Kather, J.N., Pearson, A.T., Halama, N., et al. (2019). Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25: 1054−1056. DOI: 10.1038/s41591-019-0462-y. [100] Sorich, M.J., Wiese, M.D., Rowland, A., et al. (2015). Extended RAS mutations and anti-EGFR monoclonal antibody survival benefit in metastatic colorectal cancer: A meta-analysis of randomized, controlled trials. Ann. Oncol. 26: 13−21. DOI: 10.1093/annonc/mdu378. [101] Xue, T., Peng, H., Chen, Q., et al. (2022). Preoperative prediction of KRAS mutation status in colorectal cancer using a CT-based radiomics nomogram. Br. J. Radiol. 95: 20211014. DOI: 10.1259/bjr.20211014. [102] Cui, Y., Liu, H., Ren, J., et al. (2020). Development and validation of a MRI-based radiomics signature for prediction of KRAS mutation in rectal cancer. Eur. Radiol. 30: 1948−1958. DOI: 10.1007/s00330-019-06572-3. [103] Zhang, Z., Shen, L., Wang, Y., et al. (2021). MRI Radiomics signature as a potential biomarker for predicting KRAS status in locally advanced rectal cancer patients. Front. Oncol. 11: 614052. DOI: 10.3389/fonc.2021.614052. [104] Ma, Y., Wang, J., Song, K., et al. (2021). Spatial-frequency dual-branch attention model for determining KRAS mutation status in colorectal cancer with T2-weighted MRI. Comput. Methods Programs Biomed. 209: 106311. DOI: 10.1016/j.cmpb.2021.106311. [105] Liu, H., Yin, H., Li, J., et al. (2022). A deep learning model based on MRI and clinical factors facilitates noninvasive evaluation of KRAS mutation in rectal cancer. J. Magn. Reson. Imaging 56: 1659−1668. DOI: 10.1002/jmri.28237. [106] Bilal, M., Raza, S.E.A., Azam, A., et al. (2021). Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: A retrospective study. Lancet Digit. Health 3: e763−e772. DOI: 10.1016/S2589-7500(21)00180-1. [107] Tsai, P.-C., Lee, T.-H., Kuo, K.-C., et al. (2023). Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients. Nat. Commun. 14: 2102. DOI: 10.1038/s41467-023-37179-4. [108] Sun, C., Li, B., Wei, G., et al. (2022). Deep learning with whole slide images can improve the prognostic risk stratification with stage III colorectal cancer. Comput. Methods Programs Biomed. 221: 106914. DOI: 10.1016/j.cmpb.2022.106914. [109] Liu, X., Zhang, D., Liu, Z., et al. (2021). Deep learning radiomics-based prediction of distant metastasis in patients with locally advanced rectal cancer after neoadjuvant chemoradiotherapy: A multicentre study. EBioMedicine 69: 103442. DOI: 10.1016/j.ebiom.2021.103442. [110] Lu, L., Dercle, L., Zhao, B., et al. (2021). Deep learning for the prediction of early on-treatment response in metastatic colorectal cancer from serial medical imaging. Nat. Commun. 12: 6654. DOI: 10.1038/s41467-021-26990-6. [111] Kleppe, A., Skrede, O.-J., De Raedt, S., et al. (2022). A clinical decision support system optimising adjuvant chemotherapy for colorectal cancers by integrating deep learning and pathological staging markers: A development and validation study. Lancet Oncol. 23: 1221−1232. DOI: 10.1016/S1470-2045(22)00391-6. [112] Lu, Y., Yu, Q., Gao, Y., et al. (2018). Identification of metastatic lymph nodes in MR imaging with faster region-based convolutional neural networks. Cancer Res. 78: 5135−5143. DOI: 10.1158/0008-5472.CAN-18-0494. [113] Nakanishi, R., Oki, E., Hasuda, H., et al. (2021). ASO author reflection: Radiomics-based prediction for the responder to first-line oxaliplatin-based chemotherapy in patients with colorectal liver metastasis. Ann. Surg. Oncol. 28: 2986−2987. DOI: 10.1245/s10434-020-09584-2. [114] Giannini, V., Pusceddu, L., Defeudis, A., et al. (2022). Delta-radiomics predicts response to first-line oxaliplatin-based chemotherapy in colorectal cancer patients with liver metastases. Cancers 14: 241. DOI: 10.3390/cancers14010241. [115] Dohan, A., Gallix, B., Guiu, B., et al. (2020). Early evaluation using a radiomic signature of unresectable hepatic metastases to predict outcome in patients with colorectal cancer treated with FOLFIRI and bevacizumab. Gut 69: 531−539. DOI: 10.1136/gutjnl-2018-316407. [116] Zhou, B., Khosla, A., Lapedriza, A., et al. (2016). Learning deep features for discriminative localization. Proc IEEE Conf Comput Vis Pattern Recognit pp: 2921–2929. DOI:10.1109/CVPR.2016.319. [117] Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1: 206−215. DOI: 10.1038/s42256-019-0048-x. [118] Napel, S., Mu, W., Jardim‐Perassi, B. V, et al. (2018). Quantitative imaging of cancer in the postgenomic era: Radio (geno) mics, deep learning, and habitats. Cancer 124: 4633−4649. DOI: 10.1002/cncr.31630. [119] Dextraze, K., Saha, A., Kim, D., et al. (2017). Spatial habitats from multiparametric MR imaging are associated with signaling pathway activities and survival in glioblastoma. Oncotarget 8: 112992. DOI: 10.18632/oncotarget.22947. [120] Jardim-Perassi, B. V, Huang, S., Dominguez-Viqueira, W., et al. (2019). Multiparametric MRI and coregistered histology identify tumor habitats in breast cancer mouse models. Cancer Res. 79: 3952−3964. DOI: 10.1158/0008-5472.CAN-19-0213. [121] Yao, J., Zhu, X., Jonnagaddala, J., et al. (2020). Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65: 101789. DOI: 10.1016/j.media.2020.101789. [122] Chen, R.J., Lu, M.Y., Shaban, M., et al. (2021). Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolutional networks. Proc Med Image Comput Comput Assist Interv 12908: 339−349. DOI: 10.1007/978-3-030-87237-3_33. [123] He, B., Dong, D., She, Y., et al. (2020). Predicting response to immunotherapy in advanced non-small-cell lung cancer using tumor mutational burden radiomic biomarker. J. Immunother. Cancer 8: e000550. DOI: 10.1136/jitc-2020-000550. [124] Liu, X., Liu, Z., Yan, Y., et al. (2023). Development of prognostic biomarkers by TMB-guided WSI analysis: A two-step approach. IEEE J. Biomed. Health Inform. 27: 1780−1789. DOI: 10.1109/JBHI.2023.3249354. [125] Mongan, J., Moy, L., and Kahn Jr, C.E. (2020). Checklist for artificial intelligence in medical imaging (CLAIM): A guide for authors and reviewers. Radiol. Artif. Intell. 2: e200029. DOI: 10.1148/ryai.2020200029. [126] Collins, G.S., Reitsma, J.B., Altman, D.G., et al. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. Ann. Intern. Med. 162: 55−63. DOI: 10.7326/M14-0697. [127] Ganin, Y., and Lempitsky, V. (2015). Unsupervised domain adaptation by backpropagation. Proc Int Conf Mach Learn 37: 1180−1189. DOI: 10.5555/3045118.3045244. [128] Liu, Q., Chen, C., Qin, J., et al. (2021). Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. Proc IEEE Int Conf Comput Vis pp: 1013–1023. DOI: 10.1109/CVPR46437.2021.00107. [129] Kaya, M., and Bilge, H.Ş. (2019). Deep metric learning: A survey. Symmetry 11: 1066. DOI: 10.3390/sym11091066. [130] Chen, R.J., Lu, M.Y., Weng, W.-H., et al. (2021). Multimodal co-attention transformer for survival prediction in gigapixel whole slide images. Proc IEEE Int Conf Comput Vis pp: 3995-4005. DOI: 10.1109/ICCV48922.2021.00398. [131] Bhattacharya, I., Seetharaman, A., Kunder, C., et al. (2022). Selective identification and localization of indolent and aggressive prostate cancers via CorrSigNIA: An MRI-pathology correlation and deep learning framework. Med. Image Anal. 75: 102288. DOI: 10.1016/j.media.2021.102288. [132] Yang, Y., and Soatto, S. (2020). FDA: Fourier domain adaptation for semantic segmentation. Proc IEEE Conf Comput Vis Pattern Recognit pp: 4084-4094. DOI: 10.1109/CVPR42600.2020.00414. [133] Tzeng, E., Hoffman, J., Saenko, K., et al. (2017). Adversarial discriminative domain adaptation. Proc IEEE Conf Comput Vis Pattern Recognit pp: 2962-2971. DOI: 10.1109/CVPR.2017.316. [134] Kumar, A., Sattigeri, P., Wadhawan, K., et al. (2018). Co-regularized alignment for unsupervised domain adaptation. Proc Adv neural Inf Process Syst 31: 9367−9378. DOI: 10.5555/3327546.3327607. [135] Yu, G., Sun, K., Xu, C., et al. (2021). Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images. Nat. Commun. 12: 6311. DOI: 10.1038/s41467-021-26643-8. [136] Abbet, C., Studer, L., Fischer, A., et al. (2022). Self-rule to multi-adapt: Generalized multi-source feature learning using unsupervised domain adaptation for colorectal cancer tissue detection. Med. Image Anal. 79: 102473. DOI: 10.1016/j.media.2022.102473. [137] Srinidhi, C.L., Kim, S.W., Chen, F.-D., et al. (2022). Self-supervised driven consistency training for annotation efficient histopathology image analysis. Med. Image Anal. 75: 102256. DOI: 10.1016/j.media.2021.102256. [138] Azizi, S., Culp, L., Freyberg, J., et al. (2023). Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging. Nat. Biomed. Eng. 7: 756−779. DOI: 10.1038/s41551-023-01049-7. [139] Schirris, Y., Gavves, E., Nederlof, I., et al. (2022). DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer. Med. Image Anal. 79: 102464. DOI: 10.1016/j.media.2022.102464. [140] Price, W.N., and Cohen, I.G. (2019). Privacy in the age of medical big data. Nat. Med. 25: 37−43. DOI: 10.1038/s41591-018-0272-7. [141] McMahan, B., Moore, E., Ramage, D., et al. (2017). Communication-efficient learning of deep networks from decentralized data. Proc Artif Intell Stat 54: 1273−1282. DOI: 10.48550/arXiv.1602.05629. [142] Dou, Q., So, T.Y., Jiang, M., et al. (2021). Federated deep learning for detecting COVID-19 lung abnormalities in CT: A privacy-preserving multinational validation study. NPJ Digit. Med. 4: 60. DOI: 10.1038/s41746-021-00431-6. [143] Pati, S., Baid, U., Edwards, B., et al. (2022). Federated learning enables big data for rare cancer boundary detection. Nat. Commun. 13: 7346. DOI: 10.1038/s41467-022-33407-5. [144] Ogier du Terrail, J., Leopold, A., Joly, C., et al. (2023). Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat. Med. 29: 135−146. DOI: 10.1038/s41591-022-02155-w. [145] Lu, M.Y., Chen, R.J., Kong, D., et al. (2022). Federated learning for computational pathology on gigapixel whole slide images. Med. Image Anal. 76: 102298. DOI: 10.1016/j.media.2021.102298. [146] Jiang, M., Wang, Z., and Dou, Q. (2022). Harmofl: Harmonizing local and global drifts in federated learning on heterogeneous medical images. Proc AAAI Conf Artif Intell 36: 1087−1095. DOI: 10.1609/aaai.v36i1.19993. [147] Feng, C.-M., Yan, Y., Wang, S., et al. (2022). Specificity-preserving federated learning for MR image reconstruction. IEEE Trans Med Imaging 42: 2010−2021. DOI: 10.1109/TMI.2022.3202106. [148] Yan, R., Qu, L., Wei, Q., et al. (2023). Label-efficient self-supervised federated learning for tackling data heterogeneity in medical imaging. IEEE Trans Med Imaging 42: 1932−1943. DOI: 10.1109/TMI.2022.3233574. -
ABOUT THIS ARTICLE
Cite this article:
Liu X., Zhang S., Shao L., et al., (2024). Improving prediction of treatment response and prognosis in colorectal cancer with AI-based medical image analysis. The Innovation Medicine 2(2): 100069. https://doi.org/10.59717/j.xinn-med.2024.100069
Welcome!
To request copyright permission to republish or share portions of our works, please visit Copyright Clearance Center's (CCC) Marketplace website at marketplace.copyright.com.
Export File
Citation
Liu X., Zhang S., Shao L., et al., (2024). Improving prediction of treatment response and prognosis in colorectal cancer with AI-based medical image analysis. The Innovation Medicine 2(2): 100069. https://doi.org/10.59717/j.xinn-med.2024.100069
Format
Content
-
Figure 1.
Schematic of artificial intelligence application on CRC medical imaging for clinical support
-
Figure 2.
Schematic of WSIs processing through AI methods
