Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments

Zhen Zhang; Chun Li; Wenhui Wang; Zheng Dong; Gongfa Liu; Yuhui Dong; Yi Zhang

doi:10.1016/j.xinn.2023.100539

The Innovation > 2024 Vol. 5 > No. 1

Next Article Previous Article

Review Open Access

Cite PDF

Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments

1.
National Synchrotron Radiation Laboratory, University of Science and Technology of China, Hefei 230029, China
2.
Beijing Synchrotron Radiation Facility, Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China

More Information

³These authors contributed equally
Corresponding authors: dongz@ihep.ac.cn (Z.D.); zhangyi88@ihep.ac.cn (Y.Z.)
Received Date: June 16, 2023

Accepted Date: November 13, 2023

Published Online: November 16, 2023

The Innovation 5(1), https://doi.org/10.1016/j.xinn.2023.100539 | Cite this article

GRAPHICAL ABSTRACT

GRAPHICAL ABSTRACT

DownLoad: Full size image

PUBLIC SUMMARY

PUBLIC SUMMARY
1. ■ Tomography experiments at future synchrotron beamlines face exascale big data challenges.
2. ■ Deep learning is employed for various data processing tasks at current synchrotron tomography beamlines.
3. ■ Future comprehensive and elongated data analysis processes require a full-stack deep learning pipeline.
4. ■ The data-driven full-stack deep learning pipeline based on an intelligent scheduling center (ISC) holds the key to solve the big data challenges.

ABSTRACT

ABSTRACT

Synchrotron tomography experiments are transitioning into multifunctional, cross-scale, and dynamic characterizations, enabled by new-generation synchrotron light sources and fast developments in beamline instrumentation. However, with the spatial and temporal resolving power entering a new era, this transition generates vast amounts of data, which imposes a significant burden on the data processing end. Today, as a highly accurate and efficient data processing method, deep learning shows great potential to address the big data challenge being encountered at future synchrotron beamlines. In this review, we discuss recent advances employing deep learning at different stages of the synchrotron tomography data processing pipeline. We also highlight how applications in other data-intensive fields, such as medical imaging and electron tomography, can be migrated to synchrotron tomography. Finally, we provide our thoughts on possible challenges and opportunities as well as the outlook, envisioning selected deep learning methods, curated big models, and customized learning strategies, all through an intelligent scheduling solution.

FullText(HTML)

REFERENCES

[1]	Walton, T., Gui, M., Velkova, S., et al. (2023). Axonemal structures reveal mechanoregulatory and disease mechanisms. Nature *618*, 625-633. View in Article CrossRef Google Scholar
[2]	Wu, J., Lu, Z., Jiang, D., et al. (2021). Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale. Cell *184*, 3318-3332.e17. View in Article CrossRef Google Scholar
[3]	Tałanda, M., Fernandez, V., Panciroli, E., et al. (2022). Synchrotron tomography of a stem lizard elucidates early squamate anatomy. Nature *611*, 99-104. View in Article CrossRef Google Scholar
[4]	Mangala Prasad, V., Leaman, D.P., Lovendahl, K.N., et al. (2022). Cryo-ET of Env on intact HIV virions reveals structural variation and positioning on the Gag lattice. Cell *185*, 641-653.e17. View in Article CrossRef Google Scholar
[5]	Su, Q., Chen, M., Shi, Y., et al. (2022). Cryo-EM structure of the human IgM B cell receptor. Science *377*, 875-880. View in Article CrossRef Google Scholar
[6]	Wozny, M.R., Di Luca, A., Morado, D.R., et al. (2023). In situ architecture of the ER-mitochondria encounter structure. Nature *618*, 188-192. View in Article CrossRef Google Scholar
[7]	Domínguez-Martín, M.A., Sauer, P.V., Kirst, H., et al. (2022). Structures of a phycobilisome in light-harvesting and photoprotected states. Nature *609*, 835-845. View in Article CrossRef Google Scholar
[8]	Watanabe, R., Buschauer, R., Böhning, J., et al. (2020). The in situ structure of parkinson’s disease-linked LRRK2. Cell *182*, 1508-1518.e16. View in Article CrossRef Google Scholar
[9]	Zhang, K., Liu, X., Shen, J., et al. (2020). Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell *181*, 1423-1433.e11. View in Article CrossRef Google Scholar
[10]	Ng, K.W., Boumelha, J., Enfield, K.S.S., et al. (2023). Antibodies against endogenous retroviruses promote lung cancer immunotherapy. Nature *616*, 563-573. View in Article CrossRef Google Scholar
[11]	Lee, J.H., Liu, Q., and Dadgar-Kiani, E. (2022). Solving brain circuit function and dysfunction with computational modeling and optogenetic fMRI. Science *378*, 493-499. View in Article CrossRef Google Scholar
[12]	Han, M., Bushong, E.A., Segawa, M., et al. (2023). Spatial mapping of mitochondrial networks and bioenergetics in lung cancer. Nature *615*, 712-719. View in Article CrossRef Google Scholar
[13]	Scott, M.C., Chen, C.-C., Mecklenburg, M., et al. (2012). Electron tomography at 2.4-angstrom resolution. Nature *483*, 444-447. View in Article CrossRef Google Scholar
[14]	Wang, L., Liu, T., Wu, T., et al. (2022). Strain-retardant coherent perovskite phase stabilized Ni-rich cathode. Nature *611*, 61-67. View in Article CrossRef Google Scholar
[15]	Huang, Y., Perlmutter, D., Fei-Huei Su, A., et al. (2023). Detecting lithium plating dynamics in a solid-state battery with operando X-ray computed tomography using machine learning. NPJ Comput. Mater. 9, 93. View in Article CrossRef Google Scholar
[16]	Li, Y., Yuan, G., Li, L., et al. (2023). Ductile 2-GPa steels with hierarchical substructure. Science *379*, 168-173. View in Article CrossRef Google Scholar
[17]	Possenti, E., Conti, C., Gatta, G.D., et al. (2022). Synchrotron X-ray diffraction computed tomography to non-destructively study inorganic treatments for stone conservation. iScience 25, 105112. View in Article CrossRef Google Scholar
[18]	Michelson, A., Minevich, B., Emamy, H., et al. (2022). Three-dimensional visualization of nanoparticle lattices and multimaterial frameworks. Science *376*, 203-207. View in Article CrossRef Google Scholar
[19]	Jiao, Y., Xu, G., Cui, X.H., et al. (2018). The HEPS project. J. Synchrotron Radiat. 25, 1611-1618. View in Article CrossRef Google Scholar
[20]	Nazaretski, E., Yan, H., Lauer, K., et al. (2017). Design and performance of an X-ray scanning microscope at the Hard X-ray Nanoprobe beamline of NSLS-Ⅱ. J. Synchrotron Radiat. 24, 1113-1119. View in Article CrossRef Google Scholar
[21]	Li, X., Zhang, Y., Liu, Y., et al. (2023). A high-throughput big-data orchestration and processing system for the High Energy Photon Source. J. Synchrotron Radiat. 30, 1086-1091. View in Article CrossRef Google Scholar
[22]	Pfeiffer, F. (2018). X-ray ptychography. Nat. Photonics 12, 9-17. View in Article CrossRef Google Scholar
[23]	Kuan, A.T., Phelps, J.S., Thomas, L.A., et al. (2020). Dense neuronal reconstruction through X-ray holographic nano-tomography. Nat. Neurosci. 23, 1637-1643. View in Article CrossRef Google Scholar
[24]	da Silva, J.C., Pacureanu, A., Yang, Y., et al. (2017). High-energy cryo X-ray nanoimaging at the ID16A beamline of ESRF. SPIE: X-Ray Nanoimaging: Instruments and Methods Ⅲ, 10389. View in Article Google Scholar
[25]	Walsh, C.L., Tafforeau, P., Wagner, W.L., et al. (2021). Imaging intact human organs with local resolution of cellular structures using hierarchical phase-contrast tomography. Nat. Methods 18, 1532-1541. View in Article CrossRef Google Scholar
[26]	Nazaretski, E., Yan, H., Lauer, K., et al. (2017). Design and performance of an X-ray scanning microscope at the Hard X-ray Nanoprobe beamline of NSLS-Ⅱ. J. Synchrotron Radiat. 24, 1113-1119. View in Article CrossRef Google Scholar
[27]	Schroer, C.G., Baumbach, C., Doehrmann, R., et al. (2015). Hard X-ray nanoprobe of beamline P06 at PETRA Ⅲ. NIM-A *616*, 93-97. View in Article Google Scholar
[28]	Stampanoni, M., Groso, A., Isenegger, A., et al. (2006). Trends in synchrotron-based tomographic imaging: the SLS experience. SPIE: Developments in X-Ray Tomography V, 6318. View in Article Google Scholar
[29]	Dong, Y., Li, C., Zhang, Y., et al. (2022). Exascale image processing for next-generation beamlines in advanced light sources. Nat. Rev. Phys. 4, 427-428. View in Article CrossRef Google Scholar
[30]	Kuhn, T.S. (1962). The Structure of Scientific Revolutions (The University of Chicago Press). View in Article Google Scholar
[31]	Lecun, Y., Bottou, L., Bengio, Y., et al. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278-2324. View in Article CrossRef Google Scholar
[32]	Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2017). ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84-90. View in Article CrossRef Google Scholar
[33]	Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. J. Mach. Learn. Res. 15, 315-323. View in Article Google Scholar
[34]	Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Preprint at arXiv. View in Article Google Scholar
[35]	Lin, M., Chen, Q., and Yan, S. (2013). Network in network. Preprint at arXiv. View in Article Google Scholar
[36]	Szegedy, C., Liu, W., Jia, Y., et al. (2014). Going deeper with convolutions. Preprint at arXiv. View in Article Google Scholar
[37]	He, K.M., Zhang, X.Y., Ren, S.Q., et al. (2016). Deep residual learning for image recognition. Preprint at arXiv. View in Article Google Scholar
[38]	Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2020). An image is worth 16x16 words: transformers for image recognition at scale. Preprint at arXiv. View in Article Google Scholar
[39]	Liu, Z., Lin, Y., Cao, Y., et al. (2021). Swin transformer: hierarchical vision transformer using shifted windows. Preprint at arXiv. View in Article Google Scholar
[40]	Rombach, R., Blattmann, A., Lorenz, D., et al. (2022). High-resolution image synthesis with latent diffusion models. Preprint at arXiv. View in Article Google Scholar
[41]	Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al. (2014). Generative adversarial nets. Preprint at arXiv. View in Article Google Scholar
[42]	Kirillov, A., Mintun, E., Ravi, M., et al. (2023). Segment anything. Preprint at arXiv. View in Article Google Scholar
[43]	Cheng, C.-C., Hwu, Y., and Ching, Y.-T. (2016). High-resolution and large-volume tomography reconstruction for X-ray microscopy. SPIE Med. Imaging (Portsm. R.I.). View in Article Google Scholar
[44]	Vescovi, R., Du, M., de Andrade, V., et al. (2018). Tomosaic: efficient acquisition and reconstruction of teravoxel tomography data using limited-size synchrotron X-ray beams. J. Synchrotron Radiat. 25, 1478-1489. View in Article CrossRef Google Scholar
[45]	Qin, G., Yang, C., and Du, Y. (2021). Image enhancement of optical coherence tomography using deep learning. AIID. View in Article Google Scholar
[46]	Li, M., Huang, K., Xu, Q., et al. (2012). OCTA-500: a retinal dataset for optical coherence tomography angiography study. Preprint at arXiv. View in Article Google Scholar
[47]	Guizar-Sicairos, M., Diaz, A., Holler, M., et al. (2011). Phase tomography from x-ray coherent diffractive imaging projections. Opt Express 19, 21345-21357. View in Article CrossRef Google Scholar
[48]	Liu, Y., Meirer, F., Williams, P.A., et al. (2012). TXM-Wizard: a program for advanced data collection and evaluation in full-field transmission X-ray microscopy. J. Synchrotron Radiat. 19, 281-287. View in Article CrossRef Google Scholar
[49]	Han, R., Zhang, F., Wan, X., et al. (2014). A marker-free automatic alignment method based on scale-invariant features. J. Struct. Biol. *186*, 167-180. View in Article CrossRef Google Scholar
[50]	Yu, H., Xia, S., Wei, C., et al. (2018). Automatic projection image registration for nanoscale X-ray tomographic reconstruction. J. Synchrotron Radiat. 25, 1819-1826. View in Article CrossRef Google Scholar
[51]	Zhang, Z., Bi, X., Li, P., et al. (2023). Automatic synchrotron tomographic alignment schemes based on genetic algorithms and human-in-the-loop software. J. Synchrotron Radiat. 30, 169-178. View in Article CrossRef Google Scholar
[52]	Gürsoy, D., Hong, Y.P., He, K., et al. (2017). Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection. Sci. Rep. 7, 11818. View in Article CrossRef Google Scholar
[53]	Pande, K., Donatelli, J.J., Parkinson, D.Y., et al. (2022). Joint iterative reconstruction and 3D rigid alignment for X-ray tomography. Opt Express 30, 8898-8916. View in Article CrossRef Google Scholar
[54]	Zhang, Z., Dong, Z., Yan, H., et al. (2023). A general image misalignment correction method for tomography experiments. iScience 26, 107932. View in Article CrossRef Google Scholar
[55]	Yang, X., De Carlo, F., Phatak, C., et al. (2017). A convolutional neural network approach to calibrating the rotation axis for X-ray computed tomography. J. Synchrotron Radiat. 24, 469-475. View in Article CrossRef Google Scholar
[56]	Topal, E., Löffler, M., and Zschech, E. (2020). Deep learning-based inaccuracy compensation in reconstruction of high resolution XCT data. Sci. Rep. 10, 7682. View in Article CrossRef Google Scholar
[57]	Fu, T., Zhang, K., Wang, Y., et al. (2022). Feature detection network-based correction method for accurate nano-tomography reconstruction. Appl. Opt. 61, 5695-5703. View in Article CrossRef Google Scholar
[58]	Fu, T., Zhang, K., Wang, Y., et al. (2021). Deep-learning-based image registration for nano-resolution tomographic reconstruction. J. Synchrotron Radiat. 28, 1909-1915. View in Article CrossRef Google Scholar
[59]	Liu, M., Han, Y., Xi, X., et al. (2022). Multiscale dense U-Net: a fast correction method for thermal drift artifacts in laboratory NanoCT scans of semi-conductor chips. Entropy 24, 967. View in Article CrossRef Google Scholar
[60]	Cho, S.-J., Ji, S.-W., Hong, J.-P., et al. (2021). Rethinking coarse-to-fine approach in single image deblurring. Preprint at arXiv. View in Article Google Scholar
[61]	Huang, G., Liu, Z., van der Maaten, L., et al. (2017). Densely connected convolutional networks. CVPR. View in Article Google Scholar
[62]	Oord, A.V.D., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. Preprint at arXiv. View in Article Google Scholar
[63]	He, K., Fan, H., Wu, Y., et al. (2020). Momentum contrast for unsupervised visual representation learning. CVPR. View in Article Google Scholar
[64]	Chen, X., Fan, H., Girshick, R., et al. (2020). Improved baselines with momentum contrastive learning. Preprint at arXiv. View in Article Google Scholar
[65]	Prannay, K., Teterwak, P., Wang, C., et al. (2020). Supervised contrastive learning. Preprint at arXiv. View in Article Google Scholar
[66]	Chen, T., Kornblith, S., Norouzi, M., et al. (2020). A simple framework for contrastive learning of visual representations. Preprint at arXiv. View in Article Google Scholar
[67]	Liang, K., Zhang, L., Yang, Y., et al. (2018). A self-supervised deep learning network for low-dose CT reconstruction. NSS/MIC. View in Article Google Scholar
[68]	Lee, H., Lee, J., Kim, H., et al. (2019). Deep-neural-network-based sinogram synthesis for sparse-view CT image reconstruction. IEEE Trans. Radiat. Plasma Med. Sci. 3, 109-119. View in Article CrossRef Google Scholar
[69]	Okamoto, T., Ohnishi, T., and Haneishi, H. (2022). Artifact reduction for sparse-view CT using deep learning with band patch. IEEE Trans. Radiat. Plasma Med. Sci. 6, 859-873. View in Article CrossRef Google Scholar
[70]	Jin, K.H., McCann, M.T., Froustey, E., et al. (2017). Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process. 26, 4509-4522. View in Article CrossRef Google Scholar
[71]	Okamoto, T., Kumakiri, T., and Haneishi, H. (2022). Patch-based artifact reduction for three-dimensional volume projection data of sparse-view micro-computed tomography. Radiol. Phys. Technol. 15, 206-223. View in Article CrossRef Google Scholar
[72]	Xie, S., Zheng, X., Chen, Y., et al. (2018). Artifact removal using improved GoogLeNet for sparse-view CT reconstruction. Sci. Rep. 8, 6700. View in Article CrossRef Google Scholar
[73]	Lehtinen, J., Munkberg, J., Hasselgren, J., et al. (2018). Noise2Noise: learning image restoration without clean data. Preprint at arXiv. View in Article Google Scholar
[74]	Gao, X., Su, T., Zhang, Y., et al. (2023). Attention-based dual-branch deep network for sparse-view computed tomography image reconstruction. Quant. Imag. Med. Surg. 13, 1360-1374. View in Article CrossRef Google Scholar
[75]	Raparia, D., Alessi, J., and Kponou, A. (1997). The algebraic reconstruction technique (ART). Preprint at arXiv. View in Article Google Scholar
[76]	Bappy, D.M., and Jeon, I. (2017). Modified simultaneous iterative reconstruction technique for fast, high-quality CT reconstruction. IET Image Process. 11, 701-708. View in Article CrossRef Google Scholar
[77]	Huang, Y., Wang, S., Guan, Y., et al. (2020). Limited angle tomography for transmission X-ray microscopy using deep learning. J. Synchrotron Radiat. 27, 477-485. View in Article CrossRef Google Scholar
[78]	Wang, J., Liang, J., Cheng, J., et al. (2020). Deep learning based image reconstruction algorithm for limited-angle translational computed tomography. PLoS One 15, e0226963. View in Article CrossRef Google Scholar
[79]	Yiran, J., McMichael, N., Mokarzel, P., et al. (2020). Algorithm for limited angle CT reconstruction with U-net based regularization. NSS/MIC. View in Article Google Scholar
[80]	LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature *521*, 436-444. View in Article CrossRef Google Scholar
[81]	Yang, X., Kahnt, M., Brückner, D., et al. (2020). Tomograohic reconstruction with a generative adversarial network. J. Synchrotron Radiat. 27, 486-493. View in Article CrossRef Google Scholar
[82]	Wang, C., Ding, G., Liu, Y., and Xin, H.L. (2020). 0.7 angstrom resolution electron tomography enabled by deep-learning-aided information recovery. Adv. Intell. Syst. 2, 2000152. View in Article CrossRef Google Scholar
[83]	Xin, H., Wei, H., Ding, G., et al. (2020). Artificial intelligence enabled information inpainting and artifact removal for electron tomography. Microsc. Microanal. 26, 664-665. View in Article CrossRef Google Scholar
[84]	Barutcu, S., Aslan, S., Katsaggelos, A.K., et al. (2021). Limited-angle computed tomography with deep image and physics priors. Sci. Rep. 11, 17740. View in Article CrossRef Google Scholar
[85]	Lempitsky, V., Vedaldi, A., and Ulyanov, D. (2018). Deep image prior. CVPR. View in Article Google Scholar
[86]	Qiu, B., You, Y., Huang, Z., et al. (2021). N2NSR-OCT: simultaneous denoising and super-resolution in optical coherence tomography images using semisupervised deep learning. J. Biophotonics 14, e202000282. View in Article CrossRef Google Scholar
[87]	Jing, J., Xia, W., Hou, M., et al. (2022). Training low dose CT denoising network without high quality reference data. Phys. Med. Biol. 67, 084002. View in Article CrossRef Google Scholar
[88]	Chen, H., Zhang, Y., Kalra, M.K., et al. (2017). Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE Trans. Med. Imaging 36, 2524-2535. View in Article CrossRef Google Scholar
[89]	Selim, M., Zhang, J., Brooks, M.A., et al. (2023). DiffusionCT: latent diffusion model for CT image standardization. Preprint at arXiv. View in Article Google Scholar
[90]	Liu, Z., Bicer, T., Kettimuthu, R., et al. (2020). TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion. J. Opt. Soc. Am. Opt Image Sci. Vis. 37, 422-434. View in Article CrossRef Google Scholar
[91]	Shan, H., Zhang, Y., Yang, Q., et al. (2018). 3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network. IEEE Trans. Med. Imaging 37, 1522-1534. View in Article CrossRef Google Scholar
[92]	Yang, L., Li, Z., Ge, R., et al. (2023). Low-dose CT denoising via sinogram inner-structure transformer. IEEE Trans. Med. Imaging 42, 910-921. View in Article CrossRef Google Scholar
[93]	Yin, X., Zhao, Q., Liu, J., et al. (2019). Domain progressive 3D residual convolution network to improve low-dose CT imaging. IEEE Trans. Med. Imaging 38, 2903-2913. View in Article CrossRef Google Scholar
[94]	Zhou, Z., Li, C., Bi, X., et al. (2023). A machine learning model for textured X-ray scattering and diffraction image denoising. NPJ Comput. Mater. 9, 58. View in Article CrossRef Google Scholar
[95]	Hendriksen, A.A., Pelt, D.M., and Batenburg, K.J. (2020). Noise2Inverse: self-supervised deep convolutional denoising for tomography. IEEE Trans. Comput. Imaging 6, 1320-1335. View in Article CrossRef Google Scholar
[96]	De Carlo, F., Gürsoy, D., Ching, D.J., et al. (2018). TomoBank: a tomographic data repository for computational x-ray science. Meas. Sci. Technol. 29, 034004. View in Article CrossRef Google Scholar
[97]	Yang, H., Wang, W., Shang, J., et al. (2021). Segmentation of computed tomography images and high-precision reconstruction of rubber composite structure based on deep learning. Compos. Sci. Technol. *213*, 108875. View in Article CrossRef Google Scholar
[98]	Davydzenka, T., Sinclair, D., Chawla, N., et al. (2022). Deep-layers-assisted machine learning for accurate image segmentation of complex materials. Mater. Charact. *192*, 112175. View in Article CrossRef Google Scholar
[99]	Torbati-Sarraf, H., Niverty, S., Singh, R., et al. (2021). Machine-learning-based algorithms for automated image segmentation techniques of transmission X-ray microscopy (TXM). Jom 73, 2173-2184. View in Article CrossRef Google Scholar
[100]	Lin, B., Emami, N., Santos, D.A., et al. (2022). A deep learned nanowire segmentation model using synthetic data augmentation. NPJ Comput. Mater. 8, 88. View in Article CrossRef Google Scholar
[101]	Zhou, Z.W., Siddiquee, M.M.R., Tajbakhsh, N., et al. (2018). UNet plus plus : a nested U-Net architecture for medical image segmentation. Preprint at arXiv. View in Article Google Scholar
[102]	Zhao, H.S., Shi, J.P., Qi, X.J., et al. (2017). Pyramid scene parsing network. Preprint at arXiv. View in Article Google Scholar
[103]	Chen, L.-C., Zhu, Y., Papandreou, G., et al. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. Preprint at arXiv. View in Article Google Scholar
[104]	Deng, J., Dong, W., Socher, R., et al. (2009). ImageNet: a large-scale hierarchical image database. CVPR 1. View in Article Google Scholar
[105]	He, K., Gkioxari, G., Dollar, P., et al. (2020). Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42, 386-397. View in Article CrossRef Google Scholar
[106]	Moebel, E., Martinez-Sanchez, A., Lamm, L., et al. (2021). Deep learning improves macromolecule identification in 3D cellular cryo-electron tomograms. Nat. Methods 18, 1386-1394. View in Article CrossRef Google Scholar
[107]	Liu, C., Zeng, X., Lin, R., et al. (2018). Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms. ICIP *2018*, 1578-1582. View in Article CrossRef Google Scholar
[108]	Zhu, H., Wang, C., Wang, Y., et al. (2022). Unsupervised multi-task learning for 3D subtomogram image alignment, Clustering and Segmentation. ICIP. View in Article Google Scholar
[109]	Zhou, B., Yu, H., Zeng, X., et al. (2020). One-shot learning with attention-guided segmentation in cryo-electron tomography. Front. Mol. Biosci. 7, 613347. View in Article Google Scholar
[110]	Heebner, J.E., Purnell, C., Hylton, R.K., et al. (2022). Deep learning-based segmentation of cryo-electron tomograms. J. Vis. Exp. View in Article Google Scholar
[111]	Wu, S., Liu, G., and Yang, G. (2022). Fast particle picking for cryo-electron tomography using one-stage detection. ISBI. View in Article Google Scholar
[112]	Genc, A., Kovarik, L., and Fraser, H.L. (2022). A deep learning approach for semantic segmentation of unbalanced data in electron tomography of catalytic materials. Sci. Rep. 12, 16267. View in Article CrossRef Google Scholar
[113]	Nguyen, N., Bohak, C., Engel, D., et al. (2023). Finding Nano-Otzi: cryo-electron tomography visualization guided by learned segmentation. IEEE Trans. Vis. Comput. Graph. 29, 4198-4214. View in Article CrossRef Google Scholar
[114]	Zhang, S., Liu, J., Hu, B., et al. (2023). GH-DDM: the generalized hybrid denoising diffusion model for medical image generation. Multimed. Syst. 29, 1335-1345. View in Article CrossRef Google Scholar
[115]	Wu, J., Fu, R., Fang, H., et al. (2022). MedSegDiff: medical image segmentation with diffusion probabilistic model. Preprint at arXiv. View in Article Google Scholar
[116]	Wu, J., Fu, R., Fang, H., et al. (2023). MedSegDiff-V2: diffusion based medical image segmentation with transformer. Preprint at arXiv. View in Article Google Scholar
[117]	Kopp, R., Joseph, J., Ni, X., et al. (2022). Deep learning unlocks X-ray microtomography segmentation of multiclass microdamage in heterogeneous materials. Adv. Mater. 34, 2107817. View in Article CrossRef Google Scholar
[118]	Fu, T., Monaco, F., Li, J., et al. (2022). Deep-learning-enabled crack detection and analysis in commercial lithium-ion battery cathodes. Adv. Funct. Mater. 32, 2203070. View in Article CrossRef Google Scholar
[119]	Shorten, C., and Khoshgoftaar, T.M. (2019). A survey on image data augmentation for deep learning. JBD 6, 60. View in Article Google Scholar
[120]	Thersleff, T., and Tai, C.-W. (2023). Feature-specific correlation of structural, optical, and chemical properties in the transmission electron microscope with hypermodal data fusion. Microsc. Microanal. 29, 166-179. View in Article CrossRef Google Scholar
[121]	Muto, S., and Shiga, M. (2020). Application of machine learning techniques to electron microscopic/spectroscopic image data analysis. Microscopy 69, 110-122. View in Article CrossRef Google Scholar
[122]	Nandhini Abirami, R., Durai Raj Vincent, P.M., Srinivasan, K., et al. (2022). Multimodal medical image fusion of positron emission tomography and magnetic resonance imaging using generative adversarial networks. Behav. Neurol. *2022*, 6878783. View in Article Google Scholar
[123]	Liu, M., and Mukerji, T. (2022). Multiscale fusion of digital rock images based on deep generative adversarial networks. Geophys. Res. Lett. 49. View in Article Google Scholar
[124]	Karras, T., Laine, S., Aittala, M., et al. (2020). Analyzing and improving the image quality of StyleGAN. CVPR. View in Article Google Scholar
[125]	McKenzie, E.M., Santhanam, A., Ruan, D., et al. (2020). Multimodality image registration in the head-and-neck using a deep learning-derived synthetic CT as a bridge. Med. Phys. 47, 1094-1104. View in Article CrossRef Google Scholar
[126]	Blendowski, M., Bouteldja, N., and Heinrich, M.P. (2020). Multimodal 3D medical image registration guided by shape encoder-decoder networks. IJCARS 15, 269-276. View in Article CrossRef Google Scholar
[127]	Hering, A., Kuckertz, S., Heldmann, S., et al. (2019). Memory-efficient 2.5D convolutional transformer networks for multi-modal deformable registration with weak label supervision applied to whole-heart CT and MRI scans. IJCARS 14, 1901-1912. View in Article CrossRef Google Scholar
[128]	Badea, C.T., Holbrook, M., and Clark, D.P. (2018). Multi-energy CT decomposition using convolutional neural networks. Med. Imaging. View in Article Google Scholar
[129]	Gong, H., Tao, S., Rajendran, K., et al. (2020). Deep-learning-based direct inversion for material decomposition. Med. Phys. 47, 6294-6309. View in Article CrossRef Google Scholar
[130]	Nadkarni, R., Allphin, A., Clark, D.P., et al. (2022). Material decomposition from photon-counting CT using a convolutional neural network and energy-integrating CT training labels. Phys. Med. Biol. 67, 155003. View in Article CrossRef Google Scholar
[131]	Fang, W., Wu, D., Kim, K., et al. (2020). Direct dual energy CT material decomposition using Noise2Noise prior. NSS/MIC. View in Article Google Scholar
[132]	Terzi, S., Salvo, L., Suéry, M., et al. (2009). In situ X-ray tomography observation of inhomogeneous deformation in semi-solid aluminium alloys. Scr. Mater. 61, 449-452. View in Article CrossRef Google Scholar
[133]	Yang, Z., Ren, W., Sharma, R., et al. (2017). In-situ X-ray computed tomography characterisation of 3D fracture evolution and image-based numerical homogenisation of concrete. Cem. Concr. Compos. 75, 74-83. View in Article CrossRef Google Scholar
[134]	Scott, A.E., Mavrogordato, M., Wright, P., et al. (2011). In situ fibre fracture measurement in carbon-epoxy laminates using high resolution computed tomography. Compos. Sci. Technol. 71, 1471-1477. View in Article CrossRef Google Scholar
[135]	Lorenzoni, R., Curosu, I., Léonard, F., et al. (2020). Combined mechanical and 3D-microstructural analysis of strain-hardening cement-based composites (SHCC) by in-situ X-ray microtomography. Cem. Concr. Res. *136*, 106139. View in Article CrossRef Google Scholar
[136]	Wang, T., Jiang, Z., Kemao, Q., et al. (2016). GPU accelerated digital volume correlation. Exp. Mech. 56, 297-309. View in Article CrossRef Google Scholar
[137]	Duan, X., and Huang, J. (2022). Deep learning-based digital volume correlation. Extreme Mech. Lett. 53, 101710. View in Article CrossRef Google Scholar
[138]	Lialin, V., Deshpande, V., and Rumshisky, A. (2023). Scaling down to scale up: a guide to parameter-efficient fine-tuning. Preprint at arXiv. View in Article Google Scholar
[139]	Chen, Z., Duan, Y., Wang, W., et al. (2023). Vision transformer adapter for dense predictions. Preprint at arXiv. View in Article Google Scholar
[140]	Yang, T., Zhu, Y., Xie, Y., et al. (2023). AIM: adapting image models for efficient video action recognition. View in Article Google Scholar
[141]	Zhou, K., Yang, J., Loy, C.C., et al. (2022). Learning to prompt for vision-language models. IJCV *130*, 2337-2348. View in Article CrossRef Google Scholar
[142]	Jia, M.L., Tang, L.M., Chen, B.C., et al. (2022). Visual prompt tuning. Preprint at arXiv. View in Article Google Scholar
[143]	Vijayakumar, J., Yuan, H., Mille, N., et al. (2023). Soft X-ray spectro-ptychography of boron nitride nanobamboos, carbon nanotubes and permalloy nanorods. J. Synchrotron Radiat. 30, 746-757. View in Article CrossRef Google Scholar
[144]	Radford, A., Kim, J.W., Hallacy, C., et al. (2021). Learning transferable visual models from natural language supervision. Preprint at arXiv. View in Article Google Scholar
[145]	Bai, Y., Jones, A., Ndousse, K., et al. (2022). Training a helpful and harmless assistant with reinforcement learning from human feedback. Preprint at arXiv. View in Article Google Scholar

Cited by

ABOUT THIS ARTICLE

Cite this article:

Zhen Zhang, Chun Li, Wenhui Wang, Zheng Dong, Gongfa Liu, Yuhui Dong, Yi Zhang. Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments[J]. The Innovation, 2024, 5(1). https://doi.org/10.1016/j.xinn.2023.100539

Zhen Zhang, Chun Li, Wenhui Wang, Zheng Dong, Gongfa Liu, Yuhui Dong, Yi Zhang. Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments[J]. The Innovation, 2024, 5(1). https://doi.org/10.1016/j.xinn.2023.100539

Welcome!

To request copyright permission to republish or share portions of our works, please visit Copyright Clearance Center's (CCC) Marketplace website at marketplace.copyright.com.

PDF

Figures(6)

Article Metrics

Article views(3494) PDF downloads(741)

Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments

GRAPHICAL ABSTRACT

PUBLIC SUMMARY

ABSTRACT

REFERENCES

ABOUT THIS ARTICLE

Cite this article:

Welcome!

Share

Article Metrics

Relative Articles

Cited by

Catalog

{{newsColumn.name}}

Journal links

{{newsColumn.name}}

Collections

Stay connected

Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments

GRAPHICAL ABSTRACT

PUBLIC SUMMARY

ABSTRACT

REFERENCES

ABOUT THIS ARTICLE

Cite this article:

Welcome!

Share

Article Metrics

Relative Articles

Cited by

Catalog

Export File

Citation

Format

Content

{{newsColumn.name}}

Journal links

{{newsColumn.name}}

Collections

Stay connected