Encoding biological metaverse: Advancements and challenges in neural fields from macroscopic to microscopic
-
ABSTRACT
Neural fields can efficiently encode three-dimensional (3D) scenes, providing a bridge between two-dimensional (2D) images and virtual reality. This method becomes a trendsetter in bringing the metaverse into vivo life. It has initially captured the attention of macroscopic biology, as demonstrated by computed tomography and magnetic resonance imaging, which provide a 3D field of view for diagnostic biological images. Meanwhile, it has also opened up new research opportunities in microscopic imaging, such as achieving clearer de novo protein structure reconstructions. Introducing this method to the field of biology is particularly significant, as it is refining the approach to studying biological images. However, many biologists have yet to fully appreciate the distinctive meaning of neural fields in transforming 2D images into 3D perspectives. This article discusses the application of neural fields in both microscopic and macroscopic biological images and their practical uses in biomedicine, highlighting the broad prospects of neural fields in the future biological metaverse. We stand at the threshold of an exciting new era, where the advancements in neural field technology herald the dawn of exploring the mysteries of life in innovative ways. -
-
REFERENCES
[1] Cheng, S., Zhang, Y., Li, X., et al. (2022). Roadmap toward the metaverse: An AI perspective. Innovation 3(5): 100293. https://doi.org/10.1016/j.xinn.2022.100293. [2] Mildenhall, B., Srinivasan, P.P., Tancik, M., et al. (2021). Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65(1): 99-106. https://doi.org/10.1145/3503250. [3] Molaei, A., Aminimehr, A., Tavakoli, A., et al. (2023). Implicit Neural Representation in Medical Imaging: A Comparative Survey. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (IEEE Computer Society), pp. 2373–2383. https://doi.org/10.1109/ICCVW60793.2023.00252. [4] Chen, Z., Yang, L., Lai, J.-H., et al. (2023). CuNeRF: Cube-Based Neural Radiance Field for Zero-Shot Medical Image Arbitrary-Scale Super Resolution. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (IEEE Computer Society), pp. 21128–21138. https://doi.org/10.1109/ICCV51070.2023.01937. [5] Zhong, E.D., Lerer, A., Davis, J.H., et al. (2021). CryoDRGN2: Ab initio neural reconstruction of 3D protein structures from real cryo-EM images. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (IEEE Computer Society), pp. 4046–4055. https://doi.org/10.1109/ICCV48922.2021.00403. [6] Zhong, E.D., Bepler, T., Berger, B., et al. (2021). CryoDRGN: reconstruction of heterogeneous cryo-EM structures using neural networks. Nat. Methods 18(2): 176-185. https://doi.org/10.1038/s41592-020-01049-4. [7] Xu, J., Moyer, D., Gagoski, B., et al. (2023). Nesvor: Implicit neural representation for slice-to-volume reconstruction in mri. IEEE Trans. Med. Imaging 42(6): 1707-1719. https://doi.org/10.1109/TMI.2023.3236216. [8] Wolterink, J.M., Zwienenberg, J.C., and Brune, C. (2022). Implicit Neural Representations for Deformable Image Registration. Proc. Mach. Learn. Res. 172(1-17): 1349-1359. [9] Gu, J., Tian, F., and Oh, I.-S. (2023). Retinal vessel segmentation based on self-distillation and implicit neural representation. Appl. Intell. 53(12): 15027-15044. https://doi.org/10.1007/s10489-022-04252-2. [10] Chen, H., He, B., Wang, H., et al. (2021). Nerv: Neural representations for videos. Adv. Neural Inf. Process. Syst. 34(2021): 21557-21568. -
ABOUT THIS ARTICLE
Cite this article:
Cai Y., Hu W., Pei Y., et al., (2024). Encoding biological metaverse: Advancements and challenges in neural fields from macroscopic to microscopic. The Innovation 5(3), 100627. https://doi.org/10.1016/j.xinn.2024.100627
Welcome!
To request copyright permission to republish or share portions of our works, please visit Copyright Clearance Center's (CCC) Marketplace website at marketplace.copyright.com.
Export File
Citation
Cai Y., Hu W., Pei Y., et al., (2024). Encoding biological metaverse: Advancements and challenges in neural fields from macroscopic to microscopic. The Innovation 5(3), 100627. https://doi.org/10.1016/j.xinn.2024.100627
Format
Content
DownLoad:
-
Figure 1.
Applications of neural fields in macro- and microbiology
