Molecular-substructure Deep Autoencoders cluster bioactive molecules into novel band-shaped substructure-distinguished bioactivity clusters in 3D latent space

Huazhang Ying; Xiang Wu; Chu Qin; Likun Zhang; Zhicheng Du; Jiaqi Liu; Yuzong Chen; Ying Tan

doi:10.59717/j.xinn-inform.2026.100049

The Innovation Informatics > 2026 Vol. 2 > No. 2 > 100049

Next Article Previous Article

ARTICLE Open Access

Cite PDF

Molecular-substructure Deep Autoencoders cluster bioactive molecules into novel band-shaped substructure-distinguished bioactivity clusters in 3D latent space

1.
State Key Laboratory of Chemical Oncogenomics, Institute of Biopharmaceutical and Health Engineering, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
2.
These authors contributed equally

More Information

Corresponding authors: tan.ying@sz.tsinghua.edu.cn (Y.T.); chenyuzong@sz.tsinghua.edu.cn (Y.C.)
Received Date: January 04, 2026

Accepted Date: May 12, 2026

Published Online: May 14, 2026

The Innovation Informatics 2(2), https://doi.org/10.59717/j.xinn-inform.2026.100049 | Cite this article

GRAPHICAL ABSTRACT

GRAPHICAL ABSTRACT

DownLoad: Full size image

PUBLIC SUMMARY

PUBLIC SUMMARY
1. Unsupervised DAEs for reversible 3D feature extraction and clustering of 1.9M bioactive molecules.
  
  Chempack enables rapid navigation and 3D visualization of molecular data points.
  
  Discovery of band-shaped clusters in latent space across three sets of molecular fingerprints.
  
  Identification of novel bioactive molecular clustering patterns via substructure and target correlations.

ABSTRACT

ABSTRACT

The rapid expansion of bioactive molecules (biomolecules) datasets requires scalable and interpretable chemical space analysis. However, the sparse molecular fingerprints rely on multi-step dimensionality reduction pipelines for clustering and visualization, which introduce geometric distortions and obscure intrinsic cluster patterns. In this work, we use substructure-based molecular fingerprint deep autoencoders (MolF-DAEs) to directly embed 1.9 million biomolecules into three-dimensional latent spaces (3DLSpaces), enabling locally and bioactivity-relevant functionally undistorted visualization of chemical space. By enforcing direct end-to-end autoencoder reconstruction learning, MolF-DAEs achieve high-fidelity compression (96.1-97.6% reconstruction rate) across three widely used fingerprint schemes. In 3DLSpace, the biomolecules cluster into novel band-shaped clusters, each dominated by distinct core substructure. The biomolecules within same cluster converge on a limited set of bioactivity classes. We conclude that under appropriate substructure constraints, complex bioactivity landscapes can be represented in a compact 3DLSpace with preserved structural fidelity and functional coherence. MolF-DAEs provide a robust framework for interpretable unsupervised clustering, uncovering structure-activity relationships.
- Bioactive molecule /
- MolF-DAEs /
- 3DLSpace /
- substructure

FullText(HTML)

REFERENCES

[1]	Mullowney M.W., Duncan K.R., Elsayed S.S., et al. (2023). Artificial intelligence for natural product drug discovery. Nat. Rev. Drug Discov. 22:895-916. DOI:10.1038/s41573-023-00774-7 View in Article Google Scholar
[2]	Tropsha A., Isayev O., Varnek A., et al. (2024). Integrating QSAR modelling and deep learning in drug discovery: The emergence of deep QSAR. Nat. Rev. Drug Discov. 23:141-155. DOI:10.1038/s41573-023-00832-0 View in Article Google Scholar
[3]	Mahdizadeh S.J. and Eriksson L.A. (2025). MolAI: A deep learning framework for data-driven molecular descriptor generation and advanced drug discovery applications. J. Chem. Inf. Model. 65:9892-9909. DOI:10.1021/acs.jcim.5c00491 View in Article Google Scholar
[4]	Jaeger S., Fulle S. and Turk S. (2018). Mol2vec: Unsupervised machine learning approach with chemical intuition. J. Chem. Inf. Model. 58:27-35. DOI:10.1021/acs.jcim.7b00616 View in Article Google Scholar
[5]	Levina E. and Bickel P.J. (2004). In Proceedings of the 18th International Conference on Neural Information Processing Systems, 777-784. MIT Press. View in Article Google Scholar
[6]	Song C., Liu F., Huang Y., et al. (2013). Auto-encoder based data clustering. Ruiz-Shulcloper J. and Sanniti di Baja G. (eds). Progress in pattern recognition, image analysis, computer vision, and applications: 18th Iberoamerican Congress, CIARP 2013, proceedings, Part I. (Springer), pp. 117–124. DOI:10.1007/978-3-642-41822-8_15 View in Article Google Scholar
[7]	Walters S.J. and Campbell M.J. (2004). The use of bootstrap methods for analysing health-related quality of life outcomes (particularly the SF-36). Health Qual. Life Outcomes 2:70. DOI:10.1186/1477-7525-2-70 View in Article Google Scholar
[8]	Eckhardt C.M., Madjarova S.J., Williams R.J., et al. (2023). Unsupervised machine learning methods and emerging applications in healthcare. Knee Surg. Sports Traumatol. Arthrosc. 31:376-381. DOI:10.1007/s00167-022-07233-7 View in Article Google Scholar
[9]	Altman N. and Krzywinski M. (2017). Clustering. Nat. Methods 14:545-546. DOI:10.1038/nmeth.4299 View in Article Google Scholar
[10]	Kamal I.M. and Bae H. (2022). Super-encoder with cooperative autoencoder networks. Pattern Recognit.126:108562. DOI:10.1016/j.patcog.2022.108562 View in Article Google Scholar
[11]	Hinton G.E. and Salakhutdinov R.R. (2006). Reducing the dimensionality of data with neural networks. Science 313:504-507. DOI:10.1126/science.1127647 View in Article Google Scholar
[12]	Lopez R., Regier J., Cole M.B., et al. (2018). Deep generative modeling for single-cell transcriptomics. Nat. Methods 15:1053-1058. DOI:10.1038/s41592-018-0229-2 View in Article Google Scholar
[13]	Lotfollahi M., Wolf F.A. and Theis F.J. (2019). scGen predicts single-cell perturbation responses. Nat. Methods 16:715–721. DOI:10.1038/s41592-019-0494-8 View in Article Google Scholar
[14]	Gayoso A., Steier Z., Lopez R., et al. (2021). Joint probabilistic modeling of single-cell multi-omic data with totalVI. Nat. Methods 18:272-282. https://doi.org/10.1038/s41592-020-01050-x View in Article Google Scholar
[15]	Dong K. and Zhang S. (2022). Deciphering spatial domains from spatially resolved transcriptomics with an adaptive graph attention auto-encoder. Nat. Commun. 13:1739. DOI:10.1038/s41467-022-29439-6 View in Article Google Scholar
[16]	Holmström L. (2008). Nonlinear dimensionality reduction by John A. Lee, Michel Verleysen. Int. Stat. Rev. 76:308-309. DOI:10.1111/j.1751-5823.2008.00054_10.x View in Article Google Scholar
[17]	Steinbach M., Ertöz L. and Kumar V. (2004). In Wille L.T. (ed). New Directions in Statistical Physics: Econophysics, Bioinformatics, and Pattern Recognition, 273-309. Springer Berlin Heidelberg View in Article Google Scholar
[18]	Yu W., Wang R., Nie F., et al. (2018). Multi-view embedded clustering with unsupervised trace ratio LDA. Neurocomputing 315:169-176. DOI:10.1016/j.neucom.2018.07.014 View in Article Google Scholar
[19]	Stumpfe D. and Bajorath J. (2012). Exploring activity cliffs in medicinal chemistry. J. Med. Chem. 55:2932-2942. DOI:10.1021/jm201706b View in Article Google Scholar
[20]	Han Z., Liu Z., Han J., et al. (2017). Mesh convolutional restricted Boltzmann machines for unsupervised learning of features with structure preservation on 3-D meshes. IEEE Trans. Neural Netw. Learn. Syst. 28:2268-2281. DOI:10.1109/TNNLS.2016.2582532 View in Article Google Scholar
[21]	Guo X., Liu X., Zhu E., et al. (2017). Deep clustering with convolutional autoencoders. In Liu D., Xie S., Li Y., et al. (eds.). Neural information processing - 24th international conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, proceedings, Part II. 10635:373–382. Springer. DOI:10.1007/978-3-319-70096-0_39 View in Article Google Scholar
[22]	Zdrazil B., Felix E., Hunter F., et al. (2023). The ChEMBL database in 2023: A drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic Acids Res. 52:D1180-D1192. DOI:10.1093/nar/gkad1004 View in Article Google Scholar
[23]	Shen W.X., Zeng X., Zhu F., et al. (2021). Out-of-the-box deep learning prediction of pharmaceutical properties by broadly learned knowledge-based molecular representations. Nat. Mach. Intell. 3:334-343. DOI:10.1038/s42256-021-00301-6 View in Article Google Scholar
[24]	Mendez D., Gaulton A., Bento A.P., et al. (2019). ChEMBL: Towards direct deposition of bioassay data. Nucleic Acids Res. 47:D930-D940. DOI:10.1093/nar/gky1075 View in Article Google Scholar
[25]	Gaulton A., Hersey A., Nowotka M., et al. (2017). The ChEMBL database in 2017. Nucleic Acids Res. 45:D945–D954. DOI:10.1093/nar/gkw1074 View in Article Google Scholar
[26]	Yang K., Swanson K., Jin W., et al. (2019). Analyzing learned molecular representations for property prediction. J. Chem. Inf. Model. 59:3370-3388. DOI:10.1021/acs.jcim.9b00237 View in Article Google Scholar
[27]	Shen W.X., Yang K., Liu J., et al. (2022). AggMapNet: Enhanced and explainable low-sample omics deep learning with feature-aggregated multi-channel networks. Nucleic Acids Res. 50:e45. DOI:10.1093/nar/gkac010 View in Article Google Scholar
[28]	Dong J., Yao Z.J., Zhang L., et al. (2018). PyBioMed: A python library for various molecular representations of chemicals, proteins and DNAs and their interactions. J. Cheminform. 10:16. DOI:10.1186/s13321-018-0270-2 View in Article Google Scholar
[29]	Shawe-Taylor J., Zemel R.S., Bartlett P.L., et al. (eds.) (2011). Advances in neural information processing systems 24: 25th annual conference on neural information processing systems 2011. Proceedings of a meeting held 12-14 December 2011, Granada, Spain. Curran Associates, Inc. View in Article Google Scholar
[30]	Gómez-Bombarelli R., Wei J.N., Duvenaud D., et al. (2018). Automatic chemical design using a data-driven continuous representation of molecules. ACS Cent. Sci. 4:268-276. DOI:10.1021/acscentsci.7b00572 View in Article Google Scholar
[31]	Ilnicka A. and Schneider G. (2023). Compression of molecular fingerprints with autoencoder networks. Mol. Inform. 42:2300059. DOI:10.1002/minf.202300059 View in Article Google Scholar
[32]	Moon K.R., van Dijk D., Wang Z., et al. (2019). Visualizing structure and transitions in high-dimensional biological data. Nat. Biotechnol. 37:1482-1492. DOI:10.1038/s41587-019-0336-3 View in Article Google Scholar
[33]	McInnes L. and Healy J. (2018). UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint. DOI: arXiv.1802.03426 View in Article Google Scholar
[34]	Moon K.R., van Dijk D., Wang Z., et al. (2019). Visualizing structure and transitions in high-dimensional biological data. Nat. Biotechnol. 37:1482-1492. DOI:10.1038/s41587-019-0336-3 View in Article Google Scholar
[35]	Huang H., Wang Y., Rudin C., et al. (2022). Towards a comprehensive evaluation of dimension reduction methods for transcriptomic data visualization. Commun. Biol. 5:719. DOI:10.1038/s42003-022-03628-x View in Article Google Scholar
[36]	Li Y.H., Li X.X., Hong J.J., et al. (2019). Clinical trials, progression-speed differentiating features and swiftness rule of the innovative targets of first-in-class drugs. Brief. Bioinform. 21:649-662. DOI:10.1093/bib/bby130 View in Article Google Scholar
[37]	Vasaikar S., Bhatia P., Bhatia P., et al. (2016). Complementary approaches to existing target based drug discovery for identifying novel drug targets. Biomedicines 4:27. DOI:10.3390/biomedicines4040027 View in Article Google Scholar
[38]	Attwood M.M., Fabbro D., Sokolov A.V., et al. (2021). Trends in kinase drug discovery: Targets, indications and inhibitor design. Nat. Rev. Drug Discov. 20:839-861. DOI:10.1038/s41573-021-00252-y View in Article Google Scholar
[39]	Alganem K., Hamoud A.R., Creeden J.F., et al. (2022). The active kinome: The modern view of how active protein kinase networks fit in biological research. Curr. Opin. Pharmacol. 62:117-129. DOI:10.1016/j.coph.2021.11.007 View in Article Google Scholar
[40]	Kumar V., Kaur N., Sahu S., et al. (2023). Role of tyrosine kinases and their inhibitors in cancer therapy: A comprehensive review. Curr. Med. Chem. 30:1464-1481. DOI:10.2174/0929867329666220727122952 View in Article Google Scholar
[41]	Shaban N., Kamashev D., Emelianova A., et al. (2023). Targeted inhibitors of EGFR: Structure, biology, biomarkers, and clinical applications. Cells 13:47. DOI:10.3390/cells13010047 View in Article Google Scholar
[42]	Santiago A.D.S., Couñago R.M., Ramos P.Z., et al. (2018). Structural analysis of inhibitor binding to CAMKK1 identifies features necessary for design of specific inhibitors. Sci. Rep. 8:14800. DOI:10.1038/s41598-018-33043-4 View in Article Google Scholar
[43]	Profeta G.S., dos Reis C.V., Santiago A.d.S., et al. (2019). Binding and structural analyses of potent inhibitors of the human Ca2+/calmodulin dependent protein kinase kinase 2 (CAMKK2) identified from a collection of commercially-available kinase inhibitors. Sci. Rep. 9:16452. DOI:10.1038/s41598-019-52795-1 View in Article Google Scholar
[44]	Bickerton G.R., Paolini G.V., Besnard J., et al. (2012). Quantifying the chemical beauty of drugs. Nat. Chem. 4:90-98. DOI:10.1038/nchem.1243 View in Article Google Scholar
[45]	Bondensgaard K., Ankersen M., Thøgersen H., et al. (2004). Recognition of privileged structures by G-protein coupled receptors. J. Med. Chem. 47:888-899. DOI:10.1021/jm0309452 View in Article Google Scholar
[46]	Liao J.J.-L. (2007). Molecular recognition of protein kinase binding pockets for design of potent and selective kinase inhibitors. J. Med. Chem. 50:409-424. DOI:10.1021/jm0608107 View in Article Google Scholar
[47]	van Linden O.P.J., Kooistra A.J., Leurs R., et al. (2014). KLIFS: A knowledge-based structural database to navigate kinase–ligand interaction space. J. Med. Chem. 57:249-277. DOI:10.1021/jm400378w View in Article Google Scholar
[48]	Aronov A.M., McClain B., Moody C.S., et al. (2008). Kinase-likeness and kinase-privileged fragments: Toward virtual polypharmacology. J. Med. Chem. 51:1214–1222. DOI:10.1021/jm701021b View in Article Google Scholar
[49]	Schneider P. and Schneider G. (2017). Privileged structures revisited. Angew. Chem. Int. Ed. 56:7971-7974. DOI:10.1002/anie.201702816 View in Article Google Scholar
[50]	Kanev G.K., de Graaf C., Westerman B.A., et al. (2020). KLIFS: An overhaul after the first 5 years of supporting kinase research. Nucleic Acids Res. 49:D562-D569. DOI:10.1093/nar/gkaa895 View in Article Google Scholar
[51]	Ilnicka A. and Schneider G. (2023). Designing molecules with autoencoder networks. Nat. Comput. Sci. 3:922-933. DOI:10.1038/s43588-023-00548-6 View in Article Google Scholar
[52]	Gebru T., Krause J., Wang Y., et al. (2017). Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proc. Natl. Acad. Sci. USA 114:13108-13113. DOI:10.1073/pnas.1700035114 View in Article Google Scholar
[53]	Silver D., Schrittwieser J., Simonyan K., et al. (2017). Mastering the game of Go without human knowledge. Nature 550:354-359. DOI:10.1038/nature24270 View in Article Google Scholar
[54]	Lu S. and Li R. (2021). DAC: Deep autoencoder-based clustering, a general deep learning framework of representation learning. arXiv preprint. DOI:arXiv.2102.07472 View in Article Google Scholar
[55]	Ren Y., Pu J., Yang Z., et al. (2024). Deep clustering: A comprehensive survey. IEEE Trans. Neural Netw. Learn. Syst. 35:1-21. DOI:10.1109/TNNLS.2024.3403155 View in Article Google Scholar
[56]	Xie J., Girshick R. and Farhadi A. (2016). Unsupervised deep embedding for clustering analysis. Proc. 33rd Int. Conf. Mach. Learn. 48:478-487. https://proceedings.mlr.press/v48/xieb16.html View in Article Google Scholar
[57]	Eraslan G., Simon L.M., Mircea M., et al. (2019). Single-cell RNA-seq denoising using a deep count autoencoder. Nat. Commun. 10:390. DOI:10.1038/s41467-018-07931-2 View in Article Google Scholar

Cited by

ABOUT THIS ARTICLE

Cite this article:

Ying H., Wu X., Qin C., et al. (2026). Molecular-substructure Deep Autoencoders cluster bioactive molecules into novel band-shaped substructure-distinguished bioactivity clusters in 3D latent space. The Innovation Informatics 2:100049. https://doi.org/10.59717/j.xinn-inform.2026.100049

Ying H., Wu X., Qin C., et al. (2026). Molecular-substructure Deep Autoencoders cluster bioactive molecules into novel band-shaped substructure-distinguished bioactivity clusters in 3D latent space. The Innovation Informatics 2:100049. https://doi.org/10.59717/j.xinn-inform.2026.100049

Welcome!

To request copyright permission to republish or share portions of our works, please visit Copyright Clearance Center's (CCC) Marketplace website at marketplace.copyright.com.

PDF

Figures(6)

Supplementary Information

download_supplementary
SI-Extended-TII-DOI-49

Article Metrics

Article views(645) PDF downloads(661)

Molecular-substructure Deep Autoencoders cluster bioactive molecules into novel band-shaped substructure-distinguished bioactivity clusters in 3D latent space

GRAPHICAL ABSTRACT

PUBLIC SUMMARY

ABSTRACT