Unleashing the potential of remote sensing foundation models via bridging data and computility islands

Data and computility islands in remote sensing for EO

The rapid advancement of Earth observation (EO) capabilities is driving an explosive increase in remote sensing data. There is an urgent need for advanced processing techniques to unleash their application value.1 Generalist EO intelligence refers to the ability to provide unified support for qualitative interpretation, quantitative inversion, and interactive dialogue across diverse EO data and tasks. It has attracted significant attention recently, prompting academia, industry, and government to invest substantial resources.2 Through developing remote sensing foundation models (RSFMs), generalist EO intelligence can ultimately offer humanity a shared spatial-temporal intelligence service in various fields (e.g., agriculture, forestry, and oceanography).3 However, a critical question remains: have we truly unleashed the potential of RSFMs for generalist EO intelligence? Despite the vast volume of remote sensing data, their distribution is often fragmented and decentralized due to privacy concerns, storage bottlenecks, industrial competition, and geo-information security. This fragmentation leads to data islands, which limit the full utilization of multi-source remote sensing data. Moreover, computility (i.e., computational resources) typically develops in isolation, inadequately supporting the large-scale training and application of RSFMs.

Limitation of the cloud-based architecture for RSFMs

Cloud-based architecture for remote sensing, exemplified by Google Earth Engine, is a widely adopted paradigm. It refers to the centralized storage and processing of remote sensing data on a cloud platform. However, this centralized paradigm demands substantial costs, typically affordable only by tech giants. Furthermore, this architecture raises several concerns, such as privacy concerns, monopolization, and ambiguous data ownership. Additionally, the data, storage, and computility on a single cloud platform are limited and difficult to scale, resulting in scalability challenges that hinder the growth and applicability of RSFMs.

Opportunities and challenges of the collaborative architecture for RSFMs

As we approach the upcoming era of sixth-generation mobile networks (6G), advancements in communication technologies are expected to provide significant network support, facilitating the collaborative architecture for RSFMs to address these critical challenges.4 Techniques such as federated learning and split learning can facilitate cross-cloud collaborative pre-training and fine-tuning of RSFMs, eliminating the need to transfer raw data. Recently, the success of the large language model INTELLECT-1, trained across 30 clouds distributed globally, provides valuable insights into this novel paradigm. Additionally, techniques such as multi-agent reinforcement learning and vision-and-language navigation can enhance intelligent task planning and inference. However, current research on collaborative architectures in remote sensing remains insufficient to fully support generalist EO intelligence. Challenges such as heterogeneity and trustworthiness continue to hinder effective collaboration. In practice, collaborative architectures can be integrated throughout the lifespan of RSFMs. In this commentary, we outline potential research directions in two key phases: cross-cloud collaborative training and collaborative inference of RSFMs for generalist EO intelligence.