Urban sensing in the era of large language models

Urban sensing has become increasingly important as cities evolve into the centers of human activities. Large language models (LLMs) offer new opportunities for urban sensing based on commonsense and worldview that emerged through their language-centric framework. This paper illustrates the transformative impact of LLMs, particularly in the potential of advancing next-generation urban sensing for exploring urban mechanisms. The discussion navigates through several key aspects, including enhancing knowledge transfer between humans and LLM, urban mechanisms awareness, and achieve automated decision-making with LLM agents. We emphasize the potential of LLMs to revolutionize urban sensing, offering a more comprehensive, efficient, and in-depth understanding of urban dynamics, and also acknowledge challenges in multi-modal data utilization, spatial-temporal cognition, cultural adaptability, and privacy preservation. The future of urban sensing with LLMs lies in leveraging their emerged intelligent and addressing these challenges to achieve more intelligent, responsible, and sustainable urban development.

Introduction

Urban sensing integrates and interprets multi-modal data on urban environments and human activities.1,2 Current urban sensing technologies focus on diverse urban phenomena, such as monitoring traffic flow dynamics, assessing air quality, or tracking the trajectories of human activities. However, they fall short of collaboratively sensing the interrelationships among these urban phenomena, such as the interaction between humans and the built environment.3 This limitation significantly hinders urban sensing’s ability to understand the urban mechanisms, thereby constraining urban sensing’s further development.4,5 Large language models (LLMs), represented by GPT, Gemini, and LLaMA, offer promising breakthroughs for addressing this challenge through their emerging commonsense and worldview, which encompass an inherent understanding of the world including time, space, physics, social interactions, and emotions, as well as the capacity to reason within these contexts.6 These commonsense and worldview provide LLMs with unique latent guidance in understanding the urban mechanisms through various urban phenomena, thus providing the prerequisites for forming next-generation urban sensing.7,8 This paper explores how LLMs can significantly advance the development of next-generation urban sensing (as illustrated in Figure 1) and the potential challenges during this transformative era.