AI in academic publishing no longer elephant in the room
Marked by the moment when ChatGPT and other large language models (LLMs) were released to the public, the rise of artificial intelligence (AI) around 2022 profoundly reshaped the landscape of knowledge creation and public discourse. At that early stage, The Innovation was among the first academic journals to recognize both the influence and the potential of this emerging technology. We published analyses examining the nature of LLMs and the broader concept of AI and highlighted the expansive prospects these systems might hold for scientific inquiry and broader societal transformation.
As successive generations of LLMs have appeared in rapid succession over the past several years, we have been gratified to see a technology in which we placed considerable confidence evolve with unprecedented speed. Our fundamental assessment remains unchanged: AI, as a technology built upon vast corpora of human language, has already demonstrated remarkable value (Figure 1). It enriches linguistic and intellectual expression, enables complex computation with efficiency unimaginable a decade ago, and offers new tools for humans to understand themselves and their world.
These contributions are substantial, and we see no indication that their importance will diminish. On the contrary, their relevance will only deepen in the foreseeable future.
Yet it is precisely because of this accelerating influence that a cold head is required when evaluating AI's role in natural science research. In academic publishing, authors may use AI to improve clarity, but they remain fully responsible for their content; reviewers may use it cautiously to assist evaluation without replacing judgment; and editors oversee ethical use, transparency, and integrity across the entire publication process. Many of our editors, reviewers, and authors are themselves specialists in AI, and collectively, we recognize a simple truth: today’s AI systems remain far from flawless. Bridging the gap between current capabilities and the level of reliability required for scientific inquiry will demand sustained intellectual effort from some of the most exceptional minds of our time.
A clear illustration of this gap is the now widely acknowledged issue of AI hallucination. When confronted with questions beyond its grasp, an AI system seldom admits ignorance. Instead, it may produce statements without factual support, misinterpret contextual cues, or generate fabricated details. These outputs often appear coherent, fluent, and confident—qualities that can easily mislead the inattentive reader—yet they are, at their core, erroneous or entirely invented.
This is not a trivial defect; it is rooted in how AI fundamentally operates. Rather than understanding language through meaning, intention, and context the way humans do, AI processes text by identifying and reproducing statistical patterns and correlations within large datasets, without possessing genuine semantic comprehension, conceptual awareness, or an understanding of the real-world phenomena behind the words it generates. Scientific research, however, is grounded in the rigorous understanding of mechanisms, variables, and causal relationships.
