Is a data deluge dampening our idea generation capability?

Reforming the peer review praxis

The peer-reviewed journal system has been unable to adapt to and provide a suitable system that can leverage AI technologies while controlling for downsides. AI technologies introduce properties, bottlenecks, requirements, and features that require interacting systems to undergo considerable design evolution. Each system needs to withstand the volume and complexity of AI and be able to weed out misstatements and attributional issues in submissions that may stem from unnamed, difficult-to-trace inputs.

Framing clear rules to mediate between authors, referees, and editors, integrating relatively “trustless” smart contracts and content escrows, and remuneration for referees are possibilities to be considered. Feasibility concerns still remain, however. Does decentralized peer review prevent bias? Are smart contracts really secure?

Democratization of the referee process, combined with systems to rank referees, is important. Perhaps a social-media-style open platform open to community engagement will rank referees and assign “trust scores” to papers based on the referee’s standing and public comments.

An immediate solution may be the preprint culture—if the general drift of the first few referees is in a particular direction, then the paper could be deemed suitable for approval. Publications may first be held in an intermediate compartment pending democratic review, after which a full digital object identifier (DOI) is granted. A replicability index could also be instituted, reflecting findings in successive papers, with an automatic retraction based on index thresholds.