Prompt mechanisms in medical imaging: A comprehensive survey

Public summary

* Prompt-based mechanisms effectively mitigate data scarcity and distribution shifts in medical imaging.

* This survey systematically categorizes diverse prompt mechanisms for medical imaging tasks.

* Prompts guide foundation models to enhance accuracy and adaptability without extensive retraining.

* Future directions involve automated prompting, unified multimodal models, and secure clinical integration.

Abstract

Deep learning offers transformative potential in medical imaging, yet its clinical adoption is frequently hampered by challenges such as data scarcity, distribution shifts, and the need for robust task generalization. Prompt-based methodologies have emerged as a pivotal strategy to guide deep learning models, providing flexible, domain-specific adaptations that significantly enhance model performance and adaptability without extensive retraining. This systematic review critically examines the burgeoning landscape of prompt engineering in medical imaging. We dissect diverse prompt modalities, including textual instructions, visual prompts, and learnable embeddings, and analyze their integration for core tasks such as image generation, segmentation, and classification. Our survey shows that prompt mechanisms advance medical AI on two fronts. At a performance level, they enhance accuracy, robustness, and data efficiency. Methodologically, they circumvent the need for manual feature engineering. This model guidance has the potential to enhance the interpretability of model behavior by making task guidance more explicit. Despite substantial advancements, we identify persistent challenges, particularly in prompt design optimization, data heterogeneity, and ensuring scalability for clinical deployment. Finally, this review outlines promising future trajectories, including advanced multimodal prompting and robust clinical integration, underscoring the critical role of prompt-driven AI in accelerating the revolution of diagnostics and personalized treatment planning in medicine.