Transformer-based deep learning models for disease prediction: A systematic review

Abstract

Background Transformer-based deep learning models have rapidly advanced and show promise for handling complex, multimodal healthcare data. Their application to disease prediction has expanded across clinical domains, yet questions remain about external validity, interpretability, and clinical readiness. Methods A systematic review was conducted following PRISMA 2020 guidelines. MEDLINE, Embase, Scopus, IEEE Xplore, ACM Digital Library, and arXiv were searched from inception to June 2025. Studies applying transformer-based architectures to human disease prediction were included. Data were extracted on model type, disease domain, data modality, performance metrics, validation strategy, interpretability, and implementation considerations. Risk of bias was assessed using an AI-adapted PROBAST framework, and certainty of evidence was evaluated using a modified GRADE approach. Results Forty-one studies published between 2020 and 2025 were included. Transformers were applied across neurodegenerative, cardiovascular, oncological, infectious, rare, and ophthalmic diseases, using imaging, electronic health records, clinical text, and multimodal data. Across domains, transformer-based models generally matched or outperformed convolutional and recurrent neural networks, particularly in multimodal and longitudinal tasks. However, most studies relied on internal validation, with limited external or prospective evaluation. Interpretability techniques were inconsistently applied and rarely assessed for clinical utility, while evidence of clinical implementation and regulatory readiness was minimal. Conclusion Transformer-based models demonstrate strong technical potential for disease prediction, especially when leveraging multimodal and sequential data. Nevertheless, their translation into routine healthcare remains constrained by limited external validation, uneven interpretability evaluation, and insufficient attention to equity, reproducibility, and regulatory considerations. Addressing these gaps is essential to support safe, effective, and equitable clinical adoption.