Dual-attention fusion network: A transformer-enhanced hybrid architecture for robust medical image retrieval

Abstract

Medical image retrieval (MIR) requires feature representations that capture both fine-grained local patterns and global anatomical contexts. We propose a Dual-Attention Fusion Network (DAFN), a hybrid architecture that integrates a transformer-based global feature extractor with a ResNet-50 local feature extractor using an adaptive dual-attention fusion module. To reconcile the dimensional disparity between the two branches (512-dimensional transformer features and 2048-dimensional CNN features), learnable linear projection layers map both representations into a shared embedding space prior to fusion. A composite loss function combining triplet loss with a clinical regularization term based on anatomical landmark regression further enhanced the discriminative power and clinical consistency. Experiments on MedMNIST and LIDC-IDRI demonstrated that DAFN achieved a Precision@5 of 0.89, mAP of 0.81, and AUC of 0.94, outperforming state-of-the-art methods by 4–6%. Ablation studies confirm the contribution of each module, validating DAFN as a robust tool for clinical decision support.