Deep-AquATTEN: a hybrid multi-module attention architecture for robust fish detection and classification in turbid underwater environments

Abstract

Efficient and accurate detection of fish in turbid underwater environments is crucial for aquaculture management, biomass estimation, and ecosystem monitoring. Traditional underwater analysis methods rely on handcrafted features and classical machine learning techniques; yet, they often fail to cope with the highly variable and degraded imaging conditions found in aquaculture cages. Their features lack adaptability to fluctuating lighting, backscatter effects, and rapid, overlapping movements of fish. Consequently, their performance deteriorates sharply under turbidity, color distortions, and spectral degradations, limiting the effectiveness of monitoring and automated decision-making. To address these limitations, this study introduces Deep-AquATTEN, a hybrid multi-module attention architecture derived from YOLOv11 and specifically tailored for challenging underwater conditions. The proposed pipeline incorporates several specialized modules to compensate for spectral and structural distortions. First, the Underwater Enhancement Feature Module (UEFM) is positioned between the backbone and the neck to improve spectral-spatial representations degraded by turbidity. Next, the Multi-Scale Attention Module (MSAM) is embedded into the neck after feature fusion to strengthen multi-scale contextual learning. Finally, the Dual-Branch Feature Refinement (DBFR) module refines semantic and geometric cues simultaneously before the detection head, enhancing localization and classification accuracy in complex environments. Unlike existing approaches that treat enhancement, attention, and detection separately, the proposed method introduces a tightly coupled pipeline that enables robust feature learning under severe underwater distortions. Experimental results demonstrate the effectiveness of Deep-AquATTEN, achieving Accuracy: 95.29%, mAP@0.5: 92.30%, and FPS: 63, confirming its suitability for real-time aquaculture monitoring in turbid underwater settings.