Dynamic sign language recognition based on CBAM-enhanced CNN and Bi-LSTM networks

Abstract

Abstract This paper presents a dynamic sign language recognition approach that integrates temporal encoding, Convolutional Block Attention Module (CBAM) and Bidirectional Long Short-Term Memory (Bi-LSTM) network to improve recognition accuracy and contextual understanding of sign gestures. For preprocessing, the proposed system employs a temporal image encoding technique; adopted from existing research to transform video sequences into static representations that preserve temporal information. Within the proposed architecture, a CBAM layer is embedded in each convolutional block of a CNN to extract and refine the most relevant spatial features from the WLASL video dataset. The features are then processed by a Bi-LSTM network that captures bidirectional temporal dependencies for effective recognition of sign gestures. The proposed system achieved an overall accuracy of 91.8% and was also evaluated on both classification performance and computational efficiency metrics. The results demonstrate that the integration of temporal image encoding, attention-enhanced feature extraction, and bidirectional temporal modeling leads to robustness and significant improvements in recognition accuracy in American Sign Language (ASL) word-level or isolated signs. These findings validate the effectiveness of the proposed approach, its potential for recognition of dynamic signs that exhibit bidirectional temporal correlations and support its applicability to real-world sign language recognition tasks.