<scp>CNN</scp> ‐Based Accent Similarity Detection Using Masked Spectrogram Reconstruction

Abstract

ABSTRACT The increasing reliance on voice‐based applications underscores the importance of accurate accent recognition to improve automatic speech recognition (ASR), particularly for low‐resource non‐native accents such as Ghanaian English. These varieties remain underrepresented in existing datasets, resulting in lower recognition accuracy and reduced inclusivity. This study proposes a Convolutional Neural Network‐based Masked Spectrogram Reconstruction (CNN‐MSR) framework for accent classification and similarity assessment. The model applies random spectrogram masking to learn robust accent‐specific features and incorporates a Much Lower Frame Rate (mLFR) strategy to enhance computational efficiency. Experiments on AccentDB and Ghanaian English datasets demonstrate a significant improvement in classification performance, achieving 90.71% accuracy, an absolute gain of 8.83% over the baseline CNN (81.88%). The model also reduces word error rates to 0.00 for native accents and 19.91 for non‐native accents. Embedding visualizations (t‐SNE and UMAP) and cosine similarity analysis reveal apparent clustering among native accents and overlapping patterns among non‐native accents, reflecting phonetic diversity. These results demonstrate the model's potential to improve the robustness and inclusivity of ASR systems for underrepresented accent groups.