A Robust Dot-focused Classification Approach to Convolutional Braille Recognition

Abstract

The effect of imbalanced data on the optical character recognition of Braille text is investigated by applying two techniques to a set of convolutional neural network image classification models. A multilabel classification framework is applied to identify the combination of Braille dots present in a character sample. This approach is compared to the multiclass classification framework prevalent in the literature, which directly identifies each sample as one of 64 possible Braille characters. Furthermore, data resampling methods are applied to investigate the impact of class imbalance on the multilabel and multiclass modelling approaches, respectively. The multilabel models are shown to achieve statistically significantly better performance than multiclass models, across different data resampling strategies. This includes better generalisation to out of distribution testing data from different Braille language codes, as well as robust performance under experimental image augmentation conditions. Furthermore, while multiclass models achieve better performance when trained on resampled data compared to training without resampling, this performance increase fails to rival the performance of the multilabel classification models across metrics and resampling strategies.