Classification of Multi-Label imbalanced Arabic hate speech comments using a combination of ensemble transformer sub-models

Abstract

Offensive language that has permeated the internet is a dangerous substance distinguished by its strong negative connotation. Its spread via the internet has the potential to cause harmful consequences on both people and society. In this paper, we aim to detect hate speech; Hence, we used the corpora introduced in the OSACT5 shared task on Arabic offensive language and hate speech detection (Mubarak et al., 2022). The corpora contain over 12,000 annotated comments. We base our system on several transformer models. To enhance the classification, we combined the models output within an ensemble learning model. The obtained results were pertinent; However, since the studied corpus is imbalanced, we investigated then the effect of data augmentation on the classification of low represented data. Data augmentation helped to detect more hate speech instances, the low represented category, within the dataset. Furthermore, we fine-grain the classification, to distinguish between 7 classes, by implementing a set of models based on transformers, GANs and label lightening. The strength of GANs lies in helping to generate noisy vectors that are highly similar to the real data used for training. We used the characteristics of GANs to improve the distinction between various classes. Since the data are imbalanced, we proceeded to a custom submodels approach. The process seeks to balance fine-grained classification by merging low-represented data within one class and performing binary classification, the low against the high represented class, before classifying the remaining premerged classes separately. Our system based on transfer, ensemble learning and data augmentation outperformed the best ranked teams for both tasks. We have reached 84% F-measure in the binary classification of hate speech. The fine-grained classification enhanced with label lightening has helped to achieve an improvement of 32% F-measure.