Developing an Amh3MD as a multimodal dataset for the detection of information disorders

Abstract

Nowadays, the rapid spread of misinformation, disinformation, and mal-information on social media leads to a serious societal risk, particularly in low-resourced language communities where benchmark datasets are scarce. Most publicly available multimodal benchmark datasets are dominated by high-resource language and binary classification tasks, limiting their applicability to linguistically and culturally diverse contexts. To address this gap, this study developed Amh3MD, the first publicly available multimodal benchmark dataset for information disorder detection in Amharic language. A total of 8053 annotated multimodal samples were collected from prominent national and regional media outlets, as well as from public figures and activists with over 10,000 followers on social media, comprising texts, images, and memes. The contents are classified into four classes: misinformation, disinformation, mal-information, and normal, following intent and context-based guidelines. To ensure data quality, the annotation process involved three domain experts, achieving substantial inter-annotator agreement with an overall agreement of 88.78% and a Cohen’s Kappa of 0.67. In addition, this study proposed a multimodal deep learning model that incorporates transformer-based textual representations and visual feature extraction models with feature-level fusion strategies, providing a foundation for future model implementation, evaluation, and application to multilingual languages.