A Comparative Study of Embedding Models for Offensive Language Detection in Nigeria
Offensive language detection has become a pivotal challenge in Natural Language Processing (NLP), particularly in the context of low-resource languages. Existing models, predominantly trained on high-resource languages such as English, often struggle to generalize across linguistically and culturally diverse contexts. This study addresses the limitation by developing and introducing novel datasets for the detection of offensive and hate speech in three major Nigerian languages, Hausa, Yoruba, and Igbo. Data was collected from Twitter and manually annotated using native speakers. The study leverages Afro-XLM-R embedding pre-trained on African language corpora combined with a Convolutional Neural Network (CNN) architecture for robust classification. To evaluate its effectiveness, the proposed hybrid model was benchmarked against three alternative CNN-based models utilizing FastText, word2Vec, and mBERT embedding. Experimental results demonstrate the superiority of the Afro-XLM-R + CNN approach in capturing nuanced linguistic features across the selected languages, highlighting its potential for scalable offensive language detection in low-resource settings.
Download Paper