International Conference on Computer and Knowledge Engineering

Home / 15th International Conference on Computer and Knowledge Engineering

Enhanced Hate Speech Detection Using Focal Loss and Multi-Head Attention for Imbalanced Social Media Text

Authors :

Ali Rezazadeh¹ Hadi Shahriar Shahhoseini²

1- Iran University of Science and Technology 2- Iran University of Science and Technology

Keywords :

Hate speech،Cyberbullying،NLP،Natural Language Processing،Deep Learning،Transformer،Attention

Abstract :

Hate speech detection on social media platforms faces significant challenges due to severe class imbalance, where minority classes such as neutral content are substantially underrepresented compared to hate speech and offensive language categories. Traditional deep learning approaches often achieve high overall accuracy by favoring majority classes while failing to adequately detect minority class instances, leading to biased classification systems. This paper presents an enhanced deep learning framework that addresses class imbalance through multiple complementary strategies. Our approach extends DistilBERT with an additional multi-head attention mechanism that refines transformer representations for task-specific semantic understanding. We introduce class-specific processing branches that enable specialized feature learning for each content category, coupled with a comprehensive set of 20 linguistic features capturing domain-specific patterns. To handle extreme class imbalance, we implement an advanced focal loss function with dynamic class weighting and label smoothing, combined with intelligent hybrid sampling strategies and minority class boosting mechanisms. Experimental validation on the Davidson hate speech dataset demonstrates significant improvements over state-of-the-art methods, achieving macro-averaged F1-scores of 91.57\% and weighted-averaged F1-scores of 93.73%. Our approach particularly excels in minority class detection while maintaining robust performance across all categories, with individual class F1-scores ranging from 88.36% to 95.55%. The proposed framework provides a comprehensive solution for imbalanced hate speech classification, combining architectural innovations with advanced loss functions to achieve balanced and effective content moderation capabilities.