Abstract
In the context of the proliferation of multimodal hate speech related to the Russia-Ukraine conflict, we introduce a unified multimodal fusion system for detecting hate speech and its targets in text-embedded images. Our approach leverages the Twitter-based RoBERTa and Swin Transformer V2 models to encode textual and visual modalities, and employs the Multilayer Perceptron (MLP) fusion mechanism for classification. Our system achieved macro F1 scores of 87.27% for hate speech detection and 80.05% for hate speech target detection in the Multimodal Hate Speech Event Detection Challenge 2024, securing the 1st rank in both subtasks. We open-source the trained models at https://huggingface.co/Yestin-Wang.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text (CASE 2024) |
| Editors | Ali Hurriyetoglu, Hristo Tanev, Surendrabikram Thapa, Gokce Uludogan |
| Publisher | ACL Anthology |
| Pages | 73-78 |
| Number of pages | 6 |
| ISBN (Electronic) | 9798891760707 |
| Publication status | Published - 2024 |