Cross-Domain Toxic Spans Detection

Stefan F. Schouten; Baran Barbarestani; Wondimagegnhue Tufa; Piek Vossen; Ilia Markov

Cross-Domain Toxic Spans Detection

Stefan F. Schouten
, Baran Barbarestani
, Wondimagegnhue Tufa
, Piek Vossen
, Ilia Markov

Research output: Working paper / Preprint › Preprint › Academic

36 Downloads (Pure)

Abstract

Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain.

Original language	English
Pages	1-17
Number of pages	17
Publication status	Published - 16 Jun 2023

Bibliographical note

NLDB 2023

Keywords

cs.CL
cs.LG

Access to Document

2306.09642v1Final published version, 202 KB

Persistent URL (handle)

Persistent link

Cross-Domain Toxic Spans Detection
Schouten, S. F., Barbarestani, B., Tufa, W., Vossen, P. & Markov, I., 2023, Natural Language Processing and Information Systems: 28th International Conference on Applications of Natural Language to Information Systems, NLDB 2023, Derby, UK, June 21–23, 2023, Proceedings. Métais, E., Meziane, F., Manning, W., Reiff-Marganiec, S. & Sugumaran, V. (eds.). Springer Science and Business Media Deutschland GmbH, p. 533-545 13 p. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); vol. 13913 LNCS).
Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Open Access
File
60 Downloads (Pure)

Cite this

@techreport{6606838d99a84b91b688eb5e4e740862,

title = "Cross-Domain Toxic Spans Detection",

abstract = "Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain.",

keywords = "cs.CL, cs.LG",

author = "Schouten, \{Stefan F.\} and Baran Barbarestani and Wondimagegnhue Tufa and Piek Vossen and Ilia Markov",

note = "NLDB 2023",

year = "2023",

month = jun,

day = "16",

language = "English",

pages = "1--17",

type = "WorkingPaper",

}

TY - UNPB

T1 - Cross-Domain Toxic Spans Detection

AU - Schouten, Stefan F.

AU - Barbarestani, Baran

AU - Tufa, Wondimagegnhue

AU - Vossen, Piek

AU - Markov, Ilia

N1 - NLDB 2023

PY - 2023/6/16

Y1 - 2023/6/16

N2 - Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain.

AB - Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain.

KW - cs.CL

KW - cs.LG

M3 - Preprint

SP - 1

EP - 17

BT - Cross-Domain Toxic Spans Detection

ER -