Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study

Myrthe Reuver; Suzan Verberne; Roser Morante; Antske Fokkens

Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study

Myrthe Reuver, Suzan Verberne, Roser Morante, Antske Fokkens

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

32 Downloads (Pure)

Abstract

Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics. We successfully reproduce state-of-the-art cross-topic stance detection work (Reimers et al., 2019), and systematically analyze its reproducibility. Our attention then turns to the cross-topic aspect of this work, and the specificity of topics in terms of vocabulary and socio-cultural context. We ask: To what extent is stance detection topic-independent and generalizable across topics? We compare the model’s performance on various unseen topics, and find topic (e.g. abortion, cloning), class (e.g. pro, con), and their interaction affecting the model’s performance. We conclude that investigating performance on different topics, and addressing topic-specific vocabulary and context, is a future avenue for cross-topic stance detection.

Original language	English
Title of host publication	Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]
Editors	Khalid Al-Khatib, Yufang Hou, Manfred Stede
Place of Publication	Punta Cana, the Dominican Republic
Publisher	Association for Computational Linguistics, ACL Anthology
Pages	46-56
Number of pages	10
ISBN (Electronic)	9781954085923
Publication status	Published - Nov 2021
Event	8th Workshop on Argument Mining, ArgMining 2021 - Virtual, Punta Cana, Dominican Republic Duration: 10 Nov 2021 → 11 Nov 2021

Conference

Conference	8th Workshop on Argument Mining, ArgMining 2021
Country/Territory	Dominican Republic
City	Virtual, Punta Cana
Period	10/11/21 → 11/11/21

Bibliographical note

Funding Information:
This research is funded through Open Competition Digitalization Humanities and Social Science grant nr 406.D1.19.073 awarded by the Netherlands Organization of Scientific Research (NWO). Our computing was done through SURF Research Cloud, a national supercomputer infrastructure in the Netherlands also funded by the NWO. We would like to thank dr. Nils Reimers for sending us their paper’s data. We would also like to thank the anonymous reviewers, whose very helpful comments improved the paper. All opinions and remaining errors are our own.

Publisher Copyright:
© 2021 Association for Computational Linguistics.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

2021.argmining-1.5_Is Stance Detection Topic-Independent and Cross-topic Generalizable_A Reproduction StudyFinal published version, 287 KBLicence: CC BY-NC-SA

https://aclanthology.org/2021.argmining-1.5

Persistent URL (handle)

Copy link address

Cite this

Reuver, M., Verberne, S., Morante, R., & Fokkens, A. (2021). Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study. In K. Al-Khatib, Y. Hou, & M. Stede (Eds.), Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021] (pp. 46-56). Association for Computational Linguistics, ACL Anthology. https://aclanthology.org/2021.argmining-1.5

Reuver, Myrthe ; Verberne, Suzan ; Morante, Roser et al. / Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study. Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]. editor / Khalid Al-Khatib ; Yufang Hou ; Manfred Stede. Punta Cana, the Dominican Republic : Association for Computational Linguistics, ACL Anthology, 2021. pp. 46-56

@inproceedings{53e34cc3e81444408c4316c140de8de8,

title = "Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study",

abstract = "Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics. We successfully reproduce state-of-the-art cross-topic stance detection work (Reimers et al., 2019), and systematically analyze its reproducibility. Our attention then turns to the cross-topic aspect of this work, and the specificity of topics in terms of vocabulary and socio-cultural context. We ask: To what extent is stance detection topic-independent and generalizable across topics? We compare the model{\textquoteright}s performance on various unseen topics, and find topic (e.g. abortion, cloning), class (e.g. pro, con), and their interaction affecting the model{\textquoteright}s performance. We conclude that investigating performance on different topics, and addressing topic-specific vocabulary and context, is a future avenue for cross-topic stance detection.",

author = "Myrthe Reuver and Suzan Verberne and Roser Morante and Antske Fokkens",

note = "Funding Information: This research is funded through Open Competition Digitalization Humanities and Social Science grant nr 406.D1.19.073 awarded by the Netherlands Organization of Scientific Research (NWO). Our computing was done through SURF Research Cloud, a national supercomputer infrastructure in the Netherlands also funded by the NWO. We would like to thank dr. Nils Reimers for sending us their paper{\textquoteright}s data. We would also like to thank the anonymous reviewers, whose very helpful comments improved the paper. All opinions and remaining errors are our own. Publisher Copyright: {\textcopyright} 2021 Association for Computational Linguistics.; 8th Workshop on Argument Mining, ArgMining 2021 ; Conference date: 10-11-2021 Through 11-11-2021",

year = "2021",

month = nov,

language = "English",

pages = "46--56",

editor = "Khalid Al-Khatib and Yufang Hou and Manfred Stede",

booktitle = "Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]",

publisher = "Association for Computational Linguistics, ACL Anthology",

}

Reuver, M, Verberne, S, Morante, R & Fokkens, A 2021, Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study. in K Al-Khatib, Y Hou & M Stede (eds), Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]. Association for Computational Linguistics, ACL Anthology, Punta Cana, the Dominican Republic, pp. 46-56, 8th Workshop on Argument Mining, ArgMining 2021, Virtual, Punta Cana, Dominican Republic, 10/11/21. <https://aclanthology.org/2021.argmining-1.5>

Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study. / Reuver, Myrthe; Verberne, Suzan; Morante, Roser et al.
Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]. ed. / Khalid Al-Khatib; Yufang Hou; Manfred Stede. Punta Cana, the Dominican Republic: Association for Computational Linguistics, ACL Anthology, 2021. p. 46-56.

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study

AU - Reuver, Myrthe

AU - Verberne, Suzan

AU - Morante, Roser

AU - Fokkens, Antske

N1 - Funding Information: This research is funded through Open Competition Digitalization Humanities and Social Science grant nr 406.D1.19.073 awarded by the Netherlands Organization of Scientific Research (NWO). Our computing was done through SURF Research Cloud, a national supercomputer infrastructure in the Netherlands also funded by the NWO. We would like to thank dr. Nils Reimers for sending us their paper’s data. We would also like to thank the anonymous reviewers, whose very helpful comments improved the paper. All opinions and remaining errors are our own. Publisher Copyright: © 2021 Association for Computational Linguistics.

PY - 2021/11

Y1 - 2021/11

N2 - Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics. We successfully reproduce state-of-the-art cross-topic stance detection work (Reimers et al., 2019), and systematically analyze its reproducibility. Our attention then turns to the cross-topic aspect of this work, and the specificity of topics in terms of vocabulary and socio-cultural context. We ask: To what extent is stance detection topic-independent and generalizable across topics? We compare the model’s performance on various unseen topics, and find topic (e.g. abortion, cloning), class (e.g. pro, con), and their interaction affecting the model’s performance. We conclude that investigating performance on different topics, and addressing topic-specific vocabulary and context, is a future avenue for cross-topic stance detection.

AB - Cross-topic stance detection is the task to automatically detect stances (pro, against, or neutral) on unseen topics. We successfully reproduce state-of-the-art cross-topic stance detection work (Reimers et al., 2019), and systematically analyze its reproducibility. Our attention then turns to the cross-topic aspect of this work, and the specificity of topics in terms of vocabulary and socio-cultural context. We ask: To what extent is stance detection topic-independent and generalizable across topics? We compare the model’s performance on various unseen topics, and find topic (e.g. abortion, cloning), class (e.g. pro, con), and their interaction affecting the model’s performance. We conclude that investigating performance on different topics, and addressing topic-specific vocabulary and context, is a future avenue for cross-topic stance detection.

UR - http://www.scopus.com/inward/record.url?scp=85127219245&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85127219245&partnerID=8YFLogxK

M3 - Conference contribution

SP - 46

EP - 56

BT - Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]

A2 - Al-Khatib, Khalid

A2 - Hou, Yufang

A2 - Stede, Manfred

PB - Association for Computational Linguistics, ACL Anthology

CY - Punta Cana, the Dominican Republic

T2 - 8th Workshop on Argument Mining, ArgMining 2021

Y2 - 10 November 2021 through 11 November 2021

ER -

Reuver M, Verberne S, Morante R, Fokkens A. Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study. In Al-Khatib K, Hou Y, Stede M, editors, Proceedings of the 8th Workshop on Argument Mining [ArgMining 2021]. Punta Cana, the Dominican Republic: Association for Computational Linguistics, ACL Anthology. 2021. p. 46-56 Epub 2021 Nov 10.

Is Stance Detection Topic-Independent and Cross-topic Generalizable? – A Reproduction Study

Abstract

Conference

Bibliographical note

UN SDGs

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this