TY - JOUR
T1 - Threats to validity in software engineering research
T2 - A critical reflection
AU - Verdecchia, Roberto
AU - Engström, Emelie
AU - Lago, Patricia
AU - Runeson, Per
AU - Song, Qunying
N1 - Publisher Copyright:
© 2023
PY - 2023/12
Y1 - 2023/12
N2 - Context: In the contemporary body of software engineering literature, some recurrent shortcomings characterize how threats to validity (TTV) are considered in studies. Objective: With this position paper, we aim to open a discourse on the current use of TTV sections. The goal of our position is to jointly reflect and systematically improve how we, as a research community, consider TTV in our studies. Methods: Based on our personal experience as researchers, authors, reviewers, and editors, we critically reflect on the treatment of TTV in current empirical software engineering literature. Results: We discuss the key shortcomings of TTV consideration, including the failure to acknowledge different types of validity categorizations and the tendency to treat threats just as an afterthought. For each identified problem, we propose a vision for an improved state, intending to catalyze thoughtful engagement and improvements the way our community addresses TTV. Conclusion: We posit there is an urgent need to reconsider how we approach, document, and evaluate TTV in software engineering research.
AB - Context: In the contemporary body of software engineering literature, some recurrent shortcomings characterize how threats to validity (TTV) are considered in studies. Objective: With this position paper, we aim to open a discourse on the current use of TTV sections. The goal of our position is to jointly reflect and systematically improve how we, as a research community, consider TTV in our studies. Methods: Based on our personal experience as researchers, authors, reviewers, and editors, we critically reflect on the treatment of TTV in current empirical software engineering literature. Results: We discuss the key shortcomings of TTV consideration, including the failure to acknowledge different types of validity categorizations and the tendency to treat threats just as an afterthought. For each identified problem, we propose a vision for an improved state, intending to catalyze thoughtful engagement and improvements the way our community addresses TTV. Conclusion: We posit there is an urgent need to reconsider how we approach, document, and evaluate TTV in software engineering research.
KW - Empirical research
KW - Software engineering
KW - Threats to validity
UR - http://www.scopus.com/inward/record.url?scp=85170643531&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85170643531&partnerID=8YFLogxK
U2 - 10.1016/j.infsof.2023.107329
DO - 10.1016/j.infsof.2023.107329
M3 - Article
AN - SCOPUS:85170643531
SN - 0950-5849
VL - 164
SP - 1
EP - 4
JO - Information and Software Technology
JF - Information and Software Technology
M1 - 107329
ER -