Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow

Nicholas Kozanidis; Roberto Verdecchia; Emitza Guzman Ortega

doi:10.1145/3544902.3546245

Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow

Nicholas Kozanidis, Roberto Verdecchia^*, Emitza Guzman Ortega

^*Corresponding author for this work

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Background: Numerous methodologies have been used to study technical debt. Among different data sources, Q&A sites provide an opportunity to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt through the lens of Stack Overflow.

Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions on Stack Overflow. In addition, we assess if identification strategies based on machine learning can be used to automatically identify and classify technical debt questions.

Method: We use combination of automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed both quantitatively and qualitatively to study (i) technical debt types, (ii) question length, (iii) perceived urgency,(iv) sentiment, and (v) emerging themes.Natural language processing and machine learning techniques are used to evaluate if technical debt questions can be identified and classified automatically.

Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment is mostly neutral. 29 recurrent themes emerge. Machine learning can be used to identify technical debt questions and binary urgency, but not debt types.

Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.

Original language	English
Title of host publication	ESEM '22
Subtitle of host publication	Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement
Editors	Fernanda Madeiral, Casper Lassenius, Casper Lassenius, Tayana Conte, Tomi Mannisto
Publisher	IEEE Computer Society
Pages	45-56
Number of pages	12
ISBN (Electronic)	9781450394277
DOIs	https://doi.org/10.1145/3544902.3546245
Publication status	Published - Sept 2022
Event	16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2022 - Helsinki, Finland Duration: 18 Sept 2022 → 23 Sept 2022

Publication series

Name	International Symposium on Empirical Software Engineering and Measurement
ISSN (Print)	1949-3770
ISSN (Electronic)	1949-3789

Conference

Conference	16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2022
Country/Territory	Finland
City	Helsinki
Period	18/09/22 → 23/09/22

Bibliographical note

Publisher Copyright:
© 2022 Association for Computing Machinery.

Keywords

Technical Debt
Stack Overflow
Machine Learning

Access to Document

10.1145/3544902.3546245

Persistent URL (handle)

Persistent link

Cite this

Kozanidis, N., Verdecchia, R., & Guzman Ortega, E. (2022). Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. In F. Madeiral, C. Lassenius, C. Lassenius, T. Conte, & T. Mannisto (Eds.), ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (pp. 45-56). (International Symposium on Empirical Software Engineering and Measurement). IEEE Computer Society. https://doi.org/10.1145/3544902.3546245

Kozanidis, Nicholas ; Verdecchia, Roberto ; Guzman Ortega, Emitza. / Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. editor / Fernanda Madeiral ; Casper Lassenius ; Casper Lassenius ; Tayana Conte ; Tomi Mannisto. IEEE Computer Society, 2022. pp. 45-56 (International Symposium on Empirical Software Engineering and Measurement).

@inproceedings{d7d30f05932f43a8bb56595b93fb7475,

title = "Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow",

abstract = "Background: Numerous methodologies have been used to study technical debt. Among different data sources, Q&A sites provide an opportunity to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt through the lens of Stack Overflow.Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions on Stack Overflow. In addition, we assess if identification strategies based on machine learning can be used to automatically identify and classify technical debt questions.Method: We use combination of automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed both quantitatively and qualitatively to study (i) technical debt types, (ii) question length, (iii) perceived urgency,(iv) sentiment, and (v) emerging themes.Natural language processing and machine learning techniques are used to evaluate if technical debt questions can be identified and classified automatically.Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment is mostly neutral. 29 recurrent themes emerge. Machine learning can be used to identify technical debt questions and binary urgency, but not debt types. Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.",

keywords = "Technical Debt, Stack Overflow, Machine Learning",

author = "Nicholas Kozanidis and Roberto Verdecchia and {Guzman Ortega}, Emitza",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computing Machinery.; 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2022 ; Conference date: 18-09-2022 Through 23-09-2022",

year = "2022",

month = sep,

doi = "10.1145/3544902.3546245",

language = "English",

series = "International Symposium on Empirical Software Engineering and Measurement",

publisher = "IEEE Computer Society",

pages = "45--56",

editor = "Fernanda Madeiral and Casper Lassenius and Casper Lassenius and Tayana Conte and Tomi Mannisto",

booktitle = "ESEM '22",

address = "United States",

}

Kozanidis, N, Verdecchia, R & Guzman Ortega, E 2022, Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. in F Madeiral, C Lassenius, C Lassenius, T Conte & T Mannisto (eds), ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, pp. 45-56, 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2022, Helsinki, Finland, 18/09/22. https://doi.org/10.1145/3544902.3546245

Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. / Kozanidis, Nicholas; Verdecchia, Roberto; Guzman Ortega, Emitza.
ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. ed. / Fernanda Madeiral; Casper Lassenius; Casper Lassenius; Tayana Conte; Tomi Mannisto. IEEE Computer Society, 2022. p. 45-56 (International Symposium on Empirical Software Engineering and Measurement).

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow

AU - Kozanidis, Nicholas

AU - Verdecchia, Roberto

AU - Guzman Ortega, Emitza

PY - 2022/9

Y1 - 2022/9

N2 - Background: Numerous methodologies have been used to study technical debt. Among different data sources, Q&A sites provide an opportunity to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt through the lens of Stack Overflow.Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions on Stack Overflow. In addition, we assess if identification strategies based on machine learning can be used to automatically identify and classify technical debt questions.Method: We use combination of automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed both quantitatively and qualitatively to study (i) technical debt types, (ii) question length, (iii) perceived urgency,(iv) sentiment, and (v) emerging themes.Natural language processing and machine learning techniques are used to evaluate if technical debt questions can be identified and classified automatically.Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment is mostly neutral. 29 recurrent themes emerge. Machine learning can be used to identify technical debt questions and binary urgency, but not debt types. Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.

AB - Background: Numerous methodologies have been used to study technical debt. Among different data sources, Q&A sites provide an opportunity to study how users reference and request support on technical debt. To date only few studies, focusing on narrow aspects, investigate technical debt through the lens of Stack Overflow.Aims: We aim at gaining an in-depth understanding on the characteristics of technical debt questions on Stack Overflow. In addition, we assess if identification strategies based on machine learning can be used to automatically identify and classify technical debt questions.Method: We use combination of automated and manual processes to identify technical debt questions on Stack Overflow. The final set of 415 questions is analyzed both quantitatively and qualitatively to study (i) technical debt types, (ii) question length, (iii) perceived urgency,(iv) sentiment, and (v) emerging themes.Natural language processing and machine learning techniques are used to evaluate if technical debt questions can be identified and classified automatically.Results: Architecture debt is the most recurring debt type, followed by code and design debt. Most questions display mild urgency, with frequency of higher urgency steadily declining as urgency rises. Question length varies across debt types. Sentiment is mostly neutral. 29 recurrent themes emerge. Machine learning can be used to identify technical debt questions and binary urgency, but not debt types. Conclusions: Different patterns emerge from the analysis of technical debt questions on Stack Overflow. The results provide further insights on the phenomenon, and support the adoption of a more comprehensive strategy to identify technical debt questions.

KW - Technical Debt

KW - Stack Overflow

KW - Machine Learning

UR - http://www.scopus.com/inward/record.url?scp=85139832258&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85139832258&partnerID=8YFLogxK

U2 - 10.1145/3544902.3546245

DO - 10.1145/3544902.3546245

M3 - Conference contribution

AN - SCOPUS:85139832258

T3 - International Symposium on Empirical Software Engineering and Measurement

SP - 45

EP - 56

BT - ESEM '22

A2 - Madeiral, Fernanda

A2 - Lassenius, Casper

A2 - Conte, Tayana

A2 - Mannisto, Tomi

PB - IEEE Computer Society

T2 - 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2022

Y2 - 18 September 2022 through 23 September 2022

ER -

Kozanidis N, Verdecchia R, Guzman Ortega E. Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow. In Madeiral F, Lassenius C, Lassenius C, Conte T, Mannisto T, editors, ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society. 2022. p. 45-56. (International Symposium on Empirical Software Engineering and Measurement). Epub 2022 Sept 19. doi: 10.1145/3544902.3546245

Asking about Technical Debt: Characteristics and Automatic Identification of Technical Debt Questions on Stack Overflow

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this