What Makes a Good Bug Report?

T. Zimmermann, R. Premraj, N. Bettenburg, S. Just, A. Schröter, C. Weiss

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

In software development, bug reports provide crucial information to developers. However, these reports widely differ in their quality. We conducted a survey among developers and users of APACHE, ECLIPSE, and MOZILLA to find out what makes a good bug report. The analysis of the 466 responses revealed an information mismatch between what developers need and what users supply. Most developers consider steps to reproduce, stack traces, and test cases as helpful, which are, at the same time, most difficult to provide for users. Such insight is helpful for designing new bug tracking tools that guide users at collecting and providing more helpful information. Our CUEZILLA prototype is such a tool and measures the quality of new bug reports; it also recommends which elements should be added to improve the quality. We trained CUEZILLA on a sample of 289 bug reports, rated by developers as part of the survey. The participants of our survey also provided 175 comments on hurdles in reporting and resolving bugs. Based on these comments, we discuss several recommendations for better bug tracking systems, which should focus on engaging bug reporters, better tool support, and improved handling of bug duplicates. © 2010 IEEE.
Original languageEnglish
Pages (from-to)618-643
Number of pages25
JournalIEEE Transactions on Software Engineering
Volume36
Issue number5
DOIs
Publication statusPublished - 2010

Fingerprint

Software engineering

Cite this

Zimmermann, T., Premraj, R., Bettenburg, N., Just, S., Schröter, A., & Weiss, C. (2010). What Makes a Good Bug Report? IEEE Transactions on Software Engineering, 36(5), 618-643. https://doi.org/10.1109/TSE.2010.63
Zimmermann, T. ; Premraj, R. ; Bettenburg, N. ; Just, S. ; Schröter, A. ; Weiss, C. / What Makes a Good Bug Report?. In: IEEE Transactions on Software Engineering. 2010 ; Vol. 36, No. 5. pp. 618-643.
@article{7bb08ec58a66440caa08a1c615f2469d,
title = "What Makes a Good Bug Report?",
abstract = "In software development, bug reports provide crucial information to developers. However, these reports widely differ in their quality. We conducted a survey among developers and users of APACHE, ECLIPSE, and MOZILLA to find out what makes a good bug report. The analysis of the 466 responses revealed an information mismatch between what developers need and what users supply. Most developers consider steps to reproduce, stack traces, and test cases as helpful, which are, at the same time, most difficult to provide for users. Such insight is helpful for designing new bug tracking tools that guide users at collecting and providing more helpful information. Our CUEZILLA prototype is such a tool and measures the quality of new bug reports; it also recommends which elements should be added to improve the quality. We trained CUEZILLA on a sample of 289 bug reports, rated by developers as part of the survey. The participants of our survey also provided 175 comments on hurdles in reporting and resolving bugs. Based on these comments, we discuss several recommendations for better bug tracking systems, which should focus on engaging bug reporters, better tool support, and improved handling of bug duplicates. {\circledC} 2010 IEEE.",
author = "T. Zimmermann and R. Premraj and N. Bettenburg and S. Just and A. Schr{\"o}ter and C. Weiss",
year = "2010",
doi = "10.1109/TSE.2010.63",
language = "English",
volume = "36",
pages = "618--643",
journal = "IEEE Transactions on Software Engineering",
issn = "0098-5589",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

Zimmermann, T, Premraj, R, Bettenburg, N, Just, S, Schröter, A & Weiss, C 2010, 'What Makes a Good Bug Report?' IEEE Transactions on Software Engineering, vol. 36, no. 5, pp. 618-643. https://doi.org/10.1109/TSE.2010.63

What Makes a Good Bug Report? / Zimmermann, T.; Premraj, R.; Bettenburg, N.; Just, S.; Schröter, A.; Weiss, C.

In: IEEE Transactions on Software Engineering, Vol. 36, No. 5, 2010, p. 618-643.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - What Makes a Good Bug Report?

AU - Zimmermann, T.

AU - Premraj, R.

AU - Bettenburg, N.

AU - Just, S.

AU - Schröter, A.

AU - Weiss, C.

PY - 2010

Y1 - 2010

N2 - In software development, bug reports provide crucial information to developers. However, these reports widely differ in their quality. We conducted a survey among developers and users of APACHE, ECLIPSE, and MOZILLA to find out what makes a good bug report. The analysis of the 466 responses revealed an information mismatch between what developers need and what users supply. Most developers consider steps to reproduce, stack traces, and test cases as helpful, which are, at the same time, most difficult to provide for users. Such insight is helpful for designing new bug tracking tools that guide users at collecting and providing more helpful information. Our CUEZILLA prototype is such a tool and measures the quality of new bug reports; it also recommends which elements should be added to improve the quality. We trained CUEZILLA on a sample of 289 bug reports, rated by developers as part of the survey. The participants of our survey also provided 175 comments on hurdles in reporting and resolving bugs. Based on these comments, we discuss several recommendations for better bug tracking systems, which should focus on engaging bug reporters, better tool support, and improved handling of bug duplicates. © 2010 IEEE.

AB - In software development, bug reports provide crucial information to developers. However, these reports widely differ in their quality. We conducted a survey among developers and users of APACHE, ECLIPSE, and MOZILLA to find out what makes a good bug report. The analysis of the 466 responses revealed an information mismatch between what developers need and what users supply. Most developers consider steps to reproduce, stack traces, and test cases as helpful, which are, at the same time, most difficult to provide for users. Such insight is helpful for designing new bug tracking tools that guide users at collecting and providing more helpful information. Our CUEZILLA prototype is such a tool and measures the quality of new bug reports; it also recommends which elements should be added to improve the quality. We trained CUEZILLA on a sample of 289 bug reports, rated by developers as part of the survey. The participants of our survey also provided 175 comments on hurdles in reporting and resolving bugs. Based on these comments, we discuss several recommendations for better bug tracking systems, which should focus on engaging bug reporters, better tool support, and improved handling of bug duplicates. © 2010 IEEE.

U2 - 10.1109/TSE.2010.63

DO - 10.1109/TSE.2010.63

M3 - Article

VL - 36

SP - 618

EP - 643

JO - IEEE Transactions on Software Engineering

JF - IEEE Transactions on Software Engineering

SN - 0098-5589

IS - 5

ER -

Zimmermann T, Premraj R, Bettenburg N, Just S, Schröter A, Weiss C. What Makes a Good Bug Report? IEEE Transactions on Software Engineering. 2010;36(5):618-643. https://doi.org/10.1109/TSE.2010.63