Good Bug Hunting: Inspiring and Motivating Software Testing Novices

Natalia Silvis-Cividjian, Marc Went, Robert Jansma, Viktor Bonev, Emil Apostolov

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review


Although software testing is crucial in safeguarding our rapidly digitizing society against accidental and malefic threats, students often perceive it as an unattractive activity when compared to design or coding. A more abundant exposure to bugs and their impact can improve the situation. To test this radically new hypothesis, we developed VU-BugZoo, an interactive software testing learning platform. Built upon a repository of fault-seeded executables, the platform engages students and teachers in a dynamic bug-hunting experience. What counts the most in this game is the quality of the test strategy and fault diagnosis, rather than just the quantity of devised test cases. In this paper, we describe the VU-BugZoo web-application, developed using a stack of Vue.js and Python Flask, running in Docker containers. Next, we report on its first deployment in the midst of the COVID-19 pandemic, for a remote assessment of 50 graduate CS students enrolled in an introductory software testing course. Students' evaluations show that for novices, a 100%-bug-guarantee, end-to-end testing challenge (1) makes learning more effective, (2) offers an adequate assessment instrument and (3) adds more excitement to an otherwise uninspiring chore. More research is needed to investigate the potential of our bug-centric approach to contribute to the hard mission of boosting students' interest in a testing career. We are currently extending the corpus of bugs with more fascinating and representative examples, and investigating code animation, automated grading and formative feedback generation. We intend to make the platform available for other software testing courses in academia and industry.

Original languageEnglish
Title of host publicationITiCSE '21
Subtitle of host publicationProceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1
PublisherAssociation for Computing Machinery
Number of pages7
ISBN (Electronic)9781450382144
Publication statusPublished - Jun 2021
Event26th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE 2021 - Virtual, Online, Germany
Duration: 26 Jun 20211 Jul 2021

Publication series

NameAnnual Conference on Innovation and Technology in Computer Science Education, ITiCSE
ISSN (Print)1942-647X


Conference26th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE 2021
CityVirtual, Online

Bibliographical note

Funding Information:
This project is funded by the NRO, The Netherlands Initiative for Education Research, as part of a Comenius Teaching Fellow grant. The authors would like to thank the VU Computer Science department and Stefan Schlobach in particular, for co-financing the project; Herbert Bos and Wan Fokkink for their useful suggestions and encouragements; finally, all of the CS students who participated in the software testing course, and in particular the 2020 cohort, who endured with stoicism our pilot experiment, and blessed us with their precious suggestions and encouragements.

Publisher Copyright:
© 2021 Owner/Author.

Copyright 2021 Elsevier B.V., All rights reserved.


  • COVID-19 pandemic
  • fault injection
  • remote assessment
  • software testing education


Dive into the research topics of 'Good Bug Hunting: Inspiring and Motivating Software Testing Novices'. Together they form a unique fingerprint.

Cite this