On Using Physiological Sensors and AI to Monitor Emotions in a Bug-Hunting Game

Although software testing is key to a safe society, the process itself is often perceived by students as boring and stressful. Therefore, only few consider a career in testing. The adverse effect is sub-optimally tested code, with dangerous bugs left undetected. A better understanding of what testers "feel"when learning the skill in class can remedy this situation, by means of personalized, motivating bio-feedback. In order to test our hypothesis, we propose an innovative approach that uses physiological wearable sensors (cardiac activity, respiration, and skin conductance) to monitor in real-time the affective state of testers engaged in a bug-hunting game. This is a work in progress. We present the envisioned methodology and the results of two feasibility experiments. The first experiment created a training dataset, by recording bio-signals and self-reports from eleven participants involved in a mood-induction session followed by a bug-hunting task. The second experiment showed that it is possible to use deep-learning to recognize emotions from a large set of labelled multimodal (ECG, EDA and ICG) physiological data. The classification accuracy using a binary (positive-negative) emotions model was 85%, higher than the accuracy obtained using a four-emotions (anxious, down, enthusiastic and relaxed) model (57%). Future work includes optimizing the sensory system, improving the accuracy of automated emotions recognition, increasing the validity of ground-truth emotions labelling, and investigating ways to provide individualized and formative (instead of summative) bio-feedback. The proposed approach can contribute to a more sentiment-aware education, and a more objective evaluation of the effect of teaching interventions.