Aggression recognition using overlapping speech

Automatic recognition of negative affect and aggression is key in many safety critical domains such as surveillance and health care. In this paper we explore the potential of overlapping speech for predicting aggression levels. As a first step we consider 3 categories of overlapping speech based on literature. Having an annotation of these overlap categories, we examine whether overlapping speech is a good feature for predicting aggression by using it in classification together with a set of acoustic features typically used for this purpose. Next, we explore if this fine categorization of overlap is necessary in predicting aggression levels or a more coarse representation is sufficient. Finally, we check the additive values of automatically predicted overlapping speech for aggression recognition. The experiments are performed on a dataset of dyadic interactions between professional aggression training actors (actors) and naive participants (students) interacting freely based on short role descriptions. Our findings show that overlapping speech is a key feature for predicting aggression levels, that discriminating only severe cases of overlap is a sufficient feature and that automatically predicted overlap is improving aggression recognition as well.