Human Recognition of Emotions Expressed by Human-Like Avatars on 2D Screens

Understanding emotions of others is important for effective interactions among people. Therefore, it is likely similarly important in applications where people interact with or via virtual humans. However, while some studies have examined the recognisability of expressions by virtual avatars, it is currently unclear how generalisable the findings are across technologies and designs. To empirically examine how well people (N = 100) recognise dynamic facial expressions for a set of 12 proposed avatars, the expressions are tested at high (75%) and low intensity (25%) in the context of 2D computer screens. Also, the effects of the self-reported age, gender, mood and ability to recognise emotions by the annotator are examined. Then, these findings are compared to emotion recognition literature for avatars and real people with a similar context. Finally automated recognition models are applied to test automated emotion detection, as well as to establish what facial action units may contribute to the found patterns in recognisability of the proposed avatars. We conclude that on average the emotional expressions of the proposed avatars are recognisable and confusion patterns resemble those of real people, where specific emotion pairs are more difficult to distinguish. Negative effects are found for male avatar gender and the age of the participant, while no effect is found for the self-reported mood or ability to recognise emotion. Moreover, no difference is found in the mean recognition-rate between human and avatar-based studies, yet the variation among avatar recognition studies is substantial.