Indeterminacy in Affective Computing: Considering Meaning and Context in Data Collection Practices

Automatic Affect Prediction (AAP) uses computational analysis of input data such as text, speech, images, and physiological signals to predict various affective phenomena (e.g., emotions or moods). These models are typically constructed using supervised machine-learning algorithms, which rely heavily on labeled training datasets. In this position paper, we posit that all AAP training data are derived from human Affective Interpretation Processes (AIPs resulting in a form of Affective Meaning. Research on human affect indicates a form of complexity that is fundamental to such meaning: it can possess what we refer to here broadly as Qualities of Indeterminacy (QIs)-encompassing Subjectivity (meaning depends on who is making the interpretation), Uncertainty (there is a lack of confidence regarding its correctness), Ambiguity (meaning can contain mutually exclusive concepts) and Vagueness (meaning can be situated at different levels in a nested hierarchy). Failing to appropriately consider QIs in AAP leads to results incapable of meaningful and reliable predictions in real-world settings. Based on this premise, our core argument is that a crucial step in adequately addressing indeterminacy in AAP is the development of data collection practices for modeling corpora that involve the systematic consideration of 1) a relevant set of QIs, and 2) context for the associated interpretation processes. To this end, we are 1) outlining a conceptual model of AIPs and the QIs associated with the meaning these produce, in addition to a conceptual structure of relevant context, supporting understanding of its role. Finally, we use our framework for 2) discussing examples of context-sensitivity-related challenges for addressing QIs in data collection setups. We believe our efforts can stimulate a structured discussion of both the role of aspects of indeterminacy and context in research on AAP, informing the development of better practices for data collection and analysis.