Recognizing Perceived Interdependence in Face-to-Face Negotiations through Multimodal Analysis of Nonverbal Behavior

Bernd Dudzik, Simon Columbus, Tiffany Matej Hrkalovic, Daniel Balliet, Hayley Hung

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review


Enabling computer-based applications to display intelligent behavior in complex social settings requires them to relate to important aspects of how humans experience and understand such situations. One crucial driver of peoples' social behavior during an interaction is the interdependence they perceive, i.e., how the outcome of an interaction is determined by their own and others' actions. According to psychological studies, both the nonverbal behavior displayed by Motivated by this, we present a series of experiments to automatically recognize interdependence perceptions in dyadic face-to-face negotiations using these sources. Concretely, our approach draws on a combination of features describing individuals' Facial, Upper Body, and Vocal Behavior with state-of-the-art algorithms for multivariate time series classification. Our findings demonstrate that differences in some types of interdependence perceptions can be detected through the automatic analysis of nonverbal behaviors. We discuss implications for developing socially intelligent systems and opportunities for future research.

Original languageEnglish
Title of host publicationICMI 2021
Subtitle of host publicationProceedings of the 2021 International Conference on Multimodal Interaction
EditorsZakia Hammal, Carlos Busso
PublisherAssociation for Computing Machinery, Inc
Number of pages10
ISBN (Electronic)9781450384810
Publication statusPublished - Oct 2021
Event23rd ACM International Conference on Multimodal Interaction, ICMI 2021 - Virtual, Online, Canada
Duration: 18 Oct 202122 Oct 2021


Conference23rd ACM International Conference on Multimodal Interaction, ICMI 2021
CityVirtual, Online

Bibliographical note

Funding Information:
This research was (partially) funded by the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research,, grant number 024.004.022 and the MINGLE project number 639.022.606.

Funding Information:
Data collection was funded by an ERC Starting Grant (#635356) awarded to Daniel Balliet.

Publisher Copyright:
© 2021 Owner/Author.


  • Situation Perception
  • Social Signal Processing
  • User-Modeling


Dive into the research topics of 'Recognizing Perceived Interdependence in Face-to-Face Negotiations through Multimodal Analysis of Nonverbal Behavior'. Together they form a unique fingerprint.

Cite this