A Conversational Agent that Reacts to Vocal Signals

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Conversational agents are increasingly being used for training of social skills. One of their most important benefits is their ability to provide natural interaction with humans. This work proposes to extend conversational agents’ benefits for social skills training by analysing the emotion conveyed by the user’s speech. For that, we developed a new system that captures emotions from human voice and, combined with the context of a particular situation, uses this to influence the internal state of the agent and change its behaviour. An example of the system’s use is shown and its limitations and advantages are discussed, together with the internal workflow of the system.
Original languageEnglish
Title of host publicationIntelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers
EditorsRonald Poppe, John-Jules Meyer, Remco Veltkamp, Mehdi Dastani
PublisherSpringer Verlag
Pages285-291
Number of pages7
Volume178
ISBN (Print)978-3-319-49615-3
DOIs
Publication statusPublished - 2016
Event8th International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN 2016 - Utrecht, Netherlands
Duration: 28 Jun 201630 Jun 2016

Publication series

NameLecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
PublisherSpringer
Volume178
ISSN (Print)1867-8211

Conference

Conference8th International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN 2016
CountryNetherlands
CityUtrecht
Period28/06/1630/06/16

Keywords

  • Emotions
  • Social skills training
  • Speech analysis
  • Virtual agents
  • Vocal signals

Cite this

Formolo, D., & Bosse, T. (2016). A Conversational Agent that Reacts to Vocal Signals. In R. Poppe, J-J. Meyer, R. Veltkamp, & M. Dastani (Eds.), Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers (Vol. 178, pp. 285-291). (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Vol. 178). Springer Verlag. https://doi.org/10.1007/978-3-319-49616-0
Formolo, D. ; Bosse, T. / A Conversational Agent that Reacts to Vocal Signals. Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers. editor / Ronald Poppe ; John-Jules Meyer ; Remco Veltkamp ; Mehdi Dastani. Vol. 178 Springer Verlag, 2016. pp. 285-291 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering).
@inproceedings{f6e2fef7f5544c32b03a2eaae884f745,
title = "A Conversational Agent that Reacts to Vocal Signals",
abstract = "Conversational agents are increasingly being used for training of social skills. One of their most important benefits is their ability to provide natural interaction with humans. This work proposes to extend conversational agents’ benefits for social skills training by analysing the emotion conveyed by the user’s speech. For that, we developed a new system that captures emotions from human voice and, combined with the context of a particular situation, uses this to influence the internal state of the agent and change its behaviour. An example of the system’s use is shown and its limitations and advantages are discussed, together with the internal workflow of the system.",
keywords = "Emotions, Social skills training, Speech analysis, Virtual agents, Vocal signals",
author = "D. Formolo and T. Bosse",
year = "2016",
doi = "10.1007/978-3-319-49616-0",
language = "English",
isbn = "978-3-319-49615-3",
volume = "178",
series = "Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering",
publisher = "Springer Verlag",
pages = "285--291",
editor = "Ronald Poppe and John-Jules Meyer and Remco Veltkamp and Mehdi Dastani",
booktitle = "Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers",

}

Formolo, D & Bosse, T 2016, A Conversational Agent that Reacts to Vocal Signals. in R Poppe, J-J Meyer, R Veltkamp & M Dastani (eds), Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers. vol. 178, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 178, Springer Verlag, pp. 285-291, 8th International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN 2016, Utrecht, Netherlands, 28/06/16. https://doi.org/10.1007/978-3-319-49616-0

A Conversational Agent that Reacts to Vocal Signals. / Formolo, D.; Bosse, T.

Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers. ed. / Ronald Poppe; John-Jules Meyer; Remco Veltkamp; Mehdi Dastani. Vol. 178 Springer Verlag, 2016. p. 285-291 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Vol. 178).

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - A Conversational Agent that Reacts to Vocal Signals

AU - Formolo, D.

AU - Bosse, T.

PY - 2016

Y1 - 2016

N2 - Conversational agents are increasingly being used for training of social skills. One of their most important benefits is their ability to provide natural interaction with humans. This work proposes to extend conversational agents’ benefits for social skills training by analysing the emotion conveyed by the user’s speech. For that, we developed a new system that captures emotions from human voice and, combined with the context of a particular situation, uses this to influence the internal state of the agent and change its behaviour. An example of the system’s use is shown and its limitations and advantages are discussed, together with the internal workflow of the system.

AB - Conversational agents are increasingly being used for training of social skills. One of their most important benefits is their ability to provide natural interaction with humans. This work proposes to extend conversational agents’ benefits for social skills training by analysing the emotion conveyed by the user’s speech. For that, we developed a new system that captures emotions from human voice and, combined with the context of a particular situation, uses this to influence the internal state of the agent and change its behaviour. An example of the system’s use is shown and its limitations and advantages are discussed, together with the internal workflow of the system.

KW - Emotions

KW - Social skills training

KW - Speech analysis

KW - Virtual agents

KW - Vocal signals

UR - http://www.scopus.com/inward/record.url?scp=85000785444&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85000785444&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-49616-0

DO - 10.1007/978-3-319-49616-0

M3 - Conference contribution

SN - 978-3-319-49615-3

VL - 178

T3 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

SP - 285

EP - 291

BT - Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers

A2 - Poppe, Ronald

A2 - Meyer, John-Jules

A2 - Veltkamp, Remco

A2 - Dastani, Mehdi

PB - Springer Verlag

ER -

Formolo D, Bosse T. A Conversational Agent that Reacts to Vocal Signals. In Poppe R, Meyer J-J, Veltkamp R, Dastani M, editors, Intelligent Technologies for Interactive Entertainment - 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016, Revised Selected Papers. Vol. 178. Springer Verlag. 2016. p. 285-291. (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering). https://doi.org/10.1007/978-3-319-49616-0