A realistic, multimodal virtual agent for the healthcare domain

Guido M. Linders; Julija Vaitonyte; Maryam Alimardani; Kiril O. Mitev; Max M. Louwerse

doi:10.1145/3514197.3551250

A realistic, multimodal virtual agent for the healthcare domain

Guido M. Linders, Julija Vaitonyte, Maryam Alimardani, Kiril O. Mitev, Max M. Louwerse

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

We introduce an interactive embodied conversational agent for deployment in the healthcare sector. The agent is operated by a software architecture that integrates speech recognition, dialog management, and speech synthesis, and is embodied by a virtual human face developed using photogrammetry techniques. These features together allow for real-time, face-to-face interactions with human users. Although the developed software architecture is domain-independent and highly customizable, the virtual agent will initially be applied to healtcare domain. Here we give an overview of the different components of the architecture.

Original language	English
Title of host publication	IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	9781450392488
DOIs	https://doi.org/10.1145/3514197.3551250
Publication status	Published - 6 Sept 2022
Externally published	Yes
Event	22nd ACM International Conference on Intelligent Virtual Agents, IVA 2022 - Faro, Portugal Duration: 6 Sept 2022 → 9 Sept 2022

Conference

Conference	22nd ACM International Conference on Intelligent Virtual Agents, IVA 2022
Country/Territory	Portugal
City	Faro
Period	6/09/22 → 9/09/22

Funding

This research has been funded by the grant No.: PROJ-007246 from the Operational Program South, the European Union, the Ministry of Economic Affairs, the Province of Noord-Brabant, and the municipality of Tilburg awarded to MML. Tilburg University (TiU), BlueTea, Noldus, Breda University of Applied Sciences (BUas), Royal Netherlands Aerospace Centre (NLR), and Fontys Applied University Eindhoven created the agent architecture. BUas created the photogrammetry pipeline, 3D face models, and integrated Microsoft Azure into the architecture. Spaarne Hospital together with TiU, NLR, and Noldus created bariatrics training data. TiU implemented dialog management. BlueTea and TiU implemented the virtual machine solution.

Funders	Funder number
European Commission
Ministerie van Economische Zaken

Access to Document

10.1145/3514197.3551250

Persistent URL (handle)

Persistent link

Cite this

@inproceedings{3bea55623ffe40a086dba780bd7a70a8,

title = "A realistic, multimodal virtual agent for the healthcare domain",

abstract = "We introduce an interactive embodied conversational agent for deployment in the healthcare sector. The agent is operated by a software architecture that integrates speech recognition, dialog management, and speech synthesis, and is embodied by a virtual human face developed using photogrammetry techniques. These features together allow for real-time, face-to-face interactions with human users. Although the developed software architecture is domain-independent and highly customizable, the virtual agent will initially be applied to healtcare domain. Here we give an overview of the different components of the architecture.",

author = "Linders, {Guido M.} and Julija Vaitonyte and Maryam Alimardani and Mitev, {Kiril O.} and Louwerse, {Max M.}",

year = "2022",

month = sep,

day = "6",

doi = "10.1145/3514197.3551250",

language = "English",

booktitle = "IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents",

publisher = "Association for Computing Machinery, Inc",

note = "22nd ACM International Conference on Intelligent Virtual Agents, IVA 2022 ; Conference date: 06-09-2022 Through 09-09-2022",

}

Linders, GM, Vaitonyte, J, Alimardani, M, Mitev, KO & Louwerse, MM 2022, A realistic, multimodal virtual agent for the healthcare domain. in IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents., 37, Association for Computing Machinery, Inc, 22nd ACM International Conference on Intelligent Virtual Agents, IVA 2022, Faro, Portugal, 6/09/22. https://doi.org/10.1145/3514197.3551250

A realistic, multimodal virtual agent for the healthcare domain. / Linders, Guido M.; Vaitonyte, Julija; Alimardani, Maryam et al.
IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents. Association for Computing Machinery, Inc, 2022. 37.

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - A realistic, multimodal virtual agent for the healthcare domain

AU - Linders, Guido M.

AU - Vaitonyte, Julija

AU - Alimardani, Maryam

AU - Mitev, Kiril O.

AU - Louwerse, Max M.

PY - 2022/9/6

Y1 - 2022/9/6

N2 - We introduce an interactive embodied conversational agent for deployment in the healthcare sector. The agent is operated by a software architecture that integrates speech recognition, dialog management, and speech synthesis, and is embodied by a virtual human face developed using photogrammetry techniques. These features together allow for real-time, face-to-face interactions with human users. Although the developed software architecture is domain-independent and highly customizable, the virtual agent will initially be applied to healtcare domain. Here we give an overview of the different components of the architecture.

AB - We introduce an interactive embodied conversational agent for deployment in the healthcare sector. The agent is operated by a software architecture that integrates speech recognition, dialog management, and speech synthesis, and is embodied by a virtual human face developed using photogrammetry techniques. These features together allow for real-time, face-to-face interactions with human users. Although the developed software architecture is domain-independent and highly customizable, the virtual agent will initially be applied to healtcare domain. Here we give an overview of the different components of the architecture.

UR - http://www.scopus.com/inward/record.url?scp=85138699488&partnerID=8YFLogxK

U2 - 10.1145/3514197.3551250

DO - 10.1145/3514197.3551250

M3 - Conference contribution

BT - IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents

PB - Association for Computing Machinery, Inc

T2 - 22nd ACM International Conference on Intelligent Virtual Agents, IVA 2022

Y2 - 6 September 2022 through 9 September 2022

ER -

A realistic, multimodal virtual agent for the healthcare domain

Abstract

Conference

Funding

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this