Reinforcement learning for personalized dialogue management

Floris Den Hengst, Mark Hoogendoorn, Frank Van Harmelen, Joost Bosman

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Language systems have been of great interest to the research community and have recently reached the mass market through various assistant platforms on the web. Reinforcement Learning methods that optimize dialogue policies have seen successes in past years and have recently been extended into methods that personalize the dialogue, e.g. take the personal context of users into account. These works, however, are limited to personalization to a single user with whom they require multiple interactions and do not generalize the usage of context across users. This work introduces a problem where a generalized usage of context is relevant and proposes two Reinforcement Learning (RL)-based approaches to this problem. The first approach uses a single learner and extends the traditional POMDP formulation of dialogue state with features that describe the user context. The second approach segments users by context and then employs a learner per context. We compare these approaches in a benchmark of existing non-RL and RL-based methods in three established and one novel application domain of financial product recommendation. We compare the influence of context and training experiences on performance and find that learning approaches generally outperform a handcrafted gold standard.

Original languageEnglish
Title of host publication2WI '19: IEEE/WIC/ACM International Conference on Web Intelligence
Subtitle of host publicationProceedings
EditorsPayam Barnaghi, Georg Gottlob, Yannis Manolopoulos, Theodoros Tzouramanis, Athena Vakali
PublisherAssociation for Computing Machinery, Inc
Pages59-67
Number of pages9
ISBN (Electronic)9781450369343
DOIs
Publication statusPublished - Oct 2019
Event19th IEEE/WIC/ACM International Conference on Web Intelligence, WI 2019 - Thessaloniki, Greece
Duration: 13 Oct 201917 Oct 2019

Conference

Conference19th IEEE/WIC/ACM International Conference on Web Intelligence, WI 2019
CountryGreece
CityThessaloniki
Period13/10/1917/10/19

Keywords

  • Adaptive Virtual Assistants
  • Dialogue Management
  • Personalization
  • Recommendation
  • Reinforcement Learning

Fingerprint Dive into the research topics of 'Reinforcement learning for personalized dialogue management'. Together they form a unique fingerprint.

  • Cite this

    Den Hengst, F., Hoogendoorn, M., Van Harmelen, F., & Bosman, J. (2019). Reinforcement learning for personalized dialogue management. In P. Barnaghi, G. Gottlob, Y. Manolopoulos, T. Tzouramanis, & A. Vakali (Eds.), 2WI '19: IEEE/WIC/ACM International Conference on Web Intelligence: Proceedings (pp. 59-67). Association for Computing Machinery, Inc. https://doi.org/10.1145/3350546.3352501