Narrowing reinforcement learning: Overcoming the cold start problem for personalized health interventions

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Personalization of support in health and wellbeing settings is challenging. While personalization has shown to be highly beneficial to maximize the success of interventions, often only very limited experiences are available to personalize support strategies. Because of its focus on finding suitable actions/interventions that lead to long term rewards, reinforcement learning is very suitable for personalization but requires a substantial learning period. To overcome this so-called cold start problem, we propose a novel approach called narrowing reinforcement learning. The approach exploits experiences of the nearest neighbors around a user to generate a suitable policy, expressing which action to perform in what state. Using a narrowing function, the size of the neighborhood is reduced as more experiences are collected, allowing for the most personalized experience that is possible given the amount of collected experiences. An evaluation of the approach in a realistic simulator shows that it significantly outperforms the current state-of-the-art approaches for personalization in health and wellbeing using reinforcement learning.

LanguageEnglish
Title of host publicationPRIMA 2018: Principles and Practice of Multi-Agent Systems
Subtitle of host publication21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings
EditorsNir Oren, Yuko Sakurai, Itsuki Noda, Tran Cao Son, Tim Miller, Bastin Tony Savarimuthu
PublisherSpringer/Verlag
Chapter19
Pages312-327
Number of pages16
ISBN (Electronic)9783030030988
ISBN (Print)9783030030971
DOIs
StatePublished - 2018
Event21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018 - Tokyo, Japan
Duration: 29 Oct 20182 Nov 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11224 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018
CountryJapan
CityTokyo
Period29/10/182/11/18

Fingerprint

Reinforcement learning
Reinforcement Learning
Personalization
Health
Simulators
Reward
Nearest Neighbor
Simulator
Maximise
Experience
Evaluation
Term

Keywords

  • Health
  • Personalization
  • Reinforcement learning

Cite this

Tabatabaei, S. A., Hoogendoorn, M., & van Halteren, A. (2018). Narrowing reinforcement learning: Overcoming the cold start problem for personalized health interventions. In N. Oren, Y. Sakurai, I. Noda, T. Cao Son, T. Miller, & B. T. Savarimuthu (Eds.), PRIMA 2018: Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings (pp. 312-327). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11224 LNAI). Springer/Verlag. DOI: 10.1007/978-3-030-03098-8_19
Tabatabaei, Seyed Amin ; Hoogendoorn, Mark ; van Halteren, Aart. / Narrowing reinforcement learning : Overcoming the cold start problem for personalized health interventions. PRIMA 2018: Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. editor / Nir Oren ; Yuko Sakurai ; Itsuki Noda ; Tran Cao Son ; Tim Miller ; Bastin Tony Savarimuthu. Springer/Verlag, 2018. pp. 312-327 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{d05537128f6f42298f04d5697215ba40,
title = "Narrowing reinforcement learning: Overcoming the cold start problem for personalized health interventions",
abstract = "Personalization of support in health and wellbeing settings is challenging. While personalization has shown to be highly beneficial to maximize the success of interventions, often only very limited experiences are available to personalize support strategies. Because of its focus on finding suitable actions/interventions that lead to long term rewards, reinforcement learning is very suitable for personalization but requires a substantial learning period. To overcome this so-called cold start problem, we propose a novel approach called narrowing reinforcement learning. The approach exploits experiences of the nearest neighbors around a user to generate a suitable policy, expressing which action to perform in what state. Using a narrowing function, the size of the neighborhood is reduced as more experiences are collected, allowing for the most personalized experience that is possible given the amount of collected experiences. An evaluation of the approach in a realistic simulator shows that it significantly outperforms the current state-of-the-art approaches for personalization in health and wellbeing using reinforcement learning.",
keywords = "Health, Personalization, Reinforcement learning",
author = "Tabatabaei, {Seyed Amin} and Mark Hoogendoorn and {van Halteren}, Aart",
year = "2018",
doi = "10.1007/978-3-030-03098-8_19",
language = "English",
isbn = "9783030030971",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer/Verlag",
pages = "312--327",
editor = "Nir Oren and Yuko Sakurai and Itsuki Noda and {Cao Son}, Tran and Tim Miller and Savarimuthu, {Bastin Tony}",
booktitle = "PRIMA 2018: Principles and Practice of Multi-Agent Systems",

}

Tabatabaei, SA, Hoogendoorn, M & van Halteren, A 2018, Narrowing reinforcement learning: Overcoming the cold start problem for personalized health interventions. in N Oren, Y Sakurai, I Noda, T Cao Son, T Miller & BT Savarimuthu (eds), PRIMA 2018: Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11224 LNAI, Springer/Verlag, pp. 312-327, 21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018, Tokyo, Japan, 29/10/18. DOI: 10.1007/978-3-030-03098-8_19

Narrowing reinforcement learning : Overcoming the cold start problem for personalized health interventions. / Tabatabaei, Seyed Amin; Hoogendoorn, Mark; van Halteren, Aart.

PRIMA 2018: Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. ed. / Nir Oren; Yuko Sakurai; Itsuki Noda; Tran Cao Son; Tim Miller; Bastin Tony Savarimuthu. Springer/Verlag, 2018. p. 312-327 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11224 LNAI).

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Narrowing reinforcement learning

T2 - Overcoming the cold start problem for personalized health interventions

AU - Tabatabaei,Seyed Amin

AU - Hoogendoorn,Mark

AU - van Halteren,Aart

PY - 2018

Y1 - 2018

N2 - Personalization of support in health and wellbeing settings is challenging. While personalization has shown to be highly beneficial to maximize the success of interventions, often only very limited experiences are available to personalize support strategies. Because of its focus on finding suitable actions/interventions that lead to long term rewards, reinforcement learning is very suitable for personalization but requires a substantial learning period. To overcome this so-called cold start problem, we propose a novel approach called narrowing reinforcement learning. The approach exploits experiences of the nearest neighbors around a user to generate a suitable policy, expressing which action to perform in what state. Using a narrowing function, the size of the neighborhood is reduced as more experiences are collected, allowing for the most personalized experience that is possible given the amount of collected experiences. An evaluation of the approach in a realistic simulator shows that it significantly outperforms the current state-of-the-art approaches for personalization in health and wellbeing using reinforcement learning.

AB - Personalization of support in health and wellbeing settings is challenging. While personalization has shown to be highly beneficial to maximize the success of interventions, often only very limited experiences are available to personalize support strategies. Because of its focus on finding suitable actions/interventions that lead to long term rewards, reinforcement learning is very suitable for personalization but requires a substantial learning period. To overcome this so-called cold start problem, we propose a novel approach called narrowing reinforcement learning. The approach exploits experiences of the nearest neighbors around a user to generate a suitable policy, expressing which action to perform in what state. Using a narrowing function, the size of the neighborhood is reduced as more experiences are collected, allowing for the most personalized experience that is possible given the amount of collected experiences. An evaluation of the approach in a realistic simulator shows that it significantly outperforms the current state-of-the-art approaches for personalization in health and wellbeing using reinforcement learning.

KW - Health

KW - Personalization

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85056454682&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056454682&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-03098-8_19

DO - 10.1007/978-3-030-03098-8_19

M3 - Conference contribution

SN - 9783030030971

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 312

EP - 327

BT - PRIMA 2018: Principles and Practice of Multi-Agent Systems

PB - Springer/Verlag

ER -

Tabatabaei SA, Hoogendoorn M, van Halteren A. Narrowing reinforcement learning: Overcoming the cold start problem for personalized health interventions. In Oren N, Sakurai Y, Noda I, Cao Son T, Miller T, Savarimuthu BT, editors, PRIMA 2018: Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. Springer/Verlag. 2018. p. 312-327. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Available from, DOI: 10.1007/978-3-030-03098-8_19