Personalization of health interventions using cluster-based reinforcement learning

Ali el Hassouni, Mark Hoogendoorn, Martijn van Otterlo, Eduardo Barbaro

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Research has shown that personalization of health interventions can contribute to an improved effectiveness. Reinforcement learning algorithms can be used to perform such tailoring. In this paper, we present a cluster-based reinforcement learning approach which learns optimal policies for groups of users. Such an approach can speed up the learning process while still giving a level of personalization. We apply both online and batch learning to learn policies over the clusters and introduce a publicly available simulator which we have developed to evaluate the approach. The results show batch learning significantly outperforms online learning. Furthermore, near-optimal clustering is found which proves to be beneficial in learning significantly better policies compared to learning per user and learning across all users.

Original languageEnglish
Title of host publicationPRIMA 2018 Principles and Practice of Multi-Agent Systems
Subtitle of host publication21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings
EditorsNir Oren, Yuko Sakurai, Itsuki Noda, Tran Cao Son, Tim Miller, Bastin Tony Savarimuthu
PublisherSpringer/Verlag
Pages467-475
Number of pages9
ISBN (Electronic)9783030030988
ISBN (Print)9783030030971
DOIs
Publication statusPublished - 2018
Event21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018 - Tokyo, Japan
Duration: 29 Oct 20182 Nov 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11224 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018
CountryJapan
CityTokyo
Period29/10/182/11/18

Fingerprint

Personalization
Reinforcement learning
Reinforcement Learning
Health
Learning algorithms
Simulators
Batch
Online Learning
Optimal Policy
Learning Process
Learning Algorithm
Simulator
Speedup
Learning
Clustering
Evaluate
Policy

Keywords

  • m-Health
  • Personalization
  • Reinforcement learning

Cite this

el Hassouni, A., Hoogendoorn, M., van Otterlo, M., & Barbaro, E. (2018). Personalization of health interventions using cluster-based reinforcement learning. In N. Oren, Y. Sakurai, I. Noda, T. Cao Son, T. Miller, & B. T. Savarimuthu (Eds.), PRIMA 2018 Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings (pp. 467-475). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11224 LNAI). Springer/Verlag. https://doi.org/10.1007/978-3-030-03098-8_31
el Hassouni, Ali ; Hoogendoorn, Mark ; van Otterlo, Martijn ; Barbaro, Eduardo. / Personalization of health interventions using cluster-based reinforcement learning. PRIMA 2018 Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. editor / Nir Oren ; Yuko Sakurai ; Itsuki Noda ; Tran Cao Son ; Tim Miller ; Bastin Tony Savarimuthu. Springer/Verlag, 2018. pp. 467-475 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{f12ccdc4a56e4753b988eb6a4102f0d3,
title = "Personalization of health interventions using cluster-based reinforcement learning",
abstract = "Research has shown that personalization of health interventions can contribute to an improved effectiveness. Reinforcement learning algorithms can be used to perform such tailoring. In this paper, we present a cluster-based reinforcement learning approach which learns optimal policies for groups of users. Such an approach can speed up the learning process while still giving a level of personalization. We apply both online and batch learning to learn policies over the clusters and introduce a publicly available simulator which we have developed to evaluate the approach. The results show batch learning significantly outperforms online learning. Furthermore, near-optimal clustering is found which proves to be beneficial in learning significantly better policies compared to learning per user and learning across all users.",
keywords = "m-Health, Personalization, Reinforcement learning",
author = "{el Hassouni}, Ali and Mark Hoogendoorn and {van Otterlo}, Martijn and Eduardo Barbaro",
year = "2018",
doi = "10.1007/978-3-030-03098-8_31",
language = "English",
isbn = "9783030030971",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer/Verlag",
pages = "467--475",
editor = "Nir Oren and Yuko Sakurai and Itsuki Noda and {Cao Son}, Tran and Tim Miller and Savarimuthu, {Bastin Tony}",
booktitle = "PRIMA 2018 Principles and Practice of Multi-Agent Systems",

}

el Hassouni, A, Hoogendoorn, M, van Otterlo, M & Barbaro, E 2018, Personalization of health interventions using cluster-based reinforcement learning. in N Oren, Y Sakurai, I Noda, T Cao Son, T Miller & BT Savarimuthu (eds), PRIMA 2018 Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11224 LNAI, Springer/Verlag, pp. 467-475, 21st International Conference on Principles and Practice of Multi-Agent Systems, PRIMA 2018, Tokyo, Japan, 29/10/18. https://doi.org/10.1007/978-3-030-03098-8_31

Personalization of health interventions using cluster-based reinforcement learning. / el Hassouni, Ali; Hoogendoorn, Mark; van Otterlo, Martijn; Barbaro, Eduardo.

PRIMA 2018 Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. ed. / Nir Oren; Yuko Sakurai; Itsuki Noda; Tran Cao Son; Tim Miller; Bastin Tony Savarimuthu. Springer/Verlag, 2018. p. 467-475 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11224 LNAI).

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Personalization of health interventions using cluster-based reinforcement learning

AU - el Hassouni, Ali

AU - Hoogendoorn, Mark

AU - van Otterlo, Martijn

AU - Barbaro, Eduardo

PY - 2018

Y1 - 2018

N2 - Research has shown that personalization of health interventions can contribute to an improved effectiveness. Reinforcement learning algorithms can be used to perform such tailoring. In this paper, we present a cluster-based reinforcement learning approach which learns optimal policies for groups of users. Such an approach can speed up the learning process while still giving a level of personalization. We apply both online and batch learning to learn policies over the clusters and introduce a publicly available simulator which we have developed to evaluate the approach. The results show batch learning significantly outperforms online learning. Furthermore, near-optimal clustering is found which proves to be beneficial in learning significantly better policies compared to learning per user and learning across all users.

AB - Research has shown that personalization of health interventions can contribute to an improved effectiveness. Reinforcement learning algorithms can be used to perform such tailoring. In this paper, we present a cluster-based reinforcement learning approach which learns optimal policies for groups of users. Such an approach can speed up the learning process while still giving a level of personalization. We apply both online and batch learning to learn policies over the clusters and introduce a publicly available simulator which we have developed to evaluate the approach. The results show batch learning significantly outperforms online learning. Furthermore, near-optimal clustering is found which proves to be beneficial in learning significantly better policies compared to learning per user and learning across all users.

KW - m-Health

KW - Personalization

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85056448814&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056448814&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-03098-8_31

DO - 10.1007/978-3-030-03098-8_31

M3 - Conference contribution

SN - 9783030030971

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 467

EP - 475

BT - PRIMA 2018 Principles and Practice of Multi-Agent Systems

A2 - Oren, Nir

A2 - Sakurai, Yuko

A2 - Noda, Itsuki

A2 - Cao Son, Tran

A2 - Miller, Tim

A2 - Savarimuthu, Bastin Tony

PB - Springer/Verlag

ER -

el Hassouni A, Hoogendoorn M, van Otterlo M, Barbaro E. Personalization of health interventions using cluster-based reinforcement learning. In Oren N, Sakurai Y, Noda I, Cao Son T, Miller T, Savarimuthu BT, editors, PRIMA 2018 Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings. Springer/Verlag. 2018. p. 467-475. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-03098-8_31