Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Personalisation has become omnipresent in society. For the domain of health and wellbeing such personalisation can contribute to better interventions and improved health states of users. In order for personalisation to be effective in this domain, it needs to be performed quickly and with minimal impact on the users. Reinforcement learning is one of the techniques that can be used to establish such personalisation, but it is not known to be very fast at learning. Cluster-based reinforcement learning has been proposed to improve the learning speed. Here, users who show similar behaviour are clustered and one policy is learned for each individual cluster. An important factor in this effort is the method used for clustering, which has the potential to influence the benefit of such an approach. In this paper, we propose three distance metrics based on the state of the users (Euclidean distance, Dynamic Time Warping, and high-level features) and apply different clustering techniques given these distance metrics to study their impact on the overall performance. We evaluate the different methods in a simulator with users spawned from very distinct user profiles as well as overlapping user profiles. The results show that clustering configurations using high-level features significantly outperform regular reinforcement learning without clustering (which either learn one policy for all or one policy per individual).

Original languageEnglish
Title of host publicationProceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018
EditorsSuresh Sundaram
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages813-820
Number of pages8
ISBN (Electronic)9781538692769
DOIs
Publication statusPublished - 28 Jan 2019
Event8th IEEE Symposium Series on Computational Intelligence, SSCI 2018 - Bangalore, India
Duration: 18 Nov 201821 Nov 2018

Conference

Conference8th IEEE Symposium Series on Computational Intelligence, SSCI 2018
CountryIndia
CityBangalore
Period18/11/1821/11/18

Fingerprint

Personalization
Reinforcement learning
Reinforcement Learning
Health
Clustering
User Profile
Distance Metric
Dynamic Time Warping
Simulators
Euclidean Distance
Overlapping
Simulator
Distinct
Configuration
Evaluate
Policy
Learning

Keywords

  • Health care
  • Personalization
  • Reinforcement Learning

Cite this

Grua, E. M., & Hoogendoorn, M. (2019). Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing. In S. Sundaram (Ed.), Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018 (pp. 813-820). [8628621] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SSCI.2018.8628621
Grua, Eoin Martino ; Hoogendoorn, Mark. / Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing. Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018. editor / Suresh Sundaram. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 813-820
@inproceedings{c309c9774ac6478bbf2eec1457866e2e,
title = "Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing",
abstract = "Personalisation has become omnipresent in society. For the domain of health and wellbeing such personalisation can contribute to better interventions and improved health states of users. In order for personalisation to be effective in this domain, it needs to be performed quickly and with minimal impact on the users. Reinforcement learning is one of the techniques that can be used to establish such personalisation, but it is not known to be very fast at learning. Cluster-based reinforcement learning has been proposed to improve the learning speed. Here, users who show similar behaviour are clustered and one policy is learned for each individual cluster. An important factor in this effort is the method used for clustering, which has the potential to influence the benefit of such an approach. In this paper, we propose three distance metrics based on the state of the users (Euclidean distance, Dynamic Time Warping, and high-level features) and apply different clustering techniques given these distance metrics to study their impact on the overall performance. We evaluate the different methods in a simulator with users spawned from very distinct user profiles as well as overlapping user profiles. The results show that clustering configurations using high-level features significantly outperform regular reinforcement learning without clustering (which either learn one policy for all or one policy per individual).",
keywords = "Health care, Personalization, Reinforcement Learning",
author = "Grua, {Eoin Martino} and Mark Hoogendoorn",
year = "2019",
month = "1",
day = "28",
doi = "10.1109/SSCI.2018.8628621",
language = "English",
pages = "813--820",
editor = "Suresh Sundaram",
booktitle = "Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

Grua, EM & Hoogendoorn, M 2019, Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing. in S Sundaram (ed.), Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018., 8628621, Institute of Electrical and Electronics Engineers Inc., pp. 813-820, 8th IEEE Symposium Series on Computational Intelligence, SSCI 2018, Bangalore, India, 18/11/18. https://doi.org/10.1109/SSCI.2018.8628621

Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing. / Grua, Eoin Martino; Hoogendoorn, Mark.

Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018. ed. / Suresh Sundaram. Institute of Electrical and Electronics Engineers Inc., 2019. p. 813-820 8628621.

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing

AU - Grua, Eoin Martino

AU - Hoogendoorn, Mark

PY - 2019/1/28

Y1 - 2019/1/28

N2 - Personalisation has become omnipresent in society. For the domain of health and wellbeing such personalisation can contribute to better interventions and improved health states of users. In order for personalisation to be effective in this domain, it needs to be performed quickly and with minimal impact on the users. Reinforcement learning is one of the techniques that can be used to establish such personalisation, but it is not known to be very fast at learning. Cluster-based reinforcement learning has been proposed to improve the learning speed. Here, users who show similar behaviour are clustered and one policy is learned for each individual cluster. An important factor in this effort is the method used for clustering, which has the potential to influence the benefit of such an approach. In this paper, we propose three distance metrics based on the state of the users (Euclidean distance, Dynamic Time Warping, and high-level features) and apply different clustering techniques given these distance metrics to study their impact on the overall performance. We evaluate the different methods in a simulator with users spawned from very distinct user profiles as well as overlapping user profiles. The results show that clustering configurations using high-level features significantly outperform regular reinforcement learning without clustering (which either learn one policy for all or one policy per individual).

AB - Personalisation has become omnipresent in society. For the domain of health and wellbeing such personalisation can contribute to better interventions and improved health states of users. In order for personalisation to be effective in this domain, it needs to be performed quickly and with minimal impact on the users. Reinforcement learning is one of the techniques that can be used to establish such personalisation, but it is not known to be very fast at learning. Cluster-based reinforcement learning has been proposed to improve the learning speed. Here, users who show similar behaviour are clustered and one policy is learned for each individual cluster. An important factor in this effort is the method used for clustering, which has the potential to influence the benefit of such an approach. In this paper, we propose three distance metrics based on the state of the users (Euclidean distance, Dynamic Time Warping, and high-level features) and apply different clustering techniques given these distance metrics to study their impact on the overall performance. We evaluate the different methods in a simulator with users spawned from very distinct user profiles as well as overlapping user profiles. The results show that clustering configurations using high-level features significantly outperform regular reinforcement learning without clustering (which either learn one policy for all or one policy per individual).

KW - Health care

KW - Personalization

KW - Reinforcement Learning

UR - http://www.scopus.com/inward/record.url?scp=85062780542&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062780542&partnerID=8YFLogxK

U2 - 10.1109/SSCI.2018.8628621

DO - 10.1109/SSCI.2018.8628621

M3 - Conference contribution

SP - 813

EP - 820

BT - Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018

A2 - Sundaram, Suresh

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Grua EM, Hoogendoorn M. Exploring Clustering Techniques for Effective Reinforcement Learning based Personalization for Health and Wellbeing. In Sundaram S, editor, Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 813-820. 8628621 https://doi.org/10.1109/SSCI.2018.8628621