A model a day keeps the doctor away: Reinforcement learning for personalized healthcare

Ali el Hassouni

Research output: PhD ThesisPhD-Thesis - Research and graduation internal

670 Downloads (Pure)


Reinforcement learning has shown great potential in many applications in recent years. However, many of these applications rely on simulators such as games to obtain a significant amount of interactions at a relatively low cost. Especially in the health domain data suitable for sequential decision-making is hard to obtain, while simulators are complicated to build. In this thesis, we investigate how we can use reinforcement learning for personalization in e-Health. We explored the state-of-the-art applications of reinforcement learning for personalization through a systematic literature review. Looking at the related work in the health domain, we found that reinforcement learning is an appropriate learning paradigm for personalization in general. Furthermore, we observed a lack of simulator environments and standardized datasets for the health domain. To mitigate this limitation, we developed an open-source simulation environment for e-Health problems. Using this simulator as a testbed, we proposed a cluster-based reinforcement learning approach to personalize e-Health interventions. Our approach finds similarities in behaviors and learns personalized policies optimized for long-term health behaviors. To bridge the gap towards real-life applications, we employed generative models, such as generative adversarial models, to synthesize low-level sensor data we can obtain from mobile devices in real-life. To evaluate the generated data's representativity for sequential decision-making problems, we proposed a framework consisting of two properties, structural and functional representativity. Using our realistic simulator, we used deep reinforcement learning to demonstrate that end-to-end learning on raw sensor data can improve performance. Finally, we utilized reinforcement learning for personalization for several real-world problems. We applied deep reinforcement learning for the hemodynamic optimization of patients with sepsis at the Intensive Care Unit. Finally, we applied our proposed cluster-based reinforcement learning approach for personalization to improve adherence to internet-based interventions tested in a real-life experiment using a mobile application.
Original languageEnglish
Awarding Institution
  • Vrije Universiteit Amsterdam
  • Hoogendoorn, Mark, Supervisor
  • Eiben, Gusz, Supervisor
  • Muhonen, Vesa, Co-supervisor, -
Award date18 Jan 2022
Print ISBNs9789464236040
Publication statusPublished - 18 Jan 2022


  • Reinforcement learning
  • Artificial intelligence
  • Healthcare
  • E-Health
  • Personalization
  • Simulation
  • Mental Health
  • Intensive care unit
  • Applications
  • Medical data


Dive into the research topics of 'A model a day keeps the doctor away: Reinforcement learning for personalized healthcare'. Together they form a unique fingerprint.

Cite this