Real-time assistance in suicide prevention helplines using a deep learning-based recommender system: A randomized controlled trial

Salim Salmi*, Saskia Mérelle, Nikki van Eijk, Renske Gilissen, Rob van der Mei, Sandjai Bhulai

*Corresponding author for this work

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Objective: To evaluate the effectiveness and usability of an AI-assisted tool in providing real-time assistance to counselors during suicide prevention helpline conversations. Methods: In this RCT, the intervention group used an AI-assisted tool, which generated suggestions based on sentence embeddings (i.e. BERT) from previous successful counseling sessions. Cosine similarity was used to present the top 5 chat situation to the counsellors. The control group did not have access to the tool (care as usual). Both groups completed a questionnaire assessing their self-efficacy at the end of each shift. Counselors' usage of the tool was evaluated by measuring frequency, duration and content of interactions. Results: In total, 48 counselors participated in the experiment: 27 counselors in the experimental condition and 21 counselors in the control condition. Together they rated 188 shifts. No significant difference in self-efficacy was observed between the two groups (p=0.36). However, counselors that used the AI-assisted tool had marginally lower response time and used the tool more often during conversations that had a longer duration. A deeper analysis of usage showed that the tool was frequently used in inappropriate situations, e.g. after the counselor had already provided a response to the help-seeker, defeating the purpose of the information. When the tool was employed appropriately (64 conversations), it provided usable information in 53 conversations (83%). However, counselors used the tool less frequently at optimal moments, indicating their potential lack of proficiency with using AI-assisted tools during helpline conversations or initial trust issues with the system. Conclusion: The study demonstrates benefits and pitfalls of integrating AI-assisted tools in suicide prevention for improving counselor support. Despite the lack of significant impact on self-efficacy, the support tool provided usable suggestions and the frequent use during long conversations suggests counsellors may wish to use the tool in complex or challenging interactions.

Original languageEnglish
Article number105760
Pages (from-to)1-7
Number of pages7
JournalInternational journal of medical informatics
Volume195
Early online date17 Dec 2024
DOIs
Publication statusE-pub ahead of print - 17 Dec 2024

Bibliographical note

Publisher Copyright:
© 2024 The Authors

Keywords

  • AI
  • Helpline
  • Machine learning
  • RCT
  • Support tool

Fingerprint

Dive into the research topics of 'Real-time assistance in suicide prevention helplines using a deep learning-based recommender system: A randomized controlled trial'. Together they form a unique fingerprint.

Cite this