Augmented intelligence facilitates concept mapping across different electronic health records

Tariq A. Dam; Lucas M. Fleuren; Luca F. Roggeveen; Martijn Otten; Laurens Biesheuvel; Ameet R. Jagesar; Robbert C.A. Lalisang; Robert F.J. Kullberg; Tom Hendriks; Armand R.J. Girbes; Mark Hoogendoorn; Patrick J. Thoral; Paul W.G. Elbers

doi:10.1016/j.ijmedinf.2023.105233

Augmented intelligence facilitates concept mapping across different electronic health records

Tariq A. Dam, Lucas M. Fleuren, Luca F. Roggeveen, Martijn Otten, Laurens Biesheuvel, Ameet R. Jagesar, Robbert C.A. Lalisang, Robert F.J. Kullberg, Tom Hendriks, Armand R.J. Girbes, Mark Hoogendoorn, Patrick J. Thoral, Paul W.G. Elbers

Research output: Contribution to Journal › Article › Academic › peer-review

Abstract

Introduction: With the advent of artificial intelligence, the secondary use of routinely collected medical data from electronic healthcare records (EHR) has become increasingly popular. However, different EHR systems typically use different names for the same medical concepts. This obviously hampers scalable model development and subsequent clinical implementation for decision support. Therefore, converting original parameter names to a so-called ontology, a standardized set of predefined concepts, is necessary but time-consuming and labor-intensive. We therefore propose an augmented intelligence approach to facilitate ontology alignment by predicting correct concepts based on parameter names from raw electronic health record data exports. Methods: We used the manually mapped parameter names from the multicenter “Dutch ICU data warehouse against COVID-19” sourced from three types of EHR systems to train machine learning models for concept mapping. Data from 29 intensive care units on 38,824 parameters mapped to 1,679 relevant and unique concepts and 38,069 parameters labeled as irrelevant were used for model development and validation. We used the Natural Language Toolkit (NLTK) to preprocess the parameter names based on WordNet cognitive synonyms transformed by term-frequency inverse document frequency (TF-IDF), yielding numeric features. We then trained linear classifiers using stochastic gradient descent for multi-class prediction. Finally, we fine-tuned these predictions using information on distributions of the data associated with each parameter name through similarity score and skewness comparisons. Results: The initial model, trained using data from one hospital organization for each of three EHR systems, scored an overall top 1 precision of 0.744, recall of 0.771, and F1-score of 0.737 on a total of 58,804 parameters. Leave-one-hospital-out analysis returned an average top 1 recall of 0.680 for relevant parameters, which increased to 0.905 for the top 5 predictions. When reducing the training dataset to only include relevant parameters, top 1 recall was 0.811 and top 5 recall was 0.914 for relevant parameters. Performance improvement based on similarity score or skewness comparisons affected at most 5.23% of numeric parameters. Conclusion: Augmented intelligence is a promising method to improve concept mapping of parameter names from raw electronic health record data exports. We propose a robust method for mapping data across various domains, facilitating the integration of diverse data sources. However, recall is not perfect, and therefore manual validation of mapping remains essential.

Original language	English
Article number	105233
Journal	International journal of medical informatics
Volume	179
DOIs	https://doi.org/10.1016/j.ijmedinf.2023.105233
Publication status	Published - 1 Nov 2023

Funding

The Dutch ICU Data Sharing Against COVID-19 Collaborators:, From collaborating hospitals having shared data:, Diederik Gommers, PhD, Department of Intensive Care, Erasmus Medical Center, Rotterdam, The Netherlands. Olaf L. Cremer, PhD, Intensive Care, UMC Utrecht, Utrecht, The Netherlands. Rob J. Bosman, ICU, OLVG, Amsterdam, The Netherlands. Sander Rigter, MD, Department of Anesthesiology and Intensive Care, St. Antonius Hospital, Nieuwegein, The Netherlands. Evert-Jan Wils, MD, PhD, Department of Intensive Care, Franciscus Gasthuis & Vlietland, Rotterdam, The Netherlands. Tim Frenzel, MD, PhD, Department of Intensive Care Medicine, Radboud University Medical Center, Nijmegen, The Netherlands. Dave A. Dongelmans, MD, PhD, Department of Intensive Care Medicine, Amsterdam UMC, Amsterdam, The Netherlands. Remko de Jong, MD, Intensive Care, Bovenij Ziekenhuis, Amsterdam, The Netherlands. Marco A.A. Peters, MD, Intensive Care, Canisius Wilhelmina Ziekenhuis, Nijmegen, The Netherlands. Marlijn J.A Kamps, MD, Intensive Care, Catharina Ziekenhuis Eindhoven, Eindhoven, The Netherlands. Dharmanand Ramnarain, MD, Department of Intensive Care, ETZ Tilburg, Tilburg, The Netherlands. Ralph Nowitzky, MD, Intensive Care, HagaZiekenhuis, Den Haag, The Netherlands. Fleur G.C.A. Nooteboom, MD, Intensive Care, Laurentius Ziekenhuis, Roermond, The Netherlands. Wouter de Ruijter, MD, PhD, Department of Intensive Care Medicine, Northwest Clinics, Alkmaar, The Netherlands. Louise C. Urlings-Strop, MD, PhD, Intensive Care, Reinier de Graaf Gasthuis, Delft, The Netherlands. Ellen G.M. Smit, MD, Intensive Care, Spaarne Gasthuis, Haarlem en Hoofddorp, The Netherlands. D. Jannet Mehagnoul-Schipper, MD, PhD, Intensive Care, VieCuri Medisch Centrum, Venlo, The Netherlands. Tom Dormans, MD, PhD, Intensive care, Zuyderland MC, Heerlen, The Netherlands. Cornelis P.C. de Jager, MD, PhD, Department of Intensive Care, Jeroen Bosch Ziekenhuis, Den Bosch, The Netherlands. Stefaan H.A. Hendriks, MD, Intensive Care, Albert Schweitzerziekenhuis, Dordrecht, The Netherlands. Sefanja Achterberg, MD, PhD, ICU, Haaglanden Medisch Centrum, Den Haag, The Netherlands. Evelien Oostdijk, MD, PhD, ICU, Maasstad Ziekenhuis Rotterdam, Rotterdam, The Netherlands. Auke C. Reidinga, MD, ICU, SEH, BWC, Martiniziekenhuis, Groningen, The Netherlands. Barbara Festen-Spanjer, MD, Intensive Care, Ziekenhuis Gelderse Vallei, Ede, The Netherlands. Gert B. Brunnekreef, MD, Department of Intensive Care, Ziekenhuisgroep Twente, Almelo, The Netherlands. Alexander D. Cornet, MD, PhD, FRCP, Department of Intensive Care, Medisch Spectrum Twente, Enschede, The Netherlands. Walter van den Tempel, MD, Department of Intensive Care, Ikazia Ziekenhuis Rotterdam, Rotterdam, The Netherlands. Age D. Boelens, MD, Anesthesiology, Antonius Ziekenhuis Sneek, Sneek, The Netherlands. Peter Koetsier, MD, Intensive Care, Medisch Centrum Leeuwarden, Leeuwarden, The Netherlands. Judith Lens, MD, ICU, IJsselland Ziekenhuis, Capelle aan den IJssel, The Netherlands. Harald J. Faber, MD, ICU, WZA, Assen, The Netherlands. A. Karakus, MD, Department of Intensive Care, Diakonessenhuis Hospital, Utrecht, The Netherlands. Robert Entjes, MD, Department of Intensive Care, Adrz, Goes, The Netherlands. Paul de Jong, MD, Department of Anesthesia and Intensive Care, Slingeland Ziekenhuis, Doetinchem, The Netherlands. Thijs C.D. Rettig, MD, PhD, Department of Anesthesiology, Intensive Care and Pain Medicine, Amphia Ziekenhuis, Breda, The Netherlands. Sesmu Arbous, MD, PhD, Intensivist, LUMC, Leiden, The Netherlands. Julia Koeter, MD, Intensive Care, Canisius Wilhelmina Ziekenhuis, Nijmegen, The Netherlands. Roger van Rietschote, Business Intelligence, Haaglanden MC, Den Haag,The Netherlands. M.C. Reuland, MD, Department of Intensive Care Medicine, Amsterdam UMC, Universiteit van Amsterdam, Amsterdam, The Netherlands. Laura van Manen, MD, Department of Intensive Care, BovenIJ Ziekenhuis, Amsterdam, The Netherlands. Leon Montenij, MD, PhD, Department of Anesthesiology, Pain Management and Intensive Care, Catharina Ziekenhuis Eindhoven, Eindhoven, The Netherlands. Jasper van Bommel, MD, PhD, Department of Intensive Care, Erasmus Medical Center, Rotterdam, The Netherlands. Roy van den Berg, Department of Intensive Care, ETZ Tilburg, Tilburg, The Netherlands. Ellen van Geest, Department of ICMT, Haga Ziekenhuis, Den Haag, The Netherlands. Anisa Hana, MD, PhD, Intensive Care, Laurentius Ziekenhuis, Roermond, The Netherlands. B. van den Bogaard, MD, PhD, ICU, OLVG, Amsterdam, The Netherlands. Prof. Peter Pickkers, Department of Intensive Care Medicine, Radboud University Medical Centre, Nijmegen, The Netherlands. Pim van der Heiden, MD, PhD, Intensive Care, Reinier de Graaf Gasthuis, Delft, The Netherlands. Claudia (C.W.) van Gemeren, MD, Intensive Care, Spaarne Gasthuis, Haarlem en Hoofddorp, The Netherlands. Arend Jan Meinders, MD, Department of Internal Medicine and Intensive Care, St Antonius Hospital, Nieuwegein, The Netherlands. Martha de Bruin, MD, Department of Intensive Care, Franciscus Gasthuis & Vlietland, Rotterdam, The Netherlands. Emma Rademaker, MD, MSc, Department of Intensive Care, UMC Utrecht, Utrecht, The Netherlands. Frits H.M. van Osch, PhD, Department of Clinical Epidemiology, VieCuri Medisch Centrum, Venlo, The Netherlands. Martijn D. de Kruif, MD, PhD, Department of Pulmonology, Zuyderland MC, Heerlen, The Netherlands. Nicolas Schroten, MD, Intensive Care, Albert Schweitzerziekenhuis, Dordrecht, The Netherlands. Klaas Sierk Arnold, MD, Anesthesiology, Antonius Ziekenhuis Sneek, Sneek, The Netherlands. J.W. Fijen, MD, PhD, Department of Intensive Care, Diakonessenhuis Hospital, Utrecht, The Netherland. Jacomar J.M. van Koesveld, MD, ICU, IJsselland Ziekenhuis, Capelle aan den IJssel, The Netherlands. Koen S. Simons, MD, PhD, Department of Intensive Care, Jeroen Bosch Ziekenhuis, Den Bosch, The Netherlands. Joost Labout, MD, PhD, ICU, Maasstad Ziekenhuis Rotterdam, The Netherlands. Bart van de Gaauw, MD, Martini ziekenhuis, Groningen, The Netherlands. Michael Kuiper, Intensive Care, Medisch Centrum Leeuwarden, Leeuwarden, The Netherlands. Albertus Beishuizen, MD, PhD, Department of Intensive Care, Medisch Spectrum Twente, Enschede, The Netherlands. Dennis Geutjes, Department of Information Technology, Slingeland Ziekenhuis, Doetinchem, The Netherlands. Johan Lutisan, MD, ICU, WZA, Assen, The Netherlands. Bart P. Grady, MD, PhD, Department of Intensive Care, Ziekenhuisgroep Twente, Almelo, The Netherlands. Remko van den Akker, Intensive Care, Adrz, Goes, The Netherlands. Tom A. Rijpstra, MD, Department of Anesthesiology, Intensive Care and Pain Medicine, Amphia Ziekenhuis, Breda, The Netherlands. Roos Renckens, MD, PhD, Department of Internal Medicine, Northwest Clinics, Alkmaar, the Netherlands, From collaborating hospitals having signed the data sharing agreement:, Daniël Pretorius, MD, Department of Intensive Care Medicine, Hospital St Jansdal, Harderwijk, The Netherlands. Menno Beukema, MD, Department of Intensive Care, Streekziekenhuis Koningin Beatrix, Winterswijk, The Netherlands. Bram Simons, MD, Intensive Care, Bravis Ziekenhuis, Bergen op Zoom en Roosendaal, The Netherlands. A.A. Rijkeboer, MD, ICU, Flevoziekenhuis, Almere, The Netherlands. Marcel Aries, MD, PhD, Department of Intensive Care, MUMC+, School of Mental Health and Neurosciences (MHENS), University Maastricht, Maastricht, The Netherlands. Niels C. Gritters van den Oever, MD, Intensive Care, Treant Zorggroep, Emmen, The Netherlands. Martijn van Tellingen, MD, EDIC, Department of Intensive Care Medicine, afdeling Intensive Care, ziekenhuis Tjongerschans, Heerenveen, The Netherlands. Annemieke Dijkstra, MD, Department of Intensive Care Medicine, Het Van Weel-Bethesda Ziekenhuis, Dirksland, The Netherlands. Rutger van Raalte, Department of Intensive Care, Tergooi hospital, Hilversum, The Netherlands. From the Center for Critical Care Computational Intelligence (C4I):, Fuda van Diggelen, MSc, Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands. Ali el Hassouni, PhD, Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands. David Romero Guzman, PhD, Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands. Sandjai Bhulai, PhD, Analytics and Optimization Group, Department of Mathematics, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands. Dagmar M. Ouweneel, PhD, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Ronald Driessen, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Jan Peppink, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Harm-Jan de Grooth, MD, PhD, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. G.J. Zijlstra, MD, PhD, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. A.J. van Tienhoven, MD, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Evelien van der Heiden, MD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Jan Jaap Spijkstra, MD, PhD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Hans van der Spoel, MD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Angelique M.E. de Man, MD, PhD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Thomas Klausch, PhD, Department of Clinical Epidemiology, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Heder J. de Vries, MD, Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. From Pacmed:, Sebastiaan J.J. Vonk, MSc, Pacmed, Amsterdam, The Netherlands. Willem E. Herter, BSc, Pacmed, Amsterdam, The Netherlands. Michele Tonutti, MRes, Pacmed, Amsterdam, The Netherlands. Daan P. de Bruin, MSc, Pacmed, Amsterdam, The Netherlands. Mattia Fornasa, PhD, Pacmed, Amsterdam, The Netherlands. Tomas Machado, Pacmed, Amsterdam, The Netherlands. Michael de Neree tot Babberich, Pacmed, Amsterdam, The Netherlands. Olivier Thijssens, MSc, Pacmed, Amsterdam, The Netherlands. Lot Wagemakers, Pacmed, Amsterdam, The Netherlands. Hilde G.A. van der Pol, Pacmed, Amsterdam, The Netherlands. Julie Berend, Pacmed, Amsterdam, The Netherlands. Virginia Ceni Silva, Pacmed, Amsterdam, The Netherlands. Taco Houwert, MSc, Pacmed, Amsterdam, The Netherlands. Hidde Hovenkamp, MSc, Pacmed, Amsterdam, The Netherlands. Roberto Noorduijn Londono, MSc, Pacmed, Amsterdam, The Netherlands. Davide Quintarelli, MSc, Pacmed, Amsterdam, The Netherlands. Martijn G. Scholtemeijer, MD, Pacmed, Amsterdam, The Netherlands. Aletta A. de Beer, MSc, Pacmed, Amsterdam, The Netherlands. Giovanni Cinà, PhD, Pacmed, Amsterdam, The Netherlands. Adam Izdebski, Pacmed, Amsterdam, The Netherlands. From RCCnet:, Leo Heunks, MD, PhD, Department of Intensive Care Medicine, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands. Nicole Juffermans, MD, PhD, ICU, OLVG, Amsterdam, The Netherlands. Arjen J.C. Slooter, MD, PhD, Department of Intensive Care Medicine, UMC Utrecht, Utrecht University, Utrecht, the Netherlands. From other collaborating partners:, Martijn Beudel, MD, PhD, Department of Neurology, Amsterdam UMC, Universiteit van Amsterdam, Amsterdam, The Netherlands. Tariq Dam contributed to conceptualization, data collection, processing, analysis and drafted the manuscript. Lucas Fleuren contributed to conceptualization, data collection, processing and analysis. Luca Roggeveen, Martijn Otten, Laurens Biesheuvel and Ameet Jagesar contributed to analysis and code review. Robbert Lalisang, Bob Kullberg, Tom Hendriks contributed to data collection and processing. All authors critically reviewed the manuscript. Ethics approval and consent to participate. The Medical Ethics Committee at Amsterdam UMC waived the need for patient informed consent and approved of an opt-out procedure for the collection of COVID-19 patient data during the COVID-19 crisis as documented under number 2020.156. Funding. Partially funded by the Netherlands Organization for Health Research and Development under project number 10430012010003. Data and code availability. All participating hospitals have access to the Dutch ICU Data Warehouse. External researchers can get access in collaboration with any of the participating hospitals. Contact details can be found on amsterdammedicaldatascience.nl. The code used for analysis is publicly available at github.com/tariqdam/AutoMap Partially funded by the Netherlands Organization for Health Research and Development under project number 10430012010003.

Funders	Funder number
Center for Critical Care Computational Intelligence
Den Bosch
Den Haag
Department of Clinical Epidemiology
Department of Intensive Care
Department of Intensive Care Medicine
Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence
Department of Intensive Care Medicine, Northwest Clinics
Department of Intensive Care Medicine, Radboud University Medical Center
Department of Intensive Care Medicine, Radboud University Medical Centre
ETZ Tilburg
Haaglanden MC, Den Haag
Haaglanden Medisch Centrum, Den Haag
HagaZiekenhuis
ICMT
ICU
Jeroen Bosch Ziekenhuis
MHENS
Maasstad Ziekenhuis Rotterdam
Medisch Centrum Leeuwarden
Michael de Neree tot Babberich
School of Mental Health and Neurosciences
VieCuri Medisch Centrum
Medisch Spectrum Twente
ZonMw	10430012010003
Universiteit Utrecht	2020.156
Universiteit Maastricht
Universitair Medisch Centrum Utrecht
St. Antonius Ziekenhuis
Leids Universitair Medisch Centrum

Access to Document

10.1016/j.ijmedinf.2023.105233

Persistent URL (handle)

Persistent link

Cite this

Dam, T. A., Fleuren, L. M., Roggeveen, L. F., Otten, M., Biesheuvel, L., Jagesar, A. R., Lalisang, R. C. A., Kullberg, R. F. J., Hendriks, T., Girbes, A. R. J., Hoogendoorn, M., Thoral, P. J., & Elbers, P. W. G. (2023). Augmented intelligence facilitates concept mapping across different electronic health records. International journal of medical informatics, 179, Article 105233. https://doi.org/10.1016/j.ijmedinf.2023.105233

@article{c96440b686b94f9581d2bc4b4bf2f26f,

title = "Augmented intelligence facilitates concept mapping across different electronic health records",

abstract = "Introduction: With the advent of artificial intelligence, the secondary use of routinely collected medical data from electronic healthcare records (EHR) has become increasingly popular. However, different EHR systems typically use different names for the same medical concepts. This obviously hampers scalable model development and subsequent clinical implementation for decision support. Therefore, converting original parameter names to a so-called ontology, a standardized set of predefined concepts, is necessary but time-consuming and labor-intensive. We therefore propose an augmented intelligence approach to facilitate ontology alignment by predicting correct concepts based on parameter names from raw electronic health record data exports. Methods: We used the manually mapped parameter names from the multicenter “Dutch ICU data warehouse against COVID-19” sourced from three types of EHR systems to train machine learning models for concept mapping. Data from 29 intensive care units on 38,824 parameters mapped to 1,679 relevant and unique concepts and 38,069 parameters labeled as irrelevant were used for model development and validation. We used the Natural Language Toolkit (NLTK) to preprocess the parameter names based on WordNet cognitive synonyms transformed by term-frequency inverse document frequency (TF-IDF), yielding numeric features. We then trained linear classifiers using stochastic gradient descent for multi-class prediction. Finally, we fine-tuned these predictions using information on distributions of the data associated with each parameter name through similarity score and skewness comparisons. Results: The initial model, trained using data from one hospital organization for each of three EHR systems, scored an overall top 1 precision of 0.744, recall of 0.771, and F1-score of 0.737 on a total of 58,804 parameters. Leave-one-hospital-out analysis returned an average top 1 recall of 0.680 for relevant parameters, which increased to 0.905 for the top 5 predictions. When reducing the training dataset to only include relevant parameters, top 1 recall was 0.811 and top 5 recall was 0.914 for relevant parameters. Performance improvement based on similarity score or skewness comparisons affected at most 5.23% of numeric parameters. Conclusion: Augmented intelligence is a promising method to improve concept mapping of parameter names from raw electronic health record data exports. We propose a robust method for mapping data across various domains, facilitating the integration of diverse data sources. However, recall is not perfect, and therefore manual validation of mapping remains essential.",

author = "Dam, {Tariq A.} and Fleuren, {Lucas M.} and Roggeveen, {Luca F.} and Martijn Otten and Laurens Biesheuvel and Jagesar, {Ameet R.} and Lalisang, {Robbert C.A.} and Kullberg, {Robert F.J.} and Tom Hendriks and Girbes, {Armand R.J.} and Mark Hoogendoorn and Thoral, {Patrick J.} and Elbers, {Paul W.G.}",

year = "2023",

month = nov,

day = "1",

doi = "10.1016/j.ijmedinf.2023.105233",

language = "English",

volume = "179",

journal = "International journal of medical informatics",

issn = "1386-5056",

publisher = "Elsevier Ireland Ltd",

}

Dam, TA, Fleuren, LM, Roggeveen, LF, Otten, M, Biesheuvel, L, Jagesar, AR, Lalisang, RCA, Kullberg, RFJ, Hendriks, T, Girbes, ARJ, Hoogendoorn, M, Thoral, PJ & Elbers, PWG 2023, 'Augmented intelligence facilitates concept mapping across different electronic health records', International journal of medical informatics, vol. 179, 105233. https://doi.org/10.1016/j.ijmedinf.2023.105233

TY - JOUR

T1 - Augmented intelligence facilitates concept mapping across different electronic health records

AU - Dam, Tariq A.

AU - Fleuren, Lucas M.

AU - Roggeveen, Luca F.

AU - Otten, Martijn

AU - Biesheuvel, Laurens

AU - Jagesar, Ameet R.

AU - Lalisang, Robbert C.A.

AU - Kullberg, Robert F.J.

AU - Hendriks, Tom

AU - Girbes, Armand R.J.

AU - Hoogendoorn, Mark

AU - Thoral, Patrick J.

AU - Elbers, Paul W.G.

PY - 2023/11/1

Y1 - 2023/11/1

N2 - Introduction: With the advent of artificial intelligence, the secondary use of routinely collected medical data from electronic healthcare records (EHR) has become increasingly popular. However, different EHR systems typically use different names for the same medical concepts. This obviously hampers scalable model development and subsequent clinical implementation for decision support. Therefore, converting original parameter names to a so-called ontology, a standardized set of predefined concepts, is necessary but time-consuming and labor-intensive. We therefore propose an augmented intelligence approach to facilitate ontology alignment by predicting correct concepts based on parameter names from raw electronic health record data exports. Methods: We used the manually mapped parameter names from the multicenter “Dutch ICU data warehouse against COVID-19” sourced from three types of EHR systems to train machine learning models for concept mapping. Data from 29 intensive care units on 38,824 parameters mapped to 1,679 relevant and unique concepts and 38,069 parameters labeled as irrelevant were used for model development and validation. We used the Natural Language Toolkit (NLTK) to preprocess the parameter names based on WordNet cognitive synonyms transformed by term-frequency inverse document frequency (TF-IDF), yielding numeric features. We then trained linear classifiers using stochastic gradient descent for multi-class prediction. Finally, we fine-tuned these predictions using information on distributions of the data associated with each parameter name through similarity score and skewness comparisons. Results: The initial model, trained using data from one hospital organization for each of three EHR systems, scored an overall top 1 precision of 0.744, recall of 0.771, and F1-score of 0.737 on a total of 58,804 parameters. Leave-one-hospital-out analysis returned an average top 1 recall of 0.680 for relevant parameters, which increased to 0.905 for the top 5 predictions. When reducing the training dataset to only include relevant parameters, top 1 recall was 0.811 and top 5 recall was 0.914 for relevant parameters. Performance improvement based on similarity score or skewness comparisons affected at most 5.23% of numeric parameters. Conclusion: Augmented intelligence is a promising method to improve concept mapping of parameter names from raw electronic health record data exports. We propose a robust method for mapping data across various domains, facilitating the integration of diverse data sources. However, recall is not perfect, and therefore manual validation of mapping remains essential.

AB - Introduction: With the advent of artificial intelligence, the secondary use of routinely collected medical data from electronic healthcare records (EHR) has become increasingly popular. However, different EHR systems typically use different names for the same medical concepts. This obviously hampers scalable model development and subsequent clinical implementation for decision support. Therefore, converting original parameter names to a so-called ontology, a standardized set of predefined concepts, is necessary but time-consuming and labor-intensive. We therefore propose an augmented intelligence approach to facilitate ontology alignment by predicting correct concepts based on parameter names from raw electronic health record data exports. Methods: We used the manually mapped parameter names from the multicenter “Dutch ICU data warehouse against COVID-19” sourced from three types of EHR systems to train machine learning models for concept mapping. Data from 29 intensive care units on 38,824 parameters mapped to 1,679 relevant and unique concepts and 38,069 parameters labeled as irrelevant were used for model development and validation. We used the Natural Language Toolkit (NLTK) to preprocess the parameter names based on WordNet cognitive synonyms transformed by term-frequency inverse document frequency (TF-IDF), yielding numeric features. We then trained linear classifiers using stochastic gradient descent for multi-class prediction. Finally, we fine-tuned these predictions using information on distributions of the data associated with each parameter name through similarity score and skewness comparisons. Results: The initial model, trained using data from one hospital organization for each of three EHR systems, scored an overall top 1 precision of 0.744, recall of 0.771, and F1-score of 0.737 on a total of 58,804 parameters. Leave-one-hospital-out analysis returned an average top 1 recall of 0.680 for relevant parameters, which increased to 0.905 for the top 5 predictions. When reducing the training dataset to only include relevant parameters, top 1 recall was 0.811 and top 5 recall was 0.914 for relevant parameters. Performance improvement based on similarity score or skewness comparisons affected at most 5.23% of numeric parameters. Conclusion: Augmented intelligence is a promising method to improve concept mapping of parameter names from raw electronic health record data exports. We propose a robust method for mapping data across various domains, facilitating the integration of diverse data sources. However, recall is not perfect, and therefore manual validation of mapping remains essential.

UR - http://www.scopus.com/inward/record.url?scp=85172246939&partnerID=8YFLogxK

U2 - 10.1016/j.ijmedinf.2023.105233

DO - 10.1016/j.ijmedinf.2023.105233

M3 - Article

SN - 1386-5056

VL - 179

JO - International journal of medical informatics

JF - International journal of medical informatics

M1 - 105233

ER -

Augmented intelligence facilitates concept mapping across different electronic health records

Abstract

Funding

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this