A case study of an individual participant data meta-analysis of diagnostic accuracy showed that prediction regions represented heterogeneity well

DEPRESsion Screening Data (DEPRESSD) PHQ-9 Collaboration

doi:10.1038/s41598-023-36129-w

A case study of an individual participant data meta-analysis of diagnostic accuracy showed that prediction regions represented heterogeneity well

DEPRESsion Screening Data (DEPRESSD) PHQ-9 Collaboration

World Health Organization (WHO) Collaborating Center

Research output: Contribution to Journal › Article › Academic › peer-review

Abstract

The diagnostic accuracy of a screening tool is often characterized by its sensitivity and specificity. An analysis of these measures must consider their intrinsic correlation. In the context of an individual participant data meta-analysis, heterogeneity is one of the main components of the analysis. When using a random-effects meta-analytic model, prediction regions provide deeper insight into the effect of heterogeneity on the variability of estimated accuracy measures across the entire studied population, not just the average. This study aimed to investigate heterogeneity via prediction regions in an individual participant data meta-analysis of the sensitivity and specificity of the Patient Health Questionnaire-9 for screening to detect major depression. From the total number of studies in the pool, four dates were selected containing roughly 25%, 50%, 75% and 100% of the total number of participants. A bivariate random-effects model was fitted to studies up to and including each of these dates to jointly estimate sensitivity and specificity. Two-dimensional prediction regions were plotted in ROC-space. Subgroup analyses were carried out on sex and age, regardless of the date of the study. The dataset comprised 17,436 participants from 58 primary studies of which 2322 (13.3%) presented cases of major depression. Point estimates of sensitivity and specificity did not differ importantly as more studies were added to the model. However, correlation of the measures increased. As expected, standard errors of the logit pooled TPR and FPR consistently decreased as more studies were used, while standard deviations of the random-effects did not decrease monotonically. Subgroup analysis by sex did not reveal important contributions for observed heterogeneity; however, the shape of the prediction regions differed. Subgroup analysis by age did not reveal meaningful contributions to the heterogeneity and the prediction regions were similar in shape. Prediction intervals and regions reveal previously unseen trends in a dataset. In the context of a meta-analysis of diagnostic test accuracy, prediction regions can display the range of accuracy measures in different populations and settings.

Original language	English
Article number	9275
Journal	Scientific Reports
Volume	13
Issue number	1
DOIs	https://doi.org/10.1038/s41598-023-36129-w
Publication status	Published - 1 Dec 2023

Funding

This study was funded by the Canadian Institutes of Health Research (CIHR, KRS-134297). Mr. López Malo Vázquez de Lara was supported by the Mitacs Globalink Research Internship Program. Mr. Bhandari was supported by a studentship from the Research Institute of the McGill University Health Centre. Drs. Wu and Levis were supported by Fonds de recherche du Québec—Santé (FRQS) Postdoctoral Training Fellowships. Drs. Thombs and Benedetti were supported by a FRQS researcher salary award. Ms. Neupane was supported by G.R. Caverhill Fellowship from the Faculty of Medicine, McGill University. Ms. Rice was supported by a Vanier Canada Graduate Scholarship. Ms. Riehm and Ms. Saadat were supported by CIHR Frederick Banting and Charles Best Canada Graduate Scholarship master’s awards. The primary studies by Fiest et al. and Amoozegar et al. were supported by the Cumming School of Medicine, University of Calgary, and Alberta Health Services through the Calgary Health Trust, as well as the Hotchkiss Brain Institute. Dr. Patten was supported by a Senior Health Scholar award from Alberta Innovates Health Solutions. Dr. Jetté was supported by a Canada Research Chair in Neurological Health Services Research and an AIHS Population Health Investigator Award. Collection of data for the study by Arroll et al. was supported by a project grant from the Health Research Council of New Zealand. Data collection for the study by Ayalon et al. was supported from a grant from Lundbeck International. The primary study by Khamseh et al. was supported by a grant (M-288) from Tehran University of Medical Sciences. The primary study by Bombardier et al. was supported by the Department of Education, National Institute on Disability and Rehabilitation Research, Spinal Cord Injury Model Systems: University of Washington (Grant No. H133N060033), Baylor College of Medicine (Grant No. H133N060003), and University of Michigan (Grant No. H133N060032). Collection of data for the primary study by Kiely et al. was supported by National Health and Medical Research Council (Grant Number 1002160) and Safe Work Australia. Dr. Butterworth was supported by Australian Research Council Future Fellowship (FT130101444). Collection of data for the primary study by Zhang et al. was supported by the European Foundation for Study of Diabetes, the Chinese Diabetes Society, Lilly Foundation, Asia Diabetes Foundation and Liao Wun Yuk Diabetes Memorial Fund. Dr. Conwell received support from NIMH (R24MH071604) and the Centers for Disease Control and Prevention (R49 CE002093). Collection of data for the primary study by Delgadillo et al. was supported by grant from St. Anne’s Community Services, Leeds, United Kingdom. Collection of data for the primary study by Fann et al. was supported by Grant RO1 HD39415 from the US National Center for Medical Rehabilitation Research. The primary study by Fischer et al. was funded by the German Federal Ministry of Education and Research (01GY1150). Data for the primary study by Gelaye et al. was supported by grant from the NIH (T37 MD001449). Collection of data for the primary study by Gjerdingen et al. was supported by grants from the NIMH (R34 MH072925, K02 MH65919, P30 DK50456). The primary study by Eack et al. was funded by the NIMH (R24 MH56858). Collection of data for the primary study by Hobfoll et al. was made possible in part from grants from NIMH (RO1 MH073687) and the Ohio Board of Regents. Dr. Hall received support from a grant awarded by the Research and Development Administration Office, University of Macau (MYRG2015-00109-FSS). Collection of data provided by Drs. Härter and Reuter was supported by the Federal Ministry of Education and Research (Grants No. 01 GD 9802/4 and 01 GD 0101) and by the Federation of German Pension Insurance Institute. The primary study by Henkel et al. was funded by the German Ministry of Research and Education. The primary study by Hides et al. was funded by the Perpetual Trustees, Flora and Frank Leith Charitable Trust, Jack Brockhoff Foundation, Grosvenor Settlement, Sunshine Foundation and Danks Trust. Data for the study by Razykov et al. was collected by the Canadian Scleroderma Research Group, which was funded by the CIHR (FRN 83518), the Scleroderma Society of Canada, the Scleroderma Society of Ontario, the Scleroderma Society of Saskatchewan, Sclérodermie Québec, the Cure Scleroderma Foundation, Inova Diagnostics Inc., Euroimmun, FRQS, the Canadian Arthritis Network, and the Lady Davis Institute of Medical Research of the Jewish General Hospital, Montréal, QC. Dr. Hudson was supported by a FRQS Senior Investigator Award. Collection of data for the primary study by Hyphantis et al. (2014) was supported by grant from the National Strategic Reference Framework, European Union, and the Greek Ministry of Education, Lifelong Learning and Religious Affairs (ARISTEIA-ABREVIATE, 1259). The primary study by Inagaki et al. was supported by the Ministry of Health, Labour and Welfare, Japan. The primary study by Twist et al. was funded by the UK National Institute for Health Research under its Programme Grants for Applied Research Programme (Grant Reference Number RP-PG-0606-1142). The primary study by Lamers et al. was funded by the Netherlands Organisation for Health Research and development (Grant Number 945-03-047). The primary study by Liu et al. (2011) was funded by a grant from the National Health Research Institute, Republic of China (NHRI-EX97-9706PI). The primary study by Lotrakul et al. was supported by the Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (Grant Number 49086). The primary studies by Osório et al. (2012) were funded by Reitoria de Pesquisa da Universidade de São Paulo (Grant Number 09.1.01689.17.7) and Banco Santander (Grant Number 10.1.01232.17.9). Dr. Osório was supported by Productivity Grants (PQ-CNPq-2-Number 301321/2016-7). Dr. Bernd Löwe received research grants from Pfizer, Germany, and from the medical faculty of the University of Heidelberg, Germany (Project 121/2000) for the study by Gräfe et al. Collection of data for the primary study by Williams et al. was supported by a NIMH grant to Dr. Marsh (RO1-MH069666). The primary study by Mohd Sidik et al. was funded under the Research University Grant Scheme from Universiti Putra Malaysia, Malaysia and the Postgraduate Research Student Support Accounts of the University of Auckland, New Zealand. The primary study by Santos et al. was funded by the National Program for Centers of Excellence (PRONEX/FAPERGS/CNPq, Brazil). The primary study by Muramatsu et al. (2007) was supported by an educational grant from Pfizer US Pharmaceutical Inc. Collection of primary data for the study by Pence et al. was provided by NIMH (R34MH084673). The primary study by Persoons et al. was partly funded by a grant from the Belgian Ministry of Public Health and Social Affairs and supported by a limited grant from Pfizer Belgium. The primary study by Picardi et al. was supported by funds for current research from the Italian Ministry of Health. The primary study by Rooney et al. was funded by the United Kingdom National Health Service Lothian Neuro-Oncology Endowment Fund. Dr. Shaaban was supported by funding from Universiti Sains Malaysia. The primary study by Sidebottom et al. was funded by a grant from the United States Department of Health and Human Services, Health Resources and Services Administration (Grant Number R40MC07840). Simning et al.’s research was supported in part by grants from the NIH (T32 GM07356), Agency for Healthcare Research and Quality (R36 HS018246), NIMH (R24 MH071604), and the National Center for Research Resources (TL1 RR024135). Dr. Stafford received PhD scholarship funding from the University of Melbourne. Collection of data for the studies by Turner et al. (2012) were funded by a bequest from Jennie Thomas through the Hunter Medical Research Institute. The study by van Steenbergen-Weijenburg et al. was funded by Innovatiefonds Zorgverzekeraars. The study by Wittkampf et al. was funded by The Netherlands Organization for Health Research and Development (ZonMw) Mental Health Program (nos. 100.003.005 and 100.002.021) and the Academic Medical Center/University of Amsterdam. Dr Vöhringer was supported by the Fund for Innovation and Competitiveness of the Chilean Ministry of Economy, Development and Tourism, through the Millennium Scientific Initiative (Grant Number IS130005). The primary study by Thombs et al. was done with data from the Heart and Soul Study. The Heart and Soul Study was funded by the Department of Veterans Epidemiology Merit Review Program, the Department of Veterans Affairs Health Services Research and Development service, the National Heart Lung and Blood Institute (R01 HL079235), the American Federation for Aging Research, the Robert Wood Johnson Foundation, and the Ischemia Research and Education Foundation.No other authors reported funding for primary studies or for their work on the present study. All authors have completed the ICJME uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years with the following exceptions: Dr. Chan J CN is a steering committee member and/or consultant of Astra Zeneca, Bayer, Lilly, MSD and Pfizer. She has received sponsorships and honorarium for giving lectures and providing consultancy and her affiliated institution has received research grants from these companies. Dr. Hegerl declares that within the last three years, he was an advisory board member for Lundbeck and Servier; a consultant for Bayer Pharma; a speaker for Pharma and Servier; and received personal fees from Janssen Janssen and a research grant from Medice, all outside the submitted work. Dr. Inagaki declares that he has received a grant from Novartis Pharma, and personal fees from Meiji, Mochida, Takeda, Novartis, Yoshitomi, Pfizer, Eisai, Otsuka, MSD, Technomics, and Sumitomo Dainippon, all outside of the submitted work. Dr. Ismail declares that she has received honorarium for speaker fees for educational lectures for Sanofi, Sunovion, Janssen and Novo Nordisk. All authors declare no other relationships or activities that could appear to have influenced the submitted work. No funder had any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Funders	Funder number
Asia Diabetes Foundation
Bayer Pharma
Belgian Ministry of Public Health and Social Affairs
Department of Veterans
Faculty of Medicine, Ramathibodi Hospital, Mahidol University	49086
Inova Diagnostics Inc.
Lady Davis Institute of Medical Research of the Jewish General Hospital
Liao Wun Yuk Diabetes Memorial Fund	RO1 HD39415
Lundbeck International	M-288
National Health Research Institute, Republic of China	NHRI-EX97-9706PI
National Health Service Lothian Neuro-Oncology Endowment Fund
National Program for Centers of Excellence
National Strategic Reference Framework
PRONEX
Perpetual Trustees	FRN 83518
Pfizer Belgium
Pfizer US Pharmaceutical Inc
Postgraduate Research Student Support Accounts of the University of Auckland
Research and Development Administration Office
Scleroderma Society of Canada
Scleroderma Society of Saskatchewan
Sclérodermie Québec
Vanier Canada Graduate Scholarship
National Institutes of Health	T37 MD001449
U.S. Department of Health and Human Services
National Institute of Mental Health	K02 MH65919, R24 MH56858, P30 DK50456, R34 MH072925, RO1 MH073687
National Heart, Lung, and Blood Institute	R01 HL079235
National Center for Research Resources	TL1 RR024135
Health Resources and Services Administration	T32 GM07356, R40MC07840
Agency for Healthcare Research and Quality	R36 HS018246, R24 MH071604
U.S. Department of Education
Robert Wood Johnson Foundation
American Federation for Aging Research
Scleroderma Foundation
Pfizer
Ohio Board of Regents
National Institute on Disability and Rehabilitation Research
National Center for Medical Rehabilitation Research
Ischemia Research and Education Foundation
University of Michigan	H133N060032
University of Washington	H133N060033
Baylor College of Medicine	H133N060003
McGill University
Banco Santander	PQ-CNPq-2-Number 301321/2016-7, 10.1.01232.17.9
McGill University Health Centre
Canadian Institutes of Health Research	KRS-134297
Alberta Innovates - Health Solutions
Fonds de Recherche du Québec - Santé
National Institute for Health and Care Research	RP-PG-0606-1142
European Commission
Australian Research Council	FT130101444
National Health and Medical Research Council	1002160
Health Research Council of New Zealand
European Foundation for the Study of Diabetes
Universität Heidelberg	RO1-MH069666, 121/2000
University of Melbourne
ZonMw	100.003.005, 100.002.021, 945-03-047
Bundesministerium für Bildung und Forschung	01GY1150, 01 GD 0101, 01 GD 9802/4
Ministero della Salute
Ministry of Health, Labour and Welfare
Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul
Tehran University of Medical Sciences and Health Services
Mitacs
Universiti Putra Malaysia
Universiti Sains Malaysia
Universidade de Macau	MYRG2015-00109-FSS
Ministerio de Economía, Fomento y Turismo, Chile	IS130005
Ministry of Education, Lifelong Learning and Religious Affairs
Canadian Arthritis Network
Pro-Reitoria de Pesquisa, Universidade de São Paulo	09.1.01689.17.7
Safe Work Australia
Innovatiefonds Zorgverzekeraars
Chinese Diabetes Society
Fundación Lilly
Scleroderma Society of Ontario

Access to Document

10.1038/s41598-023-36129-w

Persistent URL (handle)

Copy link address

Cite this

@article{78a3266586924dfca97014093db26b41,

title = "A case study of an individual participant data meta-analysis of diagnostic accuracy showed that prediction regions represented heterogeneity well",

abstract = "The diagnostic accuracy of a screening tool is often characterized by its sensitivity and specificity. An analysis of these measures must consider their intrinsic correlation. In the context of an individual participant data meta-analysis, heterogeneity is one of the main components of the analysis. When using a random-effects meta-analytic model, prediction regions provide deeper insight into the effect of heterogeneity on the variability of estimated accuracy measures across the entire studied population, not just the average. This study aimed to investigate heterogeneity via prediction regions in an individual participant data meta-analysis of the sensitivity and specificity of the Patient Health Questionnaire-9 for screening to detect major depression. From the total number of studies in the pool, four dates were selected containing roughly 25%, 50%, 75% and 100% of the total number of participants. A bivariate random-effects model was fitted to studies up to and including each of these dates to jointly estimate sensitivity and specificity. Two-dimensional prediction regions were plotted in ROC-space. Subgroup analyses were carried out on sex and age, regardless of the date of the study. The dataset comprised 17,436 participants from 58 primary studies of which 2322 (13.3%) presented cases of major depression. Point estimates of sensitivity and specificity did not differ importantly as more studies were added to the model. However, correlation of the measures increased. As expected, standard errors of the logit pooled TPR and FPR consistently decreased as more studies were used, while standard deviations of the random-effects did not decrease monotonically. Subgroup analysis by sex did not reveal important contributions for observed heterogeneity; however, the shape of the prediction regions differed. Subgroup analysis by age did not reveal meaningful contributions to the heterogeneity and the prediction regions were similar in shape. Prediction intervals and regions reveal previously unseen trends in a dataset. In the context of a meta-analysis of diagnostic test accuracy, prediction regions can display the range of accuracy measures in different populations and settings.",

author = "{de Lara}, {Aurelio L{\'o}pez Malo V{\'a}zquez} and Bhandari, {Parash Mani} and Yin Wu and Brooke Levis and Brett Thombs and Andrea Benedetti and Ying Sun and Chen He and Ankur Krishnan and Dipika Neupane and Zelalem Negeri and Mahrukh Imran and Rice, {Danielle B.} and Riehm, {Kira E.} and Nazanin Saadat and Marleine Azar and Jill Boruff and Pim Cuijpers and Simon Gilbody and Ioannidis, {John P. A.} and Kloda, {Lorie A.} and Dean McMillan and Patten, {Scott B.} and Ian Shrier and Ziegelstein, {Roy C.} and Akena, {Dickens H.} and Bruce Arroll and Liat Ayalon and Baradaran, {Hamid R.} and Anna Beraldi and Bombardier, {Charles H.} and Peter Butterworth and Gregory Carter and Chagas, {Marcos H.} and Chan, {Juliana C. N.} and Rushina Cholera and Neerja Chowdhary and Kerrie Clover and Yeates Conwell and {de Man-van Ginkel}, {Janneke M.} and Jaime Delgadillo and Fann, {Jesse R.} and Fischer, {Felix H.} and Daniel Fung and Bizu Gelaye and Felicity Goodyear-Smith and Greeno, {Catherine G.} and Hall, {Brian J.} and Martin H{\"a}rter and Ulrich Hegerl and Leanne Hides and Hobfoll, {Stevan E.} and Marie Hudson and Thomas Hyphantis and Masatoshi Inagaki and Khalida Ismail and Nathalie Jett{\'e} and Khamseh, {Mohammad E.} and Kiely, {Kim M.} and Yunxin Kwan and Femke Lamers and Shen-Ing Liu and Manote Lotrakul and Loureiro, {Sonia R.} and Bernd L{\"o}we and Laura Marsh and Anthony McGuire and {Mohd Sidik}, Sherina and Munhoz, {Tiago N.} and Kumiko Muramatsu and Os{\'o}rio, {Fl{\'a}via L.} and Vikram Patel and Pence, {Brian W.} and Philippe Persoons and Angelo Picardi and Katrin Reuter and Rooney, {Alasdair G.} and Santos, {In{\'a} S.} and Juwita Shaaban and Abbey Sidebottom and Adam Simning and Lesley Stafford and Sung, {Sharon C.} and Tan, {Pei Lin Lynnette} and Alyna Turner and {van der Feltz-Cornelis}, {Christina M.} and {van Weert}, {Henk C.} and V{\"o}hringer, {Paul A.} and Jennifer White and Whooley, {Mary A.} and Kirsty Winkley and Mitsuhiko Yamada and Yuying Zhang and {DEPRESsion Screening Data (DEPRESSD) PHQ-9 Collaboration}",

year = "2023",

month = dec,

day = "1",

doi = "10.1038/s41598-023-36129-w",

language = "English",

volume = "13",

journal = "Scientific Reports",

issn = "2045-2322",

publisher = "Nature Publishing Group",

number = "1",

}

TY - JOUR

T1 - A case study of an individual participant data meta-analysis of diagnostic accuracy showed that prediction regions represented heterogeneity well

AU - de Lara, Aurelio López Malo Vázquez

AU - Bhandari, Parash Mani

AU - Wu, Yin

AU - Levis, Brooke

AU - Thombs, Brett

AU - Benedetti, Andrea

AU - Sun, Ying

AU - He, Chen

AU - Krishnan, Ankur

AU - Neupane, Dipika

AU - Negeri, Zelalem

AU - Imran, Mahrukh

AU - Rice, Danielle B.

AU - Riehm, Kira E.

AU - Saadat, Nazanin

AU - Azar, Marleine

AU - Boruff, Jill

AU - Cuijpers, Pim

AU - Gilbody, Simon

AU - Ioannidis, John P. A.

AU - Kloda, Lorie A.

AU - McMillan, Dean

AU - Patten, Scott B.

AU - Shrier, Ian

AU - Ziegelstein, Roy C.

AU - Akena, Dickens H.

AU - Arroll, Bruce

AU - Ayalon, Liat

AU - Baradaran, Hamid R.

AU - Beraldi, Anna

AU - Bombardier, Charles H.

AU - Butterworth, Peter

AU - Carter, Gregory

AU - Chagas, Marcos H.

AU - Chan, Juliana C. N.

AU - Cholera, Rushina

AU - Chowdhary, Neerja

AU - Clover, Kerrie

AU - Conwell, Yeates

AU - de Man-van Ginkel, Janneke M.

AU - Delgadillo, Jaime

AU - Fann, Jesse R.

AU - Fischer, Felix H.

AU - Fung, Daniel

AU - Gelaye, Bizu

AU - Goodyear-Smith, Felicity

AU - Greeno, Catherine G.

AU - Hall, Brian J.

AU - Härter, Martin

AU - Hegerl, Ulrich

AU - Hides, Leanne

AU - Hobfoll, Stevan E.

AU - Hudson, Marie

AU - Hyphantis, Thomas

AU - Inagaki, Masatoshi

AU - Ismail, Khalida

AU - Jetté, Nathalie

AU - Khamseh, Mohammad E.

AU - Kiely, Kim M.

AU - Kwan, Yunxin

AU - Lamers, Femke

AU - Liu, Shen-Ing

AU - Lotrakul, Manote

AU - Loureiro, Sonia R.

AU - Löwe, Bernd

AU - Marsh, Laura

AU - McGuire, Anthony

AU - Mohd Sidik, Sherina

AU - Munhoz, Tiago N.

AU - Muramatsu, Kumiko

AU - Osório, Flávia L.

AU - Patel, Vikram

AU - Pence, Brian W.

AU - Persoons, Philippe

AU - Picardi, Angelo

AU - Reuter, Katrin

AU - Rooney, Alasdair G.

AU - Santos, Iná S.

AU - Shaaban, Juwita

AU - Sidebottom, Abbey

AU - Simning, Adam

AU - Stafford, Lesley

AU - Sung, Sharon C.

AU - Tan, Pei Lin Lynnette

AU - Turner, Alyna

AU - van der Feltz-Cornelis, Christina M.

AU - van Weert, Henk C.

AU - Vöhringer, Paul A.

AU - White, Jennifer

AU - Whooley, Mary A.

AU - Winkley, Kirsty

AU - Yamada, Mitsuhiko

AU - Zhang, Yuying

AU - DEPRESsion Screening Data (DEPRESSD) PHQ-9 Collaboration

PY - 2023/12/1

Y1 - 2023/12/1

N2 - The diagnostic accuracy of a screening tool is often characterized by its sensitivity and specificity. An analysis of these measures must consider their intrinsic correlation. In the context of an individual participant data meta-analysis, heterogeneity is one of the main components of the analysis. When using a random-effects meta-analytic model, prediction regions provide deeper insight into the effect of heterogeneity on the variability of estimated accuracy measures across the entire studied population, not just the average. This study aimed to investigate heterogeneity via prediction regions in an individual participant data meta-analysis of the sensitivity and specificity of the Patient Health Questionnaire-9 for screening to detect major depression. From the total number of studies in the pool, four dates were selected containing roughly 25%, 50%, 75% and 100% of the total number of participants. A bivariate random-effects model was fitted to studies up to and including each of these dates to jointly estimate sensitivity and specificity. Two-dimensional prediction regions were plotted in ROC-space. Subgroup analyses were carried out on sex and age, regardless of the date of the study. The dataset comprised 17,436 participants from 58 primary studies of which 2322 (13.3%) presented cases of major depression. Point estimates of sensitivity and specificity did not differ importantly as more studies were added to the model. However, correlation of the measures increased. As expected, standard errors of the logit pooled TPR and FPR consistently decreased as more studies were used, while standard deviations of the random-effects did not decrease monotonically. Subgroup analysis by sex did not reveal important contributions for observed heterogeneity; however, the shape of the prediction regions differed. Subgroup analysis by age did not reveal meaningful contributions to the heterogeneity and the prediction regions were similar in shape. Prediction intervals and regions reveal previously unseen trends in a dataset. In the context of a meta-analysis of diagnostic test accuracy, prediction regions can display the range of accuracy measures in different populations and settings.

AB - The diagnostic accuracy of a screening tool is often characterized by its sensitivity and specificity. An analysis of these measures must consider their intrinsic correlation. In the context of an individual participant data meta-analysis, heterogeneity is one of the main components of the analysis. When using a random-effects meta-analytic model, prediction regions provide deeper insight into the effect of heterogeneity on the variability of estimated accuracy measures across the entire studied population, not just the average. This study aimed to investigate heterogeneity via prediction regions in an individual participant data meta-analysis of the sensitivity and specificity of the Patient Health Questionnaire-9 for screening to detect major depression. From the total number of studies in the pool, four dates were selected containing roughly 25%, 50%, 75% and 100% of the total number of participants. A bivariate random-effects model was fitted to studies up to and including each of these dates to jointly estimate sensitivity and specificity. Two-dimensional prediction regions were plotted in ROC-space. Subgroup analyses were carried out on sex and age, regardless of the date of the study. The dataset comprised 17,436 participants from 58 primary studies of which 2322 (13.3%) presented cases of major depression. Point estimates of sensitivity and specificity did not differ importantly as more studies were added to the model. However, correlation of the measures increased. As expected, standard errors of the logit pooled TPR and FPR consistently decreased as more studies were used, while standard deviations of the random-effects did not decrease monotonically. Subgroup analysis by sex did not reveal important contributions for observed heterogeneity; however, the shape of the prediction regions differed. Subgroup analysis by age did not reveal meaningful contributions to the heterogeneity and the prediction regions were similar in shape. Prediction intervals and regions reveal previously unseen trends in a dataset. In the context of a meta-analysis of diagnostic test accuracy, prediction regions can display the range of accuracy measures in different populations and settings.

UR - http://www.scopus.com/inward/record.url?scp=85161208731&partnerID=8YFLogxK

U2 - 10.1038/s41598-023-36129-w

DO - 10.1038/s41598-023-36129-w

M3 - Article

SN - 2045-2322

VL - 13

JO - Scientific Reports

JF - Scientific Reports

IS - 1

M1 - 9275

ER -

A case study of an individual participant data meta-analysis of diagnostic accuracy showed that prediction regions represented heterogeneity well

Abstract

Funding

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this