Consensus on quality indicators of postgraduate medical e-learning: Delphi study

Kieran Walsh, Michiel Westerman, Fedde Scheele, R.A. de Leeuw*

*Corresponding author for this work

Research output: Contribution to JournalArticleAcademicpeer-review


Background: The progressive use of e-learning in postgraduate medical education calls for useful quality indicators. Many evaluation tools exist. However, these are diversely used and their empirical foundation is often lacking. Objective: We aimed to identify an empirically founded set of quality indicators to set the bar for “good enough” e-learning. Methods: We performed a Delphi procedure with a group of 13 international education experts and 10 experienced users of e-learning. The questionnaire started with 57 items. These items were the result of a previous literature review and focus group study performed with experts and users. Consensus was met when a rate of agreement of more than two-thirds was achieved. Results: In the first round, the participants accepted 37 items of the 57 as important, reached no consensus on 20, and added 15 new items. In the second round, we added the comments from the first round to the items on which there was no consensus and added the 15 new items. After this round, a total of 72 items were addressed and, of these, 37 items were accepted and 34 were rejected due to lack of consensus. Conclusions: This study produced a list of 37 items that can form the basis of an evaluation tool to evaluate postgraduate medical e-learning. This is, to our knowledge, the first time that quality indicators for postgraduate medical e-learning have been defined and validated. The next step is to create and validate an e-learning evaluation tool from these items.

Original languageEnglish
Article numbere13
Pages (from-to)1-7
Number of pages7
JournalJournal of Medical Internet Research
Issue number1
Early online date8 Nov 2017
Publication statusPublished - Apr 2018


  • Continuing
  • Continuing medical education
  • Distance
  • Distance education
  • E-learning
  • Education
  • Medical
  • Postgraduate medical education
  • Quality indicators
  • Quality tool


Dive into the research topics of 'Consensus on quality indicators of postgraduate medical e-learning: Delphi study'. Together they form a unique fingerprint.

Cite this