Evaluating FAIR maturity through a scalable, automated, community-governed framework

Mark D. Wilkinson, Michel Dumontier, Susanna Assunta Sansone, Luiz Olavo Bonino da Silva Santos, Mario Prieto, Dominique Batista, Peter McQuilton, Tobias Kuhn, Philippe Rocca-Serra, Mercѐ Crosas, Erik Schultes

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Transparent evaluations of FAIRness are increasingly required by a wide range of stakeholders, from scientists to publishers, funding agencies and policy makers. We propose a scalable, automatable framework to evaluate digital resources that encompasses measurable indicators, open source tools, and participation guidelines, which come together to accommodate domain relevant community-defined FAIR assessments. The components of the framework are: (1) Maturity Indicators - community-authored specifications that delimit a specific automatically-measurable FAIR behavior; (2) Compliance Tests - small Web apps that test digital resources against individual Maturity Indicators; and (3) the Evaluator, a Web application that registers, assembles, and applies community-relevant sets of Compliance Tests against a digital resource, and provides a detailed report about what a machine "sees" when it visits that resource. We discuss the technical and social considerations of FAIR assessments, and how this translates to our community-driven infrastructure. We then illustrate how the output of the Evaluator tool can serve as a roadmap to assist data stewards to incrementally and realistically improve the FAIRness of their resources.

Original languageEnglish
Article number174
Pages (from-to)1-12
Number of pages12
JournalScientific Data
Volume6
Issue number1
DOIs
Publication statusPublished - 20 Sep 2019

Fingerprint

Acceptance tests
maturity
Resources
Application programs
resources
community
Fairness
Compliance
fairness
Specifications
Web Application
Open Source
Infrastructure
funding
stakeholder
Community
Framework
Maturity
infrastructure
Specification

Cite this

Wilkinson, M. D., Dumontier, M., Sansone, S. A., Bonino da Silva Santos, L. O., Prieto, M., Batista, D., ... Schultes, E. (2019). Evaluating FAIR maturity through a scalable, automated, community-governed framework. Scientific Data, 6(1), 1-12. [174]. https://doi.org/10.1038/s41597-019-0184-5
Wilkinson, Mark D. ; Dumontier, Michel ; Sansone, Susanna Assunta ; Bonino da Silva Santos, Luiz Olavo ; Prieto, Mario ; Batista, Dominique ; McQuilton, Peter ; Kuhn, Tobias ; Rocca-Serra, Philippe ; Crosas, Mercѐ ; Schultes, Erik. / Evaluating FAIR maturity through a scalable, automated, community-governed framework. In: Scientific Data. 2019 ; Vol. 6, No. 1. pp. 1-12.
@article{df573a651f5b4709b9afd0dc53ce73b6,
title = "Evaluating FAIR maturity through a scalable, automated, community-governed framework",
abstract = "Transparent evaluations of FAIRness are increasingly required by a wide range of stakeholders, from scientists to publishers, funding agencies and policy makers. We propose a scalable, automatable framework to evaluate digital resources that encompasses measurable indicators, open source tools, and participation guidelines, which come together to accommodate domain relevant community-defined FAIR assessments. The components of the framework are: (1) Maturity Indicators - community-authored specifications that delimit a specific automatically-measurable FAIR behavior; (2) Compliance Tests - small Web apps that test digital resources against individual Maturity Indicators; and (3) the Evaluator, a Web application that registers, assembles, and applies community-relevant sets of Compliance Tests against a digital resource, and provides a detailed report about what a machine {"}sees{"} when it visits that resource. We discuss the technical and social considerations of FAIR assessments, and how this translates to our community-driven infrastructure. We then illustrate how the output of the Evaluator tool can serve as a roadmap to assist data stewards to incrementally and realistically improve the FAIRness of their resources.",
author = "Wilkinson, {Mark D.} and Michel Dumontier and Sansone, {Susanna Assunta} and {Bonino da Silva Santos}, {Luiz Olavo} and Mario Prieto and Dominique Batista and Peter McQuilton and Tobias Kuhn and Philippe Rocca-Serra and Mercѐ Crosas and Erik Schultes",
year = "2019",
month = "9",
day = "20",
doi = "10.1038/s41597-019-0184-5",
language = "English",
volume = "6",
pages = "1--12",
journal = "Scientific Data",
issn = "2052-4463",
publisher = "Nature Publishing Group",
number = "1",

}

Wilkinson, MD, Dumontier, M, Sansone, SA, Bonino da Silva Santos, LO, Prieto, M, Batista, D, McQuilton, P, Kuhn, T, Rocca-Serra, P, Crosas, M & Schultes, E 2019, 'Evaluating FAIR maturity through a scalable, automated, community-governed framework' Scientific Data, vol. 6, no. 1, 174, pp. 1-12. https://doi.org/10.1038/s41597-019-0184-5

Evaluating FAIR maturity through a scalable, automated, community-governed framework. / Wilkinson, Mark D.; Dumontier, Michel; Sansone, Susanna Assunta; Bonino da Silva Santos, Luiz Olavo; Prieto, Mario; Batista, Dominique; McQuilton, Peter; Kuhn, Tobias; Rocca-Serra, Philippe; Crosas, Mercѐ; Schultes, Erik.

In: Scientific Data, Vol. 6, No. 1, 174, 20.09.2019, p. 1-12.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - Evaluating FAIR maturity through a scalable, automated, community-governed framework

AU - Wilkinson, Mark D.

AU - Dumontier, Michel

AU - Sansone, Susanna Assunta

AU - Bonino da Silva Santos, Luiz Olavo

AU - Prieto, Mario

AU - Batista, Dominique

AU - McQuilton, Peter

AU - Kuhn, Tobias

AU - Rocca-Serra, Philippe

AU - Crosas, Mercѐ

AU - Schultes, Erik

PY - 2019/9/20

Y1 - 2019/9/20

N2 - Transparent evaluations of FAIRness are increasingly required by a wide range of stakeholders, from scientists to publishers, funding agencies and policy makers. We propose a scalable, automatable framework to evaluate digital resources that encompasses measurable indicators, open source tools, and participation guidelines, which come together to accommodate domain relevant community-defined FAIR assessments. The components of the framework are: (1) Maturity Indicators - community-authored specifications that delimit a specific automatically-measurable FAIR behavior; (2) Compliance Tests - small Web apps that test digital resources against individual Maturity Indicators; and (3) the Evaluator, a Web application that registers, assembles, and applies community-relevant sets of Compliance Tests against a digital resource, and provides a detailed report about what a machine "sees" when it visits that resource. We discuss the technical and social considerations of FAIR assessments, and how this translates to our community-driven infrastructure. We then illustrate how the output of the Evaluator tool can serve as a roadmap to assist data stewards to incrementally and realistically improve the FAIRness of their resources.

AB - Transparent evaluations of FAIRness are increasingly required by a wide range of stakeholders, from scientists to publishers, funding agencies and policy makers. We propose a scalable, automatable framework to evaluate digital resources that encompasses measurable indicators, open source tools, and participation guidelines, which come together to accommodate domain relevant community-defined FAIR assessments. The components of the framework are: (1) Maturity Indicators - community-authored specifications that delimit a specific automatically-measurable FAIR behavior; (2) Compliance Tests - small Web apps that test digital resources against individual Maturity Indicators; and (3) the Evaluator, a Web application that registers, assembles, and applies community-relevant sets of Compliance Tests against a digital resource, and provides a detailed report about what a machine "sees" when it visits that resource. We discuss the technical and social considerations of FAIR assessments, and how this translates to our community-driven infrastructure. We then illustrate how the output of the Evaluator tool can serve as a roadmap to assist data stewards to incrementally and realistically improve the FAIRness of their resources.

UR - http://www.scopus.com/inward/record.url?scp=85072522270&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072522270&partnerID=8YFLogxK

U2 - 10.1038/s41597-019-0184-5

DO - 10.1038/s41597-019-0184-5

M3 - Article

VL - 6

SP - 1

EP - 12

JO - Scientific Data

JF - Scientific Data

SN - 2052-4463

IS - 1

M1 - 174

ER -

Wilkinson MD, Dumontier M, Sansone SA, Bonino da Silva Santos LO, Prieto M, Batista D et al. Evaluating FAIR maturity through a scalable, automated, community-governed framework. Scientific Data. 2019 Sep 20;6(1):1-12. 174. https://doi.org/10.1038/s41597-019-0184-5