Funding, evaluation, and the performance of national research systems

Ulf Sandström, Peter Van den Besselaar

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Understanding the quality of science systems requires international comparative studies, which are difficult because of the lack of comparable data especially about inputs in research. In this study, we deploy an approach based on change instead of on levels of inputs and outputs: an approach that to a large extent eliminates the problem of measurement differences between countries. We firstly show that there are large differences in efficiency between national science systems, defined as the increase in output (highly cited papers) per percentage increase in input (funding). We then discuss our findings using popular explanations of performance differences: differences in funding systems (performance related or not), differences in the level of competition, differences in the level of university autonomy, and differences in the level of academic freedom. Interestingly, the available data do not support these common explanations. What the data suggest is that efficient systems are characterized by a well-developed ex post evaluation system combined with considerably high institutional funding and relatively low university autonomy (meaning a high autonomy of professionals). On the other hand, the less efficient systems have a strong ex ante control, either through a high level of so-called competitive project funding, or through strong power of the university management. Another conclusion is that more and better data are needed.

Original languageEnglish
Pages (from-to)365-384
Number of pages20
JournalJournal of Informetrics
Volume12
Issue number1
Early online date22 Feb 2018
DOIs
Publication statusPublished - Feb 2018

Fingerprint

Systems science
funding
evaluation
autonomy
performance
university
international system
science
efficiency
lack
management

Keywords

  • Bibliometrics
  • Citations
  • Input-output studies
  • Performance-based funding
  • Research efficiency
  • Research policy

Cite this

@article{793b44a5fa284ea19820aafdc5297fa2,
title = "Funding, evaluation, and the performance of national research systems",
abstract = "Understanding the quality of science systems requires international comparative studies, which are difficult because of the lack of comparable data especially about inputs in research. In this study, we deploy an approach based on change instead of on levels of inputs and outputs: an approach that to a large extent eliminates the problem of measurement differences between countries. We firstly show that there are large differences in efficiency between national science systems, defined as the increase in output (highly cited papers) per percentage increase in input (funding). We then discuss our findings using popular explanations of performance differences: differences in funding systems (performance related or not), differences in the level of competition, differences in the level of university autonomy, and differences in the level of academic freedom. Interestingly, the available data do not support these common explanations. What the data suggest is that efficient systems are characterized by a well-developed ex post evaluation system combined with considerably high institutional funding and relatively low university autonomy (meaning a high autonomy of professionals). On the other hand, the less efficient systems have a strong ex ante control, either through a high level of so-called competitive project funding, or through strong power of the university management. Another conclusion is that more and better data are needed.",
keywords = "Bibliometrics, Citations, Input-output studies, Performance-based funding, Research efficiency, Research policy",
author = "Ulf Sandstr{\"o}m and {Van den Besselaar}, Peter",
year = "2018",
month = "2",
doi = "10.1016/j.joi.2018.01.007",
language = "English",
volume = "12",
pages = "365--384",
journal = "Journal of Informetrics",
issn = "1751-1577",
publisher = "Elsevier BV",
number = "1",

}

Funding, evaluation, and the performance of national research systems. / Sandström, Ulf; Van den Besselaar, Peter.

In: Journal of Informetrics, Vol. 12, No. 1, 02.2018, p. 365-384.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - Funding, evaluation, and the performance of national research systems

AU - Sandström, Ulf

AU - Van den Besselaar, Peter

PY - 2018/2

Y1 - 2018/2

N2 - Understanding the quality of science systems requires international comparative studies, which are difficult because of the lack of comparable data especially about inputs in research. In this study, we deploy an approach based on change instead of on levels of inputs and outputs: an approach that to a large extent eliminates the problem of measurement differences between countries. We firstly show that there are large differences in efficiency between national science systems, defined as the increase in output (highly cited papers) per percentage increase in input (funding). We then discuss our findings using popular explanations of performance differences: differences in funding systems (performance related or not), differences in the level of competition, differences in the level of university autonomy, and differences in the level of academic freedom. Interestingly, the available data do not support these common explanations. What the data suggest is that efficient systems are characterized by a well-developed ex post evaluation system combined with considerably high institutional funding and relatively low university autonomy (meaning a high autonomy of professionals). On the other hand, the less efficient systems have a strong ex ante control, either through a high level of so-called competitive project funding, or through strong power of the university management. Another conclusion is that more and better data are needed.

AB - Understanding the quality of science systems requires international comparative studies, which are difficult because of the lack of comparable data especially about inputs in research. In this study, we deploy an approach based on change instead of on levels of inputs and outputs: an approach that to a large extent eliminates the problem of measurement differences between countries. We firstly show that there are large differences in efficiency between national science systems, defined as the increase in output (highly cited papers) per percentage increase in input (funding). We then discuss our findings using popular explanations of performance differences: differences in funding systems (performance related or not), differences in the level of competition, differences in the level of university autonomy, and differences in the level of academic freedom. Interestingly, the available data do not support these common explanations. What the data suggest is that efficient systems are characterized by a well-developed ex post evaluation system combined with considerably high institutional funding and relatively low university autonomy (meaning a high autonomy of professionals). On the other hand, the less efficient systems have a strong ex ante control, either through a high level of so-called competitive project funding, or through strong power of the university management. Another conclusion is that more and better data are needed.

KW - Bibliometrics

KW - Citations

KW - Input-output studies

KW - Performance-based funding

KW - Research efficiency

KW - Research policy

UR - http://www.scopus.com/inward/record.url?scp=85042382770&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042382770&partnerID=8YFLogxK

U2 - 10.1016/j.joi.2018.01.007

DO - 10.1016/j.joi.2018.01.007

M3 - Article

VL - 12

SP - 365

EP - 384

JO - Journal of Informetrics

JF - Journal of Informetrics

SN - 1751-1577

IS - 1

ER -