Reconciliation of inconsistent data sources by correction for measurement error: The feasibility of parameter re-use

P.K.P. Pankowska, Bart F.M. Bakker, D.L. Oberski, D. Pavlopoulos

Research output: Contribution to JournalArticleAcademicpeer-review

403 Downloads (Pure)

Abstract

National Statistical Institutes (NSIs) often obtain information about a single variable from separate data sources. Administrative registers and surveys, in particular, often provide overlapping information on a range of phenomena of interest to official statistics. However, even though the two sources overlap, they both contain measurement error that prevents identical units from yielding identical values. Reconciling such separate data sources and providing accurate statistics, which is an important challenge for NSIs, is typically achieved through macro-integration. In this study we investigate the feasibility of an alternative method based on the application of previously obtained results from a recently introduced extension of the Hidden Markov Model (HMM) to newer data. The method allows a reconciliation of separate error-prone data sources without having to repeat the full HMM analysis, provided the estimated measurement error processes are stable over time. As we find that these processes are indeed stable over time, the proposed method can be used effectively for macro-integration, to reconciliate both first-order statistics-e.g. the size of temporary employment in the Netherlands-and second-order statistics-e.g. the amount of mobility from temporary to permanent employment.

Original languageEnglish
Pages (from-to)317-329
Number of pages13
JournalStatistical Journal of the IAOS
Volume34
Issue number3
Early online date2017
DOIs
Publication statusPublished - 9 Aug 2018

Funding

The first author acknowledges the contribution of Statistics Netherlands for financing her PhD project and for making the data available for this research. The authors also thank Harry Ganzeboom and all other members of the SILC research group of the Vrije Uni-versiteit Amsterdam for reviewing the paper and providing valuable comments and constructive feedback.

FundersFunder number
PhD

    Keywords

    • administrative data
    • data quality
    • Hidden Markov Model
    • labour market transitions
    • measurement error
    • register data
    • survey data

    VU Research Profile

    • Governance for Society

    Fingerprint

    Dive into the research topics of 'Reconciliation of inconsistent data sources by correction for measurement error: The feasibility of parameter re-use'. Together they form a unique fingerprint.

    Cite this