Assessing the ‘Gold Standard’ of party policy placements: Is computerized replication possible?

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

The Comparative Manifestos Project (CMP) offers data on party policy positions based on a coding scheme of 56 categories. To what extent can we replicate the CMP coding results by using computerized topical coding of digitalised manifestos? Both human and computer coding have a number of strengths and weaknesses so that the combination of strengths could help to arrive at both valid and reliable party policy placements, in particular the measurement of policy movements over time. More than 1500 digitalised party manifestos in 20 democracies in the period 1960-2009 are re-coded with a computerized CMP-compatible coding scheme in order to assess the differences with human coding using exactly the same issue categories and the same left-right scale.The analysis shows that it is possible to use computer coding in order to locate the potential weakness of the human coding and the other way around. It also illustrates that the validity and reliability of policy placements is a function of the conceptualisation and operationalisation of issues, of the size of documents and of scale construction. Computerized cross-validation of the CMP-coding results offers a new and powerful tool to assess its reliability. © 2011 Elsevier Ltd.
Original languageEnglish
Pages (from-to)561-570
JournalElectoral Studies
Volume30
Issue number3
DOIs
Publication statusPublished - 2011

Fingerprint

gold standard
coding
scale construction
operationalization
democracy

Cite this

@article{456d102f2b4242ff856809c531a2da6d,
title = "Assessing the ‘Gold Standard’ of party policy placements: Is computerized replication possible?",
abstract = "The Comparative Manifestos Project (CMP) offers data on party policy positions based on a coding scheme of 56 categories. To what extent can we replicate the CMP coding results by using computerized topical coding of digitalised manifestos? Both human and computer coding have a number of strengths and weaknesses so that the combination of strengths could help to arrive at both valid and reliable party policy placements, in particular the measurement of policy movements over time. More than 1500 digitalised party manifestos in 20 democracies in the period 1960-2009 are re-coded with a computerized CMP-compatible coding scheme in order to assess the differences with human coding using exactly the same issue categories and the same left-right scale.The analysis shows that it is possible to use computer coding in order to locate the potential weakness of the human coding and the other way around. It also illustrates that the validity and reliability of policy placements is a function of the conceptualisation and operationalisation of issues, of the size of documents and of scale construction. Computerized cross-validation of the CMP-coding results offers a new and powerful tool to assess its reliability. {\circledC} 2011 Elsevier Ltd.",
author = "P. Pennings",
year = "2011",
doi = "10.1016/j.electstud.2011.05.001",
language = "English",
volume = "30",
pages = "561--570",
journal = "Electoral Studies",
issn = "0261-3794",
publisher = "Elsevier BV",
number = "3",

}

Assessing the ‘Gold Standard’ of party policy placements: Is computerized replication possible? / Pennings, P.

In: Electoral Studies, Vol. 30, No. 3, 2011, p. 561-570.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - Assessing the ‘Gold Standard’ of party policy placements: Is computerized replication possible?

AU - Pennings, P.

PY - 2011

Y1 - 2011

N2 - The Comparative Manifestos Project (CMP) offers data on party policy positions based on a coding scheme of 56 categories. To what extent can we replicate the CMP coding results by using computerized topical coding of digitalised manifestos? Both human and computer coding have a number of strengths and weaknesses so that the combination of strengths could help to arrive at both valid and reliable party policy placements, in particular the measurement of policy movements over time. More than 1500 digitalised party manifestos in 20 democracies in the period 1960-2009 are re-coded with a computerized CMP-compatible coding scheme in order to assess the differences with human coding using exactly the same issue categories and the same left-right scale.The analysis shows that it is possible to use computer coding in order to locate the potential weakness of the human coding and the other way around. It also illustrates that the validity and reliability of policy placements is a function of the conceptualisation and operationalisation of issues, of the size of documents and of scale construction. Computerized cross-validation of the CMP-coding results offers a new and powerful tool to assess its reliability. © 2011 Elsevier Ltd.

AB - The Comparative Manifestos Project (CMP) offers data on party policy positions based on a coding scheme of 56 categories. To what extent can we replicate the CMP coding results by using computerized topical coding of digitalised manifestos? Both human and computer coding have a number of strengths and weaknesses so that the combination of strengths could help to arrive at both valid and reliable party policy placements, in particular the measurement of policy movements over time. More than 1500 digitalised party manifestos in 20 democracies in the period 1960-2009 are re-coded with a computerized CMP-compatible coding scheme in order to assess the differences with human coding using exactly the same issue categories and the same left-right scale.The analysis shows that it is possible to use computer coding in order to locate the potential weakness of the human coding and the other way around. It also illustrates that the validity and reliability of policy placements is a function of the conceptualisation and operationalisation of issues, of the size of documents and of scale construction. Computerized cross-validation of the CMP-coding results offers a new and powerful tool to assess its reliability. © 2011 Elsevier Ltd.

U2 - 10.1016/j.electstud.2011.05.001

DO - 10.1016/j.electstud.2011.05.001

M3 - Article

VL - 30

SP - 561

EP - 570

JO - Electoral Studies

JF - Electoral Studies

SN - 0261-3794

IS - 3

ER -