Predicting Quality of Crowdsourced Annotations using Graph Kernels

Archana Nottamkandath; Jasper Oosterman; Gerben Klaas Dirk de Vries; Davide Ceolin; Wan Fokkink

doi:10.1007/978-3-319-18491-3_10

Predicting Quality of Crowdsourced Annotations using Graph Kernels

Archana Nottamkandath, Jasper Oosterman, Gerben Klaas Dirk de Vries, Davide Ceolin, Wan Fokkink

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler- Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75%. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.

Original language	English
Title of host publication	Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings
Editors	Y. Murayama, T. Dimitrakos, C. D. Jensen, S. Marsh
Publisher	Springer New York
Pages	134-148
Number of pages	15
Volume	454
ISBN (Print)	9783319184906
DOIs	https://doi.org/10.1007/978-3-319-18491-3_10
Publication status	Published - 2015
Event	9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015 - Hamburg, Germany Duration: 26 May 2015 → 28 May 2015

Publication series

Name	IFIP Advances in Information and Communication Technology
Volume	454
ISSN (Print)	1868-4238

Conference

Conference	9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015
Country/Territory	Germany
City	Hamburg
Period	26/05/15 → 28/05/15

Keywords

Crowdsourcing
Machine learning
RDF graph Kernels
Trust

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-319-18491-3_10

Persistent URL (handle)

Persistent link

Cite this

Nottamkandath, A., Oosterman, J., de Vries, G. K. D., Ceolin, D., & Fokkink, W. (2015). Predicting Quality of Crowdsourced Annotations using Graph Kernels. In Y. Murayama, T. Dimitrakos, C. D. Jensen, & S. Marsh (Eds.), Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings (Vol. 454, pp. 134-148). (IFIP Advances in Information and Communication Technology; Vol. 454). Springer New York. https://doi.org/10.1007/978-3-319-18491-3_10

Nottamkandath, Archana ; Oosterman, Jasper ; de Vries, Gerben Klaas Dirk et al. / Predicting Quality of Crowdsourced Annotations using Graph Kernels. Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings. editor / Y. Murayama ; T. Dimitrakos ; C. D. Jensen ; S. Marsh. Vol. 454 Springer New York, 2015. pp. 134-148 (IFIP Advances in Information and Communication Technology).

@inproceedings{6d407813a20f4a6481a3dcede2f81514,

title = "Predicting Quality of Crowdsourced Annotations using Graph Kernels",

abstract = "Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler- Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75%. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.",

keywords = "Crowdsourcing, Machine learning, RDF graph Kernels, Trust",

author = "Archana Nottamkandath and Jasper Oosterman and {de Vries}, {Gerben Klaas Dirk} and Davide Ceolin and Wan Fokkink",

year = "2015",

doi = "10.1007/978-3-319-18491-3_10",

language = "English",

isbn = "9783319184906",

volume = "454",

series = "IFIP Advances in Information and Communication Technology",

publisher = "Springer New York",

pages = "134--148",

editor = "Y. Murayama and T. Dimitrakos and Jensen, {C. D.} and S. Marsh",

booktitle = "Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings",

address = "United States",

note = "9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015 ; Conference date: 26-05-2015 Through 28-05-2015",

}

Nottamkandath, A, Oosterman, J, de Vries, GKD, Ceolin, D & Fokkink, W 2015, Predicting Quality of Crowdsourced Annotations using Graph Kernels. in Y Murayama, T Dimitrakos, CD Jensen & S Marsh (eds), Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings. vol. 454, IFIP Advances in Information and Communication Technology, vol. 454, Springer New York, pp. 134-148, 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Hamburg, Germany, 26/05/15. https://doi.org/10.1007/978-3-319-18491-3_10

Predicting Quality of Crowdsourced Annotations using Graph Kernels. / Nottamkandath, Archana; Oosterman, Jasper; de Vries, Gerben Klaas Dirk et al.
Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings. ed. / Y. Murayama; T. Dimitrakos; C. D. Jensen; S. Marsh. Vol. 454 Springer New York, 2015. p. 134-148 (IFIP Advances in Information and Communication Technology; Vol. 454).

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Predicting Quality of Crowdsourced Annotations using Graph Kernels

AU - Nottamkandath, Archana

AU - Oosterman, Jasper

AU - de Vries, Gerben Klaas Dirk

AU - Ceolin, Davide

AU - Fokkink, Wan

PY - 2015

Y1 - 2015

N2 - Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler- Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75%. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.

AB - Annotations obtained by Cultural Heritage institutions from the crowd need to be automatically assessed for their quality. Machine learning using graph kernels is an effective technique to use structural information in datasets to make predictions. We employ the Weisfeiler- Lehman graph kernel for RDF to make predictions about the quality of crowdsourced annotations in Steve.museum dataset, which is modelled and enriched as RDF. Our results indicate that we could predict quality of crowdsourced annotations with an accuracy of 75%. We also employ the kernel to understand which features from the RDF graph are relevant to make predictions about different categories of quality.

KW - Crowdsourcing

KW - Machine learning

KW - RDF graph Kernels

KW - Trust

UR - http://www.scopus.com/inward/record.url?scp=84942812433&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84942812433&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-18491-3_10

DO - 10.1007/978-3-319-18491-3_10

M3 - Conference contribution

SN - 9783319184906

VL - 454

T3 - IFIP Advances in Information and Communication Technology

SP - 134

EP - 148

BT - Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings

A2 - Murayama, Y.

A2 - Dimitrakos, T.

A2 - Jensen, C. D.

A2 - Marsh, S.

PB - Springer New York

T2 - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015

Y2 - 26 May 2015 through 28 May 2015

ER -

Nottamkandath A, Oosterman J, de Vries GKD, Ceolin D, Fokkink W. Predicting Quality of Crowdsourced Annotations using Graph Kernels. In Murayama Y, Dimitrakos T, Jensen CD, Marsh S, editors, Trust Management IX - 9th IFIP Working Group 11.11 International Conference on Trust Management, IFIPTM 2015, Proceedings. Vol. 454. Springer New York. 2015. p. 134-148. (IFIP Advances in Information and Communication Technology). doi: 10.1007/978-3-319-18491-3_10

Predicting Quality of Crowdsourced Annotations using Graph Kernels

Abstract

Publication series

Conference

Keywords

UN SDGs

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this