Combining User Reputation and Provenance Analysis for Trust Assessment

D. Ceolin, P.T. Groth, V. Maccatrozzo, W.J. Fokkink, W.R. van Hage, A. Nottamkandath

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Trust is a broad concept that in many systems is often reduced to user reputation alone. However, user reputation is just one way to determine trust. The estimation of trust can be tackled from other perspectives as well, including by looking at provenance. Here, we present a complete pipeline for estimating the trustworthiness of artifacts given their provenance and a set of sample evaluations. The pipeline is composed of a series of algorithms for (1) extracting relevant provenance features, (2) generating stereotypes of user behavior from provenance features, (3) estimating the reputation of both stereotypes and users, (4) using a combination of user and stereotype reputations to estimate the trustworthiness of artifacts and (5) selecting sets of artifacts to trust. These algorithms rely on the W3C PROV recommendations for provenance and on evidential reasoning by means of subjective logic. We evaluate the pipeline over two tagging datasets: tags and evaluations from the Netherlands Institute for Sound and Vision's Waisda? video tagging platform, as well as crowdsourced annotations from the Steve. Museum project. The approach achieves up to 85% precision when predicting tag trustworthiness. Perhaps more importantly, the pipeline provides satisfactory results using relatively little evidence through the use of provenance.
Original languageEnglish
Article number6
Pages (from-to)1-28
Number of pages28
JournalACM Journal of Data and Information Quality
Volume7
Issue number1-2
DOIs
Publication statusPublished - 2016

Fingerprint

Pipelines
Museums
Acoustic waves
Stereotypes
Trustworthiness
Tagging
Tag
Evaluation

Cite this

@article{16aab33cbc174e47964fc2cbca20d960,
title = "Combining User Reputation and Provenance Analysis for Trust Assessment",
abstract = "Trust is a broad concept that in many systems is often reduced to user reputation alone. However, user reputation is just one way to determine trust. The estimation of trust can be tackled from other perspectives as well, including by looking at provenance. Here, we present a complete pipeline for estimating the trustworthiness of artifacts given their provenance and a set of sample evaluations. The pipeline is composed of a series of algorithms for (1) extracting relevant provenance features, (2) generating stereotypes of user behavior from provenance features, (3) estimating the reputation of both stereotypes and users, (4) using a combination of user and stereotype reputations to estimate the trustworthiness of artifacts and (5) selecting sets of artifacts to trust. These algorithms rely on the W3C PROV recommendations for provenance and on evidential reasoning by means of subjective logic. We evaluate the pipeline over two tagging datasets: tags and evaluations from the Netherlands Institute for Sound and Vision's Waisda? video tagging platform, as well as crowdsourced annotations from the Steve. Museum project. The approach achieves up to 85{\%} precision when predicting tag trustworthiness. Perhaps more importantly, the pipeline provides satisfactory results using relatively little evidence through the use of provenance.",
author = "D. Ceolin and P.T. Groth and V. Maccatrozzo and W.J. Fokkink and {van Hage}, W.R. and A. Nottamkandath",
year = "2016",
doi = "10.1145/2818382",
language = "English",
volume = "7",
pages = "1--28",
journal = "ACM Journal of Data and Information Quality",
issn = "1936-1955",
publisher = "Association for Computing Machinery (ACM)",
number = "1-2",

}

Combining User Reputation and Provenance Analysis for Trust Assessment. / Ceolin, D.; Groth, P.T.; Maccatrozzo, V.; Fokkink, W.J.; van Hage, W.R.; Nottamkandath, A.

In: ACM Journal of Data and Information Quality, Vol. 7, No. 1-2, 6, 2016, p. 1-28.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - Combining User Reputation and Provenance Analysis for Trust Assessment

AU - Ceolin, D.

AU - Groth, P.T.

AU - Maccatrozzo, V.

AU - Fokkink, W.J.

AU - van Hage, W.R.

AU - Nottamkandath, A.

PY - 2016

Y1 - 2016

N2 - Trust is a broad concept that in many systems is often reduced to user reputation alone. However, user reputation is just one way to determine trust. The estimation of trust can be tackled from other perspectives as well, including by looking at provenance. Here, we present a complete pipeline for estimating the trustworthiness of artifacts given their provenance and a set of sample evaluations. The pipeline is composed of a series of algorithms for (1) extracting relevant provenance features, (2) generating stereotypes of user behavior from provenance features, (3) estimating the reputation of both stereotypes and users, (4) using a combination of user and stereotype reputations to estimate the trustworthiness of artifacts and (5) selecting sets of artifacts to trust. These algorithms rely on the W3C PROV recommendations for provenance and on evidential reasoning by means of subjective logic. We evaluate the pipeline over two tagging datasets: tags and evaluations from the Netherlands Institute for Sound and Vision's Waisda? video tagging platform, as well as crowdsourced annotations from the Steve. Museum project. The approach achieves up to 85% precision when predicting tag trustworthiness. Perhaps more importantly, the pipeline provides satisfactory results using relatively little evidence through the use of provenance.

AB - Trust is a broad concept that in many systems is often reduced to user reputation alone. However, user reputation is just one way to determine trust. The estimation of trust can be tackled from other perspectives as well, including by looking at provenance. Here, we present a complete pipeline for estimating the trustworthiness of artifacts given their provenance and a set of sample evaluations. The pipeline is composed of a series of algorithms for (1) extracting relevant provenance features, (2) generating stereotypes of user behavior from provenance features, (3) estimating the reputation of both stereotypes and users, (4) using a combination of user and stereotype reputations to estimate the trustworthiness of artifacts and (5) selecting sets of artifacts to trust. These algorithms rely on the W3C PROV recommendations for provenance and on evidential reasoning by means of subjective logic. We evaluate the pipeline over two tagging datasets: tags and evaluations from the Netherlands Institute for Sound and Vision's Waisda? video tagging platform, as well as crowdsourced annotations from the Steve. Museum project. The approach achieves up to 85% precision when predicting tag trustworthiness. Perhaps more importantly, the pipeline provides satisfactory results using relatively little evidence through the use of provenance.

U2 - 10.1145/2818382

DO - 10.1145/2818382

M3 - Article

VL - 7

SP - 1

EP - 28

JO - ACM Journal of Data and Information Quality

JF - ACM Journal of Data and Information Quality

SN - 1936-1955

IS - 1-2

M1 - 6

ER -