Time-based tags for fiction movies: comparing experts to novices using a video labeling game

Liliana Melgar Estrada, Michiel Hildebrand, Victor de Boer, Jacco van Ossenbruggen

Abstract

The cultural heritage sector has embraced social tagging as a way to increase both access to online content and to engage users with their digital collections. In this article, we build on two current lines of research. (a) We use Waisda?, an existing labeling game, to add time-based annotations to content. (b) In this context, we investigate the role of experts in human-based computation (nichesourcing). We report on a small-scale experiment in which we applied Waisda? to content from film archives. We study the differences in the type of time-based tags between experts and novices for film clips in a crowdsourcing setting. The findings show high similarity in the number and type of tags (mostly factual). In the less frequent tags, however, experts used more domain-specific terms. We conclude that competitive games are not suited to elicit real expert-level descriptions. We also confirm that providing guidelines, based on conceptual frameworks that are more suited to moving images in a time-based fashion, could result in increasing the quality of the tags, thus allowing for creating more tag-based innovative services for online audiovisual heritage.

Original languageEnglish
Pages (from-to)348-364
Number of pages17
JournalJournal of the Association for Information Science and Technology
Volume68
Issue number2
DOIs
StatePublished - 1 Feb 2017

Cite this

Estrada, Liliana Melgar; Hildebrand, Michiel; de Boer, Victor; van Ossenbruggen, Jacco / Time-based tags for fiction movies : comparing experts to novices using a video labeling game.

In: Journal of the Association for Information Science and Technology, Vol. 68, No. 2, 01.02.2017, p. 348-364.

Research output: Scientific - peer-reviewArticle

@article{1d3d6ec06af0465ca246a544dbb1236b,
title = "Time-based tags for fiction movies: comparing experts to novices using a video labeling game",
abstract = "The cultural heritage sector has embraced social tagging as a way to increase both access to online content and to engage users with their digital collections. In this article, we build on two current lines of research. (a) We use Waisda?, an existing labeling game, to add time-based annotations to content. (b) In this context, we investigate the role of experts in human-based computation (nichesourcing). We report on a small-scale experiment in which we applied Waisda? to content from film archives. We study the differences in the type of time-based tags between experts and novices for film clips in a crowdsourcing setting. The findings show high similarity in the number and type of tags (mostly factual). In the less frequent tags, however, experts used more domain-specific terms. We conclude that competitive games are not suited to elicit real expert-level descriptions. We also confirm that providing guidelines, based on conceptual frameworks that are more suited to moving images in a time-based fashion, could result in increasing the quality of the tags, thus allowing for creating more tag-based innovative services for online audiovisual heritage.",
author = "Estrada, {Liliana Melgar} and Michiel Hildebrand and {de Boer}, Victor and {van Ossenbruggen}, Jacco",
year = "2017",
month = "2",
doi = "10.1002/asi.23656",
volume = "68",
pages = "348--364",
journal = "Journal of the Association for Information Science and Technology",
issn = "2330-1635",
publisher = "John Wiley and Sons Ltd",
number = "2",

}

Time-based tags for fiction movies : comparing experts to novices using a video labeling game. / Estrada, Liliana Melgar; Hildebrand, Michiel; de Boer, Victor; van Ossenbruggen, Jacco.

In: Journal of the Association for Information Science and Technology, Vol. 68, No. 2, 01.02.2017, p. 348-364.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - Time-based tags for fiction movies

T2 - Journal of the Association for Information Science and Technology

AU - Estrada,Liliana Melgar

AU - Hildebrand,Michiel

AU - de Boer,Victor

AU - van Ossenbruggen,Jacco

PY - 2017/2/1

Y1 - 2017/2/1

N2 - The cultural heritage sector has embraced social tagging as a way to increase both access to online content and to engage users with their digital collections. In this article, we build on two current lines of research. (a) We use Waisda?, an existing labeling game, to add time-based annotations to content. (b) In this context, we investigate the role of experts in human-based computation (nichesourcing). We report on a small-scale experiment in which we applied Waisda? to content from film archives. We study the differences in the type of time-based tags between experts and novices for film clips in a crowdsourcing setting. The findings show high similarity in the number and type of tags (mostly factual). In the less frequent tags, however, experts used more domain-specific terms. We conclude that competitive games are not suited to elicit real expert-level descriptions. We also confirm that providing guidelines, based on conceptual frameworks that are more suited to moving images in a time-based fashion, could result in increasing the quality of the tags, thus allowing for creating more tag-based innovative services for online audiovisual heritage.

AB - The cultural heritage sector has embraced social tagging as a way to increase both access to online content and to engage users with their digital collections. In this article, we build on two current lines of research. (a) We use Waisda?, an existing labeling game, to add time-based annotations to content. (b) In this context, we investigate the role of experts in human-based computation (nichesourcing). We report on a small-scale experiment in which we applied Waisda? to content from film archives. We study the differences in the type of time-based tags between experts and novices for film clips in a crowdsourcing setting. The findings show high similarity in the number and type of tags (mostly factual). In the less frequent tags, however, experts used more domain-specific terms. We conclude that competitive games are not suited to elicit real expert-level descriptions. We also confirm that providing guidelines, based on conceptual frameworks that are more suited to moving images in a time-based fashion, could result in increasing the quality of the tags, thus allowing for creating more tag-based innovative services for online audiovisual heritage.

UR - http://www.scopus.com/inward/record.url?scp=84959562117&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959562117&partnerID=8YFLogxK

U2 - 10.1002/asi.23656

DO - 10.1002/asi.23656

M3 - Article

VL - 68

SP - 348

EP - 364

JO - Journal of the Association for Information Science and Technology

JF - Journal of the Association for Information Science and Technology

SN - 2330-1635

IS - 2

ER -