Evaluating unsupervised thesaurus-based labeling of audiovisual content in an archive production environment

V. de Boer, Roeland J. Ordelman, Josefien Schuurman

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

In this paper we report on a two-stage evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results for given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion is sufficiently high. We furthermore provide an analysis of the term extraction after being taken into production, where we focus on performance variation with respect to term types and television programs. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and to also assess the effect of the transformation from manual to automatic annotation from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.
LanguageEnglish
Pages189-201
Number of pages13
JournalInternational Journal on Digital Libraries
Volume17
Issue number3
DOIs
Publication statusPublished - 1 Sep 2016

Fingerprint

thesaurus
television program
workflow
evaluation
video
performance

Keywords

  • Audiovisual access
  • Audiovisual archives
  • Information extraction
  • Practice-oriented evaluation
  • Thesaurus

Cite this

@article{7619281c07b443ddaddffafef865a2db,
title = "Evaluating unsupervised thesaurus-based labeling of audiovisual content in an archive production environment",
abstract = "In this paper we report on a two-stage evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results for given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion is sufficiently high. We furthermore provide an analysis of the term extraction after being taken into production, where we focus on performance variation with respect to term types and television programs. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and to also assess the effect of the transformation from manual to automatic annotation from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.",
keywords = "Audiovisual access, Audiovisual archives, Information extraction, Practice-oriented evaluation, Thesaurus",
author = "{de Boer}, V. and Ordelman, {Roeland J.} and Josefien Schuurman",
year = "2016",
month = "9",
day = "1",
doi = "10.1007/s00799-016-0182-6",
language = "English",
volume = "17",
pages = "189--201",
journal = "International Journal on Digital Libraries",
issn = "1432-5012",
publisher = "Springer",
number = "3",

}

Evaluating unsupervised thesaurus-based labeling of audiovisual content in an archive production environment. / de Boer, V.; Ordelman, Roeland J.; Schuurman, Josefien.

In: International Journal on Digital Libraries, Vol. 17, No. 3, 01.09.2016, p. 189-201.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - Evaluating unsupervised thesaurus-based labeling of audiovisual content in an archive production environment

AU - de Boer, V.

AU - Ordelman, Roeland J.

AU - Schuurman, Josefien

PY - 2016/9/1

Y1 - 2016/9/1

N2 - In this paper we report on a two-stage evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results for given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion is sufficiently high. We furthermore provide an analysis of the term extraction after being taken into production, where we focus on performance variation with respect to term types and television programs. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and to also assess the effect of the transformation from manual to automatic annotation from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.

AB - In this paper we report on a two-stage evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results for given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion is sufficiently high. We furthermore provide an analysis of the term extraction after being taken into production, where we focus on performance variation with respect to term types and television programs. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and to also assess the effect of the transformation from manual to automatic annotation from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.

KW - Audiovisual access

KW - Audiovisual archives

KW - Information extraction

KW - Practice-oriented evaluation

KW - Thesaurus

UR - http://www.scopus.com/inward/record.url?scp=84976331378&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84976331378&partnerID=8YFLogxK

U2 - 10.1007/s00799-016-0182-6

DO - 10.1007/s00799-016-0182-6

M3 - Article

VL - 17

SP - 189

EP - 201

JO - International Journal on Digital Libraries

T2 - International Journal on Digital Libraries

JF - International Journal on Digital Libraries

SN - 1432-5012

IS - 3

ER -