A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020

António Branco, Nicoletta Calzolari, Piek Vossen, Gertjan van Noord, Dieter van Uytvank, João Silva, Luís Gomes, André Moreira, Willem Elbers

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

In this paper, we introduce a new type of shared task - which is collaborative rather than competitive - designed to support and foster the reproduction of research results. We also describe the first event running such a novel challenge, present the results obtained, discuss the lessons learned and ponder on future undertakings.

Original languageEnglish
Title of host publicationLREC 2020
Subtitle of host publicationProceedings of the 12th Conference on Language Resources and Evaluation
EditorsNicoletta Calzolari, Frederic Bechet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
PublisherEuropean Language Resources Association (ELRA)
Pages5539-5545
Number of pages7
ISBN (Electronic)9791095546344
Publication statusPublished - May 2020
Event12th Language Resources and Evaluation Conference (LERC 2020) Marseille - Marseille, France
Duration: 11 May 202016 May 2020
Conference number: 12

Conference

Conference12th Language Resources and Evaluation Conference (LERC 2020) Marseille
Abbreviated titleLERC 2020
Country/TerritoryFrance
CityMarseille
Period11/05/2016/05/20

Funding

We are very grateful to the authors that offeredtheir papers, or accepted our invitationto do so, to be the target of the reproduction tasks. They are listed in Section 3. The results reported here were partially supported by PORTULAN CLARIN—Research Infrastructure for the Science and Technologyof Language, funded by Lisboa 2020, Alentejo 2020 and FCT—Funda¸cão para a Ciência e Tec-nologia under the grant PINFRA/22117/2016. The computing environment for the replications by the technical committee has been kindly provided by EGI and the EOSC-hub H2020 project (grant agreement 777536) with the dedicated support of the CESGA, CESNET-MCC and RECAS-BARI providers.

FundersFunder number
CESNET-MCC
EOSC-hub H2020777536
Funda¸cão para a Ciência e Tec-nologia
Science and Technology of Language
EGI
Fundação para a Ciência e a TecnologiaPINFRA/22117/2016

    Keywords

    • Computational linguistics
    • Language technology
    • Natural language processing
    • Replication
    • Reproduction

    Fingerprint

    Dive into the research topics of 'A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020'. Together they form a unique fingerprint.

    Cite this