A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020

António Branco; Nicoletta Calzolari; Piek Vossen; Gertjan van Noord; Dieter van Uytvank; João Silva; Luís Gomes; André Moreira; Willem Elbers

A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020

António Branco
, Nicoletta Calzolari
, Piek Vossen
, Gertjan van Noord
, Dieter van Uytvank
, João Silva
, Luís Gomes
, André Moreira
, Willem Elbers

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

In this paper, we introduce a new type of shared task - which is collaborative rather than competitive - designed to support and foster the reproduction of research results. We also describe the first event running such a novel challenge, present the results obtained, discuss the lessons learned and ponder on future undertakings.

Original language	English
Title of host publication	LREC 2020
Subtitle of host publication	Proceedings of the 12th Conference on Language Resources and Evaluation
Editors	Nicoletta Calzolari, Frederic Bechet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Publisher	European Language Resources Association (ELRA)
Pages	5539-5545
Number of pages	7
ISBN (Electronic)	9791095546344
Publication status	Published - May 2020
Event	12th Language Resources and Evaluation Conference (LERC 2020) Marseille - Marseille, France Duration: 11 May 2020 → 16 May 2020 Conference number: 12

Conference

Conference	12th Language Resources and Evaluation Conference (LERC 2020) Marseille
Abbreviated title	LERC 2020
Country/Territory	France
City	Marseille
Period	11/05/20 → 16/05/20

Funding

We are very grateful to the authors that offered their papers, or accepted our invitation to do so, to be the target of the reproduction tasks. They are listed in Section 3. The results reported here were partially supported by PORTULAN CLARIN-Research Infrastructure for the Science and Technology of Language, funded by Lisboa 2020, Alentejo 2020 and FCT-Fundação para a Ciência e Tecnologia under the grant PINFRA/22117/2016. The computing environment for the replications by the technical committee has been kindly provided by EGI and the EOSC-hub H2020 project (grant agreement 777536) with the dedicated support of the CESGA, CESNET-MCC and RECAS-BARI providers. We are very grateful to the authors that offeredtheir papers, or accepted our invitationto do so, to be the target of the reproduction tasks. They are listed in Section 3. The results reported here were partially supported by PORTULAN CLARIN—Research Infrastructure for the Science and Technologyof Language, funded by Lisboa 2020, Alentejo 2020 and FCT—Funda¸cão para a Ciência e Tec-nologia under the grant PINFRA/22117/2016. The computing environment for the replications by the technical committee has been kindly provided by EGI and the EOSC-hub H2020 project (grant agreement 777536) with the dedicated support of the CESGA, CESNET-MCC and RECAS-BARI providers.

Funders	Funder number
CESNET-MCC
EOSC-hub H2020	777536
Funda¸cão para a Ciência e Tec-nologia
Science and Technology of Language
EGI
Fundação para a Ciência e a Tecnologia	PINFRA/22117/2016

Keywords

Computational linguistics
Language technology
Natural language processing
Replication
Reproduction

Access to Document

http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.680.pdfLicence: CC BY-NC

Persistent URL (handle)

Persistent link

Cite this

Branco, A., Calzolari, N., Vossen, P., van Noord, G., van Uytvank, D., Silva, J., Gomes, L., Moreira, A., & Elbers, W. (2020). A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020. In N. Calzolari, F. Bechet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, & S. Piperidis (Eds.), LREC 2020: Proceedings of the 12th Conference on Language Resources and Evaluation (pp. 5539-5545). European Language Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.680.pdf

Branco, António ; Calzolari, Nicoletta ; Vossen, Piek et al. / A shared task of a new, collaborative type to foster reproducibility : A first exercise in the area of language science and technology with REPROLANG2020. LREC 2020: Proceedings of the 12th Conference on Language Resources and Evaluation. editor / Nicoletta Calzolari ; Frederic Bechet ; Philippe Blache ; Khalid Choukri ; Christopher Cieri ; Thierry Declerck ; Sara Goggi ; Hitoshi Isahara ; Bente Maegaard ; Joseph Mariani ; Helene Mazo ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. European Language Resources Association (ELRA), 2020. pp. 5539-5545

@inproceedings{157f322ae99a4aecba453b2687ff47dc,

title = "A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020",

abstract = "In this paper, we introduce a new type of shared task - which is collaborative rather than competitive - designed to support and foster the reproduction of research results. We also describe the first event running such a novel challenge, present the results obtained, discuss the lessons learned and ponder on future undertakings.",

keywords = "Computational linguistics, Language technology, Natural language processing, Replication, Reproduction",

author = "Ant{\'o}nio Branco and Nicoletta Calzolari and Piek Vossen and \{van Noord\}, Gertjan and \{van Uytvank\}, Dieter and Jo{\~a}o Silva and Lu{\'i}s Gomes and Andr{\'e} Moreira and Willem Elbers",

year = "2020",

month = may,

language = "English",

pages = "5539--5545",

editor = "Nicoletta Calzolari and Frederic Bechet and Philippe Blache and Khalid Choukri and Christopher Cieri and Thierry Declerck and Sara Goggi and Hitoshi Isahara and Bente Maegaard and Joseph Mariani and Helene Mazo and Asuncion Moreno and Jan Odijk and Stelios Piperidis",

booktitle = "LREC 2020",

publisher = "European Language Resources Association (ELRA)",

note = "12th Language Resources and Evaluation Conference (LERC 2020) Marseille, LERC 2020 ; Conference date: 11-05-2020 Through 16-05-2020",

}

Branco, A, Calzolari, N, Vossen, P, van Noord, G, van Uytvank, D, Silva, J, Gomes, L, Moreira, A & Elbers, W 2020, A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020. in N Calzolari, F Bechet, P Blache, K Choukri, C Cieri, T Declerck, S Goggi, H Isahara, B Maegaard, J Mariani, H Mazo, A Moreno, J Odijk & S Piperidis (eds), LREC 2020: Proceedings of the 12th Conference on Language Resources and Evaluation. European Language Resources Association (ELRA), pp. 5539-5545, 12th Language Resources and Evaluation Conference (LERC 2020) Marseille, Marseille, France, 11/05/20. <http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.680.pdf>

A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020. / Branco, António; Calzolari, Nicoletta; Vossen, Piek et al.
LREC 2020: Proceedings of the 12th Conference on Language Resources and Evaluation. ed. / Nicoletta Calzolari; Frederic Bechet; Philippe Blache; Khalid Choukri; Christopher Cieri; Thierry Declerck; Sara Goggi; Hitoshi Isahara; Bente Maegaard; Joseph Mariani; Helene Mazo; Asuncion Moreno; Jan Odijk; Stelios Piperidis. European Language Resources Association (ELRA), 2020. p. 5539-5545.

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - A shared task of a new, collaborative type to foster reproducibility

T2 - 12th Language Resources and Evaluation Conference (LERC 2020) Marseille

AU - Branco, António

AU - Calzolari, Nicoletta

AU - Vossen, Piek

AU - van Noord, Gertjan

AU - van Uytvank, Dieter

AU - Silva, João

AU - Gomes, Luís

AU - Moreira, André

AU - Elbers, Willem

N1 - Conference code: 12

PY - 2020/5

Y1 - 2020/5

N2 - In this paper, we introduce a new type of shared task - which is collaborative rather than competitive - designed to support and foster the reproduction of research results. We also describe the first event running such a novel challenge, present the results obtained, discuss the lessons learned and ponder on future undertakings.

AB - In this paper, we introduce a new type of shared task - which is collaborative rather than competitive - designed to support and foster the reproduction of research results. We also describe the first event running such a novel challenge, present the results obtained, discuss the lessons learned and ponder on future undertakings.

KW - Computational linguistics

KW - Language technology

KW - Natural language processing

KW - Replication

KW - Reproduction

UR - https://www.scopus.com/pages/publications/85096516698

UR - https://www.scopus.com/pages/publications/85096516698#tab=citedBy

UR - http://www.lrec-conf.org/proceedings/lrec2020/index.html

M3 - Conference contribution

AN - SCOPUS:85096516698

SP - 5539

EP - 5545

BT - LREC 2020

A2 - Calzolari, Nicoletta

A2 - Bechet, Frederic

A2 - Blache, Philippe

A2 - Choukri, Khalid

A2 - Cieri, Christopher

A2 - Declerck, Thierry

A2 - Goggi, Sara

A2 - Isahara, Hitoshi

A2 - Maegaard, Bente

A2 - Mariani, Joseph

A2 - Mazo, Helene

A2 - Moreno, Asuncion

A2 - Odijk, Jan

A2 - Piperidis, Stelios

PB - European Language Resources Association (ELRA)

Y2 - 11 May 2020 through 16 May 2020

ER -

Branco A, Calzolari N, Vossen P, van Noord G, van Uytvank D, Silva J et al. A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020. In Calzolari N, Bechet F, Blache P, Choukri K, Cieri C, Declerck T, Goggi S, Isahara H, Maegaard B, Mariani J, Mazo H, Moreno A, Odijk J, Piperidis S, editors, LREC 2020: Proceedings of the 12th Conference on Language Resources and Evaluation. European Language Resources Association (ELRA). 2020. p. 5539-5545

A shared task of a new, collaborative type to foster reproducibility: A first exercise in the area of language science and technology with REPROLANG2020

Abstract

Conference

Funding

Keywords

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this