Abstract
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018) |
| Editors | Hitoshi Isahara, Bente Maegaard, Stelios Piperidis, Christopher Cieri, Thierry Declerck, Koiti Hasida, Helene Mazo, Khalid Choukri, Sara Goggi, Joseph Mariani, Asuncion Moreno, Nicoletta Calzolari, Jan Odijk, Takenobu Tokunaga |
| Place of Publication | Miyazaki |
| Publisher | European Language Resources Association (ELRA) |
| Pages | 1101-1111 |
| Number of pages | 11 |
| ISBN (Electronic) | 9791095546009 |
| Publication status | Published - 2018 |
| Event | 11th International Conference on Language Resources and Evaluation, LREC 2018 - Miyazaki, Japan Duration: 7 May 2018 → 12 May 2018 |
Conference
| Conference | 11th International Conference on Language Resources and Evaluation, LREC 2018 |
|---|---|
| Country | Japan |
| City | Miyazaki |
| Period | 7/05/18 → 12/05/18 |
Fingerprint
Keywords
- Annotation consistency
- Events
- Resource interoperability
Cite this
}
Resource Interoperability for Sustainable Benchmarking: The Case of Events : The case of events. / van Son, C.M.; Inel, O.A.; Morante Vallejo, R.; Aroyo, L.M.; Vossen, P.T.J.M.
Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018). ed. / Hitoshi Isahara; Bente Maegaard; Stelios Piperidis; Christopher Cieri; Thierry Declerck; Koiti Hasida; Helene Mazo; Khalid Choukri; Sara Goggi; Joseph Mariani; Asuncion Moreno; Nicoletta Calzolari; Jan Odijk; Takenobu Tokunaga. Miyazaki : European Language Resources Association (ELRA), 2018. p. 1101-1111.Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review
TY - GEN
T1 - Resource Interoperability for Sustainable Benchmarking: The Case of Events
T2 - The case of events
AU - van Son, C.M.
AU - Inel, O.A.
AU - Morante Vallejo, R.
AU - Aroyo, L.M.
AU - Vossen, P.T.J.M.
PY - 2018
Y1 - 2018
N2 - With the continuous growth of benchmark corpora, which often annotate the same documents, there is a range of opportunities to compare and combine similar and complementary annotations. However, these opportunities are hampered by a wide range of problems that are related to the lack of resource interoperability. In this paper, we illustrate these problems by assessing aspects of interoperability at the document-level across a set of 20 corpora annotated with (aspects of) events. The issues range from applying different document naming conventions, to mismatches in textual content and structural/conceptual differences among annotation schemes. We provide insight into the exact document intersections between the corpora by mapping their document identifiers and perform an empirical analysis of event annotations showing their compatibility and consistency in and across the corpora. This way, we aim to make the community more aware of the challenges and opportunities and to inspire working collaboratively towards interoperable resources.
AB - With the continuous growth of benchmark corpora, which often annotate the same documents, there is a range of opportunities to compare and combine similar and complementary annotations. However, these opportunities are hampered by a wide range of problems that are related to the lack of resource interoperability. In this paper, we illustrate these problems by assessing aspects of interoperability at the document-level across a set of 20 corpora annotated with (aspects of) events. The issues range from applying different document naming conventions, to mismatches in textual content and structural/conceptual differences among annotation schemes. We provide insight into the exact document intersections between the corpora by mapping their document identifiers and perform an empirical analysis of event annotations showing their compatibility and consistency in and across the corpora. This way, we aim to make the community more aware of the challenges and opportunities and to inspire working collaboratively towards interoperable resources.
KW - Annotation consistency
KW - Events
KW - Resource interoperability
UR - http://www.scopus.com/inward/record.url?scp=85059886152&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85059886152&partnerID=8YFLogxK
UR - http://lrec2018.lrec-conf.org/en/
M3 - Conference contribution
SP - 1101
EP - 1111
BT - Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018)
A2 - Isahara, Hitoshi
A2 - Maegaard, Bente
A2 - Piperidis, Stelios
A2 - Cieri, Christopher
A2 - Declerck, Thierry
A2 - Hasida, Koiti
A2 - Mazo, Helene
A2 - Choukri, Khalid
A2 - Goggi, Sara
A2 - Mariani, Joseph
A2 - Moreno, Asuncion
A2 - Calzolari, Nicoletta
A2 - Odijk, Jan
A2 - Tokunaga, Takenobu
PB - European Language Resources Association (ELRA)
CY - Miyazaki
ER -