Abstract
LSTM-based language models have been shown effective in Word Sense Disambiguation
(WSD). In particular, the technique proposed by Yuan et al. (2016) returned state-of-the-art performance
in several benchmarks, but neither the training data nor the source code was released.
This paper presents the results of a reproduction study and analysis of this technique using only
openly available datasets (GigaWord, SemCor, OMSTI) and software (TensorFlow). Our study
showed that similar results can be obtained with much less data than hinted at by Yuan et al.
(2016). Detailed analyses shed light on the strengths and weaknesses of this method. First,
adding more unannotated training data is useful, but is subject to diminishing returns. Second,
the model can correctly identify both popular and unpopular meanings. Finally, the limited sense
coverage in the annotated datasets is a major limitation. All code and trained models are made
freely available.
(WSD). In particular, the technique proposed by Yuan et al. (2016) returned state-of-the-art performance
in several benchmarks, but neither the training data nor the source code was released.
This paper presents the results of a reproduction study and analysis of this technique using only
openly available datasets (GigaWord, SemCor, OMSTI) and software (TensorFlow). Our study
showed that similar results can be obtained with much less data than hinted at by Yuan et al.
(2016). Detailed analyses shed light on the strengths and weaknesses of this method. First,
adding more unannotated training data is useful, but is subject to diminishing returns. Second,
the model can correctly identify both popular and unpopular meanings. Finally, the limited sense
coverage in the annotated datasets is a major limitation. All code and trained models are made
freely available.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the International Conference on Computational Linguistics (COLING 2018) |
| Publisher | International Conference on Computational Linguistics (COLING) |
| Pages | 354-365 |
| Number of pages | 12 |
| ISBN (Print) | 9781948087506 |
| Publication status | Published - 2018 |
| Event | 27th International Conference on Computational Linguistics COLING 2018 - Santa Fe, NM Duration: 20 Aug 2018 → 26 Aug 2018 Conference number: 27 |
Conference
| Conference | 27th International Conference on Computational Linguistics COLING 2018 |
|---|---|
| Abbreviated title | COLING 2018 |
| City | Santa Fe, NM |
| Period | 20/08/18 → 26/08/18 |