Abstract
WordNet is a database that represents relations between words and concepts as an abstraction
of the contexts in which words are used. Contextualized language models represent words
in contexts but leave the underlying concepts implicit. In this paper, we investigate how dif-
ferent layers of a pre-trained language model shape the abstract lexical relationship toward
the actual contextual concept. Can we define the amount of contextualized concept forming needed given the abstracted representation of a word? Specifically, we consider samples of
words with different polysemy profiles shared across three languages, assuming that words
with a different polysemy profile require a different degree of concept shaping by context.
We conduct probing experiments to investigate the impact of prior polysemy profiles on the
representation in different layers. We analyze how contextualized models can approximate
meaning through context and examine crosslingual interference effects.
of the contexts in which words are used. Contextualized language models represent words
in contexts but leave the underlying concepts implicit. In this paper, we investigate how dif-
ferent layers of a pre-trained language model shape the abstract lexical relationship toward
the actual contextual concept. Can we define the amount of contextualized concept forming needed given the abstracted representation of a word? Specifically, we consider samples of
words with different polysemy profiles shared across three languages, assuming that words
with a different polysemy profile require a different degree of concept shaping by context.
We conduct probing experiments to investigate the impact of prior polysemy profiles on the
representation in different layers. We analyze how contextualized models can approximate
meaning through context and examine crosslingual interference effects.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 12th Global Wordnet Conference [GWC2023] |
| Subtitle of host publication | San Sebastian, Spain, January 23-27, 2023 |
| Publisher | ACL Anthology |
| Number of pages | 11 |
| Publication status | Published - 2023 |
