Toward Measuring the Resemblance of Embedding Models for Evolving Ontologies

Romana Pernisch, Daniele Dell'aglio, Abraham Bernstein

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review


Updates on ontologies affect the operations built on top of them. But not all changes are equal: some updates drastically change the result of operations; others lead to minor variations, if any. Hence, estimating the impact of a change ex-ante is highly important, as it might make ontology engineers aware of the consequences of their action during editing. However, in order to estimate the impact of changes, we need to understand how to measure them. To address this gap for embeddings, we propose a new measure called Embedding Resemblance Indicator (ERI), which takes into account both the stochasticity of learning embeddings as well as the shortcomings of established comparison methods. We base ERI on (i) a similarity score, (ii) a robustness factor $\hatμ $ (based on the embedding method, similarity measure, and dataset), and (iii) the number of added or deleted entities to the embedding computed with the Jaccard index. To evaluate ERI, we investigate its usage in the context of two biomedical ontologies and three embedding methods - -GraRep, LINE, and DeepWalk - -as well as the two standard benchmark datasets - -FB15k-237 and Wordnet-18-RR - -with TransE and RESCAL embeddings. To study different aspects of ERI, we introduce synthetic changes in the knowledge graphs, generating two test-cases with five versions each and compare their impact with the expected behaviour. Our studies suggests that ERI behaves as expected and captures the similarity of embeddings based on the severity of changes. ERI is crucial for enabling further studies into impact of changes on embeddings.

Original languageEnglish
Title of host publicationK-CAP 2021
Subtitle of host publicationProceedings of the 11th Knowledge Capture Conference
PublisherAssociation for Computing Machinery, Inc
Number of pages8
ISBN (Electronic)9781450384575
Publication statusPublished - Dec 2021
Event11th ACM International Conference on Knowledge Capture, K-CAP 2021 - Virtual, Online, United States
Duration: 2 Dec 20213 Dec 2021


Conference11th ACM International Conference on Knowledge Capture, K-CAP 2021
Country/TerritoryUnited States
CityVirtual, Online

Bibliographical note

Funding Information:
We thank the Swiss National Science Foundation for their partial support under contract number 407550_167177. This work was also partially funded by Elsevier’s Discovery Lab.

Publisher Copyright:
© 2021 ACM.


  • embedding similarity
  • knowledge graph embeddings
  • ontology evolution


Dive into the research topics of 'Toward Measuring the Resemblance of Embedding Models for Evolving Ontologies'. Together they form a unique fingerprint.

Cite this