GEval: A Modular and Extensible Evaluation Framework for Graph Embedding Techniques

Maria Angela Pellegrino*, Abdulrahman Altabba, Martina Garofalo, Petar Ristoski, Michael Cochez

*Corresponding author for this work

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

66 Downloads (Pure)


While RDF data are graph shaped by nature, most traditional Machine Learning (ML) algorithms expect data in a vector form. To transform graph elements to vectors, several graph embedding approaches have been proposed. Comparing these approaches is interesting for 1) developers of new embedding techniques to verify in which cases their proposal outperforms the state-of-art and 2) consumers of these techniques in choosing the best approach according to the task(s) the vectors will be used for. The comparison could be delayed (and made difficult) by the choice of tasks, the design of the evaluation, the selection of models, parameters, and needed datasets. We propose GEval, an evaluation framework to simplify the evaluation and the comparison of graph embedding techniques. The covered tasks range from ML tasks (Classification, Regression, Clustering), semantic tasks (entity relatedness, document similarity) to semantic analogies. However, GEval is designed to be (easily) extensible. In this article, we will describe the design and development of the proposed framework by detailing its overall structure, the already implemented tasks, and how to extend it. In conclusion, to demonstrate its operating approach, we consider the parameter tuning of the KGloVe algorithm as a use case.

Original languageEnglish
Title of host publicationThe Semantic Web
Subtitle of host publication17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31–June 4, 2020, Proceedings
EditorsAndreas Harth, Sabrina Kirrane, Axel-Cyrille Ngonga Ngomo, Heiko Paulheim, Anisa Rula, Anna Lisa Gentile, Peter Haase, Michael Cochez
Number of pages18
ISBN (Electronic)9783030494612
ISBN (Print)9783030494605
Publication statusPublished - 2020
Event17th Extended Semantic Web Conference, ESWC 2020 - Heraklion, Greece
Duration: 31 May 20204 Jun 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12123 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference17th Extended Semantic Web Conference, ESWC 2020


  • Evaluation framework
  • Knowledge graph embedding
  • Machine Learning
  • Semantic tasks


Dive into the research topics of 'GEval: A Modular and Extensible Evaluation Framework for Graph Embedding Techniques'. Together they form a unique fingerprint.

Cite this