WebPIE: A Web-scale parallel inference engine using MapReduce: A Web-scale Parallel Inference Engine using MapReduce

J. Urbani, S. Kotoulas, J. Maassen, F.A.H. van Harmelen, H.E. Bal

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

The large amount of Semantic Web data and its fast growth pose a significant computational challenge in performing efficient and scalable reasoning. On a large scale, the resources of single machines are no longer sufficient and we are required to distribute the process to improve performance. In this article, we propose a distributed technique to perform materialization under the RDFS and OWL ter Horst semantics using the MapReduce programming model. We will show that a straightforward implementation is not efficient and does not scale. Our technique addresses the challenge of distributed reasoning through a set of algorithms which, combined, significantly increase performance. We have implemented WebPIE (Web-scale Inference Engine) and we demonstrate its performance on a cluster of up to 64 nodes. We have evaluated our system using very large real-world datasets (Bio2RDF, LLD, LDSR) and the LUBM synthetic benchmark, scaling up to 100 billion triples. Results show that our implementation scales linearly and vastly outperforms current systems in terms of maximum data size and inference speed.

Original languageEnglish
Pages (from-to)59-75
Number of pages17
JournalJournal of Web Semantics
Volume10
DOIs
Publication statusPublished - 1 Jan 2012

Keywords

  • Distributed computing
  • High performance
  • MapReduce
  • Reasoning
  • Semantic Web

Fingerprint

Dive into the research topics of 'WebPIE: A Web-scale parallel inference engine using MapReduce: A Web-scale Parallel Inference Engine using MapReduce'. Together they form a unique fingerprint.

Cite this