MARVIN: Distributed reasoning over large-scale Semantic Web data

E. Oren, S. Kotoulas, G. Anadiotis, R.M. Siebes, A.C.M. ten Teije, F.A.H. van Harmelen

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Many Semantic Web problems are difficult to solve through common divide-and-conquer strategies, since they are hard to partition. We present Marvin, a parallel and distributed platform for processing large amounts of RDF data, on a network of loosely coupled peers. We present our divide-conquer-swap strategy and show that this model converges towards completeness. Within this strategy, we address the problem of making distributed reasoning scalable and load-balanced. We present SpeedDate, a routing strategy that combines data clustering with random exchanges. The random exchanges ensure load balancing, while the data clustering attempts to maximise efficiency. SpeedDate is compared against random and deterministic (DHT-like) approaches, on performance and load-balancing. We simulate parameters such as system size, data distribution, churn rate, and network topology. The results indicate that SpeedDate is near-optimally balanced, performs in the same order of magnitude as a DHT-like approach, and has an average throughput per node that scales with sqrt(i) for i items in the system. We evaluate our overall Marvin system for performance, scalability, load balancing and efficiency. © 2009 Elsevier B.V. All rights reserved.
Original languageEnglish
JournalJournal of Web Semantics
Volume7
Issue number4
DOIs
Publication statusPublished - 2009

Fingerprint

Dive into the research topics of 'MARVIN: Distributed reasoning over large-scale Semantic Web data'. Together they form a unique fingerprint.

Cite this