RStore: A Direct-Access DRAM-based Data Store

Animesh Trivedi, Patrick Stuedi, Bernard Metzler, Clemens Lutz, Martin Schmatz, Thomas R. Gross

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Distributed DRAM stores have become an attractive option for providing fast data accesses to analytics applications. To accelerate the performance of these stores, researchers have proposed using RDMA technology. RDMA offers high bandwidth and low latency data access by carefully separating resource setup from IO operations, and making IO operations fast by using rich network semantics and offloading. Despite recent interest, leveraging the full potential of RDMA in a distributed environment remains a challenging task. In this paper, we present RDMA Store or RStore, a DRAM-based data store that delivers high performance by extending RDMA's separation philosophy to a distributed setting. RStore achieves high aggregate bandwidth (705 Gb/s) and close-to-hardware latency on our 12-machine testbed. We developed a distributed graph processing framework and a Key-Value sorter using RStore's unique memory-like API. The graph processing framework, which relies on RStore for low-latency graph access, outperforms state-of-the-art systems by margins of 2.6-4.2× when calculating Page Rank. The Key-Value sorter can sort 256 GB of data in 31.7 sec, which is 8× better than Hadoop TeraSort in a similar setting.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE 35th International Conference on Distributed Computing Systems, ICDCS 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages674-685
Number of pages12
ISBN (Electronic)9781467372145
DOIs
Publication statusPublished - 22 Jul 2015
Externally publishedYes
Event35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015 - Columbus, United States
Duration: 29 Jun 20152 Jul 2015

Publication series

NameProceedings - International Conference on Distributed Computing Systems
Volume2015-July

Conference

Conference35th IEEE International Conference on Distributed Computing Systems, ICDCS 2015
CountryUnited States
CityColumbus
Period29/06/152/07/15

Keywords

  • Data processing
  • Data storage systems
  • Next generation networking

Fingerprint

Dive into the research topics of 'RStore: A Direct-Access DRAM-based Data Store'. Together they form a unique fingerprint.

Cite this