Data Blocks: Hybrid OLTP and OLAP on compressed storage using both vectorization and compilation

Harald Lang, Tobias Mühlbauer, Florian Funke, Peter Boncz, Thomas Neumann, Alfons Kemper

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

This work aims at reducing the main-memory footprint in high performance hybrid OLTP&OLAP databases, while retaining high query performance and transactional throughput. For this purpose, an innovative compressed columnar storage format for cold data, called Data Blocks is introduced. Data Blocks further incorporate a new light-weight index structure called Positional SMA that narrows scan ranges within Data Blocks even if the entire block cannot be ruled out. To achieve highest OLTP performance, the compression schemes of Data Blocks are very light-weight, such that OLTP transactions can still quickly access individual tuples. This sets our storage scheme apart from those used in specialized analytical databases where data must usually be bit-unpacked. Up to now, high-performance analytical systems use either vectorized query execution or "just-in-time" (JIT) query compilation. The fine-grained adaptivity of Data Blocks necessitates the integration of the best features of each approach by an interpreted vectorized scan subsystem feeding into JIT-compiled query pipelines. Experimental evaluation of HyPer, our full-edged hybrid OLTP&OLAP database system, shows that Data Blocks accelerate performance on a variety of query workloads while retaining high transaction throughput.

Original languageEnglish
Title of host publicationSIGMOD 2016 - Proceedings of the 2016 International Conference on Management of Data
PublisherAssociation for Computing Machinery (ACM)
Pages311-326
Number of pages16
Volume26-June-2016
ISBN (Electronic)9781450335317
DOIs
Publication statusPublished - 26 Jun 2016
Event2016 ACM SIGMOD International Conference on Management of Data, SIGMOD 2016 - San Francisco, United States
Duration: 26 Jun 20161 Jul 2016

Conference

Conference2016 ACM SIGMOD International Conference on Management of Data, SIGMOD 2016
CountryUnited States
CitySan Francisco
Period26/06/161/07/16

    Fingerprint

Cite this

Lang, H., Mühlbauer, T., Funke, F., Boncz, P., Neumann, T., & Kemper, A. (2016). Data Blocks: Hybrid OLTP and OLAP on compressed storage using both vectorization and compilation. In SIGMOD 2016 - Proceedings of the 2016 International Conference on Management of Data (Vol. 26-June-2016, pp. 311-326). Association for Computing Machinery (ACM). https://doi.org/10.1145/2882903.2882925