Efficient Estimation of Read Density when Caching for Big Data Processing

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Big data processing systems are becoming increasingly more present in cloud workloads. Consequently, they are starting to incorporate more sophisticated mechanisms from traditional database and distributed systems. We focus in this work on the use of caching policies, which for big data raise important new challenges. Not only they must respond to new variants of the trade-off between hit rate, response time, and the space consumed by the cache, but they must do so at possibly higher volume and velocity than web and database workloads. Previous caching policies have not been tested experimentally with big data workloads. We address these challenges in this work. We propose the Read Density family of policies, which is a principled approach to quantify the utility of cached objects through a family of utility functions that depend on the frequency of reads of an object. We further design the Approximate Histogram, which is a policy-based technique based on an array of counters. This technique promises to achieve runtime-space efficient computation of the metric required by the cache policy. We evaluate through trace-based simulation the caching policies from the Read Density family, and compare them with over ten state-of-the-art alternatives. We use two workload traces representative for big data processing, collected from commercial Spark and MapReduce deployments. While we achieve comparable performance to the state-of-art with less parameters, meaningful performance improvement for big data workloads remain elusive.

Original languageEnglish
Title of host publicationINFOCOM 2019 - IEEE Conference on Computer Communications Workshops, INFOCOM WKSHPS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages502-507
Number of pages6
ISBN (Electronic)9781728118789
DOIs
Publication statusPublished - Apr 2019
Event2019 INFOCOM IEEE Conference on Computer Communications Workshops, INFOCOM WKSHPS 2019 - Paris, France
Duration: 29 Apr 20192 May 2019

Conference

Conference2019 INFOCOM IEEE Conference on Computer Communications Workshops, INFOCOM WKSHPS 2019
Country/TerritoryFrance
CityParis
Period29/04/192/05/19

Fingerprint

Dive into the research topics of 'Efficient Estimation of Read Density when Caching for Big Data Processing'. Together they form a unique fingerprint.

Cite this