Understanding ephemeral storage for serverless analytics

Ana Klimovic, Yawen Wang, Christos Kozyrakis, Patrick Stuedi, Jonas Pfefferle, Animesh Trivedi

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Serverless computing frameworks allow users to launch thousands of concurrent tasks with high elasticity and fine-grain resource billing without explicitly managing computing resources. While already successful for IoT and web microservices, there is increasing interest in leveraging serverless computing to run data-intensive jobs, such as interactive analytics. A key challenge in running analytics workloads on serverless platforms is enabling tasks in different execution stages to efficiently communicate data between each other via a shared data store. In this paper, we explore the suitability of different cloud storage services (e.g., object stores and distributed caches) as remote storage for serverless analytics. Our analysis leads to key insights to guide the design of an ephemeral cloud storage system, including the performance and cost efficiency of Flash storage for serverless application requirements and the need for a pay-what-you-use storage service that can support the high throughput demands of highly parallel applications.

Original languageEnglish
Title of host publicationProceedings of the 2018 USENIX Annual Technical Conference, USENIX ATC 2018
PublisherUSENIX Association
Pages789-794
Number of pages6
ISBN (Electronic)9781939133021
Publication statusPublished - 1 Jan 2020
Externally publishedYes
Event2018 USENIX Annual Technical Conference, USENIX ATC 2018 - Boston, United States
Duration: 11 Jul 201813 Jul 2018

Publication series

NameProceedings of the 2018 USENIX Annual Technical Conference, USENIX ATC 2018

Conference

Conference2018 USENIX Annual Technical Conference, USENIX ATC 2018
Country/TerritoryUnited States
CityBoston
Period11/07/1813/07/18

Funding

We thank the ATC reviewers for their feedback. We thank Sadjad Fouladi and Qian Li for the insightful technical discussions. This work is supported by the Stanford Platform Lab, Samsung and Huawei. Ana Klimovic is supported by a Stanford Graduate Fellowship.

Fingerprint

Dive into the research topics of 'Understanding ephemeral storage for serverless analytics'. Together they form a unique fingerprint.

Cite this