Abstract
Serverless computing frameworks allow users to launch thousands of concurrent tasks with high elasticity and fine-grain resource billing without explicitly managing computing resources. While already successful for IoT and web microservices, there is increasing interest in leveraging serverless computing to run data-intensive jobs, such as interactive analytics. A key challenge in running analytics workloads on serverless platforms is enabling tasks in different execution stages to efficiently communicate data between each other via a shared data store. In this paper, we explore the suitability of different cloud storage services (e.g., object stores and distributed caches) as remote storage for serverless analytics. Our analysis leads to key insights to guide the design of an ephemeral cloud storage system, including the performance and cost efficiency of Flash storage for serverless application requirements and the need for a pay-what-you-use storage service that can support the high throughput demands of highly parallel applications.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2018 USENIX Annual Technical Conference, USENIX ATC 2018 |
Publisher | USENIX Association |
Pages | 789-794 |
Number of pages | 6 |
ISBN (Electronic) | 9781939133021 |
Publication status | Published - 1 Jan 2020 |
Externally published | Yes |
Event | 2018 USENIX Annual Technical Conference, USENIX ATC 2018 - Boston, United States Duration: 11 Jul 2018 → 13 Jul 2018 |
Publication series
Name | Proceedings of the 2018 USENIX Annual Technical Conference, USENIX ATC 2018 |
---|
Conference
Conference | 2018 USENIX Annual Technical Conference, USENIX ATC 2018 |
---|---|
Country/Territory | United States |
City | Boston |
Period | 11/07/18 → 13/07/18 |
Funding
We thank the ATC reviewers for their feedback. We thank Sadjad Fouladi and Qian Li for the insightful technical discussions. This work is supported by the Stanford Platform Lab, Samsung and Huawei. Ana Klimovic is supported by a Stanford Graduate Fellowship.