Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3542929.3563464acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article
Public Access

Cypress: input size-sensitive container provisioning and request scheduling for serverless platforms

Published: 07 November 2022 Publication History

Abstract

The growing popularity of the serverless platform has seen an increase in the number and variety of applications (apps) being deployed on it. The majority of these apps process user-provided input to produce the desired results. Existing work in the area of input-sensitive profiling has empirically shown that many such apps have input size-dependent execution times which can be determined through modelling techniques. Nevertheless, existing serverless resource management frameworks are agnostic to the input size-sensitive nature of these apps. We demonstrate in this paper that this can potentially lead to container over-provisioning and/or end-to-end Service Level Objective (SLO) violations. To address this, we propose Cypress, an input size-sensitive resource management framework, that minimizes the containers provisioned for apps, while ensuring a high degree of SLO compliance. We perform an extensive evaluation of Cypress on top of a Kubernetes-managed cluster using 5 apps from the AWS Serverless Application Repository and/or Open-FaaS Function Store with real-world traces and varied input size distributions. Our experimental results show that Cypress spawns up to 66% fewer containers, thereby, improving container utilization and saving cluster-wide energy by up to 2.95X and 23%, respectively, versus state-of-the-art frameworks, while remaining highly SLO-compliant (up to 99.99%).

References

[1]
2020. Email Length Best Practices. https://www.campaignmonitor.com/blog/email-marketing/email-length-best-practices-for-email-marketers-and-email-newbies/.
[2]
2020. Establishing Effective SLOs. https://www.datadoghq.com/blog/establishing-service-level-objectives/.
[3]
2020. Findings about Search Engine Optimization. https://backlinko.com/search-engine-ranking.
[4]
2020. Twitter Stream traces. https://archive.org/details/twitterstream. Accessed: 2020-05-07.
[5]
2021. AWS Lambda Cold Starts. https://mikhail.io/serverless/coldstarts/aws/.
[6]
2021. Azure Functions Cold Starts. https://mikhail.io/serverless/coldstarts/azure/.
[7]
2021. Image Compression with Lambda. https://dev.to/aarongarvey/size-matters-image-compression-with-lambda-and-s3-40bf.
[8]
2022. Amazon Automated Chat Moderation. https://aws.amazon.com/blogs/business-productivity/automated-moderation-and-sentiment-analysis-with-amazon-chime-sdk-messaging/.
[9]
2022. AWS Lambda. https://aws.amazon.com/lambda/.
[10]
2022. AWS Lambda for Sentiment Analysis. https://aws.amazon.com/blogs/compute/using-aws-lambda-and-amazon-comprehend-for-sentiment-analysis/.
[11]
2022. AWS Serverless Application Repository. https://aws.amazon.com/serverless/serverlessrepo/.
[12]
2022. Google Cloud Functions. https://cloud.google.com/functions/docs/.
[13]
2022. HappyScribe: Online Audio Transaltion Service. https://www.happyscribe.com/.
[14]
2022. Intel CPU Energy Meter. https://github.com/sosy-lab/cpu-energy-meter.
[15]
2022. Kubernetes. https://kubernetes.io/.
[16]
2022. Microsoft Azure Serverless Functions. https://azure.microsoft.com/en-us/services/functions/.
[17]
2022. OpenFaaS. https://www.openfaas.com/.
[18]
2022. OpenFaaS Function Store. https://github.com/openfaas/store.
[19]
2022. Provisioned Concurrency. https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html.
[20]
2022. Sentiment Analysis of Social Media. https://docs.aws.amazon.com/whitepapers/latest/big-data-analytics-options/example-3-sentiment-analysis-of-social-media.html.
[21]
2022. Serverless Speech to Text. https://awscloudfeed.com/whats-new/videos/serverless-speech-to-text-using-s3-and-lambda.
[22]
2022. The State of Serverless. https://www.datadoghq.com/state-of-serverless/.
[23]
2022. Translating documents with Amazon Translate and AWS Lambda. https://aws.amazon.com/blogs/machine-learning/translating-documents-with-amazon-translate-aws-lambda-and-the-new-batch-translate-api/.
[24]
Istemi Ekin Akkus et al. 2018. SAND: Towards High-Performance Serverless Computing. In ATC.
[25]
A. Alourani, M.A.N. Bikas, and M. Grechanik. 2016. Input-Sensitive Profiling: A Survey. Advances in Computers, Vol. 103. Elsevier, 31--52.
[26]
Daniel S. Berger, Benjamin Berg, Timothy Zhu, Siddhartha Sen, and Mor Harchol-Balter. 2018. RobinHood: Tail Latency Aware Caching - Dynamic Reallocation from Cache-Rich to Cache-Poor. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 195--212. https://www.usenix.org/conference/osdi18/presentation/berger
[27]
Vivek M. Bhasi, Jashwant Raj Gunasekaran, Prashanth Thinakaran, Cyan Subhra Mishra, Mahmut Taylan Kandemir, and Chita Das. 2021. Kraken: Adaptive Container Provisioning for Deploying Dynamic DAGs in Serverless Platforms. In Proceedings of the ACM Symposium on Cloud Computing (Seattle, WA, USA) (SoCC '21). Association for Computing Machinery, New York, NY, USA, 153--167.
[28]
Marc Brooker, Andreea Florescu, Diana-Maria Popa, Rolf Neugebauer, Alexandru Agache, Alexandra Iordache, Anthony Liguori, and Phil Piwonka. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In NSDI.
[29]
James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, and Jonathan Appavoo. 2020. SEUSS: skip redundant paths to make serverless fast. In Proceedings of the Fifteenth European Conference on Computer Systems. 1--15.
[30]
Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2019. Cirrus: A Serverless Framework for End-to-End ML Workflows. In Proceedings of the ACM Symposium on Cloud Computing (Santa Cruz, CA, USA) (SoCC '19). Association for Computing Machinery, New York, NY, USA, 13--24.
[31]
Emilio Coppa, Camil Demetrescu, and Irene Finocchi. 2012. Input-Sensitive Profiling. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (Beijing, China) (PLDI '12). Association for Computing Machinery, New York, NY, USA, 89--98.
[32]
Nilanjan Daw, Umesh Bellur, and Purushottam Kulkarni. 2020. Xanadu: Mitigating cascading cold starts in serverless function chain deployments. In Proceedings of the 21st International Middleware Conference. 356--370.
[33]
Jeffrey Dean and Luiz André Barroso. 2013. The Tail at Scale. Commun. ACM 56, 2 (feb 2013), 74--80.
[34]
Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. 2007. Dynamo: Amazon's Highly Available Key-Value Store. In Proceedings of Twenty-First ACM SIGOPS Symposium on Operating Systems Principles (Stevenson, Washington, USA) (SOSP '07). Association for Computing Machinery, New York, NY, USA, 205--220.
[35]
Simon Eismann, Johannes Grohmann, Erwin van Eyk, Nikolas Herbst, and Samuel Kounev. 2020. Predicting the Costs of Serverless Workflows. In Proceedings of the ACM/SPEC International Conference on Performance Engineering (Edmonton AB, Canada) (ICPE '20). Association for Computing Machinery, New York, NY, USA, 265--276.
[36]
Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, Shuvo Chatterjee, Christos Kozyrakis, Matei Zaharia, and Keith Winstein. 2019. From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). USENIX Association, Renton, WA, 475--488. http://www.usenix.org/conference/atc19/presentation/fouladi
[37]
Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 363--376. https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/fouladi
[38]
Simon F. Goldsmith, Alex S. Aiken, and Daniel S. Wilkerson. 2007. Measuring Empirical Computational Complexity. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering (Dubrovnik, Croatia) (ESEC-FSE '07). Association for Computing Machinery, New York, NY, USA, 395--404.
[39]
Jashwant Raj Gunasekaran, Prashanth Thinakaran, Mahmut Taylan Kandemir, Bhuvan Urgaonkar, George Kesidis, and Chita Das. 2019. Spock: Exploiting Serverless Functions for SLO and Cost Aware Resource Procurement in Public Cloud. In 2019 IEEE 12th International Conference on Cloud Computing (CLOUD). 199--208.
[40]
Jashwant Raj Gunasekaran, Prashanth Thinakaran, Nachiappan C Nachiappan, Mahmut Taylan Kandemir, and Chita R Das. 2020. Fifer: Tackling Resource Underutilization in the Serverless Era. In Proceedings of the 21st International Middleware Conference. 280--295.
[41]
Ling Huang, Jinzhu Jia, Bin Yu, Byung-gon Chun, Petros Maniatis, and Mayur Naik. 2010. Predicting Execution Time of Computer Programs Using Sparse Polynomial Regression. In Advances in Neural Information Processing Systems (NeurIPS), J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta (Eds.), Vol. 23. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2010/file/995665640dc319973d3173a74a03860c-Paper.pdf
[42]
David S Johnson. 1973. Near-Optimal Bin Packing Algorithms. Ph.D. Dissertation. Massachusetts Institute of Technology.
[43]
Ram Srivatsa Kannan, Lavanya Subramanian, Ashwin Raju, Jeongseob Ahn, Jason Mars, and Lingjia Tang. 2019. GrandSLAm: Guaranteeing SLAs for Jobs in Microservices Execution Frameworks. In EuroSys.
[44]
Kate Keahey, Jason Anderson, Zhuo Zhen, Pierre Riteau, Paul Ruth, Dan Stanzione, Mert Cevik, Jacob Colleran, Haryadi S. Gunawi, Cody Hammock, Joe Mambretti, Alexander Barnes, François Halbach, Alex Rocha, and Joe Stubbs. 2020. Lessons Learned from the Chameleon Testbed. In Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC '20). USENIX Association.
[45]
Bernhard Korte and Jens Vygen. 2018. Bin-Packing. In Combinatorial Optimization. Springer, 489--507.
[46]
Amirhossein Mirhosseini, Brendan L. West, Geoffrey W. Blake, and Thomas F. Wenisch. 2020. Q-Zilla: A Scheduling Framework and Core Microarchitecture for Tail-Tolerant Microservices. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). 207--219.
[47]
Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Naren Nayak, and Vadim Sukhomlinov. 2019. Agile cold starts for scalable serverless. In 11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 19).
[48]
Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. In USENIX ATC.
[49]
Ruth Rettie and Lisa Chittenden. 2003. Email marketing: Success factors. Kingston Business School, Kingston University.
[50]
Johann Schleier-Smith, Vikram Sreekanti, Anurag Khandelwal, Joao Carreira, Neeraja J. Yadwadkar, Raluca Ada Popa, Joseph E. Gonzalez, Ion Stoica, and David A. Patterson. 2021. What Serverless Computing is and Should Become: The next Phase of Cloud Computing. Commun. ACM 64, 5 (apr 2021), 76--84.
[51]
Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 205--218. https://www.usenix.org/conference/atc20/presentation/shahrad
[52]
Aakash Sharma, Saravanan Dhakshinamurthy, George Kesidis, and Chita R.Das. 2021. CASH: A Credit Aware Scheduling for Public Cloud Platforms. In 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid). 227--236.
[53]
Christy Sich. 2017. A Comparison of Traditional Book Reviews and Amazon.com Book Reviews of Fiction Using a Content Analysis Approach. Evidence Based Library and Information Practice 12, 1 (Mar. 2017), 85--96.
[54]
Paulo Silva, Daniel Fireman, and Thiago Emmanuel Pereira. 2020. Prebaking Functions to Warm the Serverless Cold Start. In Proceedings of the 21st International Middleware Conference. 1--13.
[55]
Arjun Singhvi, Arjun Balasubramanian, Kevin Houck, Mohammed Danish Shaikh, Shivaram Venkataraman, and Aditya Akella. 2021. Atoll: A Scalable Low-Latency Serverless Platform. In Proceedings of the ACM Symposium on Cloud Computing (Seattle, WA, USA) (SoCC '21). Association for Computing Machinery, New York, NY, USA, 138--152.
[56]
Amoghavarsha Suresh and Anshul Gandhi. 2021. ServerMore: Opportunistic Execution of Serverless Functions in the Cloud. In Proceedings of the ACM Symposium on Cloud Computing (Seattle, WA, USA) (SoCC '21). Association for Computing Machinery, New York, NY, USA, 570--584.
[57]
Ali Tariq, Austin Pahl, Sharat Nimmagadda, Eric Rozner, and Siddharth Lanka. 2020. Sequoia: Enabling quality-of-service in serverless computing. In Proceedings of the 11th ACM Symposium on Cloud Computing. 311--327.
[58]
Prashanth Thinakaran, Jashwant Raj Gunasekaran, Bikash Sharma, Mahmut Taylan Kandemir, and Chita R. Das. 2017. Phoenix: A Constraint-Aware Scheduler for Heterogeneous Datacenters. In 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS). 977--987.
[59]
Prashanth Thinakaran, Jashwant Raj Gunasekaran, Bikash Sharma, Mahmut Taylan Kandemir, and Chita R. Das. 2019. Kube-Knots: Resource Harvesting through Dynamic Container Orchestration in GPU-based Datacenters. In 2019 IEEE International Conference on Cluster Computing (CLUSTER). 1--13.
[60]
Guido Urdaneta, Guillaume Pierre, and Maarten Van Steen. 2009. Wikipedia workload analysis for decentralized hosting. Computer Networks (2009).
[61]
Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking Behind the Curtains of Serverless Platforms. In ATC.
[62]
Yunjing Xu, Zachary Musgrave, Brian Noble, and Michael Bailey. 2013. Bobtail: Avoiding Long Tails in the Cloud. In 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13). USENIX Association, Lombard, IL, 329--341. https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/xu_yunjing
[63]
Hailong Yang, Quan Chen, Moeiz Riaz, Zhongzhi Luan, Lingjia Tang, and Jason Mars. 2017. PowerChief: Intelligent power allocation for multi-stage applications to improve responsiveness on power constrained CMP. In Computer Architecture News.
[64]
Yiming Zhang, Jon Crowcroft, Dongsheng Li, Chengfen Zhang, Huiba Li, Yaozheng Wang, Kai Yu, Yongqiang Xiong, and Guihai Chen. 2018. KylinX: a dynamic library operating system for simplified and efficient cloud virtualization. In 2018 USENIX Annual Technical Conference. 173--186.
[65]
Yanqi Zhang, Íñigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, and Ricardo Bianchini. 2021. Faster and Cheaper Serverless Computing on Harvested Resources. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (Virtual Event, Germany) (SOSP '21). Association for Computing Machinery, New York, NY, USA, 724--739.

Cited By

View all
  • (2024)FAAStloop: Optimizing Loop-Based Applications for Serverless ComputingProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698560(943-960)Online publication date: 20-Nov-2024
  • (2024)Streamlining Cloud-Native Application Development and Deployment with Robust EncapsulationProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698552(847-865)Online publication date: 20-Nov-2024
  • (2024)FaaSBatch: Boosting Serverless Efficiency With In-Container Parallelism and Resource MultiplexingIEEE Transactions on Computers10.1109/TC.2024.335283473:4(1071-1085)Online publication date: 11-Jan-2024
  • Show More Cited By

Index Terms

  1. Cypress: input size-sensitive container provisioning and request scheduling for serverless platforms

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SoCC '22: Proceedings of the 13th Symposium on Cloud Computing
    November 2022
    574 pages
    ISBN:9781450394147
    DOI:10.1145/3542929
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 November 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. input size
    2. resource-management
    3. scheduling
    4. serverless

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    SoCC '22
    Sponsor:
    SoCC '22: ACM Symposium on Cloud Computing
    November 7 - 11, 2022
    California, San Francisco

    Acceptance Rates

    Overall Acceptance Rate 169 of 722 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)443
    • Downloads (Last 6 weeks)81
    Reflects downloads up to 19 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)FAAStloop: Optimizing Loop-Based Applications for Serverless ComputingProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698560(943-960)Online publication date: 20-Nov-2024
    • (2024)Streamlining Cloud-Native Application Development and Deployment with Robust EncapsulationProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698552(847-865)Online publication date: 20-Nov-2024
    • (2024)FaaSBatch: Boosting Serverless Efficiency With In-Container Parallelism and Resource MultiplexingIEEE Transactions on Computers10.1109/TC.2024.335283473:4(1071-1085)Online publication date: 11-Jan-2024
    • (2024)Paldia: Enabling SLO-Compliant and Cost-Effective Serverless Computing on Heterogeneous Hardware2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00018(100-113)Online publication date: 27-May-2024
    • (2024)HeROcache: Storage-Aware Scheduling in Heterogeneous Serverless Edge – The Case of IDS2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00071(587-597)Online publication date: 6-May-2024
    • (2023)ParrotfishProceedings of the 2023 ACM Symposium on Cloud Computing10.1145/3620678.3624654(177-192)Online publication date: 30-Oct-2023
    • (2023)Libra: Harvesting Idle Resources Safely and Timely in Serverless ClustersProceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3588195.3592996(181-194)Online publication date: 7-Aug-2023
    • (2023)Optimizing CPU Performance for Recommendation Systems At-ScaleProceedings of the 50th Annual International Symposium on Computer Architecture10.1145/3579371.3589112(1-15)Online publication date: 17-Jun-2023
    • (2023)Online Function Caching in Serverless Edge Computing2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00308(2295-2302)Online publication date: 17-Dec-2023
    • (2023)FaaSBatch: Enhancing the Efficiency of Serverless Computing by Batching and Expanding Functions2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS57875.2023.00040(372-382)Online publication date: Jul-2023
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media