Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3445814.3446757acmconferencesArticle/Chapter ViewAbstractPublication PagesasplosConference Proceedingsconference-collections
research-article

FaasCache: keeping serverless computing alive with greedy-dual caching

Published: 17 April 2021 Publication History

Abstract

Functions as a Service (also called serverless computing) promises to revolutionize how applications use cloud resources. However, functions suffer from cold-start problems due to the overhead of initializing their code and data dependencies before they can start executing. Keeping functions alive and warm after they have finished execution can alleviate the cold-start overhead. Keep-alive policies must keep functions alive based on their resource and usage characteristics, which is challenging due to the diversity in FaaS workloads.
Our insight is that keep-alive is analogous to caching. Our caching-inspired Greedy-Dual keep-alive policy can be effective in reducing the cold-start overhead by more than 3× compared to current approaches. Caching concepts such as reuse distances and hit-ratio curves can also be used for auto-scaled server resource provisioning, which can reduce the resource requirement of FaaS providers by 30% for real-world dynamic workloads. We implement caching-based keep-alive and resource provisioning policies in our FaasCache system, which is based on OpenWhisk. We hope that our caching analogy opens the door to more principled and optimized keep-alive and resource provisioning techniques for future FaaS workloads and platforms.

References

[1]
[n.d.]. AWS Lambda Limits. https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html.
[2]
[n.d.]. Keeping Functions Warm - How To Fix AWS Lambda Cold Start Issues. https://serverless.com/blog/keep-your-lambdas-warm/.
[3]
[n.d.]. PID Controllers. https://en.wikipedia.org/wiki/PID_controller.
[4]
2015. Docker. https://www.docker.com/.
[5]
2017. How long does AWS Lambda keep your idle functions around before a cold start? https://read.acloud.guru/how-long-does-aws-lambda-keep-your-idle-functions-around-before-a-cold-start-bf715d3b810.
[6]
2018. Lambda Warmer: Optimize AWS Lambda Function Cold Starts. https://www.jeremydaly.com/lambda-warmer-optimize-aws-lambda-function-cold-starts/.
[7]
2019. AWS Lambda predictable start-up times with provisioned concurrency. https://aws.amazon.com/blogs/compute/new-for-aws-lambda-predictable-start-up-times-with-provisioned-concurrency/.
[8]
2019. Azure Functions Warm-up trigger. https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-warmup.
[9]
2020. Apache OpenWhisk: Open Source Serverless Cloud Platform. https://openwhisk.apache.org/.
[10]
2020. AWS Lambda. https://aws.amazon.com/lambda/.
[11]
2020. Azure Functions. https://azure.microsoft.com/en-us/services/functions/ .
[12]
2020. Google Cloud Functions. https://cloud.google.com/functions .
[13]
2020. Google Cloud Functions Tips and Tricks. https://cloud.google.com/functions/docs/bestpractices/tips.
[14]
2020. OpenFaaS : Server Functions, Made Simple. https://www.openfaas.com.
[15]
Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th $\$USENIX$\$ Symposium on Networked Systems Design and Implementation ($\$NSDI$\$ 20). 419?434.
[16]
Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-Performance Serverless Computing. USENIX ATC (2018), 14.
[17]
Erwan Alliaume and Benjamin Le Roux. 2018. Cold start / Warm start with AWS Lambda. https://blog.octo.com/en/cold-start-warm-start-with-aws-lambda/.
[18]
Soumya Basu, Aditya Sundarrajan, Javad Ghaderi, Sanjay Shakkottai, and Ramesh Sitaraman. 2017. Adaptive TTL-based caching for content delivery. In Proceedings of the 2017 ACM SIGMETRICS/International Conference on Measurement and Modeling of Computer Systems. 45?46.
[19]
Pei Cao and Sandy Irani. 1997. Cost-Aware WWW Proxy Caching Algorithms. In Proceedings of the USENIX Symposium on Internet Technologies and Systems. 15.
[20]
Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2018. A case for serverless machine learning. In Workshop on Systems for ML and Open Source Software at NeurIPS, Vol. 2018.
[21]
Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2019. Cirrus: a Serverless Framework for End-to-end ML Workflows. In Proceedings of the ACM Symposium on Cloud Computing - SoCC '19. ACM Press, Santa Cruz, CA, USA, 13?24. isbn:978-1-4503-6973-2 https://doi.org/10.1145/3357223.3362711
[22]
Benjamin Carver, Jingyuan Zhang, Ao Wang, and Yue Cheng. 2019. In Search of a Fast and Efficient Serverless DAG Engine. arXiv:1910.05896 [cs] (Oct. 2019). http://arxiv.org/abs/1910.05896 arXiv: 1910.05896.
[23]
Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, and Kyle Chard. 2020. FuncX: A Federated Function Serving Fabric for Science. In Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (Stockholm, Sweden) (HPDC '20). Association for Computing Machinery, New York, NY, USA, 65?76. isbn:9781450370523 https://doi.org/10.1145/3369583.3392683
[24]
Hao Che, Ye Tung, and Zhijun Wang. 2002. Hierarchical web caching systems: Modeling, design and experimental results. IEEE journal on Selected Areas in Communications 20, 7 (2002), 1305?1314.
[25]
Kai Cheng and Yahiko Kambayashi. 2000. LRU-SP: a size-adjusted and popularity-aware LRU replacement algorithm for web caching. In Proceedings 24th Annual International Computer Software and Applications Conference. COMPSAC2000. IEEE, 48?53.
[26]
Ludmila Cherkasova. 1998. Improving WWW Proxies Performance with Greedy-Dual-Size-Frequency Caching Policy. In HP Labs Technical Report 98-69 (R.1).
[27]
Ludmila Cherkasova and Gianfranco Ciardo. 2001. Role of Aging, Frequency, and Size in Web Cache Replacement Policies. In High-Performance Computing and Networking, G. Goos, J. Hartmanis, J. van Leeuwen, Bob Hertzberger, Alfons Hoekstra, and Roy Williams (Eds.). Vol. 2110. Springer Berlin Heidelberg, Berlin, Heidelberg, 114?123. isbn:978-3-540-42293-8 978-3-540-48228-4 https://doi.org/10.1007/3-540-48228-8_12 Series Title: Lecture Notes in Computer Science.
[28]
Ludmila Cherkasova and Gianfranco Ciardo. 2001. Role of aging, frequency, and size in web cache replacement policies. In International Conference on High-Performance Computing and Networking. Springer, 114?123.
[29]
Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qixuan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems. 467?481.
[30]
Gil Einziger, Roy Friedman, and Ben Manes. 2017. Tinylfu: A highly efficient cache admission policy. ACM Transactions on Storage (ToS) 13, 4 (2017), 1?31.
[31]
Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, and Shuvo Chatterjee. 2019. From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers. USENIX ATC (2019), 15.
[32]
Anshul Gandhi, Mor Harchol-Balter, Ram Raghunathan, and Michael A Kozuch. 2012. Autoscale: Dynamic, robust capacity management for multi-tier data centers. ACM Transactions on Computer Systems (TOCS) 30, 4 (2012), 1?26.
[33]
Bishakh Chandra Ghosh, Sourav Kanti Addya, Nishant Baranwal Somy, Shubha Brata Nath, Sandip Chakraborty, and Soumya K. Ghosh. 2019. Caching Techniques to Improve Latency in Serverless Architectures. arXiv:1911.07351 [cs] (Nov. 2019). http://arxiv.org/abs/1911.07351 arXiv: 1911.07351.
[34]
Scott Hendrickson, Stephen Sturdevant, Tyler Harter, Venkateshwaran Venkataramani, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2016. Serverless computation with OpenLambda. In 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 16).
[35]
Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Chen Ding, and Zhenlin Wang. 2016. Kinetic modeling of data eviction in cache. In 2016 USENIX Annual Technical Conference (USENIX ATC 16)). 351?364.
[36]
Bo Jiang, Philippe Nain, and Don Towsley. 2018. On the convergence of the ttl approximation for an lru cache under independent stationary request processes. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS) 3, 4 (2018), 1?31.
[37]
Aji John, Kristiina Ausmees, Kathleen Muenzen, Catherine Kuhn, and Amanda Tan. 2019. SWEEP: Accelerating Scientific Research Through Scalable Serverless Workflows. In Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion - UCC '19 Companion. ACM Press, Auckland, New Zealand, 43?50. isbn:978-1-4503-7044-8 https://doi.org/10.1145/3368235.3368839
[38]
Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the cloud: Distributed computing for the 99\ Cloud Computing. ACM, 445?451.
[39]
Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Yadwadkar, Joseph E. Gonzalez, Raluca Ada Popa, Ion Stoica, and David A. Patterson. 2019. Cloud Programming Simplified: A Berkeley View on Serverless Computing. arXiv:1902.03383 [cs] (Feb. 2019). http://arxiv.org/abs/1902.03383 arXiv: 1902.03383.
[40]
Jeongchul Kim and Kyungyong Lee. 2019. FunctionBench: A Suite of Workloads for Serverless Cloud Function Service. In 2019 IEEE 12th International Conference on Cloud Computing (CLOUD). 502?504. https://doi.org/10.1109/CLOUD.2019.00091 ISSN: 2159-6182.
[41]
Ping-Min Lin and Alex Glikson. 2019. Mitigating Cold Starts in Serverless Platforms: A Pool-Based Approach. arXiv:1903.12221 [cs] (March 2019). http://arxiv.org/abs/1903.12221 arXiv: 1903.12221.
[42]
Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: Library Operating Systems for the Cloud. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (Houston, Texas, USA) (ASPLOS '13). ACM, New York, NY, USA, 461?472. isbn:978-1-4503-1870-9 https://doi.org/10.1145/2451116.2451167
[43]
Johannes Manner, Martin EndreB, Tobias Heckel, and Guido Wirtz. 2018. Cold Start Influencing Factors in Function as a Service. In 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion). IEEE, Zurich, 181?188. isbn:978-1-72810-359-4 https://doi.org/10.1109/UCC-Companion.2018.00054
[44]
Nimrod Megiddo and Dharmendra S Modha. 2003. ARC: A Self-Tuning, Low Overhead Replacement Cache. In USENIX FAST, Vol. 3. 115?130.
[45]
Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Vadim Sukhomlinov, and Naren Nayak. 2019. Agile Cold Starts for Scalable Serverless. USENIX Workshop on Hot Topics in Cloud Computing (HotCloud) (2019), 6.
[46]
Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. USENIX ATC (2018), 14.
[47]
Elizabeth J O'neil, Patrick E O'neil, and Gerhard Weikum. 1993. The LRU-K page replacement algorithm for database disk buffering. Acm Sigmod Record 22, 2 (1993), 297?306.
[48]
Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. (July 2020), 205?218. isbn:978-1-939133-14-4 http://arxiv.org/abs/2003.03423
[49]
Vaishaal Shankar, Karl Krauth, Qifan Pu, Eric Jonas, Shivaram Venkataraman, Ion Stoica, Benjamin Recht, and Jonathan Ragan-Kelley. 2018. Numpywren: Serverless linear algebra. arXiv preprint arXiv:1810.09679 (2018).
[50]
Prateek Sharma, Ahmed Ali-Eldin, and Prashant Shenoy. 2019. Resource Deflation: A New Approach For Transient Resource Reclamation. In Proceedings of the Fourteenth EuroSys Conference 2019 (Dresden, Germany) (EuroSys '19). ACM, New York, NY, USA, Article 33, 17 pages. isbn:978-1-4503-6281-8 https://doi.org/10.1145/3302424.3303945
[51]
Aditya Sundarrajan, Mingdong Feng, Mangesh Kasbekar, and Ramesh K Sitaraman. 2017. Footprint descriptors: Theory and practice of cache provisioning in a global cdn. In Proceedings of the 13th International Conference on emerging Networking EXperiments and Technologies. 55?67.
[52]
Amoghavarsha Suresh, Gagan Somashekar, Anandh Varadarajan, Veerendra Ramesh Kakarla, Hima Upadhyay, and Anshul Gandhi. 2020. ENSURE: Efficient Scheduling and Autonomous Resource Management in Serverless Environments. (2020), 10.
[53]
Erwin van Eyk, Alexandru Iosup, Simon Seif, and Markus Thommes. 2017. The SPEC cloud group's research vision on FaaS and serverless architectures. In Proceedings of the 2nd International Workshop on Serverless Computing - WoSC '17. ACM Press, Las Vegas, Nevada, 1?4. isbn:978-1-4503-5434-9 https://doi.org/10.1145/3154847.3154848
[54]
Carl A Waldspurger, Nohhyun Park, Alexander Garthwaite, and Irfan Ahmad. 2015. Efficient MRC Construction with SHARDS. In 13th USENIX Conference on File and Storage Technologies (FAST 15). 95?110.
[55]
Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In 2018 USENIX Annual Technical Conference. 133?146.
[56]
B. P. Welford. 1962. Note on a Method for Calculating Corrected Sums of Squares and Products. Technometrics 4, 3 (1962), 419?420. https://doi.org/10.1080/00401706.1962.10490022 arxiv:https://www.tandfonline.com/doi/pdf/10.1080/00401706.1962.10490022
[57]
Jake Wires, Stephen Ingram, Zachary Drudi, Nicholas JA Harvey, and Andrew Warfield. 2014. Characterizing storage workloads with counter stacks. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). 335?349.
[58]
N. Young. 1994. The K-server dual and loose competitiveness for paging. Algorithmica 11, 6 (June 1994), 525?541. issn:0178-4617, 1432-0541 https://doi.org/10.1007/BF01189992
[59]
Neal E Young. 2002. On-line file caching. Algorithmica 33, 3 (2002), 371?383.
[60]
Yu Zhang, Ping Huang, Ke Zhou, Hua Wang, Jianying Hu, Yongguang Ji, and Bin Cheng. 2020. OSCA: An Online-Model Based Cache Allocation Scheme in Cloud Block Storage Systems. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 785?798. isbn:978-1-939133-14-4 https://www.usenix.org/conference/atc20/presentation/zhang-yu
[61]
Yu Zhang, Ping Huang, Ke Zhou, Hua Wang, Jianying Hu, Yongguang Ji, and Bin Cheng. 2020. OSCA: An Online-Model Based Cache Allocation Scheme in Cloud Block Storage Systems. In 2020 USENIX Annual Technical Conference. 785?798.

Cited By

View all
  • (2025)Medusa: Accelerating Serverless LLM Inference with MaterializationProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 110.1145/3669940.3707285(653-668)Online publication date: 3-Feb-2025
  • (2024)A secure, fast, and resource-efficient serverless platform with function REWINDProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692029(597-613)Online publication date: 10-Jul-2024
  • (2024)Harmonizing efficiency and practicabilityProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3691993(1-17)Online publication date: 10-Jul-2024
  • Show More Cited By

Index Terms

  1. FaasCache: keeping serverless computing alive with greedy-dual caching

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
    April 2021
    1090 pages
    ISBN:9781450383172
    DOI:10.1145/3445814
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 17 April 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    Author Tags

    1. Caching
    2. Cloud Computing
    3. Functions as a Service
    4. Serverless Computing

    Qualifiers

    • Research-article

    Conference

    ASPLOS '21
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 535 of 2,713 submissions, 20%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)478
    • Downloads (Last 6 weeks)49
    Reflects downloads up to 04 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Medusa: Accelerating Serverless LLM Inference with MaterializationProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 110.1145/3669940.3707285(653-668)Online publication date: 3-Feb-2025
    • (2024)A secure, fast, and resource-efficient serverless platform with function REWINDProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692029(597-613)Online publication date: 10-Jul-2024
    • (2024)Harmonizing efficiency and practicabilityProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3691993(1-17)Online publication date: 10-Jul-2024
    • (2024)SabreProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691939(1-18)Online publication date: 10-Jul-2024
    • (2024)An Autoscalable Approach to Optimize Energy Consumption using Smart Meters data in Serverless ComputingScience and Technology for Energy Transition10.2516/stet/2024078Online publication date: 3-Sep-2024
    • (2024)Data pipeline approaches in serverless computing: a taxonomy, review, and research trendsJournal of Big Data10.1186/s40537-024-00939-011:1Online publication date: 11-Jun-2024
    • (2024)Cold Start Latency in Serverless Computing: A Systematic Review, Taxonomy, and Future DirectionsACM Computing Surveys10.1145/370087557:3(1-36)Online publication date: 11-Nov-2024
    • (2024)Accountable Carbon Footprints and Energy Profiling For Serverless FunctionsProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698531(522-541)Online publication date: 20-Nov-2024
    • (2024)On-demand and Parallel Checkpoint/Restore for GPU ApplicationsProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698510(415-433)Online publication date: 20-Nov-2024
    • (2024)Pre-Warming is Not Enough: Accelerating Serverless Inference With Opportunistic Pre-LoadingProceedings of the ACM Symposium on Cloud Computing10.1145/3698038.3698509(178-195)Online publication date: 20-Nov-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media