research-article

Open access

My VM is Lighter (and Safer) than your Container

Authors:

Florian Schmidt,

Kenichi Yasukata,

Felipe HuiciAuthors Info & Claims

SOSP '17: Proceedings of the 26th Symposium on Operating Systems Principles

Pages 218 - 233

https://doi.org/10.1145/3132747.3132763

Published: 14 October 2017 Publication History

Abstract

Containers are in great demand because they are lightweight when compared to virtual machines. On the downside, containers offer weaker isolation than VMs, to the point where people run containers in virtual machines to achieve proper isolation. In this paper, we examine whether there is indeed a strict tradeoff between isolation (VMs) and efficiency (containers). We find that VMs can be as nimble as containers, as long as they are small and the toolstack is fast enough.

We achieve lightweight VMs by using unikernels for specialized applications and with Tinyx, a tool that enables creating tailor-made, trimmed-down Linux virtual machines. By themselves, lightweight virtual machines are not enough to ensure good performance since the virtualization control plane (the toolstack) becomes the performance bottleneck. We present LightVM, a new virtualization solution based on Xen that is optimized to offer fast boot-times regardless of the number of active VMs. LightVM features a complete redesign of Xen's control plane, transforming its centralized operation to a distributed one where interactions with the hypervisor are reduced to a minimum. LightVM can boot a VM in 2.3ms, comparable to fork/exec on Linux (1ms), and two orders of magnitude faster than Docker. LightVM can pack thousands of LightVM guests on modest hardware with memory and CPU usage comparable to that of processes.

Supplementary Material

MP4 File (my_vm_lighter.mp4)

Download
2266.49 MB

References

[1]

Amazon Web Services {n. d.}. Amazon EC2 Container Service. https://aws.amazon.com/ecs/. ({n. d.}).

[2]

Amazon Web Services {n. d.}. AWS Lambda - Serverless Compute. https://aws.amazon.com/lambda. ({n. d.}).

[3]

Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. 2003. Xen and the Art of Virtualization. SIGOPS Open Syst. Rev. 37, 5 (Oct. 2003), 164--177.

Digital Library

[4]

J. Clark. {n. d.}. Google: "EVERYTHING at Google runs in a container". http//:www.theregister.co.uk/2014/05/23/google_containerizationtwobillion/. ({n. d.}).

[5]

Patrick Colp, Mihir Nanavati, Jun Zhu, William Aiello, George Coker, Tim Deegan, Peter Loscocco, and Andrew Warfield. 2011. Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles (SOSP '11). ACM, New York, NY, USA, 189--202.

Digital Library

[6]

Docker {n. d.}. The Docker Containerization Platform. https://www.docker.com/. ({n. d.}).

[7]

John R. Douceur, Jeremy Elson, Jon Howell, and Jacob R. Lorch. 2008. Leveraging Legacy Code to Deploy Desktop Applications on the Web. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI'08). USENIX Association, Berkeley, CA, USA, 339--354. http://dl.acm.org/citation.cfm?id=1855741.1855765

Digital Library

[8]

D. R. Engler, M. F. Kaashoek, and J. O'Toole, Jr. 1995. Exokernel: An Operating System Architecture for Application-level Resource Management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (SOSP '95). ACM, New York, NY, USA, 251--266.

Digital Library

[9]

Erlang on Xen 2012. Erlang on Xen. http://erlangonxen.org/. (July 2012).

[10]

Google Cloud Platform {n. d.}. The Google Cloud Platform Container Engine. https://cloud.google.com/container-engine. ({n. d.}).

[11]

A. Grattafiori. {n. d.}. Understanding and Hardening Linux Containers. https://www.nccgroup.trust/us/our-research/understanding-and-hardening-linux-containers/. ({n. d.}).

[12]

Cameron Hamilton-Rich. {n. d.}. axTLS Embedded SSL. http://axtls.sourceforge.net. ({n. d.}).

[13]

Poul henning Kamp and Robert N. M. Watson. 2000. Jails: Confining the omnipotent root. In In Proc. 2nd Intl. SANE Conference.

[14]

J. Hertz. {n. d.}. Abusing Privileged and Unprivileged Linux Containers. https://www.nccgroup.tmst/uk/our-research/abusing-privileged-and-unprivileged-linux-containers/, ({n. d.}).

[15]

Jon Howell, Bryan Parno, and John R. Douceur. 2013. Embassies: Radically Refactoring the Web. In Presented as part of the 10th USENTX Symposium on Networked Systems Design and Implementation (NSDI13). USENIX, Lombard, IL, 529--545. https://www.usenix.org/conference/nsdil3/technical-sessions/presentation/howell

Digital Library

[16]

Yun Chao Hu, Milan Patel, Dario Sabella, Nurit Sprecher, and Valerie Young. 2015. Mobile Edge Computing - A key technology towards 5G. ETSI White Paper No. 11, First edition (2015).

[17]

IBM. {n. d.}. Docker at insane scale on IBM Power Systems. https://www.ibm.com/blogs/bluemix/2015/ll/docker-insane-scale-on-ibm-power-systems. ({n. d.}).

[18]

IBM developerWorks Open {n. d.}. Solo5 Unikernel. https://developer.ibm.com/open/openprojects/solo5-unikernel/. ({n. d.}).

[19]

Intel. {n. d.}. Intel Clear Containers: A Breakthrough Combination of Speed and Workload Isolation. https://clearlinux.org/sites/default/files/vmscontainers_wp_v5.pdf. ({n. d.}).

[20]

Avi Kivity, Yaniv Kamay, Dor Laor, Uri Lublin, and Anthony Liguori. 2007. KVM: the Linux Virtual Machine Monitor. In In Proc. 2007 Ottawa Linux Symposium (OLS '07).

[21]

Avi Kivity, Dor Laor, Glauber Costa, Pekka Enberg, Nadav Har'El, Don Marti, and Vlad Zolotarov. 2014. OSv---Optimizing the Operating System for Virtual Machines. In Proceedings of the 2014 USENTX Annual Technical Conference (USENIX ATC '14). USENIX Association, Philadelphia, PA, 61--72. https://www.usenix.org/conference/atcl4/technical-sessions/presentation/kivity

Digital Library

[22]

E. Kovacs. {n. d.}. Docker Fixes Vulnerabilities, Shares Plans For Making Platform Safer. http//:www.securityweek.com/docker-fixes-vulnerabilities-shares-plans-making-platform-safer. ({n. d.}).

[23]

Simon Kuenzer, Anton Ivanov, Filipe Manco, Jose Mendes, Yuri Volchkov, Florian Schmidt, Kenichi Yasukata, Michio Honda, and Felipe Huici. 2017. Unikernels Everywhere: The Case for Elastic CDNs. In Proceedings of the 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '17). ACM, New York, NY, USA, 15--29.

Digital Library

[24]

Horacio Andrés Lagar-Cavilla, Joseph Andrew Whitney, Adin Matthew Scannell, Philip Patchin, Stephen M. Rumble, Eyal de Lara, Michael Brudno, and Mahadev Satyanarayanan. 2009. SnowFlock: Rapid Virtual Machine Cloning for Cloud Computing. In Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys '09). ACM, New York, NY, USA, 1--12.

Digital Library

[25]

LinuxContainers.org {n. d.}. LinuxContainers.org. https://linuxcontainers.org. ({n. d.}).

[26]

Anil Madhavapeddy, Thomas Leonard, Magnus Skjegstad, Thomas Gazagnaire, David Sheets, Dave Scott, Richard Mortier, Amir Chaudhry, Balraj Singh, Jon Ludlam, Jon Crowcroft, and Ian Leslie. 2015. Jitsu: Just-In-Time Summoning of Unikernels. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI '15). USENIX Association, Oakland, CA, 559--573. https://www.usenix.org/conference/nsdil5/technical-sessions/presentation/madhavapeddy

Digital Library

[27]

Anil Madhavapeddy and David J. Scott. 2013. Unikernels: Rise of the Virtual Library Operating System. Queue 11, 11, Article 30 (Dec. 2013), 15 pages.

Digital Library

[28]

Y. Mao, J. Zhang, and K. B. Letaief. 2016. Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices. IEEE Journal on Selected Areas in Communications 34, 12 (Dec 2016), 3590--3605.

Digital Library

[29]

Joao Martins, Mohamed Ahmed, Costin Raiciu, Vladimir Olteanu, Michio Honda, Roberto Bifulco, and Felipe Huici. 2014. ClickOS and the Art of Network Function Virtualization. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI '14). USENIX Association, Seattle, WA, 459--473. https://www.usenix.org/conference/nsdil4/technical-sessions/presentation/martins

Digital Library

[30]

McAffee. 2016. Mobile Threat Report. https://www.mcafee.com/us/resources/reports/rp-mobile-threat-report-2016.pdf. (2016).

[31]

MicroPython {n. d.}. MicroPython. https://micropython.org/. ({n. d.}).

[32]

Microsoft. {n. d.}. Azure Container Service. https://azure.microsoft.com/en-us/services/container-service/. ({n. d.}).

[33]

Microsoft Research. {n. d.}. Drawbridge. https://www.microsoft.com/en-us/research/project/drawbridge/. ({n. d.}).

[34]

minios {n. d.}. Mini-OS. https://wiki.xenproject.org/wiki/Mini-OS. ({n. d.}).

[35]

A. Mourat. {n. d.}. 5 security concerns when using Docker. https://www.oreilly.com/ideas/five-security-concerns-when-using-docker. ({n. d.}).

[36]

Vlad Nitu, Pierre Olivier, Alain Tchana, Daniel Chiba, Antonio Barbalace, Daniel Hagimont, and Binoy Ravindran. 2017. Swift Birth and Quick Death: Enabling Fast Parallel Guest Boot and Destruction in the Xen Hypervisor. In Proceedings of the 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '17). ACM, New York, NY, USA, 1--14.

Digital Library

[37]

MAN page. {n. d.}. Linux system calls list. http://man7.org/linux/man-pages/man2/syscalls.2.html. ({n. d.}).

[38]

Rumpkernel.org {n. d.}. Rump Kernels. http://rumpkernel.org/. ({n. d.}).

[39]

Sandvine. {n. d.}. Internet traffic encryption. https://www.sandvine.com/trends/encryption.html. ({n. d.}).

[40]

Mahadev Satyanarayanan, Paramvir Bahl, Ramón Caceres, and Nigel Davies. 2009. The Case for VM-Based Cloudlets in Mobile Computing. IEEE Pervasive Computing 8, 4 (Oct. 2009), 14--23.

Digital Library

[41]

Justine Sherry, Shaddi Hasan, Colin Scott, Arvind Krishnamurthy, Sylvia Ratnasamy, and Vyas Sekar. 2012. Making Middleboxes Someone Else's Problem: Network Processing As a Cloud Service. In Proceedings of the ACM SIGCOMM 2012 Conference on Computer Communication (SIGCOMM '12). ACM, New York, NY, USA, 13--24.

Digital Library

[42]

Stephen Soltesz, Herbert Pötzl, Marc E. Fiuczynski, Andy Bavier, and Larry Peterson. 2007. Container-based Operating System Virtualization: A Scalable, High-performance Alternative to Hypervisors. SIGOPS Oper. Syst. Rev. 41, 3 (March 2007), 275--287.

Digital Library

[43]

S. Stabellini. {n. d.}. Xen on ARM. http//:www.slideshare.net/xen_com_mgr/alsf13-stabellini. ({n. d.}).

[44]

Udo Steinberg and Bernhard Kauer. 2010. NOVA: A Microhypervisorbased Secure Virtualization Architecture. In Proceedings of the 5th European Conference on Computer Systems (EuroSys '10). ACM, New York, NY, USA, 209--222.

Digital Library

[45]

A. van de Ven. {n. d.}. An introduction to Clear Containers. https://lwn.net/Articles/644675/. ({n. d.}).

[46]

Akshat Verma, Gargi Dasgupta, Tapan Kumar Nayak, Pradipta De, and Ravi Kothari. 2009. Server Workload Analysis for Power Minimization Using Consolidation. In Proceedings of the 2009 USENIX Annual Technical Conference (USENIX ATC '09). USENIX Association, Berkeley, CA, USA, 28--28. http://dl.acm.org/citation.cfm?id=1855807.1855835

Digital Library

[47]

VMWare. {n. d.}. vSphere ESXi Bare-Metal Hypervisor. http//:www.vmware.com/products/esxi-and-esx.html. ({n. d.}).

[48]

Michael Vrable, Justin Ma, Jay Chen, David Moore, Erik Vandekieft, Alex C. Snoeren, Geoffrey M. Voelker, and Stefan Savage. 2005. Scalability, Fidelity, and Containment in the Potemkin Virtual Honey-farm. SIGOPS Oper. Syst. Rev. 39, 5 (Oct. 2005), 148--162.

Digital Library

[49]

Andrew Whitaker, Marianne Shaw, and Steven D. Gribble. 2002. Scale and Performance in the Denali Isolation Kernel. SIGOPS Oper. Syst. Rev. 36, SI (Dec. 2002), 195--209.

Digital Library

[50]

Dan Williams and Ricardo Koller. 2016. Unikernel Monitors: Extending Minimalism Outside of the Box. In 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud '16). USENIX Association, Denver, CO. https://www.usenix.org/conference/hotcloud16/workshop-program/presentation/williams

Digital Library

[51]

Wei Zhang, Jinho Hwang, Shriram Rajagopalan, K.K. Ramakrishnan, and Timothy Wood. 2016. Flurries: Countless Fine-Grained NFs for Flexible Per-Flow Customization. In Proceedings of the 12th International on Conference on Emerging Networking EXperiments and Technologies (CoNEXT '16). ACM, New York, NY, USA, 3--17.

Digital Library

Cited By

Liu YAdkar MHolzmann GKuenning GLiu PSmolka SSu WZadok EMa XWon Y(2024)MetisProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650705(123-140)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650705
Mallellu SMaheswari V. Aluvalu RKantipudi M(2024)Function as a Service (FaaS) for Fast, Efficient, Scalable SystemsServerless Computing Concepts, Technology and Architecture10.4018/979-8-3693-1682-5.ch008(134-151)Online publication date: 5-Apr-2024
https://doi.org/10.4018/979-8-3693-1682-5.ch008
Tarasiuk STraczuk DSzczepaniuk KStoń PSmołka J(2024)Performance evaluation of designated containerization and virtualization solutions using a synthetic benchmarkJournal of Computer Sciences Institute10.35784/jcsi.623132(157-162)Online publication date: 30-Sep-2024
https://doi.org/10.35784/jcsi.6231
Show More Cited By

Index Terms

My VM is Lighter (and Safer) than your Container
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
      2. Software infrastructure
        Virtual machines

Recommendations

Transparently bridging semantic gap in CPU management for virtualized environments

Consolidated environments are progressively accommodating diverse and unpredictable workloads in conjunction with virtual desktop infrastructure and cloud computing. Unpredictable workloads, however, aggravate the semantic gap between the virtual ...
Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors
EuroSys '07: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007

Hypervisors, popularized by Xen and VMware, are quickly becoming commodity. They are appropriate for many usage scenarios, but there are scenarios that require system virtualization with high degrees of both isolation and efficiency. Examples include ...
Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors
EuroSys'07 Conference Proceedings

Hypervisors, popularized by Xen and VMware, are quickly becoming commodity. They are appropriate for many usage scenarios, but there are scenarios that require system virtualization with high degrees of both isolation and efficiency. Examples include ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSP '17: Proceedings of the 26th Symposium on Operating Systems Principles

October 2017

677 pages

ISBN:9781450350853

DOI:10.1145/3132747

Copyright © 2017 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems
USENIX Assoc: USENIX Assoc

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020

Conference

SOSP '17

Sponsor:

SIGOPS
USENIX Assoc

SOSP '17: ACM SIGOPS 26th Symposium on Operating Systems Principles

October 28, 2017

Shanghai, China

Acceptance Rates

Overall Acceptance Rate 174 of 961 submissions, 18%

Upcoming Conference

SOSP '25

Sponsor:
sigops

ACM SIGOPS 31st Symposium on Operating Systems Principles

October 13 - 16, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

201
Total Citations
View Citations
26,086
Total Downloads

Downloads (Last 12 months)2,799
Downloads (Last 6 weeks)246

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu YAdkar MHolzmann GKuenning GLiu PSmolka SSu WZadok EMa XWon Y(2024)MetisProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650705(123-140)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650705
Mallellu SMaheswari V. Aluvalu RKantipudi M(2024)Function as a Service (FaaS) for Fast, Efficient, Scalable SystemsServerless Computing Concepts, Technology and Architecture10.4018/979-8-3693-1682-5.ch008(134-151)Online publication date: 5-Apr-2024
https://doi.org/10.4018/979-8-3693-1682-5.ch008
Tarasiuk STraczuk DSzczepaniuk KStoń PSmołka J(2024)Performance evaluation of designated containerization and virtualization solutions using a synthetic benchmarkJournal of Computer Sciences Institute10.35784/jcsi.623132(157-162)Online publication date: 30-Sep-2024
https://doi.org/10.35784/jcsi.6231
Leis VDietrich C(2024)Cloud-Native Database Systems and Unikernels: Reimagining OS Abstractions for Modern HardwareProceedings of the VLDB Endowment10.14778/3659437.365946217:8(2115-2122)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.14778/3659437.3659462
Copik MCalotoiu ARethy GBöhringer RBruno RHoefler T(2024)Process-as-a-Service: Unifying Elastic and Stateful Clouds with Serverless ProcessesProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698567(223-242)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698567
Parola FQi SNarappa ARamakrishnan KRisso F(2024)SURE: Secure Unikernels Make Serverless Computing Rapid and EfficientProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698558(668-688)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698558
Lertpongrujikorn PNguyen HSalehi M(2024)Streamlining Cloud-Native Application Development and Deployment with Robust EncapsulationProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698552(847-865)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698552
Misono MOkelmann PMainas CBhatotia P(2024)uIO: Lightweight and Extensible UnikernelsProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698518(580-599)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698518
Szekely ABelay AMorris RKaashoek MWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)Unifying serverless and microservice workloads with SigmaOSProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695947(385-402)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695947
Zeno LChen ASilberstein MSekar VYu MSeneviratne AVeitch D(2024)In-Network Address Caching for Virtual NetworksProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672213(735-749)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672213
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents