research-article

Hecate: Lifting and Shifting On-Premises Workloads to an Untrusted Cloud

Authors:

Weidong CuiAuthors Info & Claims

CCS '22: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security

Pages 1231 - 1242

https://doi.org/10.1145/3548606.3560592

Published: 07 November 2022 Publication History

Abstract

Despite the recent exponential growth in cloud adoption, businesses that handle sensitive data (e.g., health and financial sectors) are hesitant to migrate their on-premises IT infrastructure to the public cloud due to the lack of trust on the cloud provider. Confidential computing aims to move the cloud provider out of the trusted computing base. New hardware features such as AMD's SEV-SNP can run a full virtual machine (VM) with confidentiality and integrity protection against the cloud. However, there exist challenges in supporting legacy operating systems and enforcing security policies (e.g., firewalls) in confidential VMs.

In this paper, we present Hecate, an L1 hypervisor that runs inside a confidential VM enabled by SEV-SNP. Hecate can support legacy operating systems by running them in a nested VM and enforce various security policies on the nested VM based on the virtualization boundary. The key challenge in designing Hecate is that it cannot rely on the untrusted L0 hypervisor for nested virtualization. To solve it, we repurpose SEV-SNP's newly added privilege dimension called Virtual Machine Privilege Levels (VMPLs) to enable virtualization for a single nested VM.

We have built a prototype of Hecate based on the Linux KVM virtualization stack. Our prototype is capable of running MS-DOS, FreeBSD and vanilla Linux without any modification. It also supports security checks on the nested VM such as network firewalls and kernel integrity. When compared with a regular, non-confidential VM, the nested VM enabled by Hecate can achieve a throughput between 57% and 85% for real-world applications such as the Nginx web server and the MySQL database.

References

[1]

AMD64 Architecture Programmer's Manual Volume 2: System Programming. https://developer.amd.com/resources/developer-guides-manuals.

[2]

ARM Confidential Compute Architecture. https://www.arm.com/why-arm/ architecture/security-features/arm-confidential-compute-architecture.

[3]

AWS Nitro Enclaves. https://aws.amazon.com/ec2/nitro/nitro-enclaves/.

[4]

Hypervisor-Protected Code Integrity (HVCI). https://docs.microsoft.com/en- us/windows-hardware/drivers/bringup/device-guard-and-credential-guard.

[5]

Intel Trust Domain Extensions (Intel TDX). https://www.intel.com/content/ www/us/en/developer/articles/technical/intel-trust-domain-extensions.html.

[6]

SEV-ES Guest-Hypervisor Communication Block Standardization. https:// developer.amd.com/wp-content/resources/56421.pdf.

[7]

MacVTap. https://virt.kernelnewbies.org/MacVTap.

[8]

AMD SEV-SNP: Strengthening VM Isolation with Integrity Protection and More. https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf.

[9]

AMD Server Vulnerabilities. https://www.amd.com/en/corporate/product- security/bulletin/amd-sb-1021.

[10]

Using SQL Server in Windows. https://docs.microsoft.com/en-us/troubleshoot/ sql/general/use-sql-server-in-windows.

[11]

SEV Secure Nested Paging Firmware ABI Specification. https://www.amd.com/ system/files/TechDocs/56860.pdf

[12]

Ahmed M Azab, Kirk Swidowski, Rohan Bhutkar, Jia Ma, Wenbo Shen, Ruowen Wang, and Peng Ning. 2016. SKEE: A Lightweight Secure Kernel-level Execution Environment for ARM. In Proceedings of the 2016 Network and Distributed System Security Symposium (NDSS).

[13]

Andrew Baumann, Marcus Peinado, and Galen Hunt. 2014. Shielding Applications from an Untrusted Cloud with Haven. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI).

[14]

Muli Ben-Yehuda, Michael D Day, Zvi Dubitzky, Michael Factor, Nadav Har'El, Abel Gordon, Anthony Liguori, Orit Wasserman, and Ben-Ami Yassour. 2010. The Turtles Project: Design and Implementation of Nested Virtualization. In Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI).

[15]

Martim Carbone, Matthew Conover, Bruce Montague, and Wenke Lee. 2012. Secure and Robust Monitoring of Virtual Machines Through Guest-Assisted Introspection. In Proceedings of the 15th International Workshop on Recent Advances in Intrusion Detection (RAID).

Digital Library

[16]

Cilium. BPF and XDP Reference Guide. https://docs.cilium.io/en/latest/bpf/.

[17]

Cilium. Cilium Network Policies. https://docs.cilium.io/en/stable/policy/.

[18]

Cilium. eBPF-based Networking Observability and Security. https://cilium.io.

[19]

Brendan Dolan-Gavitt, Tim Leek, Michael Zhivich, Jonathon Giffin, and Wenke Lee. 2011. Virtuoso: Narrowing the Semantic Gap in Virtual Machine Introspection. In Proceedings of the 32nd IEEE Symposium on Security and Privacy (Oakland).

Digital Library

[20]

Yangchun Fu and Zhiqiang Lin. 2012. Space Traveling Across VM: Automatically Bridging the Semantic Gap in Virtual Machine Introspection via Online Kernel Data Redirection. In Proceedings of the 33rd IEEE Symposium on Security and Privacy (Oakland).

Digital Library

[21]

Tal Garfinkel and Mendel Rosenblum. 2003. A Virtual Machine Introspection based Architecture for Intrusion Detection. In Proceedings of the 2003 Network and Distributed System Security Symposium (NDSS).

[22]

Xinyang Ge, Hayawardh Vijayakumar, and Trent Jaeger. 2014. Sprobes: Enforcing Kernel Code Integrity on the TrustZone Architecture. In Proceedings of the 3rd Mobile Security Technology Workshop (MOST).

[23]

Zhongshu Gu, Zhui Deng, Dongyan Xu, and Xuxian Jiang. 2011. Process Implanting: A New Active Introspection Framework for Virtualization. In Proceedings of the 2011 Network and Distributed System Security Symposium (NDSS).

Digital Library

[24]

Le Guan, Peng Liu, Xinyu Xing, Xinyang Ge, Shengzhi Zhang, Meng Yu, and Trent Jaeger. 2017. TrustShadow: Secure Execution of Unmodified Applications with ARM TrustZone. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys).

Digital Library

[25]

Jin Soo Jang, Sunjune Kong, Minsu Kim, Daegyeong Kim, and Brent Byunghoon Kang. 2015. SeCReT: Secure Channel between Rich Execution Environment and Trusted Execution Environment. In Proceedings of the 2015 Network and Distributed System Security Symposium (NDSS).

[26]

Avi Kivity, Yaniv Kamay, Dor Laor, Uri Lublin, and Anthony Liguori. 2007. KVM: the Linux Virtual Machine Monitor. In Proceedings of the 2007 Ottawa Linux Symposium.

[27]

Sangho Lee, Ming-Wei Shih, Prasun Gera, Taesoo Kim, Hyesoon Kim, and Marcus Peinado. 2017. Inferring Fine-Grained Control Flow Inside SGX Enclaves with Branch Shadowing. In Proceedings of the 26th USENIX Security Symposium.

Digital Library

[28]

Dingji Li, Zeyu Mi, Yubin Xia, Binyu Zang, Haibo Chen, and Haibing Guan. 2021. TwinVisor: Hardware-Isolated Confidential Virtual Machines for ARM. In Proceedings of the 28th Symposium on Operating Systems Principles (SOSP).

Digital Library

[29]

Mengyuan Li, Yinqian Zhang, and Zhiqiang Lin. 2021. CROSSLINE: Breaking "Security-by-Crash" based Memory Isolation in AMD SEV. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS).

Digital Library

[30]

Mengyuan Li, Yinqian Zhang, Zhiqiang Lin, and Yan Solihin. 2019. Exploiting Un- protected I/O Operations in AMD's Secure Encrypted Virtualization. In Proceed- ings of the 28th USENIX Security Symposium (Santa Clara, CA). 1257--1272. https: //www.usenix.org/conference/usenixsecurity19/presentation/li-mengyuan

[31]

Mengyuan Li, Yinqian Zhang, Huibo Wang, Kang Li, and Yueqiang Cheng. 2021. CIPHERLEAKS: Breaking Constant-time Cryptography on AMD SEV via the Ciphertext Side Channel. In Proceedings of the 30th USENIX Security Symposium.

[32]

Jin Tack Lim and Jason Nieh. 2020. Optimizing Nested Virtualization Performance using Direct Virtual Hardware. In Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS).

Digital Library

[33]

Yutao Liu, Yubin Xia, Haibing Guan, Binyu Zang, and Haibo Chen. 2014. Concur- rent and Consistent Virtual Machine Introspection with Hardware Transactional Memory. In Proceedings of the 20th International Symposium on High Performance Computer Architecture (HPCA).

[34]

Mathias Morbitzer, Manuel Huber, and Julian Horsch. 2019. Extracting Secrets from Encrypted Virtual Machines. In Proceedings of the 9th ACM Conference on Data and Application Security and Privacy. 221--230. https://doi.org/10.1145/ 3292006.3300022

Digital Library

[35]

Alireza Saberi, Yangchun Fu, and Zhiqiang Lin. 2014. Hybrid-Bridge: Efficiently Bridging the Semantic Gap in Virtual Machine Introspection via Decoupled Execution and Training Memoization. In Proceedings of the 2014 Network and Distributed System Security Symposium (NDSS).

[36]

Arvind Seshadri, Mark Luk, Ning Qu, and Adrian Perrig. 2007. SecVisor: A Tiny Hypervisor to Provide Lifetime Kernel Code Integrity for Commodity OSes. In Proceedings of the 21st Symposium on Operating Systems Principles (SOSP).

Digital Library

[37]

Monirul I Sharif, Wenke Lee, Weidong Cui, and Andrea Lanzi. 2009. Secure In-VM Monitoring Using Hardware Virtualization. In Proceedings of the 16th ACM Conference on Computer and Communications Security (CCS).

Digital Library

[38]

Youren Shen, Hongliang Tian, Yu Chen, Kang Chen, Runji Wang, Yi Xu, Yubin Xia, and Shoumeng Yan. 2020. Occlum: Secure and Efficient Multitasking inside a Single Enclave of Intel SGX. In Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS).

Digital Library

[39]

Deepa Srinivasan, Zhi Wang, Xuxian Jiang, and Dongyan Xu. 2011. Process Out-Grafting: an Efficient "Out-of-VM" Approach for Fine-Grained Process Execution Monitoring. In Proceedings of the 18th ACM Conference on Computer and Communications Security (CCS).

Digital Library

[40]

Chia-Che Tsai, Donald E Porter, and Mona Vij. 2017. Graphene-SGX: A Practical Library OS for Unmodified Applications on SGX. In Proceedings of the 2017 USENIX Annual Technical Conference (ATC).

[41]

Jo Van Bulck, Marina Minkin, Ofir Weisse, Daniel Genkin, Baris Kasikci, Frank Piessens, Mark Silberstein, Thomas F Wenisch, Yuval Yarom, and Raoul Strackx. 2018. Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution. In Proceedings of the 27th USENIX Security Symposium.

Digital Library

[42]

Jo Van Bulck, Nico Weichbrodt, Rüdiger Kapitza, Frank Piessens, and Raoul Strackx. 2017. Telling Your Secrets without Page Faults: Stealthy Page Table-Based Attacks on Enclaved Execution. In Proceedings of the 26th USENIX Security Symposium.

[43]

Yuanzhong Xu, Weidong Cui, and Marcus Peinado. 2015. Controlled-Channel Attacks: Deterministic Side Channels for Untrusted Operating Systems. In Proceedings of the 36th IEEE Symposium on Security and Privacy.

Digital Library

[44]

Fengzhe Zhang, Jin Chen, Haibo Chen, and Binyu Zang. 2011. Cloudvisor: Retrofitting Protection of Virtual Machines in Multi-Tenant Cloud with Nested Virtualization. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP).

Digital Library

[45]

Siqi Zhao, Xuhua Ding, Wen Xu, and Dawu Gu. 2017. Seeing Through the Same Lens: Introspecting Guest Address Space at Native Speed. In Proceedings of the 26th USENIX Security Symposium

Cited By

Schlüter BSridhara SBertschi AShinde S(2024)WeSee: Using Malicious #VC Interrupts to Break AMD SEV-SNP2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00262(4220-4238)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00262
Lohmöller JPennekamp JMatzutt RSchneider CVlad ETrautwein CWehrle K(2024)The unresolved need for dependable guarantees on security, sovereignty, and trust in data ecosystemsData & Knowledge Engineering10.1016/j.datak.2024.102301151:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.datak.2024.102301
Ye MRuocco ABuono DBottomley JFranke H(2023)Free the Turtles: Removing Nested Virtualization for Performance and Confidentiality in the Cloud2023 IEEE 16th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD60044.2023.00039(275-281)Online publication date: Jul-2023
https://doi.org/10.1109/CLOUD60044.2023.00039

Index Terms

Hecate: Lifting and Shifting On-Premises Workloads to an Untrusted Cloud
1. Security and privacy
  1. Systems security
    1. Operating systems security
      1. Trusted computing
      2. Virtualization and security

Recommendations

TwinVisor: Hardware-isolated Confidential Virtual Machines for ARM
SOSP '21: Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles

Confidential VM, which offers an isolated execution environment for cloud tenants with limited trust in the cloud provider, has recently been deployed in major clouds such as AWS and Azure. However, while ARM has become increasingly popular in cloud ...
Trustworthy confidential virtual machines for the masses
Middleware '23: Proceedings of the 24th International Middleware Conference

Confidential computing alleviates the concerns of distrustful customers by removing the cloud provider from their trusted computing base and resolves their disincentive to migrate their workloads to the cloud. This is facilitated by new hardware ...
Designing and Implementing Live Migration Support for Arm-based Confidential VMs
APSys '24: Proceedings of the 15th ACM SIGOPS Asia-Pacific Workshop on Systems

Confidential virtual machines (CVMs) are increasingly deployed to protect users' code and data in use against attackers with hypervisor privileges. Given Arm's growing adoption, various implementations have extended commodity hypervisors like KVM to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '22: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security

November 2022

3598 pages

ISBN:9781450394505

DOI:10.1145/3548606

General Chairs:
Heng Yin
University of California, Riverside
,
Angelos Stavrou
Virginia Tech
,
Program Chairs:
Cas Cremers
CISPA Helmholtz Center for Information Security
,
Elaine Shi
Carnegie Mellon University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS '22

Sponsor:

SIGSAC

CCS '22: 2022 ACM SIGSAC Conference on Computer and Communications Security

November 7 - 11, 2022

CA, Los Angeles, USA

Acceptance Rates

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
778
Total Downloads

Downloads (Last 12 months)218
Downloads (Last 6 weeks)24

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schlüter BSridhara SBertschi AShinde S(2024)WeSee: Using Malicious #VC Interrupts to Break AMD SEV-SNP2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00262(4220-4238)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00262
Lohmöller JPennekamp JMatzutt RSchneider CVlad ETrautwein CWehrle K(2024)The unresolved need for dependable guarantees on security, sovereignty, and trust in data ecosystemsData & Knowledge Engineering10.1016/j.datak.2024.102301151:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.datak.2024.102301
Ye MRuocco ABuono DBottomley JFranke H(2023)Free the Turtles: Removing Nested Virtualization for Performance and Confidentiality in the Cloud2023 IEEE 16th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD60044.2023.00039(275-281)Online publication date: Jul-2023
https://doi.org/10.1109/CLOUD60044.2023.00039

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents