Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3466752.3480093acmconferencesArticle/Chapter ViewAbstractPublication PagesmicroConference Proceedingsconference-collections
research-article

Network-on-Chip Microarchitecture-based Covert Channel in GPUs

Published: 17 October 2021 Publication History

Abstract

As GPUs are becoming widely deployed in the cloud infrastructure to support different application domains, the security concerns of GPUs are becoming increasingly important. In particular, the support for multiprogramming in modern GPUs has led to new vulnerabilities since multiple kernels in a GPU can be executed at the same time. In this work, we propose a new microarchitectural timing covert channel for GPUs that can be established based on the shared, on-chip interconnect channels. We first reverse-engineer the organization of the on-chip networks in modern GPUs to understand the core placements throughout the GPU. The hierarchical organization of the GPU results in the sharing of interconnect bandwidth between neighboring cores. Based on this understanding, we identify how contention for the interconnect bandwidth can be exploited for a novel covert channel attack. We propose two types of interconnect-based covert channels that exploit the on-chip network hierarchy. Unlike cache-based covert channels, no states of the on-chip network need to be modified for communication in our interconnect-based covert channel and the impact of contention is very predictable. By exploiting the parallelism of GPUs, our proposed covert channel results in very high bandwidth – achieving approximately 24 Mbps of bandwidth on NVIDIA Volta GPUs and results in one of the highest known microarchitectural covert channel bandwidth.

References

[1]
Dennis Abts and Deborah Weisser. 2007. Age-Based Packet Arbitration in Large-Radix k-ary n-cubes. In Proceedings of the 2007 ACM/IEEE conference on Supercomputing. 1–11.
[2]
Jacob T. Adriaens, Katherine Compton, Nam Sung Kim, and Michael J. Schulte. 2012. The Case for GPGPU Spatial Multitasking. In 2012 IEEE International Symposium on High-Performance Computer Architecture (HPCA). 1–12.
[3]
Advanced Micro Devices, Inc.2019. Introducing RDNA Architecture The all new Radeon gaming architecture powering “Navi”.
[4]
Advanced Micro Devices, Inc.2020. “AMD Instinct MI100” Instruction Set Architecture Reference Guide.
[5]
Jaeguk Ahn, Cheolgyu Jin, Jiho Kim, Minsoo Rhu, Yunsi Fei, David Kaeli, and John Kim. 2021. Trident: A Hybrid Correlation-Collision GPU Cache Timing Attack for AES Key Recovery. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). 332–344.
[6]
Amazon Web Services, Inc.2021. Amazon Elastic Compute Cloud: User Guide for Linux Instances.
[7]
Mihir Awatramani, Joseph Zambreno, and Diane Rover. 2013. Increasing GPU Throughput using Kernel Interleaved Thread Block Scheduling. In 2013 IEEE 31st International Conference on Computer Design (ICCD). 503–506.
[8]
Ali Bakhoda, John Kim, and Tor M. Aamodt. 2010. Throughput-Effective On-Chip Networks for Manycore Accelerators. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE Computer Society, USA, 421–432.
[9]
Travis H. Boraten and Avinash K. Kodi. 2018. Securing NoCs Against Timing Attacks with Non-Interference Based Adaptive Routing. In 2018 12th IEEE/ACM International Symposium on Networks-on-Chip (NOCS). 1–8.
[10]
Jie Chen and Guru Venkataramani. 2014. CC-Hunter: Uncovering Covert Timing Channels on Shared Processor Hardware. In 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 216–228.
[11]
Li-Jhan Chen, Hsiang-Yun Cheng, Po-Han Wang, and Chia-Lin Yang. 2017. Improving GPGPU Performance via Cache Locality Aware Thread Block Scheduling. IEEE Computer Architecture Letters 16, 2 (2017), 127–131.
[12]
Jack Choquette. 2017. VOLTA: Programmability and Performance. https://www.old.hotchips.org/wp-content/uploads/hc_archives/hc29/HC29.21-Monday-Pub/HC29.21.10-GPU-Gaming-Pub/HC29.21.132-Volta-Choquette-NVIDIA-Final3.pdf.
[13]
Angelo Corana. 2015. Architectural Evolution of NVIDIA GPUs for High-Performance Computing. https://doi.org/10.13140/RG.2.1.1496.1042
[14]
William J. Dally and Brian Towles. 2001. Route Packets, Not Wires: On-Chip Inteconnection Networks. In Proceedings of the 38th Annual Design Automation Conference (DAC). 684–689.
[15]
William J. Dally and Brian Towles. 2004. Principles and Practices of Interconnection Networks. Elsevier.
[16]
Sankha Baran Dutta, Hoda Naghibijouybari, Nael Abu-Ghazaleh, Andres Márquez, and Kevin Barker. 2021. Leaky Buddies: Cross-Component Covert Channels on Integrated CPU-GPU Systems. In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). 972–984.
[17]
Dmitry Evtyushkin, Dmitry Ponomarev, and Nael Abu-Ghazaleh. 2015. Covert Channels through Branch Predictors: A Feasibility Study. In Proceedings of the Fourth Workshop on Hardware and Architectural Support for Security and Privacy(Portland, Oregon) (HASP ’15). Association for Computing Machinery, New York, NY, USA, Article 5, 8 pages.
[18]
Radeon Technologies group. 2017. AMD’s Redeon Next Generation GPU Architecture. https://www.old.hotchips.org/wp-content/uploads/hc_archives/hc29/HC29.21-Monday-Pub/HC29.21.10-GPU-Gaming-Pub/HC29.21.120-Radeon-Vega10-Mantor-AMD-f1.pdf.
[19]
Daniel Gruss, Clémentine Maurice, Klaus Wagner, and Stefan Mangard. 2016. Flush+Flush: A Fast and Stealthy Cache Attack. In Proceedings of the 13th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment - Volume 9721 (San Sebastián, Spain) (DIMVA 2016). Springer-Verlag, Berlin, Heidelberg, 279–299.
[20]
Youngkwang Han and John Kim. 2019. A Novel Covert Channel Attack Using Memory Encryption Engine Cache. In 2019 56th ACM/IEEE Design Automation Conference (DAC). 1–6.
[21]
Casen Hunger, Mikhail Kazdagli, Ankit Rawat, Alex Dimakis, Sriram Vishwanath, and Mohit Tiwari. 2015. Understanding Contention-Based Channels and Using Them for Defense. In 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA). 639–650.
[22]
Natalie Enright Jerger and Li-Shiuan Peh. 2009. On-Chip Networks, Synthesis Lectures on Computer Architecture. Morgan & cLaypool publishers(2009).
[23]
Zhen H. Jiang, Yunsi Fei, and David Kaeli. 2016. A Complete Key Recovery Timing Attack on a GPU. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA). 394–405.
[24]
Zhen H. Jiang, Yunsi Fei, and David Kaeli. 2019. Exploiting Bank Conflict-Based Side-Channel Timing Leakage of GPUs. ACM Trans. Archit. Code Optim. 16, 4, Article 42 (Nov. 2019), 24 pages.
[25]
Gurunath Kadam, Danfeng Zhang, and Adwait Jog. 2018. RCoal: Mitigating GPU Timing Attack via Subwarp-Based Randomized Coalescing Techniques. In 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA). 156–167.
[26]
Gurunath Kadam, Danfeng Zhang, and Adwait Jog. 2020. BCoal: Bucketing-Based Memory Coalescing for Efficient and Secure GPUs. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). 570–581.
[27]
Elmira Karimi, Yunsi Fei, and David Kaeli. 2020. Hardware/Software Obfuscation against Timing Side-channel Attack on a GPU. In 2020 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). 122–131.
[28]
Stephen W. Keckler, William J. Dally, Brucek Khailany, Michael Garland, and David Glasco. 2011. GPUs and the Future of Parallel Computing. IEEE Micro 31, 5 (2011), 7–17.
[29]
Mahmoud Khairy, Vadim Nikiforov, David Nellans, and Timothy G. Rogers. 2020. Locality-Centric Data and Threadblock Management for Massive GPUs. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 1022–1036.
[30]
Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt, and Timothy G. Rogers. 2020. Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). 473–486.
[31]
Prabhat Kumar, Yan Pan, John Kim, Gokhan Memik, and Alok Choudhary. 2009. Exploring Concentration and Channel Slicing in On-chip Network Router. In 2009 3rd ACM/IEEE International Symposium on Networks-on-Chip (NOCS). 276–285.
[32]
Butler W. Lampson. 1973. A Note on the Confinement Problem. Commun. ACM 16, 10 (Oct. 1973), 613–615.
[33]
Minseok Lee, Gwangsun Kim, John Kim, Woong Seo, Yeongon Cho, and Soojung Ryu. 2016. iPAWS: Instruction-Issue Pattern-Based Adaptive Warp Scheduling for GPGPUs. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA). 370–381.
[34]
Minseok Lee, Seokwoo Song, Joosik Moon, John Kim, Woong Seo, Yeongon Cho, and Soojung Ryu. 2014. Improving GPGPU Resource Utilization Through Alternative Thread Block Scheduling. In 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA). 260–271.
[35]
Erik Lindholm, John Nickolls, Stuart Oberman, and John Montrym. 2008. NVIDIA Tesla: A Unified Graphics and Computing Architecture. IEEE Micro 28, 2 (2008), 39–55.
[36]
Bartosz Lipinski, Wojciech Mazurczyk, and Krzysztof Szczypiorski. 2014. Improving Hard Disk Contention-Based Covert Channel in Cloud Computing. In 2014 IEEE Security and Privacy Workshops. 100–107.
[37]
Fangfei Liu, Yuval Yarom, Qian Ge, Gernot Heiser, and Ruby B. Lee. 2015. Last-Level Cache Side-Channel Attacks are Practical. In 2015 IEEE Symposium on Security and Privacy. 605–622.
[38]
Chao Luo, Yunsi Fei, Pei Luo, Saoni Mukherjee, and David Kaeli. 2015. Side-Channel Power Analysis of a GPU AES Implementation. In 2015 33rd IEEE International Conference on Computer Design (ICCD). 281–288.
[39]
Robert Martin, John Demme, and Simha Sethumadhavan. 2012. TimeWarp: Rethinking Timekeeping and Performance Monitoring Mechanisms to Mitigate Side-Channel Attacks. In 2012 39th Annual International Symposium on Computer Architecture (ISCA). 118–129.
[40]
Clémentine Maurice, Christoph Neumann, Olivier Heen, and Aurélien Francillon. 2015. C5: Cross-Cores Cache Covert Channel. In Proceedings of the 12nd International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment(DIMVA 2015). 46–64.
[41]
David N. Muchene, Klevis Luli, and Craig A. Shue. 2013. Reporting Insider Threats via Covert Channels. In 2013 IEEE Security and Privacy Workshops. 68–71.
[42]
Hoda Naghibijouybari, Khaled N. Khasawneh, and Nael Abu-Ghazaleh. 2017. Constructing and Characterizing Covert Channels on GPGPUs. In 2017 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 354–366.
[43]
Hoda Naghibijouybari, Ajaya Neupane, Zhiyun Qian, and Nael Abu-Ghazaleh. 2018. Rendered Insecure: GPU Side Channel Attacks Are Practical. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (Toronto, Canada) (CCS ’18). Association for Computing Machinery, New York, NY, USA, 2139–2153.
[44]
Nan Jiang, Daniel U. Becker, George Michelogiannakis, James Balfour, Brian Towles, John Kim, and William J. Dally. 2013. A Detailed and Flexible Cycle-Accurate Network-on-Chip Simulator. In 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 86–96.
[45]
NVIDIA Corporation. 2016. Nvidia Tesla P100 The Most Advanced Datacenter Accelerator Ever Built Featuring pascal GP100, the World’s Fastest GPU.
[46]
NVIDIA Corporation. 2017. Nvidia Tesla V100 GPU Architecture, The World’s Most Advanced Data Center GPU.
[47]
NVIDIA Corporation. 2018. CUDA C Programming Guide.
[48]
NVIDIA Corporation. 2018. Nvidia Turing GPU Architecture, Graphics Reinvented.
[49]
NVIDIA Corporation. 2020. Multi-Process Service.
[50]
NVIDIA Corporation. 2020. NVIDIA Multi-Instance GPU User Guide.
[51]
Riccardo Paccagnella, Licheng Luo, and Christopher W. Fletcher. 2021. Lord of the Ring(s): Side Channel Attacks on the CPU On-Chip Ring Interconnect Are Practical. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 645–662.
[52]
Colin Percival. 2005. Cache Missing for Fun and Profit.
[53]
Peter Pessl, Daniel Gruss, Clémentine Maurice, Michael Schwarz, and Stefan Mangard. 2016. DRAMA: Exploiting DRAM Addressing for Cross-CPU Attacks. In 25th USENIX Security Symposium (USENIX Security 16). 565–581.
[54]
Lili Qiu, Yin Zhang, Feng Wang, Mi Kyung, and Han Ratul Mahajan. 1985. Trusted Computer System Evaluation Criteria. In National Computer Security Center. Citeseer.
[55]
Steve Rennich. 2011. CUDA C/C++ Streams and Concurrency. In GPU Technology Conference.
[56]
Thomas Ristenpart, Eran Tromer, Hovav Shacham, and Stefan Savage. 2009. Hey, You, Get Off of My Cloud: Exploring Information Leakage in Third-Party Compute Clouds. In Proceedings of the ACM Conference on Computer and Communications Security(CCS ’09). 199–212.
[57]
Timothy G. Rogers, Mike O’Connor, and Tor M. Aamodt. 2012. Cache-Conscious Wavefront Scheduling. In 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 72–83.
[58]
Timothy G. Rogers, Mike O’Connor, and Tor M. Aamodt. 2013. Divergence-Aware Warp Scheduling. In 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 99–110.
[59]
Benjamin Semal, Konstantinos Markantonakis, Raja Naeem Akram, and Jan Kalbantner. 2020. Leaky Controller: Cross-VM Memory Controller Covert Channel on Multi-Core Systems. In IFIP International Conference on ICT Systems Security and Privacy Protection. 3–16.
[60]
Ankit Sethia, Davoud A. Jamshidi, and Scott Mahlke. 2015. Mascar: Speeding up GPU Warps by Reducing Memory Pitstops. In 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA). 174–185.
[61]
William Stallings. 2006. Cryptography and Network Security, 4/E. Pearson Education India.
[62]
Dean Sullivan, Orlando Arias, Travis Meade, and Yier Jin. 2018. Microarchitectural Minefields: 4K-Aliasing Covert Channel and Multi-Tenant Detection in Iaas Clouds. In NDSS.
[63]
Yao Wang and G. Edward Suh. 2012. Efficient Timing Channel Protection for On-Chip Networks. In 2012 6th IEEE/ACM International Symposium on Networks-on-Chip (NOCS). 142–151.
[64]
Zhenghong Wang and Ruby B. Lee. 2006. Covert and Side Channels Due to Processor Architecture. In 2006 22nd Annual Computer Security Applications Conference (ACSAC’06). 473–482.
[65]
Zhenghong Wang and Ruby B. Lee. 2008. A Novel Cache Architecture with Enhanced Performance and Security. In 2008 41st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 83–93.
[66]
Zhenning Wang, Jun Yang, Rami Melhem, Bruce Childers, Youtao Zhang, and Minyi Guo. 2016. Simultaneous Multikernel GPU: Multi-tasking Throughput Processors via Fine-Grained Sharing. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA). 358–369.
[67]
Hassan MG Wassel, Ying Gao, Jason K Oberg, Ted Huffmire, Ryan Kastner, Frederic T Chong, and Timothy Sherwood. 2013. SurfNoC: A Low Latency and Provably Non-Interfering Approach to Secure Networks-On-Chip. ACM SIGARCH Computer Architecture News 41, 3 (2013), 583–594.
[68]
Zhenyu Wu, Zhang Xu, and Haining Wang. 2012. Whispers in the Hyper-space: High-speed Covert Channel Attacks in the Cloud. In 21st USENIX Security Symposium (USENIX Security 12). USENIX Association, Bellevue, WA, 159–173.
[69]
Qiumin Xu, Hyeran Jeon, Keunsoo Kim, Won W. Ro, and Murali Annavaram. 2016. Warped-Slicer: Efficient Intra-SM Slicing through Dynamic Resource Partitioning for GPU Multiprogramming. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA). 230–242.
[70]
Qiumin Xu, Hoda Naghibijouybari, Shibo Wang, Nael Abu-Ghazaleh, and Murali Annavaram. 2019. GPUGuard: Mitigating Contention Based Side and Covert Channel Attacks on GPUs. In Proceedings of the ACM International Conference on Supercomputing (Phoenix, Arizona) (ICS ’19). Association for Computing Machinery, New York, NY, USA, 497–509.
[71]
Yunjing Xu, Michael Bailey, Farnam Jahanian, Kaustubh Joshi, Matti Hiltunen, and Richard Schlichting. 2011. An Exploration of L2 Cache Covert Channels in Virtualized Environments. In Proceedings of the 3rd ACM Workshop on Cloud Computing Security Workshop (Chicago, Illinois, USA) (CCSW ’11). Association for Computing Machinery, New York, NY, USA, 29–40.
[72]
Mengjia Yan, Yasser Shalabi, and Josep Torrellas. 2016. ReplayConfusion: Detecting Cache-Based Covert Channel Attacks Using Record and Replay. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 1–14.

Cited By

View all
  • (2024)Write+Sync: Software Cache Write Covert Channels Exploiting Memory-Disk SynchronizationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341425519(8066-8078)Online publication date: 2024
  • (2024)Veiled Pathways: Investigating Covert and Side Channels Within GPU Uncore2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00088(1169-1183)Online publication date: 2-Nov-2024
  • (2024)Ghost Arbitration: Mitigating Interconnect Side-Channel Timing Attacks in GPU2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00086(1138-1152)Online publication date: 2-Nov-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
MICRO '21: MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture
October 2021
1322 pages
ISBN:9781450385572
DOI:10.1145/3466752
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Covert Channel
  2. GPU
  3. Network-on-Chip

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

MICRO '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 484 of 2,242 submissions, 22%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)265
  • Downloads (Last 6 weeks)21
Reflects downloads up to 14 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Write+Sync: Software Cache Write Covert Channels Exploiting Memory-Disk SynchronizationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341425519(8066-8078)Online publication date: 2024
  • (2024)Veiled Pathways: Investigating Covert and Side Channels Within GPU Uncore2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00088(1169-1183)Online publication date: 2-Nov-2024
  • (2024)Ghost Arbitration: Mitigating Interconnect Side-Channel Timing Attacks in GPU2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00086(1138-1152)Online publication date: 2-Nov-2024
  • (2024)Uncovering Real GPU NoC Characteristics: Implications on Interconnect Architecture2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00070(885-898)Online publication date: 2-Nov-2024
  • (2024)Salus: Efficient Security Support for CXL-Expanded GPU Memory2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00027(1-15)Online publication date: 2-Mar-2024
  • (2024)Whispering Pixels: Exploiting Uninitialized Register Accesses in Modern GPUs2024 IEEE 9th European Symposium on Security and Privacy (EuroS&P)10.1109/EuroSP60621.2024.00026(345-360)Online publication date: 8-Jul-2024
  • (2024)Owl: Differential-Based Side-Channel Leakage Detection for CUDA Applications2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58291.2024.00044(362-376)Online publication date: 24-Jun-2024
  • (2024)SideLink: Exposing NVLink to Covert and Side-Channel Attacks Official Work-in-Progress PaperSecurity, Privacy, and Applied Cryptography Engineering10.1007/978-3-031-80408-3_2(6-15)Online publication date: 9-Dec-2024
  • (2023)The First Concept and Real-world Deployment of a GPU-based Thermal Covert Channel: Attack and Countermeasures2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10137090(1-6)Online publication date: Apr-2023
  • (2023)Spy in the GPU-box: Covert and Side Channel Attacks on Multi-GPU SystemsProceedings of the 50th Annual International Symposium on Computer Architecture10.1145/3579371.3589080(1-13)Online publication date: 17-Jun-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media