research-article

Open access

Breakfast of champions: towards zero-copy serialization with NIC scatter-gather

Authors:

Deepti Raghavan,

Irene ZhangAuthors Info & Claims

HotOS '21: Proceedings of the Workshop on Hot Topics in Operating Systems

Pages 199 - 205

https://doi.org/10.1145/3458336.3465287

Published: 03 June 2021 Publication History

Abstract

Microsecond I/O will make data serialization a major bottleneck for datacenter applications. Serialization is fundamentally about data movement: serialization libraries coalesce and flatten in-memory data structures into a single transmittable buffer. CPU-based serialization approaches will hit a performance limit due to data movement overheads and be unable to keep up with modern networks.

We observe that widely deployed NICs possess scatter-gather capabilities that can be re-purposed to accelerate serialization's core task of coalescing and flattening in-memory data structures. It is possible to build a completely zero-copy, zero-allocation serialization library with commodity NICs. Doing so introduces many research challenges, including using the hardware capabilities efficiently for a wide variety of non-uniform data structures, making application memory available for zero-copy I/O, and ensuring memory safety.

References

[1]

Apache Software Foundation. Hadoop. https://hadoop.apache.org.

[2]

Apache Software Foundation. Apache avro. https://avro.apache.org/, 2012.

[3]

Apache Sofware Foundation. Apache thrift. https://thrift.apache.org/download, 2017.

[4]

K. Ashcraft and D. Engler. Using programmer-written compiler extensions to catch security holes. In IEEE Symposium on Security and Privacy, 2002.

Digital Library

[5]

L. Barroso, M. Marty, D. Patterson, and P. Ranganathan. Attack of the killer microseconds. Communicatons of the ACM, 2017.

Digital Library

[6]

I. Calciu, I. Puddu, A. Kolli, A. Nowatzyk, J. Gandhi, O. Mutlu, and P. Subrahmanyam. Project pberry: Fpga acceleration for remote memory. In HotOS, 2019.

Digital Library

[7]

lwIP - A Lightweight TCP/IP stack - Summary. https://savannah.nongnu.org/projects/lwip/.

[8]

A. Gainaru, R. L. Graham, A. Polyakov, and G. Shainer. Using infiniband hardware gather-scatter capabilities to optimize mpi all-to-all. In EuroMPI 2016, 2016.

Digital Library

[9]

A. Gallatin, J. Chase, and K. Yocum. Trapeze/ip: Tcp/ip at near-gigabit speeds. In ATC, 1999.

Digital Library

[10]

Y. Gan, Y. Zhang, D. Cheng, A. Shetty, P. Rathi, N. Katarki, A. Bruno, J. Hu, B. Ritchken, B. Jackson, et al. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In ASPLOS, 2019.

Digital Library

[11]

mprotect(2) - linux manual page. https://man7.org/linux/man-pages/man2/mprotect.2.html.

[12]

writev(2) - linux man page. https://linux.die.net/man/2/writev.

[13]

gRPC Authors. grpc: A high-performance, open source universal rpc framework. https://grpc.io/.

[14]

Storage performance development kit. https://spdk.io/.

[15]

J. Jang, S. J. Jung, S. Jeong, J. Heo, H. Shin, T. J. Ham, and J. W. Lee. A specialized architecture for object serialization with applications to big data analytics. In ISCA, 2020.

Digital Library

[16]

S. Jha, J. Behrens, T. Gkountouvas, M. Milano, W. Song, E. Tremel, R. V. Renesse, S. Zink, and K. P. Birman. Derecho: Fast state machine replication for cloud services. ACM Transactions on Computer Systems, 2019.

Digital Library

[17]

A. Kalia, M. Kaminsky, and D. Andersen. Datacenter rpcs can be general and fast. In NSDI, 2019.

Digital Library

[18]

S. Kanev, J. P. Darago, K. Hazelwood, P. Ranganathan, T. Moseley, G.-Y. Wei, and D. Brooks. Profiling a warehouse-scale computer. In ISCA, 2015.

Digital Library

[19]

A. Kesavan, R. Ricci, and R. Stuntsman. To copy or not to copy: Making in-memory databases fast on modern nics. https://rstutsman.github.io/papers/copy-not-to-copy.pdf.

[20]

Kyro. https://github.com/EsotericSoftware/kryo, Accessed January 23, 2021.

[21]

G. Langdale and D. Lemire. Parsing gigabytes of json per second. The VLDB Journal, 2019.

[22]

A. Narayan, A. Panda, M. Alizadeh, H. Balakrishnan, A. Krishnamurthy, and S. Shenker. Bertha: Tunneling through the network api. In HotNets, 2020.

Digital Library

[23]

K. Nguyen, L. Fang, C. Navasca, G. Xu, B. Demsky, and S. Lu. Skyway: Connecting managed heaps in distributed big data systems. In ASPLOS, 2018.

Digital Library

[24]

S. Novakovic, A. Daglis, E. Bugnion, B. Falsafi, and B. Grot. Scale-out numa. In ASPLOS, 2014.

Digital Library

[25]

Nvidia. Connectx-5. advanced offload capabilities for the most demanding applications. https://www.nvidia.com/en-us/networking/ethernet/connectx-5/.

[26]

A. Ousterhout, J. Fried, J. Behrens, A. Belay, and H. Balakrishnan. Shenango: Achieving high CPU efficiency for latency-sensitive datacenter workloads. In NSDI, 2019.

Digital Library

[27]

S. Peter, J. Li, I. Zhang, D. R. K. Ports, D. Woos, A. Krishnamurthy, T. Anderson, and T. Roscoe. Arrakis: The operating system is the control plane. In OSDI, 2014.

Digital Library

[28]

A. Pourhabibi, S. Gupta, H. Kassir, M. Sutherland, Z. Tian, M. P. Drumond, B. Falsafi, and C. Koch. Optimus prime: Accelerating data transformation in servers. In ASPLOS, 2020.

Digital Library

[29]

G. Prekas, M. Kogias, and E. Bugnion. Zygos: Achieving low tail latency for microsecond-scale networked tasks. In SOSP, 2017.

Digital Library

[30]

A rdma protocol specification. http://rdmaconsortium.org/, 2009.

[31]

redis labs. Redis. https://redis.io/.

[32]

G. Santhanaraman, J. Wu, W. Huang, and D. K. Panda. Designing zero-copy message passing interface derived datatype communication over infiniband: Alternative approaches and performance evaluation. The International Journal of High Performance Computing Applications, 2005.

Digital Library

[33]

R. Taft, I. Sharif, A. Matei, N. VanBenschoten, J. Lewis, T. Grieger, K. Niemi, A. Woods, A. Birzin, R. Poss, P. Bardea, A. Ranade, B. Darnell, B. Gruneir, J. Jaffray, L. Zhang, and P. Mattis. Cockroachdb: The resilient geo-distributed sql database. In SIGMOD, 2020.

Digital Library

[34]

Dpdk: Data plane development kit. https://www.dpdk.org/.

[35]

H.-W. Tseng, Q. Zhao, Y. Zhou, M. Gahagan, and S. Swanson. Morpheus: Creating application objects efficiently for heterogeneous computing. In ISCA, 2016.

Digital Library

[36]

W. Van Oortmerssen. Flatbuffers: a memory efficient serialization library. https://opensource.googleblog.com/2014/06/flatbuffers-memory-efficient.html, 2014.

[37]

K. Varda. Protocol buffers: Google's data interchange form. https://opensource.googleblog.com/2008/07/protocol-buffers-googles-data.html, 2008.

[38]

K. Varda. Cap'n proto. https://capnproto.org/, 2020 (Accessed October 22, 2020).

[39]

A. Wolnikowski, S. Ibanez, J. Stone, C. Kim, R. Manohar, and R. Soulé. Zerializer: Towards zero-copy serialization. In HotOS, 2021.

Digital Library

[40]

B. Yi, J. Xia, L. Chen, and K. Chen. Towards zero copy dataflows using rdma. In SIGCOMM Posters and Demos, 2017.

Digital Library

[41]

M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauly, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In NSDI, 2012.

Digital Library

[42]

I. Zhang, J. Liu, A. Austin, M. L. Roberts, and A. Badam. I'm not dead yet! the role of the operating system in a kernel-bypass era. In HotOS, 2019.

Digital Library

Cited By

Kolokasis IEvdorou GAkram SKozanitis CPapagiannis AZakkak FPratikakis PBilas A(2024)TeraHeap: Exploiting Flash Storage for Mitigating DRAM Pressure in Managed Big Data FrameworksACM Transactions on Programming Languages and Systems10.1145/370059346:4(1-37)Online publication date: 13-Dec-2024
https://dl.acm.org/doi/10.1145/3700593
Gao YHe YLi XZhao BLin HLiang YZhong JZhang HWang JZeng YGui KTong JYang MRoychoudhury APaiva AAbreu RStorey M(2024)An Empirical Study on Low GPU Utilization of Deep Learning JobsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639232(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639232
Qi SMonis LZeng ZWang IRamakrishnan K(2024)SPRIGHT: High-Performance eBPF-Based Event-Driven, Shared-Memory Processing for Serverless ComputingIEEE/ACM Transactions on Networking10.1109/TNET.2024.336656132:3(2539-2554)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1109/TNET.2024.3366561
Show More Cited By

Index Terms

Breakfast of champions: towards zero-copy serialization with NIC scatter-gather
1. Networks
  1. Network architectures
    1. Programming interfaces

Recommendations

Cornflakes: Zero-Copy Serialization for Microsecond-Scale Networking
SOSP '23: Proceedings of the 29th Symposium on Operating Systems Principles

Data serialization is critical for many datacenter applications, but the memory copies required to move application data into packets are costly. Recent zero-copy APIs expose NIC scatter-gather capabilities, raising the possibility of offloading this ...
Alto: lightweight VMs using virtualization-aware managed runtimes
ManLang '18: Proceedings of the 15th International Conference on Managed Languages & Runtimes

Virtualization enables datacenter operators to safely run computations that belong to untrusted tenants. An ideal virtual machine has three properties: a small memory footprint; strong isolation from other VMs and the host OS; and the ability to maintain ...
Rethinking Design Metrics for Datacenter DRAM
MEMSYS '15: Proceedings of the 2015 International Symposium on Memory Systems

Over the years, the evolution of DRAM has provided a little improvement in access latencies, but has been optimized to deliver greater peak bandwidths from the devices. The combined bandwidth in a contemporary multi-socket server system runs into ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HotOS '21: Proceedings of the Workshop on Hot Topics in Operating Systems

June 2021

251 pages

ISBN:9781450384384

DOI:10.1145/3458336

General Chair:
Sebastian Angel
University of Pennsylvania
,
Program Chairs:
Baris Kasikci
University of Michigan
,
Eddie Kohler
Harvard University

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

HotOS '21

Sponsor:

SIGOPS

HotOS '21: Workshop on Hot Topics in Operating Systems

June 1 - 3, 2021

Michigan, Ann Arbor

Upcoming Conference

HOTOS '25

Sponsor:
sigops

Workshop on Hot Topics in Operating Systems

May 14 - 16, 2025

Banff , AB , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,190
Total Downloads

Downloads (Last 12 months)288
Downloads (Last 6 weeks)38

Reflects downloads up to 09 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kolokasis IEvdorou GAkram SKozanitis CPapagiannis AZakkak FPratikakis PBilas A(2024)TeraHeap: Exploiting Flash Storage for Mitigating DRAM Pressure in Managed Big Data FrameworksACM Transactions on Programming Languages and Systems10.1145/370059346:4(1-37)Online publication date: 13-Dec-2024
https://dl.acm.org/doi/10.1145/3700593
Gao YHe YLi XZhao BLin HLiang YZhong JZhang HWang JZeng YGui KTong JYang MRoychoudhury APaiva AAbreu RStorey M(2024)An Empirical Study on Low GPU Utilization of Deep Learning JobsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639232(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639232
Qi SMonis LZeng ZWang IRamakrishnan K(2024)SPRIGHT: High-Performance eBPF-Based Event-Driven, Shared-Memory Processing for Serverless ComputingIEEE/ACM Transactions on Networking10.1109/TNET.2024.336656132:3(2539-2554)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1109/TNET.2024.3366561
Kamath APeter S(2024) (MC) 2 : Lazy MemCopy at the Memory Controller 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00084(1112-1128)Online publication date: 29-Jun-2024
https://doi.org/10.1109/ISCA59077.2024.00084
Seemakhupt KStephens BKhan SLiu SWassel HYeganeh SSnoeren AKrishnamurthy ACuller DLevy HDruschel PKaufmann AMace JFlinn JSeltzer M(2023)A Cloud-Scale Characterization of Remote Procedure CallsProceedings of the 29th Symposium on Operating Systems Principles10.1145/3600006.3613156(498-514)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3600006.3613156
Raghavan DRavi SYuan GThaker PSrivastava SMurray MPenna POusterhout ALevis PZaharia MZhang IDruschel PKaufmann AMace JFlinn JSeltzer M(2023)Cornflakes: Zero-Copy Serialization for Microsecond-Scale NetworkingProceedings of the 29th Symposium on Operating Systems Principles10.1145/3600006.3613137(200-215)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3600006.3613137
Ghemawat SGrandl RPetrovic SWhittaker MPatel PPosva IVahdat ABaumann ACrooks NSchwarzkopf M(2023)Towards Modern Development of Cloud ApplicationsProceedings of the 19th Workshop on Hot Topics in Operating Systems10.1145/3593856.3595909(110-117)Online publication date: 22-Jun-2023
https://dl.acm.org/doi/10.1145/3593856.3595909
Kumar ASivasubramaniam AZhu T(2023)SplitRPC: A {Control + Data} Path Splitting RPC Stack for ML Inference ServingProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/35899747:2(1-26)Online publication date: 22-May-2023
https://dl.acm.org/doi/10.1145/3589974
Kolokasis IEvdorou GAkram SKozanitis CPapagiannis AZakkak FPratikakis PBilas AAamodt TJerger NSwift M(2023)TeraHeap: Reducing Memory Pressure in Managed Big Data FrameworksProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582045(694-709)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582045
Krakowski FRuhland FSchöttner M(2023)High-Performance Object Serialization based on Ahead-of-Time Schema Generation2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00335(2378-2385)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TrustCom60117.2023.00335
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten