research-article

Open access

Grafting sockets for fast container networking

Authors:

Hajime TazakiAuthors Info & Claims

ANCS '18: Proceedings of the 2018 Symposium on Architectures for Networking and Communications Systems

Pages 15 - 27

https://doi.org/10.1145/3230718.3230723

Published: 23 July 2018 Publication History

Abstract

This paper proposes a novel approach for fast container networking that enables applications in containers to utilize host network stacks directly with proper access control. Our approach, called socket-grafting, offers a new socket-layer communication channel by grafting sockets in containers onto sockets in host network stacks. In contrast to recent approaches that make network stacks faster by using high-speed packet I/O techniques, socket-grafting makes container networking faster by bypassing a container's network stack and virtual interfaces. As a concrete implementation of socket-grafting, we designed AF_GRAFT, which is a new address family for the socket layer. AF_GRAFT performs interconnection between the two socket layers in the container and host network stacks. In this paper, we describe socket-grafting, AF_GRAFT design, and its implementation on Linux. Our evaluation results demonstrate that the proposed method doubled throughput and reduced latency by 23% compared with traditional NAT-based container networking, and improved the network performance of containerized HTTP servers and message queues.

References

[1]

2016. XDP - IO Visor Project. (2016). https://www.iovisor.org/technology/xdp

[2]

2017. Intel Data Plane Development Kit. (2017). http://dpdk.org/

[3]

2017. Linux Containers. (2017). https://linuxcontainers.org/

[4]

2018. Apache OpenWhisk is a serverless event-based programming service and an Apache Incubator project. (March 2018). https://github.com/apache/incubator-openwhisk

[5]

2018. Cilium. (2018). https://cilium.io/

[6]

2018. Container Network Interface - networking for Linux containers. (2018). https://github.com/containernetworking/cni

[7]

2018. Kubernetes | Production-Grade Container Orchestration. (2018). https://kubernetes.io/

[8]

2018. networking:iproute2 {Linux Foundation Wiki}. (Feb. 2018). https://wiki.linuxfoundation.org/networking/iproute2

[9]

2018. rkt, a security-minded, standards-based container engine. (2018). https://coreos.com/rkt/

[10]

2018. upa/af-graft. (2018). https://github.com/upa/af-graft

[11]

Jake Adriaens. 2017. Google Cloud Platform Blog: Andromeda 2.1 reduces GCP's intra-zone latency by 40%. (Nov. 2017). https://cloudplatform.googleblog.com/2017/11/Andromeda-2-1-reduces-GCPs-intra-zone-latency-by-40-percent.html

[12]

J. Anderson, H. Hu, U. Agarwal, C. Lowery, H. Li, and A. Apon. 2016. Performance considerations of network functions virtualization using containers. In 2016 International Conference on Computing, Networking and Communications (ICNC). 1--7.

[13]

A. Balalaie, A. Heydarnoori, and P. Jamshidi. 2016. Microservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture. IEEE Software 33, 3 (May 2016), 42--52.

Digital Library

[14]

J. Claassen, R. Koning, and P. Grosso. 2016. Linux containers networking: Performance and scalability of kernel modules. In NOMS 2016 - 2016 IEEE/MP Network Operations and Management Symposium. 713--717.

[15]

Jonathan Corbet. 2007. Large receive offload {LWN.net}. (Aug. 2007). https://lwn.net/Articles/243949/

[16]

Hideki Eiraku, Yasushi Shinjo, Calton Pu, Younggyun Koh, and Kazuhiko Kato. 2009. Fast Networking with Socket-outsourcing in Hosted Virtual Machine Environments. In Proceedings of the 2009 ACM Symposium on Applied Computing (SAC '09). ACM, New York, NY, USA, 310--317.

Digital Library

[17]

W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. 2015. An updated performance comparison of virtual machines and Linux containers. In 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 171--172.

[18]

Sean Hefty. 2012. RSOCKETS. (March 2012). http://downloads.openfabrics.org/Media/Monterey_2012/2012_Workshop_Mon_Rsockets.pdf

[19]

Michio Honda, Felipe Huici, Costin Raiciu, Joao Araujo, and Luigi Rizzo. 2014. Rekindling Network Protocol Innovation with User-level Stacks. SIGCOMM Comput. Commun. Rev. 44, 2 (April 2014), 52--58.

Digital Library

[20]

Yang Hu, Mingcong Song, and Tao Li. 2017. Towards "Full Containerization" in Containerized Network Function Virtualization. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '17). ACM, New York, NY, USA, 467--481.

Digital Library

[21]

Junichiro Itoh. 2003. Implementing AF-independent application. (May 2003). http://www.kame.net/newsletter/19980604/

[22]

James Lewis and Martin Fowler. 2014. Microservices. (March 2014). https://martinfowler.com/articles/microservices.html

[23]

Steven McCanne and Van Jacobson. 1993. The BSD Packet Filter: A New Architecture for User-level Packet Capture. In Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings (USENIX'93). USENIX Association, Berkeley, CA, USA, 2--2. http://dl.acm.org/citation.cfm?id=1267303.1267305

Digital Library

[24]

Dirk Merkel. 2014. Docker: Lightweight Linux Containers for Consistent Development and Deployment. Linux Journal. 2014, 239, Article 2 (March 2014). http://dl.acm.org/citation.cfm?id=2600239.2600241

Digital Library

[25]

R. Morabito, J. KjÃd'llman, and M. Komu. 2015. Hypervisors vs. Lightweight Virtualization: A Performance Comparison. In 2015 IEEE International Conference on Cloud Engineering. 386--393.

Digital Library

[26]

Cristina Opriceana and Hajime Tazaki.2017. Network stack personality in Android phone. (2017). https://www.netdevconf.org/2.2/papers/tazaki-mptcp-talk.pdf

[27]

Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2014. Arrakis: The Operating System is the Control Plane. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, CO, 1--16. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/peter

Digital Library

[28]

Kevin Pulo. 2009. Fun with LD_PRELOAD. (Jan. 2009). https://linux.org.au/conf/2009/slides/172.pdf

[29]

Luigi Rizzo. 2012. netmap: A Novel Framework for Fast Packet I/O. In 2012 USENIX Annual Technical Conference (USENIX ATC 12). USENIX Association, Boston, MA, 101--112. https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo

Digital Library

[30]

Jay Schulist, Daniel Borkmann, and Alexei Starovoitov. 2018. Linux Socket Filtering aka Berkeley Packet Filter (BPF). (Jan. 2018). https://www.kernel.org/doc/Documentation/networking/filter.txt

[31]

Amazon Web Services. 2018. AWS Lambda - Serverless Compute. (2018). https://aws.amazon.com/lambda/?nc1=h_ls

[32]

Solarflare. 2013. OpenOnload. (2013). http://www.openonload.org/

[33]

Dinesh Subhraveti, Sri Goli, Serge Hallyn, Ravi Chamarthy, and Christos Kozyrakis. 2017. AppSwitch: Resolving the Application Identity Crisis. (2017). arXiv:arXiv:1711.02294

[34]

Mellanox Technologies. 2018. Mellanox/sockperf: Network Benchmarking Utility. (March 2018). https://github.com/Mellanox/sockperf

[35]

Kenichi Yasukata, Michio Honda, Douglas Santry, and Lars Eggert. 2016. StackMap: Low-Latency Networking with the OS Stack and Dedicated NICs. In 2016 USENIX Annual Technical Conference (USENIX ATC 16). USENIX Association, Denver, CO. https://www.usenix.org/conference/atc16/technical-sessions/presentation/stackmap-low-latency-networking-os-stack-and

Digital Library

[36]

Tianlong Yu, Shadi Abdollahian Noghabi, Shachar Raindel, Hongqiang Liu, Jitu Padhye, and Vyas Sekar. 2016. FreeFlow: High Performance Container Networking. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (HotNets '16). ACM, New York, NY, USA, 43--49.

Digital Library

[37]

Wei Zhang, Guyue Liu, Wenhui Zhang, Neel Shah, Phillip Lopreiato, Gregoire Todeschi, K.K. Ramakrishnan, and Timothy Wood. 2016. OpenNetVM: A Platform for High Performance Network Service Chains. In Proceedings of the 2016 Workshop on Hot Topics in Middleboxes and Network Function Virtualization (HotMIddlebox '16). ACM, New York, NY, USA, 26--31. https://doi.org/2940147.2940155

[38]

Yang Zhao, Nai Xia, Chen Tian, Bo Li, Yizhou Tang, Yi Wang, Gong Zhang, Rui Li, and Alex X. Liu. 2017. Performance of Container Networking Technologies. In Proceedings of the Workshop on Hot Topics in Container Networking and Networked Systems (HotConNet '17). ACM, New York, NY, USA, 1--6.

Digital Library

Cited By

Matsumoto NSuda A(2024)bypass4netns: Accelerating TCP/IP Communications in Rootless ContainersProceedings of the Asian Internet Engineering Conference 202410.1145/3674213.3674221(67-75)Online publication date: 9-Aug-2024
https://dl.acm.org/doi/10.1145/3674213.3674221
Zhang LPang KXu J(2024)High-performance microservice differentiated domain communication technologyComputing10.1007/s00607-023-01232-3106:11(3503-3535)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1007/s00607-023-01232-3
Lee KLee KPark HHwang JYoo C(2023)Autothrottle: Satisfying Network Performance Requirements for ContainersIEEE Transactions on Cloud Computing10.1109/TCC.2022.318639711:2(2096-2109)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TCC.2022.3186397
Show More Cited By

Index Terms

Grafting sockets for fast container networking
1. Networks
  1. Network services
    1. Cloud computing
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Communications management

Recommendations

Automated Multi-Swarm Networking with Open Baton NFV MANO Framework
Euro-Par 2018: Parallel Processing Workshops
Abstract
Container-based Network Functions Virtualization (NFV) and multi-site/multi-cluster service orchestration are a critical topic in the field of ICT infrastructure. Academia, Industry and Open Source projects are actively working on the technology. ...
Optimization of yard remarshalling operations in automated container terminals
Abstract
When an automated container terminal (ACT) yard is busy, the loading, unloading, container retrieval, and grounding of different liners may be performed in the same time window, which can lead to conflicts in yard crane operations on the seaside ...
Analysis on container port capacity: a Markovian modeling approach

Container ports handle outbound, inbound, and transshipment containers plying between the area for vessels on the quay and the storage space in the yard. Port operators typically concentrate their efforts on the container handling process with the aims ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ANCS '18: Proceedings of the 2018 Symposium on Architectures for Networking and Communications Systems

July 2018

181 pages

ISBN:9781450359023

DOI:10.1145/3230718

Program Chairs:
Theophilus Benson
Brown University
,
Noa Zilberman
University of Cambridge

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ANCS '18

Sponsor:

ANCS '18: Symposium on Architectures for Networking and Communications Systems

July 23 - 24, 2018

New York, Ithaca

Acceptance Rates

Overall Acceptance Rate 88 of 314 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
1,892
Total Downloads

Downloads (Last 12 months)198
Downloads (Last 6 weeks)24

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Matsumoto NSuda A(2024)bypass4netns: Accelerating TCP/IP Communications in Rootless ContainersProceedings of the Asian Internet Engineering Conference 202410.1145/3674213.3674221(67-75)Online publication date: 9-Aug-2024
https://dl.acm.org/doi/10.1145/3674213.3674221
Zhang LPang KXu J(2024)High-performance microservice differentiated domain communication technologyComputing10.1007/s00607-023-01232-3106:11(3503-3535)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1007/s00607-023-01232-3
Lee KLee KPark HHwang JYoo C(2023)Autothrottle: Satisfying Network Performance Requirements for ContainersIEEE Transactions on Cloud Computing10.1109/TCC.2022.318639711:2(2096-2109)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TCC.2022.3186397
Szalay MMátray PToka L(2023)Real-Time FaaS: Towards a Latency Bounded Serverless CloudIEEE Transactions on Cloud Computing10.1109/TCC.2022.315146911:2(1636-1650)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TCC.2022.3151469
Zha ZWang AGuo YLi QSun KChen S(2022)EZPath: Expediting Container Network Traffic via Programmable Switches2022 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking55013.2022.9829818(1-8)Online publication date: 13-Jun-2022
https://doi.org/10.23919/IFIPNetworking55013.2022.9829818
Choochotkaew SChiba TTrent SAmaral M(2022)Bypass Container Overlay Networks with Transparent BPF-driven Socket Replacement2022 IEEE 15th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD55607.2022.00033(134-143)Online publication date: Jul-2022
https://doi.org/10.1109/CLOUD55607.2022.00033
Kim ELee KYoo C(2022)DepCon: Achieving Network SLO for High Performance CloudsEuro-Par 2021: Parallel Processing Workshops10.1007/978-3-031-06156-1_27(339-351)Online publication date: 9-Jun-2022
https://doi.org/10.1007/978-3-031-06156-1_27
Grigoryan GKwon MRafique M(2020)Extending the Control Plane of Container Orchestrators for I/O Virtualization2020 2nd International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC)10.1109/CANOPIEHPC51917.2020.00006(1-7)Online publication date: Nov-2020
https://doi.org/10.1109/CANOPIEHPC51917.2020.00006
Bacou MTodeschi GHagimont DTchana A(2019)Nested Virtualization Without the NestProceedings of the 48th International Conference on Parallel Processing10.1145/3337821.3337840(1-10)Online publication date: 5-Aug-2019
https://dl.acm.org/doi/10.1145/3337821.3337840

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents