US20230179484A1

US20230179484A1 - Automatic configuring of vlan and overlay logical switches for container secondary interfaces

Info

Publication number: US20230179484A1
Application number: US18/102,700
Authority: US
Inventors: Danting LIU; Jianjun SHEN; Wenfeng Liu; Rui Cao; Ran Gu; Donghai HAN
Original assignee: VMware LLC
Current assignee: VMware LLC
Priority date: 2021-06-11
Filing date: 2023-01-28
Publication date: 2023-06-08
Also published as: US11606254B2; US20220400053A1

Abstract

The method of some embodiments allocates a secondary network interface for a pod, which has a primary network interface, in a container network operating on an underlying logical network. The method receives an ND that designates a network segment. The method receives the pod, wherein the pod includes an identifier of the ND. The method then creates a secondary network interface for the pod and connects the secondary network interface to the network segment. In some embodiments, the pods include multiple ND identifiers that each identify a network segment. The method of such embodiments creates multiple secondary network interfaces and attaches the multiple network segments to the multiple secondary network interfaces.

Description

Container networks (e.g., Kubernetes) are an increasingly popular type of network system for deploying applications in datacenters. The pods of containers produced by such a system can be deployed more rapidly than virtual machines (VMs) or physical computers. Therefore, a deployment can be scaled up or down to meet demand more rapidly than is typical for VMs or physical computers. In addition, a set of containers in a container network system has less overhead and can generally perform the same tasks faster than a corresponding VM would.
In present container based network systems (e.g., Kubernetes) pods are instantiated with an automatically configured primary interface for communicating with outside devices (e.g., physical or virtual machines or containers separate from the pod). However, existing container based network systems do not have a convenient way of adding secondary interfaces to a pod. For some container network based applications, multiple interfaces for a single pod are necessary. However, in the existing art, there is no way to automatically add additional interfaces to a pod. Therefore, there is a need in the art for an automated way to add secondary interfaces to a pod.

BRIEF SUMMARY

The method of some embodiments allocates a secondary network interface for a pod, which has a primary network interface, in a container network operating on an underlying logical network. The method receives a network attachment definition (ND) that designates a network segment. The method receives the pod, wherein the pod includes an identifier of the ND. The method then creates a secondary network interface for the pod and connects the secondary network interface to the network segment. In some embodiments, the pods include multiple ND identifiers that each identify a network segment. The method of such embodiments creates multiple secondary network interfaces and attaches the multiple network segments to the multiple secondary network interfaces.
Designating the network segment includes identifying a network segment created on the logical network before the ND is received in some embodiments. The method may further include directing the logical network to modify the network segment according to a set of attributes in the received ND.
Designating the network segment includes providing a set of attributes of the network segment in some embodiments. The method of such embodiments further includes directing the logical network to create the network segment according to the received set of attributes. The set of attributes may include a network type, where the network type is a VLAN-backed network segment or an overlay-backed network segment.
In some embodiments in which a pod includes multiple ND identifiers, for one set of NDs, each ND designates a network segment by identifying a network segment created on the logical network before the ND is received while for another set of NDs, each ND designates a network segment by providing a set of attributes of the network segment. The method of such embodiments further includes directing the logical network to create the second set of network segments according to the received set of attributes.
The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, the Detailed Description, the Drawings, and the Claims is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, the Detailed Description, and the Drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appended claims. However, for purposes of explanation, several embodiments of the invention are set forth in the following figures.

FIG. 1 illustrates an example of a control system of some embodiments of the invention.

FIG. 2 illustrates an example of a logical network for a virtual private cloud.

FIG. 3 illustrates pods implemented on VMs of a host computer.

FIG. 4 conceptually illustrates pods with interfaces to one or more network segments.

FIG. 5 illustrates a communication sequence of some embodiments for adding a secondary interface to a pod.

FIG. 6 conceptually illustrates a process of some embodiments for allocating a secondary network interface for a pod with a primary network interface.

FIG. 7 conceptually illustrates a computer system with which some embodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.
The method of some embodiments allocates a secondary network interface for a pod, which has a primary network interface, in a container network operating on an underlying logical network. The method receives an ND that designates a network segment. The method receives the pod, wherein the pod includes an identifier of the ND. The method then creates a secondary network interface for the pod and connects the secondary network interface to the network segment. In some embodiments, the pods include multiple ND identifiers that each identify a network segment. The method of such embodiments creates multiple secondary network interfaces and attaches the multiple network segments to the multiple secondary network interfaces.
Designating the network segment includes identifying a network segment created on the logical network before the ND is received in some embodiments. The method may further include directing the logical network to modify the network segment according to a set of attributes in the received ND.
Designating the network segment includes providing a set of attributes of the network segment in some embodiments. The method of such embodiments further includes directing the logical network to create the network segment according to the received set of attributes. The set of attributes may include a network type, where the network type is a VLAN-backed network segment or an overlay-backed network segment.
In some embodiments in which a pod includes multiple ND identifiers, for one set of NDs, each ND designates a network segment by identifying a network segment created on the logical network before the ND is received while for another set of NDs, each ND designates a network segment by providing a set of attributes of the network segment. The method of such embodiments further includes directing the logical network to create the second set of network segments according to the received set of attributes.
Many of the embodiments described herein are described with relation to a Kubernetes system, sometimes abbreviated “Kubes” or “K8s.” However, one of ordinary skill in the art will understand that this is merely one example of a container network system that embodies the inventions described herein and that other embodiments may apply to other container network systems.
In the Kubernetes system, a container in a container network is a lightweight executable image that contains software and all of its dependencies (e.g., libraries, etc.). Containers are executed in pods. A pod is the smallest deployable unit a user can create in a Kubernetes system. A pod may have one or more containers running in it. The containers of a pod may use shared storage and network resources. The pod includes a specification for how to run the containers. A pod's contents in some embodiments are always stored together and executed together. A pod provides an application-specific logical host. The logical host contains one or more application containers. One of the potential shared resources of a pod is a secondary interface.
In addition to the templates and code that is supplied by the original programmers of the Kubernetes system, the system allows a user to create customized resources. The network control system of some embodiments processes one or more Custom Resource Definitions (CRDs) that define attributes of custom-specified network resources. The CRDs define extensions to the Kubernetes networking requirements. Some embodiments use the following CRDs: network-attachment-definition (NDs), Virtual Network Interfaces (VIF) CRDs, Virtual Network CRDs, Endpoint Group CRDs, security CRDs, Virtual Service Object (VSO) CRDs, and Load Balancer CRDs.
FIG. 1 illustrates an example of a control system 100 of some embodiments of the invention. This system 100 processes APIs that use the Kubernetes-based declarative model to describe the desired state of (1) the machines to deploy, and (2) the connectivity, security and service operations that are to be performed for the deployed machines (e.g., private and public IP addresses connectivity, load balancing, security policies, etc.). To process these APIs, the control system 100 uses one or more CRDs to define some of the resources referenced in the APIs. The system 100 performs automated processes to deploy a logical network that connects the deployed machines and segregates these machines from other machines in the datacenter set. The machines are connected to the deployed logical network of a virtual private cloud (VPC) in some embodiments.
As shown, the control system 100 includes an API processing cluster 105, a software defined network (SDN) manager cluster 110, an SDN controller cluster 115, and compute managers and controllers 117. The API processing cluster 105 includes two or more API processing nodes 135, with each node comprising an API processing server 140, a Kubelet 142 node agent, and a network controller plugin (NCP) 145. The API processing server 140 receives intent-based API calls and parses these calls. In some embodiments, the received API calls are in a declarative, hierarchical Kubernetes format, and may contain multiple different requests.
The API processing server 140 parses each received intent-based API request into one or more individual requests. When the requests relate to the deployment of machines, the API server provides these requests directly to compute managers and controllers 117, or indirectly provide these requests to the compute managers and controllers 117 through the Kubelet 142 and/or the NCP 145 running on the Kubernetes master node 135. The compute managers and controllers 117 then deploy VMs and/or Pods on host computers in the availability zone.
The kubelet 142 node agent on a node can register the node with the API server 140 using one of: the hostname; a flag to override the hostname; or specific logic for a cloud provider. The kubelet 142 receives PodSpecs, YAML (a data serialization language) or JavaScript Object Notation (JSON) formatted objects that each describe a pod. The kubelet 142 uses a set of PodSpecs to create (e.g., using the compute managers and controllers 117) the pods that are provided by various mechanism elements (e.g., from the API server 140) and ensures that the containers described in those PodSpecs are running and healthy.
The API calls can also include requests that require network elements to be deployed. In some embodiments, these requests explicitly identify the network elements to deploy, while in other embodiments the requests can also implicitly identify these network elements by requesting the deployment of compute constructs (e.g., compute clusters, containers, etc.) for which network elements have to be defined by default. As further described below, the control system 100 uses the NCP 145 to identify the network elements that need to be deployed, and to direct the deployment of these network elements.
In some embodiments, the API calls refer to extended resources that are not defined per se by the baseline Kubernetes system. For these references, the API processing server 140 uses one or more CRDs 120 to interpret the references in the API calls to the extended resources. As mentioned above, the CRDs in some embodiments include the NDs, VIF, Virtual Network, Endpoint Group, Security Policy, Admin Policy, and Load Balancer and VSO CRDs. In some embodiments, the CRDs are provided to the API processing server 140 in one stream with the API calls.
NCP 145 is the interface between the API server 140 and the SDN manager cluster 110 that manages the network elements that serve as the forwarding elements (e.g., switches, routers, bridges, etc.) and service elements (e.g., firewalls, load balancers, etc.) in an availability zone. The SDN manager cluster 110 directs the SDN controller cluster 115 to configure the network elements to implement the desired forwarding elements and/or service elements (e.g., logical forwarding elements and logical service elements) of one or more logical networks. As further described below, the SDN controller cluster 115 interacts with local controllers on host computers and edge gateways to configure the network elements in some embodiments.
In some embodiments, NCP 145 registers for event notifications with the API server 140, e.g., sets up a long-pull session with the API server to receive all CRUD (Create, Read, Update, and Delete) events for various CRDs that are defined for networking. In some embodiments, the API server 140 is a Kubernetes master VM, and the NCP 145 runs in this VM as a Pod. NCP 145 in some embodiments collects realization data from the SDN resources for the CRDs and provides this realization data as it relates to the CRD status. In some embodiments, the NCP 145 communicates directly with the API server 140 and/or through the Kubelet 142.
In some embodiments, NCP 145 processes the parsed API requests relating to NDs, VIFs, virtual networks, load balancers, endpoint groups, security policies, and VSOs, to direct the SDN manager cluster 110 to implement (1) the NDs to designate network segments for use with secondary interfaces of pods, (2) the VIFs needed to connect VMs and Pods to forwarding elements on host computers, (3) the virtual networks to implement different segments of a logical network of the VPC, (4) the load balancers to distribute the traffic load to endpoint machines, (5) the firewalls to implement security and admin policies, and (6) the exposed ports to access services provided by a set of machines in the VPC to machines outside and inside of the VPC.
The API server 140 provides the CRDs 120 that have been defined for these extended network constructs to the NCP 145 for it to process the APIs that refer to the corresponding network constructs (e.g., network segments). The API server 140 also provides configuration data from the configuration storage 125 to the NCP 145. The configuration data in some embodiments includes parameters that adjust the pre-defined template rules that the NCP 145 follows to perform its automated processes. In some embodiments, the configuration data includes a configuration map. The configuration map of some embodiments may be generated from one or more directories, files, or literal values. The configuration map (or “ConfigMap”) is discussed further with respect to the device plugin 144, below.
The NCP 145 performs these automated processes to execute the received API requests in order to direct the SDN manager cluster 110 to deploy the network elements for the VPC. For a received API, the control system 100 performs one or more automated processes to identify and deploy one or more network elements that are used to implement the logical network for a VPC. The control system performs these automated processes without an administrator performing any action to direct the identification and deployment of the network elements after an API request is received.
The SDN managers 110 and controllers 115 can be any SDN managers and controllers available today. In some embodiments, these managers and controllers are the NSX-T managers and controllers licensed by VMware Inc. In such embodiments, NCP 145 detects network events by processing the data supplied by its corresponding API server 140, and uses NSX-T APIs to direct the NSX-T manager 110 to deploy and/or modify NSX-T network constructs needed to implement the network state expressed by the API calls. The communication between the NCP and NSX-T manager 110 is an asynchronous communication, in which NCP provides the desired state to NSX-T managers, which then relay the desired state to the NSX-T controllers to compute and disseminate the state asynchronously to the host computer, forwarding elements and service nodes in the availability zone (i.e., to the SDDC set controlled by the controllers 115).
After receiving the APIs from the NCPs 145, the SDN managers 110 in some embodiments direct the SDN controllers 115 to configure the network elements to implement the network state expressed by the API calls. In some embodiments, the SDN controllers serve as the central control plane (CCP) of the control system 100.
In some embodiments, a device plug-in 144 identifies resources available to the pods on a node based on a configuration map of the node. The configuration map in some embodiments is received from the API server 140. In some embodiments, the configuration map is generated from files in the configuration storage 125, from data received by the API server from the NCP and/or from data generated by the SDN manager 110. In some embodiments, the device plug-in receives the configuration map directly from the API server 140. In other embodiments, the device plug-in receives the configuration map through the kubelet 142. The configuration map in some embodiments includes identifiers of pre-created network segments of the logical network.
A network segment, sometimes called a logical switch, logical network segment, or a transport zone, acts in a manner similar to a subnet, e.g., a layer 2 broadcast zone. Individual pods can interface with a network segment and communicate with other pods or devices configured to interface with the network segment. However, one of ordinary skill in the art will understand that a network segment (or logical switch) does not operate as a physical switch connecting devices that are both directly connected to the same switch, but for example as a VPN tunnel or VLAN, allowing pods or devices that are not directly connected to communicate as though they are all connected to a common switch.
FIG. 2 illustrates an example of a logical network for a virtual private cloud. FIG. 2 depicts the SDN controllers 115, acting as the CCP, computing high level configuration data (e.g., port configuration, policies, forwarding tables, service tables, etc.). In such capacity, the SDN controllers 115 push the high-level configuration data to the local control plane (LCP) agents 220 on host computers 205, LCP agents 225 on edge appliances 210 and TOR (top-of-rack) agents 230 of TOR switches 215. The CCP and LCPs configure managed physical forwarding elements (PFEs), e.g., switches, routers, bridges, etc., to implement logical forwarding elements (LFEs). A typical LFE spans multiple PFEs running on multiple physical devices (e.g., computers, etc.).
Based on the received configuration data, the LCP agents 220 on the host computers 205 configure one or more software switches 250 and software routers 255 to implement distributed logical switches, routers, bridges and/or service nodes (e.g., service VMs or hypervisor service engines) of one or more logical networks with the corresponding switches and routers on other host computers 205, edge appliances 210, and TOR switches 215. On the edge appliances, the LCP agents 225 configure packet processing stages 270 of these appliance to implement the logical switches, routers, bridges and/or service nodes of one or more logical networks along with the corresponding switches and routers on other host computers 205, edge appliances 210, and TOR switches 215.
For the TORs 215, the TOR agents 230 configure one or more configuration tables 275 of TOR switches 215 through an OVSdb server 240. The data in the configuration tables is then used to configure the hardware ASIC packet-processing pipelines 280 to perform the desired forwarding operations to implement the desired logical switching, routing, bridging and service operations. U.S. patent application Ser. No. 14/836,802, filed Aug. 26, 2015, now issued as U.S. Pat. No. 10,554,484, U.S. patent application Ser. No. 15/342,921, filed Nov. 3, 2016, now issued as U.S. Pat. No. 10,250,553, U.S. patent application Ser. No. 14/815,839, filed Jul. 31, 2015, now issued as U.S. Pat. No. 9,847,938, and U.S. patent application Ser. No. 13/589,077, filed Aug. 17, 2021, now issued as U.S. Pat. No. 9,178,833 describe CCPs, LCPs and TOR agents in more detail, and are incorporated herein by reference.
After the host computers 205 are configured along with the edge appliances 210 and/or TOR switches 215, they can implement one or more logical networks, with each logical network segregating the machines and network traffic of the entity for which it is deployed from the machines and network traffic of other entities in the same availability zone. FIG. 2 illustrates an example of a logical network 295 that defines a VPC for one entity, such as one corporation in a multi-tenant public datacenter, or one department of one corporation in a private datacenter.
As shown, the logical network 295 includes multiple logical switches 284 with each logical switch connecting different sets of machines and serving as a different network segment. Each logical switch has a port 252 that connects with (i.e., is associated with) a virtual interface 265 of a machine 260. The machines 260 in some embodiments include VMs and Pods, with each Pod having one or more containers. The logical network 295 also includes a logical router 282 that connects the different network segments defined by the different logical switches 284. In some embodiments, the logical router 282 serves as a gateway for the deployed VPC in FIG. 2 .
FIG. 3 illustrates pods 260 implemented on VM 360 of a host computer 205. The pods 365 are connected to a software forwarding element (SFE) 370. In some embodiments the SFE 370 is a software switch, a software bridge, or software code that enables the pods to share the virtual network interface card (VNIC) 375 of the VM 360. The connection between the pods 365 and the SFE 370 is initiated by an NSX node agent 380 that performs the functions of an NCP (e.g., as part of a distributed NCP) on the VM 360. The SFE 370 in turn passes communications between the pods 365 and the VNIC 375. The VNIC 375 connects to the port 385 of the software switch 250 that is configured by the LCP 220.
The LCP 220 acts as a local agent of a CCP and, in some embodiments, configures the software switch 250 to implement one or more network segments. As mentioned above, a network segment (or logical switch) allows multiple pods to communicate as though they were on a common switch, but the logical switch itself is implemented by multiple software switches 250 that operate on different host computers, VMs, etc. In some embodiments, a single software switch 250 may implement part of multiple different network segments.
Pods of some embodiments may require multiple interfaces to provide multiple avenues of communication that require different characteristics. For example, in some embodiments a pod may implement part of a telecommunications application, the primary interface of the pod may connect to the main telecommunications network (e.g., to handle one or more of telecommunications control functions, voice data, etc.) while a secondary interface of the pod may provide a high performance link for data traffic. Such a high performance link may be used in some embodiments to connect to a Single Root I/O Virtualization (SR-IOV) system. In some embodiments, the pods are not limited to just the primary and one secondary interfaces, but may have an arbitrary number of interfaces up to the capacity of the logical network to provide network segments.
FIG. 4 conceptually illustrates pods 405, 410, and 415 with interfaces to one or more network segments. Pod 405 is limited to a single interface 407, connecting to network segment 420. The network segment 420 is a logical construct provided by a software switch (not shown) that enables the pod 405 to communicate (e.g., through a VLAN or tunnel in some embodiments) with other pods that interface with the network segment 420 such as pod 410. Pod 410 may be implemented by the same VM as pod 405, or a different VM on the same host, on a VM on a different host, or even directly on a physical computer without a VM. Pod 410 also has a primary interface 412 that connects it to network segment 420. However, pod 410 also has secondary interfaces 413 and 414 connecting pod 410 to network segments 430 and 440, respectively. Pod 415 has primary interface 417 and secondary interface 418 connecting pod 415 to network segments 430 and 440, respectively. Thus pods 410 and 415 can communicate using either network segment 430 or network segment 440. The logical router 282 connects the network segments 420-440.
Some embodiments provide a sequence for providing resources (including interfaces) to pods, using a device plugin to identify the resources for a kubelet creating the pods. Although the discussion below is limited to a list of network segments, in some embodiments, the device plugin supplies lists of other devices in addition to network segments. FIG. 5 illustrates a communication sequence 500 of some embodiments for adding a secondary interface to a pod. The communication sequence includes several steps, numbered (1) to (7) in the diagram and the following description. The communication sequence 500 begins when the API server 140 (1) sends a list of network segments and, for each network segment, a list of interfaces of the segment to the device plugin 144. The device plugin 144 then determines which interfaces are available and (2) provides the list of available interfaces for each network segment to the kubelet 142. In some embodiments, the device plugin 144 determines available interfaces of a network segment by retrieving an interface list from a specific file location (e.g., sys/class/net) and comparing this interface list with the interface names of the network segment. If an interface name from the interface list matches an interface name of the network segment then the device plugin 144 identifies it as available and that it is a list of such available interfaces that is sent to the kubelet 142 in step (2).
At some point after the kubelet 142 receives the network segment and available interface lists, the API server (3) sends a pod definition to the kubelet 142 that the kubelet 142 will use to create a pod. The pod definition in some embodiments contains a name or other identifier of a secondary network segment to attach the pod to. In some embodiments, the pod includes an internal identifier of the secondary interface to identify the interface to containers of the pod. One of ordinary skill in the art will understand that this internal identifier is a separate and generally distinct identifier from the list of available interfaces identified by the device plugin.
The kubelet 142, in some embodiments, then sends (4) a request for an interface ID of an unallocated interface of the network segment identified in the pod definition, to the device plugin 144. The device plugin 144 then sends (5) an interface ID of an unallocated interface of the identified network segment to the kubelet 142. The device plugin 144 monitors the allocated interface IDs in the embodiment in FIG. 5 however, in other embodiments, the kubelet 142 or the NCP 145 monitors the allocated interface IDs. In some embodiments, when a pod is deleted, whichever element monitors the allocated interface IDs updates the status of the secondary interface(s) allocated to that pod to “unallocated.” The NCP 145 queries (6) the kubelet 142 for any pods with secondary interfaces to be attached and receives (7) the interface ID from the kubelet 142. The NCP 145 then creates an interface for the pod and attaches the interface to the identified network segment.
Although the communications sequence of FIG. 5 includes a particular set of messages sent in a particular order, in other embodiments, different messages may be sent or the order may be different. For example, in some embodiments, rather than a device plugin tracking the allocated and unallocated interface IDs of a network segment, a kubelet or the NCP tracks which interfaces are allocated and unallocated. In some embodiments, the kubelet receives a pod definition with a network segment identified and creates the pod. Then an NCP determines that the pod includes a network segment identifier, creates a secondary network interface for the pod and connects the secondary interface to the identified network segment. FIG. 6 conceptually illustrates a process 600 of some embodiments for allocating a secondary network interface for a pod with a primary network interface. In some embodiments, the process 600 is performed by an NCP.
The process 600 begins by receiving (at 605) a pod. In some embodiments, receiving a pod means receiving at the NCP a notification that a pod has been created (e.g., by a kubelet). The process 600 determines (at 610) that the pod includes an identifier of a network attachment definition (ND). An ND designates a network segment to attach to a secondary network interface of the pod. In some embodiments, designating a network segment may include identifying, in the ND, a pre-created network segment of a logical network and/or providing attributes in the ND that allow an NCP to command a network manager or controller to dynamically create a network segment in the logical network. When the pod includes an identifier of an ND, the NCP uses that identifier (e.g., in operation 620) to determine which ND designates the network segment to be attached to a secondary interface of the pod.
This is an example of a pod definition that includes an identifier of an ND:


	kind: Pod
	metadata:
	name: my-pod
	namespace: my-namespace
	annotations:
	k8s.v1.cni.cncf.io/networks: \|
	[
	{
	“name”: “net-nsx”,
	# The name of network attachment CRD
	“interface”: “eth1”,
	# (optional)The name of interface within the pod.
	“ips”: [“1.2.3.4/24”],
	# (optional)IP/prefix_length and mac addresses for the
	interface, optional.
	“mac”: “aa:bb:cc:dd:ee:ff” #(optional)
	},
	]

In the above example (pod example 1) the pod includes one ND identifier indicating that the pod should have one secondary network interface. However, in some embodiments, pods may include multiple ND identifiers, indicating that the pods should have multiple secondary network interfaces attached to multiple network segments. The identified ND has an identifier called a name, in this example, “net-nsx”. However, in some embodiments the ND may have other designations such as a number, code, or other type of identifier. Some examples of NDs that might designate the secondary network segments to attach to the pod of the pod example are provided below.
The process 600 creates (at 615) a secondary interface for the pod. The process 600 then connects (at 620) the secondary network interface created in the pod in operation 615 to the network segment designated by the ND identified in operation 610. The network segment, in some embodiments may be a pre-created network segment. Pre-created network segments are created independently on the logical network without the use of an ND. When a user codes the corresponding ND, the user adds a network identifier, used by the logical network to identify the pre-created network segment, to the ND.
Here is an example of an ND corresponding to the name: net-nsx (the identifier in the pod example above). The ND designates the network segment to be attached when a pod uses the ND identifier “net-nsx”. This ND example, and the subsequent dynamically created network segment examples include the name: net-nsx. However, unlike the dynamic network segments, this example of an ND that designates a pre-created network segment includes an identifier of an existing, pre-created network segment:


	ND example 1:
	apiVersion: k8s.cni.cncf.io/v1
	kind: NetworkAttachmentDefinition
	metadata:
	name: net-nsx
	spec:
	config: ‘{
	“cniVersion”: “0.3.0”,
	“type”: “nsx”,
	# NCP CNI plugin type
	“networkID”: “071c3745-f982-45ba-91b2-3f9c22af0240”,
	# ID of pre-created NSXT Segment
	“ipam”: {
	# “ipam” is optional
	“subnet”: “192.168.0.0/24”,
	# required in “ipam”
	“rangeStart”: “192.168.0.2”,
	# optional, default value is the secondary IP of subnet
	“rangeEnd”: “192.168.0.254”,
	# optional, default value is the penultimate IP of subnet
	“gateway”: “192.168.0.1”
	# optional, default value is the first IP of subnet
	}
	}’

In Example ND 1, the networkID: “071c3745-f982-45ba-91b2-3f9c22af0240” is an ID used by the logical network to identify a pre-created network segment of the logical network. The identified network segment was created (e.g., at the instructions of the user, using the logical network) without the ND and selected by the user (e.g., using the network ID placed in the ND when it was coded) to be used as the network segment for pods using that ND. The NDs of some embodiments with pre-created network segment IDs may also contain additional attributes that modify the pre-created network and/or the interface of the pod on the network segment.
In some embodiments, in addition to or instead of connecting pre-created network segments to pods, the process 600 in operation 620 may connect network segments that are dynamically created according to network attributes provided in an ND. In some embodiments, these network attributes may merely identify the type of network (e.g., VLAN, overlay, MACVLAN, IPVLAN, ENS, etc.) to create or may include additional network attributes. The following are examples of NDs for creating a VLAN-backed network segment and an overlay-backed network segment.


ND example 2:
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: net-nsx
spec:
config: ‘{
“cniVersion”: “0.3.0”,
“type”: “nsx”,
# NCP CNI plugin type
“networkID”: “”,
# ID of pre-created NSXT Segment
“networkType”: “vlan”,
“vlanID”: 100,
“ipam”: {
# “ipam” is optional
“subnet”: “192.168.0.0/24”,
# required in “ipam”
“rangeStart”: “192.168.0.2”,
# optional, default value is the secondary IP of subnet
“rangeEnd”: “192.168.0.254”,
# optional, default value is the penultimate IP of subnet
“gateway”: “192.168.0.1”
# optional, default value is the first IP of subnet
}
}’
ND example 3:
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: net-nsx
spec:
config: ‘{
“cniVersion”: “0.3.0”,
“type”: “nsx”,
# NCP CNI plugin type
“networkID”: “”
# ID of pre-created NSXT Segment
“networkType”: “overlay”,
“gatewayID”: “081c3745-d982-45bc-91c2-3f9c22af0249”,
# Optional. ID of NSX-T Gateway to which the created Segment should
be connected.
“ipam”: {
# “ipam” is optional
“subnet”: “192.168.0.0/24”,
# required in “ipam”
“rangeStart”: “192.168.0.2”,
# optional, default value is the secondary IP of subnet
“rangeEnd”: “192.168.0.254”,
# optional, default value is the penultimate IP of subnet
“gateway”: “192.168.0.1”
# optional, default value is the first IP of subnet
}
}’

In ND example 2, there is no networkID as the ND CRD does not specify a pre-created network segment. In ND example 2, the ND includes a network type (vlan) and a vlanID number (100). In ND example 3, the ND includes a network type (overlay) and an ID of a logical network Gateway to which the created segment should be connected (081c3745-d982-45bc-91c2-3f9c22af0249).
The illustrated embodiment of FIG. 6 handles both pre-created and dynamically created network segments. In some embodiments, the dynamically created network segments are created by the NCP directing the logical network to create the network segments: (1) when the Kubernetes system is being brought online, (2) when the NDs are initially provided to the Kubernetes API (e.g., for NDs coded after the Kubernetes system is started), and/or (3) the first time a pod identifying a particular ND that designates a particular network segment is first received by the NCP. In some embodiments, the NCP provides, to the logical network, default attributes for a dynamic network segment to supplement any attributes supplied by the ND. In some embodiments, these default attributes are supplied in one or more CRDs (that are not network attachment definition CRDs).
As previously mentioned, in some embodiments, a pod may have more than one secondary interface. Therefore, the process 600 determines (at 625) whether the ND identifier was the last ND identifier of the pod. If the ND identifier was not the last one in the pod, the process 600 loops back to operation 615. If the network segment identifier was the last one in the pod, the process 600 ends.
Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as computer-readable medium). When these instructions are executed by one or more processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. The computer-readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.
In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.
FIG. 7 conceptually illustrates a computer system 700 with which some embodiments of the invention are implemented. The computer system 700 can be used to implement any of the above-described hosts, controllers, gateway and edge forwarding elements. As such, it can be used to execute any of the above-described processes. This computer system 700 includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. Computer system 700 includes a bus 705, processing unit(s) 710, a system memory 725, a read-only memory 730, a permanent storage device 735, input devices 740, and output devices 745.
The bus 705 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the computer system 700. For instance, the bus 705 communicatively connects the processing unit(s) 710 with the read-only memory 730, the system memory 725, and the permanent storage device 735.
From these various memory units, the processing unit(s) 710 retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments. The read-only-memory (ROM) 730 stores static data and instructions that are needed by the processing unit(s) 710 and other modules of the computer system. The permanent storage device 735, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the computer system 700 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 735.
Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device 735. Like the permanent storage device 735, the system memory 725 is a read-and-write memory device. However, unlike storage device 735, the system memory 725 is a volatile read-and-write memory, such as random access memory. The system memory 725 stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention's processes are stored in the system memory 725, the permanent storage device 735, and/or the read-only memory 730. From these various memory units, the processing unit(s) 710 retrieve instructions to execute and data to process in order to execute the processes of some embodiments.
The bus 705 also connects to the input and output devices 740 and 745. The input devices 740 enable the user to communicate information and select commands to the computer system 700. The input devices 740 include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices 745 display images generated by the computer system 700. The output devices 745 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as touchscreens that function as both input and output devices 740 and 745.
Finally, as shown in FIG. 7 , bus 705 also couples computer system 700 to a network 765 through a network adapter (not shown). In this manner, the computer 700 can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet), or a network of networks (such as the Internet). Any or all components of computer system 700 may be used in conjunction with the invention.
Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.
While the above discussion primarily refers to microprocessors or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.
As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms “display” or “displaying” mean displaying on an electronic device. As used in this specification, the terms “computer-readable medium,” “computer-readable media,” and “machine-readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals.
While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. For instance, several of the above-described embodiments deploy gateways in public cloud datacenters. However, in other embodiments, the gateways are deployed in a third-party's private cloud datacenters (e.g., datacenters that the third-party uses to deploy cloud gateways for different entities in order to deploy virtual networks for these entities). Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.

Claims

1-8. (canceled)

9. A method for configuring a pod, the method comprising:

performing an automated process to associate the pod with at least two network segments, said performing comprising:

detecting deployment of the pod associated with a first network segment, said pod having a default primary interface for connecting to the first network segment;

from a definition of the pod, determining that the pod is to be associated with a second network segment;

identifying a secondary interface for the pod; and

associating the secondary interface with the pod and the second network segment, said secondary interface allowing the pod to communicate with the second network segment.

10. The method of claim 9, wherein said determining comprises:

receiving a network attachment definition (NAD) that designates a network segment; and

determining that the pod definition includes an identifier of the NAD;

11. The method of claim 10, wherein identifying the secondary interface comprises creating the secondary network interface for the pod based on the determination that the pod definition includes the identifier of the NAD.

12. The method of claim 10, wherein the NAD designate a pre-created network segment.

13. The method of claim 10, wherein the NAD designates attributes of a network segment that the automated process needs to dynamically deploy and to associate the secondary interface with this dynamically deployed network segment.

14. The method of claim 13, wherein the dynamically deployed network segment is a VLAN (virtual local area network) segment or an overlay network segment.

15. The method of claim 10, wherein the pod definition references a plurality of identifiers for a plurality of NADs to indicate that the pod has a plurality of secondary network interfaces attached to a plurality of network segments.

16. The method of claim 10, wherein the NAD comprises a name that is used to identify a network attachment.

17. The method of claim 10, wherein the NAD comprises a number or code identifier that identifies the network attachment.

18. The method of claim 9, wherein said secondary network interface serves as high-performance interface for data traffic.

19. A non-transitory machine readable medium storing a program which when executed by at least one processing unit configures a pod, the program comprising sets of instructions for:

identifying a secondary interface for the pod; and

20. The non-transitory machine readable medium of claim 19, wherein the set of instructions for said determining comprises sets of instructions for:

determining that the pod definition includes an identifier of the NAD;

21. The non-transitory machine readable medium of claim 20, wherein the set of instructions for identifying the secondary interface comprises a set of instructions for creating the secondary network interface for the pod based on the determination that the pod definition includes the identifier of the NAD.

22. The non-transitory machine readable medium of claim 20, wherein the NAD designate a pre-created network segment.

23. The non-transitory machine readable medium of claim 20, wherein the NAD designates attributes of a network segment that the automated process needs to dynamically deploy and to associate the secondary interface with this dynamically deployed network segment.

24. The non-transitory machine readable medium of claim 23, wherein the dynamically deployed network segment is a VLAN (virtual local area network) segment or an overlay network segment.

25. The non-transitory machine readable medium of claim 20, wherein the pod definition references a plurality of identifiers for a plurality of NADs to indicate that the pod has a plurality of secondary network interfaces attached to a plurality of network segments.

26. The non-transitory machine readable medium of claim 20, wherein the NAD comprises a name that is used to identify a network attachment.

27. The non-transitory machine readable medium of claim 20, wherein the NAD comprises a number or code identifier that identifies the network attachment.

28. The non-transitory machine readable medium of claim 19, wherein said secondary network interface serves as high-performance interface for data traffic.