CN112714069A

CN112714069A - Method for lowering shunting module to network card hardware in IPSec security gateway environment

Info

Publication number: CN112714069A
Application number: CN202110011389.XA
Authority: CN
Inventors: 李健; 李令成; 管海兵
Original assignee: Shanghai Jiaotong University
Current assignee: Shanghai Jiaotong University
Priority date: 2021-01-06
Filing date: 2021-01-06
Publication date: 2021-04-27

Abstract

The invention discloses a method for lowering a shunting module to network card hardware in an IPSec security gateway environment, and relates to the field of network encryption communication. In the shunting of IPSec flow, an IPSec triple containing an SPI value is used for replacing a quintuple of a non-encrypted network data packet to carry out hashing; establishing logic of a shunting module of an IPSec security gateway in a network card, carrying out hash by utilizing an IPSec triple, and distributing IPSec flows into a multi-queue; and establishing a scheduling module of a control layer, circularly monitoring the load of each working core, scheduling the IPSec flow on the working core with the load exceeding the threshold value to the working core with the lightest load, controlling a shunting module, and redirecting the IPSec flow to a new working core. The invention transfers the original software shunting module to the network card, thereby realizing better expandability and throughput and realizing load balance.

Description

Method for lowering shunting module to network card hardware in IPSec security gateway environment

Technical Field

The invention relates to the field of network encryption communication, in particular to a method for lowering a shunting module to network card hardware in an IPSec security gateway environment.

Background

With the rise of cloud computing, more and more customers place services in the cloud. The large amount of network traffic makes cloud service manufacturers increasingly demanding various performance requirements on computing networks. Meanwhile, more and more clients pay more attention to the security and privacy of data, so that more and more network data packets are encrypted and then transmitted.

Among them, the IPSec protocol is the most common encryption protocol for communication between two private networks. Mainstream cloud service manufacturers provide a network function module, namely an IPSec security gateway, which encrypts network packets by using an IPSec protocol. Because IPSec encryption and decryption are computationally intensive computations and the computational power of a single core is difficult to achieve with high throughput, existing IPSec gateways generally use multiple processors to process IPSec streams in parallel.

For ease and flexibility, this type of IPSec security gateway is divided into three parts: the IPSec encryption and decryption operations are decoupled to form a working module which is placed in a plurality of working cores for parallel processing; the scheduling module is used as a control plane to make a strategy for scheduling the network packet, the workload is light, global information is required, and therefore the scheduling module is placed at an independent control node; the shunting module local dispatching strategy forwards the network packet to a corresponding working core, the work is heavy in load and needs to interact with the working core, so the network packet should be placed in an independent core close to the working core for processing.

In the existing framework, the IPSec security gateway uses software to implement a distribution module to distribute IPSec streams to a plurality of worker cores. The use of software to implement the shunting module can cause two problems:

1. occupying the computational resources of the processor. In a computationally intensive module such as an IPSec security gateway, the computational resources of the processor are valuable and should be allocated to logic such as upper layer applications as much as possible. The fewer computational resources used for the transport or network protocol stack should be the better. The load of the shunting module occupies the computing resources of the working core and the upper-layer application, and the throughput is reduced.

2. The offload module can become a bottleneck for scalability of the IPSec security gateway. When network traffic increases, offloading using a single processor can result in heavy loading, which can be a performance bottleneck for the entire IPSec security gateway. By increasing the number of processors of the shunting module, the shunting module not only occupies more processor computing resources, but also needs to design complex scheduling logic, thereby realizing the flow load balance on the multi-core shunting module. This makes the software-implemented offload module become the scalability performance bottleneck of the IPSec security gateway when the traffic increases.

For unencrypted network data packets, the current network card already implements the function of the shunting module. The network card distributes the data packet to a plurality of queues of a plurality of network cards through a five-tuple (source IP, destination IP, source port number, destination port number and protocol number) of a Hash network packet by an RSS (received side scaling) load distribution method. The function of distributing the network packet to a plurality of processors, namely a shunting module, can be realized by corresponding the queues to the processors one by one.

However, directly putting the offloading module of the IPSec security gateway down into the network card may cause two problems:

problem one, a network protocol packet encrypted by the IPSec protocol would have three elements in the five tuple of the network packet: the source port number, the destination port number, the protocol number are encrypted, and only two elements are reserved: the source IP and the destination IP are unencrypted data. When the network card receives a network packet encrypted by the IPSec protocol, the network card cannot hash the data packet in a five-tuple manner, but only hash two unencrypted packets, which may cause IPSec streams in the same cluster (i.e., IPSec streams having the same source IP and destination IP but different source port numbers and destination port numbers) to be allocated to the same working core for processing, resulting in uneven overall allocation.

And secondly, the shunting module needs to be matched with the scheduling module, and the network flows are reasonably distributed according to the throughput of the network flows, so that each work core has load balance.

Therefore, those skilled in the art are dedicated to develop a method for lowering a offloading module to network card hardware in an IPSec security gateway environment, and in an existing IPSec security gateway, an original software offloading module is lowered to a network card, so as to achieve better scalability and throughput, and achieve load balancing.

Disclosure of Invention

In view of the above defects in the prior art, the technical problem to be solved by the present invention is to reduce the occupation of computing resources of a processor, avoid the performance bottleneck of the scalability of a security gateway, achieve better scalability and throughput, and achieve load balancing.

In order to achieve the above object, the present invention provides a method for lowering a offload module to network card hardware in an IPSec security gateway environment, comprising the following steps:

step 1, in the shunt of IPSec flow, an IPSec triple containing an SPI value is used for replacing a quintuple of a non-encrypted network data packet to carry out hash;

step 2, establishing logic of a shunting module of an IPSec security gateway in a network card, carrying out hash by utilizing an IPSec triple in the network card, and distributing IPSec streams into a multi-queue;

step 3, establishing a scheduling module of a control layer, circularly monitoring the load of each working core and scheduling the IPSec flow on the working core with the load exceeding a threshold value to the working core with the lightest load; and controls the offload module to redirect the IPSec stream to the new worker core.

Further, the IPSec triplet in step 1 includes a source IP, a destination IP, and an SPI domain.

Further, in step 1, the receiving end and the sending end negotiate to enable the sending end to correspond the SPI field in the IPSec to the quintuple of the network data packet one by one when the sending end performs IPSec encryption on the network data packet.

Further, in step 1, the receiving end and the sending end negotiate by using a first-stage handshake protocol in the IPSec protocol.

Further, the SPI value is negotiated in the handshake protocol in step 1, and the corresponding relationship between the SPI value and the network data packet is designated by the transmitting end.

Further, the step 1 of specifying the corresponding relationship between the SPI value and the network data packet includes setting by manual configuration and by a configuration file.

Further, the step 2 is realized by using a self-defined RSS module of the network card and multiple queues.

Further, the step 3 scheduling module monitors the load of the working core, and reads the load state of the working core through "/proc/stat".

Further, the workload threshold of step 3 is 99%.

Further, the scheduling module in step 3 acquires a relationship correspondence table in the offloading module during initialization, and constructs a correspondence between the IPSec stream and the working core.

In the preferred embodiment of the present invention, the SPI field of IPSec is used as a new element, and the SPI field and the original unencrypted binary form an IPSec triplet together for hashing, and the IPSec triplet is hashed by using a flow director hardware mechanism in the network card for shunting. Meanwhile, the logic of the control panel is designed in the aspect of software, so that the control panel can monitor the states of all the working cores, formulate a scheduling algorithm and control the network card to carry out shunting operation. The invention is divided into three parts: IPSec triples, a shunting module and a scheduling module.

First, a new key different from the quintuple is established as the key for hashing the IPSec flow. The SPI domain in the IPSec packet header is selected, and the communication of the sending end enables the SPI domain to establish a one-to-one correspondence relationship with the quintuple. The SPI value is the identification number that the IPSec protocol adds to the IPSec packet header after encrypting the network packet. After encrypting the network packet, the sending end adds an IPSec packet header to the packet header of the network packet and records the SPI value. The receiving end finds information such as the encrypted key according to the SPI value and decrypts the IPSec packet. In the initialization stage, the receiving end notifies the sending end, negotiates to make the SPI domain encrypted by IPSec correspond to the quintuple one by one, and the sending end modifies the encryption and decryption key correspondence table to generate rules that the SPI domain corresponds to the quintuple one by one. By the rule, the sending end sends the encrypted network packet, and the receiving end can replace the original hash quintuple scheme by the hash SPI value. Although this scheme requires coordination of the transmitting end, the overhead is small: in the IPSec protocol, the SPI values may be generated by hashing a quintuple, i.e., the SPI values and the quintuple are in one-to-one correspondence, and the sending end needs to modify the configuration file to make the SPI fields and the quintuple in one-to-one correspondence, without introducing additional changes to the sending end code. Meanwhile, the measures can enable the receiving end to utilize the hash of the SPI value to avoid the problem that the load is too heavy due to the fact that the same cluster IPSec flow is distributed to a single working core.

Secondly, a shunting module is established at a network card of a receiving end, and the IPSec stream is shunted through a Hash IPSec ternary group. This section is the logic that implements the software offload module in the network card. By utilizing a mechanism of self-defining RSS keywords in the network card, a Hash mode based on IPSec triples (source end IP, destination end IP and SPI) can be established. Besides the default RSS form, the multi-queue network card generally has the setting of a self-defined RSS hash domain, and sets the keyword of hash as a source IP, a destination IP and an SPI value, so as to realize hash of IPSec streams. After the network card hashes the IPSec streams, the IPSec streams are distributed to multiple queues of the network card and distributed to multiple work cores.

Thirdly, writing a scheduling module of the control layer. In order to make each working core obtain balanced load, the invention realizes dynamic scheduling according to the load condition of each working core. In the initialization stage, the scheduling module establishes a corresponding relationship that each IPSec stream is allocated to the working core. And cyclically monitors the processor usage load of each worker core. When detecting that a certain working core is overloaded, the scheduling module redistributes an IPSec flow on the core to the working core with the lightest load, and notifies the shunting module to revise the mapping relationship again, and distributes the IPSec flow to a new queue of the working core.

Compared with the prior art, the invention has the following obvious substantive characteristics and obvious advantages:

1. saving processor resources. A computationally intensive project such as an IPSec security gateway should use more processor computing power for working logic such as user applications. The software implementation consumes processor computing power, and the throughput of the whole IPSec security gateway is reduced. The invention lowers the shunting module to the network card hardware, so that the required processor resource is reduced and the total throughput is increased.

2. And the expandability is improved. With the rising throughput requirements of IPSec security gateways, running the offload module with a single processor can result in heavy load, and the use of multiple processors requires complex scheduling algorithms. This can cause the offload module of the IPSec security gateway to become a performance bottleneck for scalability under the demand for increased throughput. And the shunting module is put down to the network card hardware to obtain better throughput. The current network card can reach the flow of 10G and 40G, and the expandability under large throughput is easy to realize.

3. The IPSec flows in the same cluster (namely the IPSec flows with the same source IP and the same destination IP but different source port numbers and destination port numbers) are prevented from being distributed to the same working core for processing, and the overall distribution is not uniform.

4. The distribution module is matched with the scheduling module, and the network flows are reasonably distributed according to the throughput of the network flows, so that each work core has balanced load.

The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.

Drawings

FIG. 1 is an overall block diagram of a preferred embodiment of the present invention;

FIG. 2 is a diagram of data flow allocation logic in accordance with a preferred embodiment of the present invention;

fig. 3 is a flow chart of rescheduling according to a preferred embodiment of the present invention.

Detailed Description

The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.

In the drawings, structurally identical elements are represented by like reference numerals, and structurally or functionally similar elements are represented by like reference numerals throughout the several views. The size and thickness of each component shown in the drawings are arbitrarily illustrated, and the present invention is not limited to the size and thickness of each component. The thickness of the components may be exaggerated where appropriate in the figures to improve clarity.

In the existing IPSec security gateway, the original software distribution module is put down to the network card, so that better expandability and throughput are realized. The overall design of the invention is shown in figure 1. Firstly, it is proposed that in the splitting of IPSec streams, IPSec triplets (source IP, destination IP, SPI) containing SPI values are used to replace quintuple of unencrypted network packets for hashing, thereby avoiding the problem that the triplets in IPSec stream quintuple cannot be identified by being encrypted. And secondly, establishing logic of a shunting module of the IPSec security gateway in the network card. And carrying out hashing by using the IPSec triples in the network card, and distributing the IPSec flows into the multi-queue. And thirdly, establishing a scheduling module of a control layer, circularly monitoring the load of each working core and scheduling the IPSec flow on the working core with the heavy load to the core with the light load. And controls the offload module to redirect the IPSec stream to the new worker core.

The invention is divided into three parts: IPSec triples, a shunting module and a scheduling module.

The first part is an IPSec triplet to implement a hashing scheme using an SPI field in the IPSec packet header instead of a quintuple. The receiving end needs to negotiate with the sending end, so that the sending end can correspond the SPI domain in IPSec to the quintuple one by one when the sending end carries out IPSec encryption on the network packet. The sending end may use the first-stage handshake protocol in the IPSec protocol to complete this part of the work. When negotiating the SPI value in the handshake protocol, the corresponding relation between the SPI value and the network packet needs to be appointed, and at the moment, the sending end can make the SPI value and the quintuple correspond one to one through manual configuration or setting through a configuration file.

The second part is to realize the logic of the shunting module in the network card. The distribution module is used for realizing the function of distributing the network flow to a plurality of working cores on the data plane level. And the self-defined RSS module of the network card and the multi-queue are used for realizing. The mainstream network card has a hash option of a user-defined keyword except for a basic RSS module which takes a quintuple as a keyword for hashing. In the case of IPSec flows, IPSec triplets (source IP, destination IP and SPI fields) are selected as the key to the hash. The shunting module stores a table of the corresponding relation between the IPSec triple and the output queue number. The network packets may be distributed into multiple queues of network cards by table lookup. The work core will obtain the network packet from the corresponding queue of the network card and perform the IPSec encryption and decryption operation.

The third part is a scheduling module of the control plane. The scheduling module acquires the table in the shunting module during initialization, and constructs the corresponding relation between the current IPSec flow and the working core. The scheduling module may then continuously monitor the load of each of the cores. Because IPSec encryption and decryption is a computationally intensive module, processor utilization can be a criterion for whether the worker core is busy. When the processor utilization of a working core is too high to exceed the threshold, it can be determined that the load on the working core is too heavy, and the scheduling module remaps a certain IPSec stream on the module to the working core with the lightest load. And then the dispatching module informs the shunting module to modify the corresponding relation table of the IPSec triple and the working core, and redirects the IPSec flow to a new working core. The notification mode adopts an Ethtool system tool. Ethtool is a common tool for communication between network cards and systems. And modifying the rear end of the Ethtool at the network card drive so that the Ethtool can control the forwarding of the IPSec network packet. The scheduling module can notify the shunting module of the network card by using the Ethtool.

The specific work flow is divided into three parts: initialization section, stream allocation section and rescheduling section:

the initialization part comprises the following steps:

1. the user needs to customize the initial shunting strategy. The user establishes a corresponding relationship between the triple of the IPSec streams to be controlled and the expected destination working core in the configuration file, and the default configuration is to allocate all the IPSec streams to the working core 0.

2. The scheduling module reads the configuration file, and informs the shunting module of the network card of the shunting strategy defined by the user through an Ethtool command 'ethotool-config-ntuple' to increase the rule of the corresponding relation between the IPSec triple and the working card.

3. And the receiving end and the notification sending end use a configuration mode that the quintuple corresponds to the SPI one by one when establishing IPSec communication.

4. And the sending end configures the rule of the encryption and decryption key corresponding relation table through 'ip xfrm sp'.

The flow allocation part of the data plane is divided into the following steps, and the flow chart is shown in fig. 2:

1. when the sending end encrypts the network packet, the sending end encrypts the network packet according to the IPSec protocol and the configuration information, and sends the IPSec packet with the IPSec packet header.

2. At the receiving end, the shunting module of the network card can check the IPSec zone bit to judge whether the network packet is the IPSec protocol network packet after receiving the network packet, and if not, the traditional RSS mode is used for shunting.

3. If the IPSec flow is found, inquiring a corresponding relation table of the IPSec triple in the shunting module and the target working core, and distributing the IPSec flow to a queue corresponding to the target working core.

4. And the working core takes the IPSec flow from the corresponding queue and performs IPSec encryption and decryption operations.

The rescheduling part of the control plane is divided into the following steps, and the flow chart is shown in fig. 3:

1. the working core monitoring part of the scheduling module monitors the load of each working core, reads the load state of each working core by reading "/proc/stat", and adds the four parts of "user, priority, system and idle" in the load state, namely the load condition of the working core.

2. When the load of a certain processor exceeds a threshold value (the default is 99%), the load is considered to be too heavy.

3. And the dispatching module searches the IPSec flow on the core through the corresponding relation table of the IPSec triple and the working core. And selecting one queue of the IPSec flow corresponding to the IPSec flow, and scheduling the queue to the work core with the lightest load.

4. And the scheduling module informs the shunting module through the Ethtool command and modifies the corresponding relation table of the shunting module. This IPSec flow may be redirected to the new worker core.

5. If the elimination of the overweight core cannot be realized through a certain number of adjustments (default set to 20), the overall load of the system is judged to be overweight. The administrator should assign more work cores to relieve security gateway pressure.

The three modules can realize the function of putting the shunting module of the IPSec safety module down to the network card hardware, and realize dynamic scheduling according to the load of the working core so as to ensure the scheduling capability of load balance.

The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims

1. A method for lowering a shunting module to network card hardware in an IPSec security gateway environment is characterized by comprising the following steps:

2. The method of claim 1, wherein the IPSec triplet in step 1 comprises the source IP, destination IP, and SPI fields.

3. The method according to claim 1, wherein the receiving end and the transmitting end negotiate in step 1 to make the transmitting end correspond SPI fields in IPSec to quintuple of the network data packet one to one when the transmitting end performs IPSec encryption on the network data packet.

4. The method according to claim 3, wherein the receiving end and the sending end in step 1 negotiate with a first-stage handshake protocol in the IPSec protocol.

5. The method according to claim 4, wherein the SPI value is negotiated in the handshake protocol in step 1, and the sender specifies a correspondence between the SPI value and the network packet.

6. The method of claim 5, wherein the step 1 of assigning the SPI value to network packet correspondence comprises setting via a configuration file via manual configuration.

7. The method of claim 1, wherein step 2 is implemented using a custom RSS module of a network card and multiple queues.

8. The method of claim 1, wherein the step 3 scheduler module monitors the workload of the worker core and reads the workload status of the worker core via "/proc/stat".

9. The method of lowering a offload module to network card hardware in an IPSec security gateway environment as in claim 1, wherein the workload threshold of step 3 is 99%.

10. The method according to claim 1, wherein the scheduling module in step 3 acquires a relationship correspondence table in the offloading module during initialization, and constructs a correspondence between the IPSec stream and the working core.