CN111865996A

CN111865996A - Data detection method and device and electronic equipment

Info

Publication number: CN111865996A
Application number: CN202010721769.8A
Authority: CN
Inventors: 李亚敏; 张胜; 蒋家堂
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2020-07-24
Filing date: 2020-07-24
Publication date: 2020-10-30

Abstract

The present disclosure provides a data detection method, a device and an electronic device, wherein the method comprises the following steps: receiving a data stream, the data stream comprising at least one first data packet; processing at least one first data packet based on a firewall policy, and determining at least one second data packet, wherein the first number of the first data packets is more than or equal to the second number of the second data packets; detecting data in at least one second data packet by using an attack detection model to obtain a detection result, wherein the detection result comprises an attack type; and if the detection result is the attack type, outputting prompt information to prompt that the second data packet corresponding to the detection result comprises the attack type data.

Description

Data detection method and device and electronic equipment

Technical Field

The present disclosure relates to the field of internet technologies, and in particular, to a data detection method, an apparatus, and an electronic device.

Background

In order to promote the healthy and orderly development of internet information technology, users and information service providers need to comply with local regulations, such as internet information service management methods. Blocking is required for information transmission activities that do not meet regulatory requirements, e.g. blocking of data flow from certain source IP addresses is required in firewalls.

In the course of implementing the disclosed concept, the inventors found that there are at least the following problems in the prior art: in the related technology, the firewall mainly issues a firewall policy to perform basic data packet detection and the like, and the deep detection of the content of the data packet is less, so that the firewall detection is not comprehensive enough.

Disclosure of Invention

One aspect of the present disclosure provides a data detection method performed by an electronic device, including: receiving a data stream, the data stream comprising at least one first data packet; processing at least one first data packet based on a firewall policy, and determining at least one second data packet, wherein the first number of the first data packets is more than or equal to the second number of the second data packets; detecting data in at least one second data packet by using an attack detection model to obtain a detection result, wherein the detection result comprises an attack type; and if the detection result is the attack type, outputting prompt information to prompt that the second data packet corresponding to the detection result comprises the attack type data.

According to an embodiment of the present disclosure, detecting data in at least one second data packet using an attack detection model includes: performing flow segmentation on at least one second data packet to obtain at least one file, wherein each file is a second data packet group taking a session as a unit; converting the at least one file into at least one binary image, the binary image comprising a specified number of pixels, each byte of the file having a corresponding pixel, the third number of the at least one file being less than or equal to the fourth number of the at least one binary image; converting the at least one binary image into at least one feature vector; and processing at least one feature vector by using the attack detection model to obtain a detection result.

According to an embodiment of the present disclosure, the attack detection model comprises at least one sub-model for a specific attack type. Correspondingly, the method further comprises the following steps: receiving a user operation instruction; and responding to a user operation instruction, selecting at least one sub-model from the attack detection model, and detecting with at least one second data packet.

Another aspect of the present disclosure provides a data detection apparatus, including: the device comprises a data receiving module, a first detection module, a second detection module and a prompt module. The data receiving module is used for receiving a data stream, and the data stream comprises at least one first data packet; the first detection module is used for processing at least one first data packet based on the firewall policy and determining at least one second data packet, wherein the second quantity of the second data packets is less than or equal to the first quantity of the first data packets; the second detection module is used for detecting data in at least one second data packet by using the attack detection model to obtain a detection result, wherein the detection result comprises an attack type and a non-attack type; and the prompting module is used for outputting prompting information to prompt that the second data packet corresponding to the detection result comprises attack type data if the detection result is the attack type.

Another aspect of the present disclosure provides an electronic device including: memory, a processor and a computer program stored on the memory and executable on the processor for implementing the method as described above when the processor executes the computer program.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.

Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.

According to the embodiment of the disclosure, after the data packet is detected and processed based on the firewall policy, the content of the data packet is also subjected to security detection, so that the firewall detection depth and the firewall detection effect are effectively improved.

Drawings

For a more complete understanding of the present disclosure and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

fig. 1 schematically illustrates a schematic diagram of an application scenario applicable to a data detection method, apparatus and electronic device according to an embodiment of the present disclosure;

fig. 2 schematically illustrates an exemplary system architecture of an electronic device and application data detection method, apparatus, and according to embodiments of the disclosure;

FIG. 3 schematically illustrates a flow chart of a data detection method according to an embodiment of the present disclosure;

FIG. 4 schematically illustrates a diagram of filtering a first packet according to an embodiment of the disclosure;

FIG. 5 schematically illustrates a schematic diagram of server load balancing according to an embodiment of the disclosure;

FIG. 6 schematically shows a schematic diagram of a binary image according to an embodiment of the disclosure;

FIG. 7 schematically shows a schematic diagram of an attack detection model and submodels according to an embodiment of the disclosure;

FIG. 8 schematically shows a structural schematic of a sub-model according to an embodiment of the disclosure;

FIG. 9 schematically shows a schematic structural diagram of a data detection apparatus according to an embodiment of the present disclosure;

FIG. 10 schematically illustrates a block diagram of a second detection module according to an embodiment of the disclosure;

FIG. 11 schematically illustrates a data detection logic diagram for a data detection apparatus according to an embodiment of the present disclosure; and

FIG. 12 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "a or B" should be understood to include the possibility of "a" or "B", or "a and B". The terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features.

To facilitate a better understanding of embodiments of the present disclosure, firewall technology is first described. In the related art, functions such as a firewall function and load balancing may be provided by a server. Firewalls or other security measures may be employed to separate internal networks or personal computers and the like from external networks. However, firewalls are not very secure against external attacks, including, for example: viruses, worms, trojans, or other forms of malicious code, as well as computer hacker intrusion, insider attacks, errors, and possible omissions, etc., make the network vulnerable. Therefore, users desire to enhance the protection capabilities of firewalls and to reduce network resources consumed by the protection functions provided.

It should be noted that in order to implement the protection function of the firewall, a specific set of firewall policies may be set in the firewall according to the needs of the user. For example, firewall policies may include specific firewall rules, address translations, addresses that may or may not be allowed, blocking signatures, anti-virus signatures, and the like.

The embodiment of the disclosure provides a data detection method, a data detection device and electronic equipment. The method includes a head detection process and a body detection process. In the header detection process, a data stream is received firstly, the data stream comprises at least one first data packet, then the at least one first data packet is processed based on a firewall policy, at least one second data packet is determined, and the first number of the first data packets is more than or equal to the second number of the second data packets. And after the head detection process is finished, entering a main body detection process, and detecting data in at least one second data packet by using an attack detection model to obtain a detection result, wherein the detection result comprises an attack type. This also makes it possible to: and if the detection result is the attack type, outputting prompt information to prompt that the second data packet corresponding to the detection result comprises the attack type data. On the basis of detecting the head, when the data traffic passes through the firewall, the embodiment of the disclosure further performs deep security detection on the content of the data traffic. For example, in addition to detecting a blacklist, a DDOS and the like, the detection of deep contents such as an attack mode and the like can be realized, the comprehensiveness of the detected contents is effectively improved, and the user experience is improved.

Fig. 1 schematically illustrates an application scenario applicable to a data detection method, apparatus and electronic device according to an embodiment of the present disclosure.

As shown in fig. 1, when a plurality of first packets included in a network data flow pass through a firewall, the firewall first detects the first packets and may determine a destination address of the first packets. In the process of detecting the first data packet, a part of the first data packet can be filtered out to form a second data packet. At the moment, the data of the second data packet is processed by using the attack detection model, so that the second data packet is deeply detected to determine whether the attack data is included. In the process of processing the second data packet by using the attack detection model, information included by the second data packet does not need to be restored, and the network data stream can be rapidly detected on the premise of low network resource consumption, so that the firewall detection effect is effectively improved.

Fig. 2 schematically illustrates an exemplary system architecture of an electronic device and application data detection method, apparatus, and device according to an embodiment of the disclosure.

It should be noted that fig. 2 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.

As shown in fig. 2, a system architecture 200 according to an embodiment of the present disclosure may include

terminal devices

201, 202, 203, a network 204, a firewall 205, and a server 206. The

end devices

201, 202, 203, the firewall 205 and the server 206 may be connected by a network 204, and the network 204 may comprise various connection types, such as wired, wireless communication links or fiber optic cables, etc.

The

terminal devices

201, 202, 203 may be a variety of electronic devices that support network transport functions including, but not limited to, smart phones, tablets, laptop portable computers, mainframe and desktop computers, and the like. According to the embodiment of the present disclosure, the

terminal devices

201, 202, 203 may transmit data therebetween through a local area network, a wide area network, or the like, for example.

The firewall 205 may be connected to the

terminal devices

201, 202, 203, for example, and network traffic and data packets flowing into and out of the

terminal devices

201, 202, 203 may all pass through the firewall 205.

The server 206 may be a server providing various services, such as a background management server (for example only) providing support for requests initiated by users with the

terminal devices

201, 202, 203. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that the data detection method provided by the embodiment of the present disclosure may be generally executed by the firewall 205. Accordingly, the data detection apparatus provided by the embodiments of the present disclosure may be generally disposed in the firewall 205. The data detection method provided by the embodiment of the present disclosure may also be performed by a server or a server cluster that is different from the firewall 205 and is capable of communicating with the

terminal devices

201, 202, 203 and/or the firewall 205. Accordingly, the data detection apparatus provided in the embodiment of the present disclosure may also be disposed in a server or a server cluster that is different from the firewall 205 and is capable of communicating with the

terminal devices

201, 202, and 203 and/or the firewall 205.

It should be understood that the number of end devices, firewalls, and servers in fig. 2 are merely illustrative. There may be any number of end devices, firewalls, and servers, as desired for implementation.

Fig. 3 schematically shows a flow chart of a data detection method according to an embodiment of the present disclosure.

As shown in fig. 3, the data detection method performed by the server side may include operations S301 to S307.

In operation S301, a data stream is received, the data stream including at least one first data packet.

In this embodiment, the first data packet may include a data packet generated by handshaking based on a specified protocol between the electronic device and the server, and a data packet transmitted after the connection is established. The designated protocol may include an HTTPS protocol, a Transport Layer Security (TLS) protocol, a Secure Socket Layer (SSL) protocol, and other communication protocols that can encrypt a data stream.

For example, at least a portion of the first data packet may include header information (e.g., a frame header, an Internet Protocol (IP) header), etc.). The header information may include information such as an IP Address, a Media access control Address (MAC Address), a port, and the like, so as to accurately transmit the packet to the target object. In addition, the data packet may further include a data portion, and the data portion is used for carrying transmitted information. However, the data portion may include data of attack type, which brings security risk to the internet. The firewall in the related art cannot detect the data of the attack type included in the data portion well.

In operation S303, at least one first packet is processed based on the firewall policy, and at least one second packet is determined, where a first number of the first packets is greater than or equal to a second number of the second packets.

The firewall policy may be set by the user, for example, the firewall policy may include a black list and a white list for the IP address, and sensitive word filtering.

In one embodiment, processing the at least one first packet based on the firewall policy and determining the at least one second packet may include the following. First, header information of at least one first packet is determined. Then, the at least one first data packet is filtered based on the header information and the firewall policy to obtain at least one second data packet. For example, the frame header and the IP header are parsed, and basic security detection is performed. The ethernet frame includes a destination MAC, a source MAC, and a protocol type. The destination MAC is placed in the source MAC, so that the rapid forwarding can be realized. Only the destination MAC address is searched, and the forwarding is directly carried out based on the searched MAC address, so that the forwarding speed can be effectively increased by putting the destination MAC in the process of searching in the front. The source MAC address portion may include 6 bytes. The protocol type (which functions to identify higher layer protocols) portion may comprise 2 bytes, for example, a common protocol identification includes: IPV 40 x0800, IPX 0x8137, ARP 0x0806, IPV 60 x86 DD.

The message structure content of the IP header may include: IPV4 header information has a minimum of 20 bytes and a maximum of 60 bytes, and IP headers are arranged in units of 4 bytes (32 bits). The first four bytes are version information. The second four bytes represent an offset, such as the several-chip info and FLAG (FLAG) field in a long message. The third four bytes include time-to-live, protocol, header checksum information. The fourth four bytes is a 32-bit source IP address. The fifth four bytes is a 32-bit destination IP address.

This allows determining a plurality of address information of the first packet at the same time after performing operation S303, so as to facilitate subsequent packet forwarding.

It should be noted that, in order to increase the attack detection speed of the data packet including the header information (for example, when the data packet during handshake has not resolved the information such as the source IP address, etc., blacklist filtering cannot be implemented, which easily causes missing detection), the source IP address, the domain name, etc. may be determined by resolving the digital certificate, so that blacklist filtering may be implemented on the data packet during handshake. For example, the SSL certificate may include the following information.

Issuing:

common Name (CN) · icbc

Tissue (O) INDUSTRIAL AND COMMERCIAL BARK OF CHINA

……

(the content of the above SSL certificate is merely illustrative)

This helps to improve the comprehensiveness of the detection.

Fig. 4 schematically illustrates a schematic diagram of filtering a first data packet according to an embodiment of the present disclosure.

As shown in fig. 4, the firewall policies set by the user include: the processing mode of the first data packet adapted to the blacklist is to discard, and the processing mode of the first data packet adapted to the whitelist is to pass, so as to prevent Distributed Denial of Service (DDOS) attack and the like. The blacklist and the whitelist may include a plurality of IP addresses, and the IP address may be a source IP address or a destination IP address. And if the matching is successful, performing corresponding data packet discarding operation or subsequent attack detection operation.

In one embodiment, data transmission efficiency is improved because data transmission can be performed between two electronic devices in a session manner. A session may include a plurality of data packets, and when at least one data packet in a session includes data of an attack type, it indicates that there is a hidden danger in the security of the data transmitted based on the session.

For example, the above method may further include the following operations. After obtaining at least one second data packet, for each second data packet, if a session corresponding to the second data packet does not exist, creating a first session corresponding to the second data packet, so as to transmit the second data packet based on the first session. Specifically, for the case of no session, first, blacklist matching may be performed, and if the source address of the second data packet hits the blacklist, the second data packet is discarded, and the subsequent process is ended. If the source Address of the second data packet does not hit the blacklist, performing routing table query, Network Address Translation (NAT) policy matching and the like, and finally performing session creation.

For another example, the method may further include the following operations. After at least one second data packet is obtained, for each second data packet, if a second session corresponding to the second data packet exists, the second session is refreshed, then, server load balancing is carried out at least based on the second session, a destination address is obtained, and the second data packet is sent to the destination address. Specifically, for the case of a session, operations such as session refreshing, server load balancing, routing table query, and the like may be performed, and blacklist matching is performed, and if the source address of the second data packet hits the blacklist, the second data packet is discarded, and the subsequent process is ended. And if the source address of the second data packet is not in black, carrying out user redirection and the like. In the related art, if the source address of the second data packet is not black, the second data packet is released to be sent to the destination address, and in the embodiment of the disclosure, deep content detection is performed on data of the second data packet before the second data packet is released, so as to improve network security.

Fig. 5 schematically illustrates a schematic diagram of performing server load balancing according to an embodiment of the present disclosure.

As shown in fig. 5, load balancing may distribute a packet to a plurality of operation units, such as a Web server, a File Transfer Protocol (FTP) server, an application server cluster, and other mission-critical servers, so as to collectively complete a work task. Load balancing routing mode, bridging mode, and service direct return mode. In fig. 5, IP addresses, domain names, ports, and the like such as x.x.x.x, y.y.y, z.z.z.z are set in the load balancing, so that the firewall can distribute the packets based on the packet allocation condition, the task completion condition, and the like of each address after the received packets 1, 2, 3, and the like.

In operation S305, data in the at least one second data packet is detected by using the attack detection model, and a detection result is obtained, where the detection result includes an attack type.

In this embodiment, the performance of the firewall can be effectively improved by performing attack detection on the data portion of the second data packet. Wherein, the detection of the data portion of the data packet may employ a variety of correlation detection techniques. For example, the data of the data portion is first restored (e.g., decoded, etc.) to obtain information carried by the data portion, so that whether the data portion contains data of an attack type, such as whether the data portion contains malicious code, a malicious function, etc., can be determined based on the parsed information.

Since the data portion of the data packet is transmitted in the form of binary digits, in order to improve the detection efficiency, it is also possible to determine whether there is data of the attack type by converting the data of the data portion into a binary map.

In one embodiment, detecting data in the at least one second data packet using the attack detection model may include the following operations. Firstly, carrying out flow segmentation on at least one second data packet to obtain at least one file, wherein each file is a second data packet group taking a session as a unit. Then, the at least one file is converted into at least one binary image, the binary image includes a specified number of pixels, each byte of the file has a corresponding pixel, and the third number of the at least one file is less than or equal to the fourth number of the at least one binary image. Next, the at least one binary image is converted into at least one feature vector. And then, processing at least one characteristic vector by using the attack detection model to obtain a detection result.

For example, with respect to traffic splitting, it is used to split a continuous original network data stream to add discrete traffic units. The input data format for traffic splitting may be Pcap and split in units of sessions, (e.g., using session + All for traffic splitting), and the output data format for traffic splitting is still Pcap. The Pcap file format is a standard format for capturing network data packets, and the Pcap file is a binary stream file and consists of a header and a plurality of data packets (including a data packet header and data information), wherein the header is 24 bits, and the data packet header is 16 bits. The data information in the data packet is an ethernet frame, such as an ethernet frame header (16 bits), an IP header (20 bits), and a TCP data header (20 bits), followed by the data to be transmitted.

For example, with respect to image generation, all files are first sorted to a uniform length. If the file size is larger than 784 bytes, it is clipped to 784 bytes. If the file size is less than 784 bytes, 0x00 is added at the end to make up to 784 bytes. So that the same size result file is converted into a gray image. Each byte of the original file represents a pixel, e.g., 0x00 is black and 0xff is white. It should be noted that the file size of 784 bytes is only an exemplary illustration, and may be, for example, 100 bytes, 200 bytes, 550 bytes, 900 bytes, and the like, which is not limited herein.

Fig. 6 schematically shows a schematic diagram of a binary image according to an embodiment of the present disclosure.

As shown in fig. 6, "0 x 00" is represented by the number "0", i.e., the grid in which the number "0" is located in fig. 6 may be a black pixel. "0 xff" is represented by the number "1", i.e., the grid in which the number "1" is located in fig. 6 may be a white pixel. The 36 bytes shown in the left diagram of fig. 6 may be only a representation of a part in the file, and correspondingly, the 36 pixels shown in the right diagram of fig. 6 may be only a representation of a part in the image.

For example, regarding feature (vector) extraction, it can be implemented based on Index (IDX) conversion. The image is converted into an IDX format file. The IDX file contains all pixels of a group of pictures and statistics (e.g., the number of "0" s, the number of "1" s). As in the image of fig. 6, multiple sets of feature vectors can be obtained by IDX conversion: 0,1,1,1,0,0 … …; 0,0,1,1,0,0 … …; 0,1,1,1,0,1 … …; 1,0,1,1,1,0 … …; 0,1,1,1,0,1 … …; 0,0,1,1,1,0 … …; … …, etc.

In one embodiment, the attack detection model may include at least one sub-model for a particular attack type. That is, the attack detection model may be composed of a plurality of submodels, and each submodel may perform detection of a specific attack type, respectively. It should be noted that the attack detection model may be called as a whole, or each sub-model may be called separately, which is convenient for satisfying the diversified demands of the user. For example, in a specific scenario, if a user wants to perform only web backdoor (webshell) detection and Structured Query Language (SQL) injection detection, and does not perform detection of other attack types, the user may subscribe to a webshell detection model and an SQL injection detection model. In addition, the user can order a plurality of or all attack detection models and then select the required sub-models by himself or herself when using the attack detection models.

Specifically, the above method may further include the following operations. First, a user operation instruction is received. Then, responding to the user operation instruction, at least one sub-model is selected from the attack detection model, and at least one second data packet is used for detection. Therefore, convenience of the user in carrying out targeted detection on the network data traffic can be effectively improved. In addition, the detection speed can be effectively improved, and if a user determines that attack data detection of a command execution type is not needed in a specific scene, corresponding detection can be performed without wasting time.

FIG. 7 schematically shows a schematic diagram of an attack detection model and submodels according to an embodiment of the disclosure.

As shown in fig. 7, the attack detection model may include: the SQL injection detection model and the file inclusion detection model are at least one of a SQL injection detection model, a command execution detection model, a code execution detection model, a webshell detection model, a Cross site scripting (XSS) detection model, a Cross Site Request Forgery (CSRF) detection model, a malicious crawler detection model and the like. The user can select the required detection model by himself when arranging the firewall.

In an embodiment, to improve attack detection efficiency, the method may further include: after obtaining the at least one file, for the at least one file, empty files and/or duplicate files are removed.

For example, with respect to traffic flushing, empty and duplicate files may be deleted. The data format in this step is unchanged, i.e., is still in Pcap format. Since the operation is to perform security detection on the data in the data packet, whether the data includes attack type data is determined. Repeated detection is not needed for repeated files so as to improve the detection efficiency. It should be noted that, when it is determined that the data packet includes repeated data and the repeated data is deleted, the data packet identifier of the data packet in which the deleted file is located needs to be stored, so as to perform backtracking, for example, after the file a in the data packet a and the file B in the data packet B are repeated files, after the file B is deleted, the file B needs to be stored to be the same as the file a, and the file B is information of the data in the data packet B, so that after it is determined that the file a includes data of the attack type, it is convenient to determine that the data packet a and the data packet B include files of the attack type.

Furthermore, before removing the empty file and/or the duplicate file, the method may further include the operations of: and randomizing the MAC address and/or the IP address in the file. For example, after said obtaining at least one file, for at least one said file, the MAC address in at least one said file is first randomized at the data link layer, and/or the IP address in at least one said file is randomized at the IP layer. Then, at least one of the files after randomization is removed of empty files and/or duplicate files. Therefore, data anonymization operation can be realized, interference of different MAC addresses and IP addresses on subsequent attack model judgment results is avoided (the data input into an attack detection model only contains contents related to attack technique characteristics as much as possible, and other irrelevant contents are reduced). By randomizing the MAC address and the IP address, contents irrelevant to the characteristics of the attack technique, such as the MAC geology and the IP geology, can be anonymized, and the interference of the contents on the model training result is reduced. Regarding the topology of the attack detection model. In one embodiment, the attack detection model includes an input layer, at least one pair of convolutional layers and pooling layers arranged in an overlapping arrangement, and at least one fully-connected layer adjacent to the pooling layers. It should be noted that, when the attack detection model includes a plurality of submodels, the topology of each submodel may be the same or different, for example, a topology such as a convolutional neural network may be adopted. In addition, various other types of topologies may also be employed, such as an inverse deep convolutional inverse graph network, a recurrent neural network, a deep residual error network, and so forth.

Fig. 8 schematically shows a structural schematic of a sub-model according to an embodiment of the disclosure.

As shown in fig. 8, the topology of each submodel may be the same. For example, each sub-model may include, in order: input layer, convolutional layer, pooling layer, and multiple fully-connected layers. The convolution kernels used by the first convolution layer and the second convolution layer can be the same or different, and after convolution operation, the dimensionality of the feature vector can be reduced.

For example, model training with respect to attack detection models. The model training can be in an off-line training mode, and the trained model can be used by the firewall in a issuing mode. Specifically, in the model training, a back propagation algorithm may be used for the model training. The training data used in the model training may be positive samples (e.g., data with correct labeling information) and/or negative samples (e.g., data with incorrect labeling information).

The SQL injection detection model is taken as an example for illustration. The topological structure of the SQL injection detection model can use the architecture of a Convolutional Neural Network (CNN) to mark and preprocess 5000 positive samples and 5000 negative samples, and then model training is carried out on the IDX file obtained by preprocessing so as to output the final model judgment result. Reading 28 x 1 data flow images from the IDX file and starting from the first convolution layer, performing a convolution operation, using 20 kernels of size 5 x 5, can result in 20 feature maps of size 24 x 24. After the first convolutional layer, a 2 x 2 max pool operation is performed on the pooled layer, resulting in 20 12 x 12 feature maps. The second convolutional layer used 50 kernels of 5 x 5, resulting in 50 feature maps of size 8 x 8. After the second 2 x 2 max pool operation, 50 feature maps of size 4 x 4 were generated. The last two layers are fully connected layers, the result sizes are 500 and 10, respectively, and the softmax function is used to output the corresponding probability results. And finally, performing model training according to a preset accuracy rate to obtain model parameters, and performing operations such as effect verification and the like.

In operation S307, if the detection result is the attack type, a hint information is output to hint that the second data packet corresponding to the detection result includes the attack type data.

In this embodiment, the operation required for the next step can be determined according to the detection result of the model. And if the attack detection model identifies that the flow packet is SQL injection attack, sending out a related alarm prompt of SQL injection.

In one embodiment, the detection result includes a non-attack type. Accordingly, the above method may further include the following operations. And if the detection result is of a non-attack type, releasing the second data packet corresponding to the detection result. For example, if the identification result is normal traffic, the packet is forwarded normally.

The data detection method provided by the embodiment of the disclosure performs deep security detection on data of a data packet based on artificial intelligence on the basis of a traditional firewall, for example, when data flow passes through the firewall, input data of an attack detection model is obtained by performing data segmentation, flow cleaning, image generation, IDX conversion and other processing on the data flow received by the firewall, and performs security detection on at least one attack type on the input data (capable of representing data packet content) based on a transmitted attack detection model, so that alarm prompt can be respectively performed on different attack types.

On the one hand, prevent hot wall's safety inspection mode, from relying on artifical maintenance prevent that hot wall static policy issues, change into and can further carry out the degree of depth detection based on artificial intelligence training's dynamic model, not only make the enterprise can customize and multiplex to preventing hot wall safety inspection model according to the demand, strengthen preventing hot wall's customization function, reduce protection cost, through introducing the safety inspection model based on artificial intelligence, also make prevent hot wall's intellectuality and detection efficiency to promote.

On one hand, the safety detection content of the firewall is converted from the basic content detection of the blacklist, the DDOS and the like into the deep content detection of the blacklist, the DDOS and the attack mode, so that the detection content is more comprehensive.

On the one hand, the training of the attack detection model is carried out in an off-line mode, so that not only can the model training and the effect verification be carried out under the condition of ensuring the normal use of the firewall, but also the reliability of detection of different attack types can be greatly improved through the setting of the accuracy and the training of the iteration model.

In one embodiment, if the detection result is the attack type, outputting the hint information to hint that the second data packet corresponding to the detection result includes the attack type data may include the following operations.

And if the detection result is the attack type, intercepting a second data packet corresponding to the detection result. If the second data packet includes data of an attack type such as SQL injection, XSS and the like, the second data packet corresponding to the detection result can be intercepted. It should be noted that the second data packet is not directly discarded, and the user experience may be degraded if the second data packet is directly discarded due to the possibility of inaccurate detection result. However, the second packet may not be directly released in order to secure the network.

And then, sending interception prompt information to the destination address of the intercepted second data packet. This allows the user to determine whether the second data packet needs to be received, reducing the risk of loss of the second data packet that the user demands.

And then, in response to receiving a data release request from the destination address, sending the intercepted second data packet to the destination address, wherein the data release request is generated after the user operates the interception prompt message. For example, if the user determines that it is desired to receive the second packet after seeing the presented interception prompting message, a data passing request may be sent by the user to the firewall to receive the second packet from the firewall.

Fig. 9 schematically shows a structural diagram of a data detection apparatus according to an embodiment of the present disclosure.

As shown in fig. 9, the data detection apparatus 900 may include a data receiving module 910, a first detection module 920, a second detection module 930, and a prompt module 940.

The data receiving module 910 is configured to receive a data stream, where the data stream includes at least one first data packet.

The first detection module 920 is configured to process at least one first packet based on the firewall policy and determine at least one second packet, where a second number of the second packets is less than or equal to the first number of the first packets.

The second detecting module 930 is configured to detect data in the at least one second data packet by using the attack detection model, so as to obtain a detection result, where the detection result includes an attack type and a non-attack type.

The prompting module 940 is configured to output prompting information to prompt that the second data packet corresponding to the detection result includes the attack type data if the detection result is the attack type.

FIG. 10 schematically illustrates a block diagram of a second detection module according to an embodiment of the disclosure.

As shown in fig. 10, the second detection module 930 may include: the device comprises a flow dividing unit, an image generating unit, a feature extracting unit and a model detecting unit.

The flow dividing unit is used for dividing the flow of at least one second data packet to obtain at least one file, and each file is a second data packet group taking a session as a unit.

The image generation unit is used for converting at least one file into at least one binary image, the binary image comprises a specified number of pixels, each byte of the file has a corresponding pixel, and the third number of the at least one file is less than or equal to the fourth number of the at least one binary image.

The feature extraction unit is used for converting at least one binary image into at least one feature vector.

The model detection unit is used for processing at least one feature vector by using the attack detection model to obtain a detection result.

In addition, the second detection module 930 may further include: the device comprises a flow cleaning unit, an IDX conversion unit, a model determination unit and a result output unit. Such as a traffic purge unit, for deleting duplicate or empty packets. The IDX conversion unit is used for converting the files in the data packet into files in an IDX format. The model determining unit is used for determining the sub-models based on user requirements so as to detect specific attack behaviors corresponding to the selected sub-models. And the result output unit is used for outputting whether the second data packet comprises the data of the attack type.

In one embodiment, the first detection module 920 may include a base processing unit, a first packet processing unit, and a subsequent packet processing unit.

The basic processing unit mainly functions to perform basic processing on the data packet, such as parsing a frame header and an IP header, basic security detection, and the like, and determine whether to create a session subsequently. If a session needs to be created, the session enters a first packet processing unit, operations such as blacklist matching, routing table query, NAT policy matching and the like are performed, and the session is finally created. And if the session does not need to be created, entering a subsequent packet processing unit, and performing operations such as session refreshing, server load balancing, routing table query, blacklist, user redirection and the like. Accordingly, the second detection module 930 may perform preprocessing and attack type determination on the second data packet based on the trained model, and output the determination result.

FIG. 11 schematically illustrates a data detection logic diagram of a data detection apparatus according to an embodiment of the present disclosure.

As shown in fig. 11, after receiving the network traffic, the data receiving module 910 performs basic processing on the second data packet by the basic processing unit, including parsing a frame header and an IP packet header of the network packet, and performing basic security detection such as DDOS according to some information of the header.

The first packet processing unit firstly performs blacklist matching for the situation that a session needs to be created, if a source address hits the blacklist, the message is discarded, and the subsequent flow is ended. And if the blacklist is not hit, performing routing table query, NAT strategy matching and the like, and finally performing session creation.

The subsequent packet processing unit mainly includes operations such as session update, server load balancing, routing table query, blacklist, user redirection (for example, forwarding to an address not exactly the same as a destination address), and the like, for a case where a new session does not need to be created.

The second detection module 930 may be configured to implement functions including second packet preprocessing, attack detection model issuing, second packet detection, and the like. And outputting a model judgment result of whether the second data packet is of a relevant attack type by processing the second data packet and detecting the specific attack type. Wherein, the attack detection model can be trained in a mode of notifying offline training in advance. For example, the attack detection model may include at least one of: the SQL injection detection model and the file inclusion detection model, the command execution detection model, the code execution detection model, the webshell detection model, the XSS detection model, the CSRF detection model, the malicious crawler detection model and the like are used for identifying the corresponding attack types. In addition, the second detection module 930 performs image processing on the second data packet, so in terms of selection of the model training algorithm, it needs to be ensured that the data format after preprocessing of the second data packet can be processed, such as various neural network algorithms for training image data.

Specifically, the second detecting module 930 is configured to implement the following functions: the processes of flow segmentation, flow cleaning, image generation, IDX conversion, etc. preprocess the second data packet into content that the detection model can process to identify, such as converting the original flow data (e.g., Pcap format) into CNN input data (e.g., IDX format).

The data detection device provided by the embodiment of the disclosure carries out deep content detection on a second data packet through a second detection module based on a issued attack detection model on the basis of a traditional firewall, improves the customization and intelligent degree of the firewall, improves the detection efficiency and accuracy and reduces the enterprise protection cost on the basis of improving the comprehensiveness of the detection content. If the attack detection model aiming at the specific attack data is issued by the second detection module before the second data packet is forwarded, the specific attack data is judged and identified, and an alarm prompt is given. Therefore, model training and accuracy setting can be flexibly carried out on different types of attack flows according to user requirements, the issued models can be multiplexed, the labor and time cost invested by enterprise training models is reduced, the detection advantages of the firewall on the flows are utilized, and operation and maintenance personnel are helped to know an attack alarm prompt on the firewall layer of flow access so as to facilitate subsequent flow interception and tracing. Even at least part of network nodes of the intranet do not need to be configured with a protection tool and the like aiming at a specific attack mode.

The operations executed by the modules can refer to the related contents of the method part as shown above, and are not described in detail here.

Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.

For example, any number of the data receiving module 910, the first detecting module 920, the second detecting module 930, and the prompting module 940 may be combined in one module to be implemented, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the data receiving module 910, the first detecting module 920, the second detecting module 930, and the prompting module 940 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware, and firmware, or implemented by a suitable combination of any several of them. Alternatively, at least one of the data receiving module 910, the first detecting module 920, the second detecting module 930 and the prompting module 940 may be at least partially implemented as a computer program module, which when executed, may perform a corresponding function.

FIG. 12 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure. The electronic device shown in fig. 12 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 12, an electronic apparatus 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1202 or a program loaded from a storage section 1208 into a Random Access Memory (RAM) 1203. The processor 1201 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 1201 may also include on-board memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the present disclosure.

In the RAM1203, various programs and data necessary for the operation of the electronic apparatus 1200 are stored. The processor 1201, the ROM 1202, and the RAM1203 are connected to each other by a bus 1204. The processor 1201 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1202 and/or the RAM 1203. Note that the programs may also be stored in one or more memories other than the ROM 1202 and the RAM 1203. The processor 1201 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

Electronic device 1200 may also include input/output (I/O) interface 1205, according to an embodiment of the disclosure, input/output (I/O) interface 1205 also connected to bus 1204. The electronic device 1200 may also include one or more of the following components connected to the I/O interface 1205: an input section 1206 including a keyboard, a mouse, and the like; an output portion 1207 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 1208 including a hard disk and the like; and a communication section 1209 including a network interface card such as a LAN card, a modem, or the like. The communication section 1209 performs communication processing via a network such as the internet. A driver 1210 is also connected to the I/O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is mounted on the drive 1210 as necessary, so that a computer program read out therefrom is mounted into the storage section 1208 as necessary.

According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 1209, and/or installed from the removable medium 1211. The computer program, when executed by the processor 1201, performs the above-described functions defined in the electronic device of the embodiments of the present disclosure. According to embodiments of the present disclosure, the electronic devices, apparatuses, devices, modules, units, and the like described above may be realized by computer program modules.

The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 1202 and/or the RAM1203 and/or one or more memories other than the ROM 1202 and the RAM1203 described above.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. A data detection method performed by an electronic device, comprising:

receiving a data stream, the data stream comprising at least one first data packet;

processing at least one first data packet based on a firewall policy, and determining at least one second data packet, wherein the first number of the first data packets is more than or equal to the second number of the second data packets;

detecting data in at least one second data packet by using an attack detection model to obtain a detection result, wherein the detection result comprises an attack type; and

and if the detection result is the attack type, outputting prompt information to prompt that the second data packet corresponding to the detection result comprises attack type data.

2. The method of claim 1, wherein the detecting data in at least one of the second data packets using an attack detection model comprises:

performing flow segmentation on at least one second data packet to obtain at least one file, wherein each file is a second data packet group taking a session as a unit;

converting at least one of the files into at least one binary image, the binary image including a specified number of pixels, each byte of the file having a corresponding pixel, the third number of at least one of the files being less than or equal to the fourth number of at least one of the binary images;

converting at least one of the binary images into at least one feature vector; and

and processing at least one feature vector by using an attack detection model to obtain a detection result.

3. The method of claim 2, wherein:

the attack detection model comprises at least one sub-model for a specific attack type;

the method further comprises the following steps:

receiving a user operation instruction; and

and responding to the user operation instruction, and selecting at least one sub-model from the attack detection model to detect at least one second data packet.

4. The method of claim 2, further comprising: after said obtaining at least one file, for at least one of said files,

removing empty files and/or duplicate files;

or

Randomizing MAC addresses in at least one of said files at the data link layer and/or randomizing IP addresses in at least one of said files at the IP layer; and

and removing the empty file and/or the repeated file in at least one file after randomization.

5. The method of claim 1, wherein the processing at least one of the first packets based on a firewall policy and determining at least one second packet comprises:

determining header information of at least one of the first data packets; and

and filtering at least one first data packet based on the header information and a firewall policy to obtain at least one second data packet.

6. The method of claim 5, further comprising: after said obtaining of the at least one second data packet, for each second data packet,

and if the session corresponding to the second data packet does not exist, creating a first session corresponding to the second data packet to transmit the second data packet based on the first session.

7. The method of claim 5, further comprising: after said obtaining of the at least one second data packet, for each second data packet,

if the second session corresponding to the second data packet exists, refreshing the second session; and

and balancing the server load at least based on the second session to obtain a destination address, so as to send the second data packet to the destination address.

8. The method of any one of claims 1 to 7, wherein the attack detection model comprises an input layer, at least one pair of overlapping convolutional and pooling layers, and at least one fully-connected layer adjacent to the pooling layer.

9. The method according to any one of claims 1 to 7, wherein the outputting prompt information to prompt that the second data packet corresponding to the detection result includes attack type data if the detection result is an attack type comprises:

if the detection result is the attack type, intercepting a second data packet corresponding to the detection result;

transmitting interception prompt information to a destination address of the intercepted second data packet;

and responding to a received data release request from the destination address, and sending the intercepted second data packet to the destination address, wherein the data release request is generated after a user operates the interception prompt information.

10. The method of any one of claims 1 to 7, wherein the detection result comprises a non-attack type;

the method further comprises the following steps:

and if the detection result is of a non-attack type, releasing a second data packet corresponding to the detection result.

11. A data detection apparatus comprising:

the data receiving module is used for receiving a data stream, and the data stream comprises at least one first data packet;

the first detection module is used for processing at least one first data packet based on a firewall policy and determining at least one second data packet, wherein the second quantity of the second data packets is less than or equal to the first quantity of the first data packets;

the second detection module is used for detecting data in at least one second data packet by using an attack detection model to obtain a detection result, wherein the detection result comprises an attack type and a non-attack type; and

and the prompting module is used for outputting prompting information to prompt that the second data packet corresponding to the detection result comprises attack type data if the detection result is the attack type.

12. An electronic device, comprising:

one or more processors;

a storage device for storing executable instructions which, when executed by the processor, implement the method of any one of claims 1 to 10.