CN108900374B - Data processing method and device applied to DPI equipment - Google Patents
Data processing method and device applied to DPI equipment Download PDFInfo
- Publication number
- CN108900374B CN108900374B CN201810654859.2A CN201810654859A CN108900374B CN 108900374 B CN108900374 B CN 108900374B CN 201810654859 A CN201810654859 A CN 201810654859A CN 108900374 B CN108900374 B CN 108900374B
- Authority
- CN
- China
- Prior art keywords
- target
- data stream
- application type
- target data
- layer protocol
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/026—Capturing of monitoring data using flow identification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/028—Capturing of monitoring data by filtering
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a data processing method and device applied to DPI equipment, and belongs to the technical field of data processing. The method comprises the following steps: when a target data stream is received, determining a target application layer protocol adopted by the target data stream; acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to the target application layer protocol; and distributing the target data stream to target processing equipment corresponding to the target application type. By adopting the invention, the processing efficiency of the data stream can be improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus applied to a DPI device.
Background
Currently, the internet shows a development trend of high traffic and high bandwidth, and under the development trend, DPI (Deep Packet Inspection) equipment is widely applied. The DPI device is a traffic detection device, and is generally deployed at a key node in a network, and is capable of performing detection and identification on a data flow passing through the key node, and then sending an identification result to a corresponding analysis platform, where the analysis platform is capable of performing analysis statistics on the data flow passing through the key node based on the identification result.
After receiving the data stream, the DPI device may identify, based on a packet in the data stream, an application layer Protocol used by the data stream, such as a Protocol like FTP (File Transfer Protocol), SMTP (Simple Mail Transfer Protocol), HTTP (Hyper Text Transfer Protocol), and features of each packet, such as a packet length and information carried in each packet field. The DPI device may then generate a log file based on the identified information and send it to the analysis platform. Because the existing DPI device is mainly used to identify application layer protocols and packet features corresponding to data streams, and the data processing is rough, a better data processing method applied to the DPI device is urgently needed.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present invention provide a data processing method and apparatus applied to a DPI device. The technical scheme is as follows:
in a first aspect, a data processing method applied to a DPI device is provided, where the method includes:
when a target data stream is received, determining a target application layer protocol adopted by the target data stream;
acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to the target application layer protocol;
and distributing the target data stream to target processing equipment corresponding to the target application type.
Further, after determining the target application layer protocol adopted by the target data stream, the method further includes:
judging whether the target application layer protocol is an application layer protocol to be processed;
and if the target application layer protocol is the application layer protocol to be processed, acquiring the target message characteristics of the target data stream.
Further, after identifying the target application type corresponding to the target packet feature, the method further includes:
recording the target application type as an application type of the target data stream, wherein the target application type is an identified application type or an unidentified application type;
and counting the proportion of the data streams of the identified application type or the unidentified application type in all the data streams according to the application types of all the recorded data streams.
Further, the method further comprises:
determining a target area to which the target data stream belongs according to the target message characteristics, and updating the target message characteristics to a target database corresponding to the target area;
and generating a regional statistical result of the target region based on the target database, and displaying the regional statistical result on a page.
Further, the method further comprises:
determining a target data channel to which the target data stream belongs according to the target message characteristics;
counting target flow information of the target data channel, and calculating estimated bandwidth capacity of the target data channel based on the target flow information;
and if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, generating an adjustment instruction for the target data channel.
In a second aspect, a data processing apparatus for DPI device is provided, the apparatus comprising:
the device comprises a determining module, a judging module and a judging module, wherein the determining module is used for determining a target application layer protocol adopted by a target data stream when the target data stream is received;
the identification module is used for acquiring the target message characteristics of the target data stream and identifying the target application type corresponding to the target message characteristics by combining with the preset identification rule corresponding to the target application layer protocol;
and the distribution module is used for distributing the target data stream to the target processing equipment corresponding to the target application type.
Further, the apparatus further includes a determining module, configured to:
judging whether the target application layer protocol is an application layer protocol to be processed;
and if the target application layer protocol is the application layer protocol to be processed, acquiring the target message characteristics of the target data stream.
Further, the apparatus further comprises a statistics module configured to:
recording the target application type as an application type of the target data stream, wherein the target application type is an identified application type or an unidentified application type;
and counting the proportion of the data streams of the identified application type or the unidentified application type in all the data streams according to the application types of all the recorded data streams.
Further, the apparatus further comprises a display module configured to:
determining a target area to which the target data stream belongs according to the target message characteristics, and updating the target message characteristics to a target database corresponding to the target area;
and generating a regional statistical result of the target region based on the target database, and displaying the regional statistical result on a page.
Further, the apparatus further comprises a generating module configured to:
determining a target data channel to which the target data stream belongs according to the target message characteristics;
counting target flow information of the target data channel, and calculating estimated bandwidth capacity of the target data channel based on the target flow information;
and if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, generating an adjustment instruction for the target data channel.
In a third aspect, there is provided a network device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement the data processing method applied to a DPI device according to the first aspect.
In a fourth aspect, there is provided a computer readable storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by a processor to implement the data processing method as described in the first aspect applied to a DPI device.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
in this embodiment, when a target data stream is received, a target application layer protocol adopted by the target data stream is determined; acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to the target application layer protocol; and distributing the target data stream to target processing equipment corresponding to the target application type. Therefore, the DPI equipment can not only detect and identify the data stream, but also distribute the data stream with the identified application type to the processing equipment for processing the application type, thereby fully exerting the processing capacity of each processing equipment and improving the processing efficiency of the data streams with different application types. Moreover, the DPI equipment can also display the regional statistical result in a management interface corresponding to the DPI equipment in the form of a picture, a table, characters and the like, so that the time and the energy required by manual sorting can be saved, and higher accuracy is achieved. Meanwhile, the module architecture of the DPI equipment provided by the embodiment of the invention can integrate various data processing functions, so that the DPI equipment can be flexibly adapted to various application scenes, and the management complexity and the management cost of the DPI equipment can be effectively reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is an application scenario diagram of a DPI device according to an embodiment of the present invention;
fig. 2 is a flowchart of a data processing method applied to a DPI device according to an embodiment of the present invention;
figure 3 is a block architecture diagram of a DPI device according to an embodiment of the present invention;
FIG. 4 is a block diagram of the device configuration module of FIG. 3;
fig. 5 is a schematic structural diagram of a data processing apparatus applied to a DPI device according to an embodiment of the present invention;
figure 6 is a schematic structural diagram of a DPI device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The embodiment of the invention provides a data processing method applied to a DPI device, wherein an execution main body of the method can be the DPI device, the DPI device can detect and identify data streams and can be deployed in a plurality of application scenes such as a backbone network, a metropolitan area network, an enterprise internal network and the like, as shown in figure 1, the DPI device can be directly deployed in a network device link formed by a network card and a plurality of network devices, and thus, the data streams output by the network card can be returned to the network device of the next hop after being detected and identified by the DPI device. The DPI device may include a processor, a memory, and a transceiver, wherein the processor may be configured to perform various processing on data in the following procedures, the memory may be configured to store data required and generated in the following procedures, and the transceiver may be configured to receive and transmit related data in the following procedures. In a certain situation, the function of the DPI device may be implemented by any device embedded with a DPI functional assembly, and the embodiment takes the DPI device as an execution main body as an example for description, and the rest of the situations are similar and will not be described again.
The following will describe in detail a processing flow of the data processing method applied to the DPI device shown in fig. 2 with reference to the specific embodiment, and the content may be as follows:
step 201: when the target data stream is received, a target application layer protocol adopted by the target data stream is determined.
In implementation, a large amount of data traffic can flow into or out of a machine room of a service provider at every moment, and the data traffic can be transmitted between devices in a data stream form, for example, a Network machine room deployed by a CDN (Content Delivery Network) in a certain area can receive request data for different services sent by different users in the area, and can return corresponding service data to each user. The service provider may deploy the DPI device at a node externally contacted by the computer room, so that data traffic flowing into or out of the computer room can pass through the DPI device, and the DPI device can collect multiple data flows, and then the DPI device can perform various data processing such as identification and management on the data flows.
Specifically, when the DPI device receives any data flow (which may be referred to as a target data flow), the DPI device may first perform packet parsing on a packet of the target data flow in a network layer and a transport layer to obtain the following information of the packet: a source IP (Internet Protocol) address, a source port number, a destination IP address, a destination port number, and a Protocol type, where the Protocol type may be a transport layer Protocol such as TCP (Transmission Control Protocol), UDP (User Datagram Protocol), ICMP (Internet Control Message Protocol), and the like. The DPI device may then determine the application-layer protocol (which may be referred to as the target application-layer protocol) to be used by the target data stream by means of common port identification or feature word identification. Taking a common port identification manner as an example, since many standard protocol types support protocol identification using some standards or ports defined or recommended by an operator, for example, the FTP protocol often employs port 20/21, the SMTP protocol often employs port 25, and the HTTP protocol often employs port 80/8080, the DPI device may determine a target application layer protocol employed by a target data flow according to the common port identification manner by resolving a port number of the target data flow.
Optionally, the DPI device may selectively process the received data stream, and accordingly, the specific processing may be as follows: judging whether the target application layer protocol is an application layer protocol to be processed; and if the target application layer protocol is the application layer protocol to be processed, acquiring the target message characteristics of the target data stream.
In an implementation, the DPI device may set whether to process the data flow based on an application layer protocol employed by the data flow. For example, the DPI device may set certain application layer protocols as pending application layer protocols, and the DPI device may then only process data flows that employ the pending application layer protocols. Thus, after determining the target application layer protocol of the target data stream, the DPI device can determine whether the target application layer protocol is the application layer protocol to be processed, and if the target application layer protocol is the application layer protocol to be processed, the DPI device can continue to obtain the target message characteristics of the target data stream according to the subsequent processing flow; if the target application layer protocol is not the application layer protocol to be processed, the DPI device may not perform subsequent processing on the target data stream, such as discarding the data stream, or directly pass through the data stream.
Step 202: and acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to a target application layer protocol.
In implementation, a service provider may preset a preset identification rule corresponding to each application layer protocol in the DPI device, and the DPI device may identify application types corresponding to different packet features, such as video applications, voice applications, web applications, sub-applications flagged by an application provider, and the like, by using the preset identification rule. Taking a preset identification rule corresponding to the HTTP protocol and used for identifying a certain video application as an example, if information carried by fields of a host, uri, origin, and the like of a packet can be successfully matched with the preset identification rule, if the host field carries a website or a keyword of the video application, the matching is indicated to be successful, and the DPI device can identify an application type corresponding to the packet feature as the video application. In this way, after determining the target application layer protocol of the target data flow, the DPI device may perform application layer parsing on the packet of the target data flow based on the target application layer protocol, and then the DPI device may obtain packet characteristics (which may be referred to as target packet characteristics) of the target data flow, where the packet characteristics may be source IP and destination IP of the packet, URL carried in the packet, host field of the packet, and the like. Then, the DPI device may identify a target application type corresponding to the target packet feature in combination with a preset identification rule corresponding to the target application layer protocol.
Optionally, after the DPI device identifies the target application type corresponding to the target packet feature, it may also count a ratio of a certain type of data flow to all data flows, and the corresponding processing may be as follows: recording the target application type as the application type of the target data stream, wherein the target application type is an identified application type or an unidentified application type; and counting the proportion of the data streams with the identified application type or the unidentified application type in all the data streams according to the recorded application types of all the data streams.
In implementation, since new applications are layered endlessly, so that the types of the applications are extremely many, the DPI device may often recognize only a part of the application types, and therefore the DPI device may classify the application types of the data streams into recognized application types or unrecognized application types based on the recognition result, and further the DPI device may count the recognition rate of a certain application type, that is, count the proportion of the data streams of the certain application type in all the data streams. In this way, after the DPI device identifies the target application type corresponding to the target packet feature, for example, the identifiable application type or the unidentified application type, the identified target application type may be recorded as the application type of the target data stream. Then, the DPI device may count the proportion of the data flow of the identified application type to all the data flows, or count the proportion of the data flow of the unidentified application type to all the data flows, in real time or according to a preset period, such as ten minutes or half hour, based on the recorded application types of all the data flows.
Step 203: and distributing the target data stream to the target processing equipment corresponding to the target application type.
In implementation, the nodes in the machine room that are connected to outside may be connected to multiple processing devices in a wired or wireless manner, and at the same time, different processing devices may be configured to process data streams of a certain application type, for example, a server for processing a video-type application, a server for processing a voice-type application, a server or a service cluster for processing all sub-applications under a certain application provider may be configured according to application functions, and a server for processing a great wall bandwidth service, a server for processing a bandwidth services in a song hua may be configured according to a network operator. In this way, after the DPI device identifies the target application type corresponding to the target packet feature, the DPI device can distribute the target data stream to the target processing device corresponding to the target application type, thereby fully exerting the processing capability of each processing device and improving the processing efficiency of the data streams of different application types. It should be noted that, if the target application type is an unidentified application type, the DPI device may distribute the target data stream to a processing device corresponding to the unidentified application type, such as a general processing device for processing any application.
Optionally, the DPI device may perform statistics on packet characteristics of the data stream, and perform page display on the statistical result, where the corresponding processing may be as follows: determining a target area to which the target data stream belongs according to the target message characteristics, and updating the target message characteristics to a target database corresponding to the target area; and generating a regional statistical result of the target region based on the target database, and displaying the regional statistical result on a page.
In implementation, the area where the processing device receiving the data stream is located may be referred to as an area to which the data stream belongs, for example, the area may be divided into a beijing area, a building area, and the like according to geographic locations, meanwhile, different areas may correspond to respective databases, and the message characteristics of all data streams corresponding to the areas may be recorded in the databases. In this way, the DPI device may determine a target area to which the target data stream belongs according to the target packet characteristics, and then may update the target packet characteristics to a target database corresponding to the target area. For example, the DPI device may analyze packet characteristics of a destination IP or a specific message field of the target data flow to obtain regional information, and if the destination IP points to a building door region or a specific message field carries regional information related to a building door, the DPI device may determine that the target data flow belongs to the building door region. Then, the DPI device may generate a region statistical result of the target region based on the packet features of all data flows in the region recorded by the target database, and then the DPI device may perform page display on the region statistical result. Specifically, the DPI device may generate regional statistical results of multiple dimensions, such as a traffic proportion of each district and county of a building, a traffic proportion of each district and county of a building in different application types in each district and county of the building, and the like, according to the message characteristics of all data flows in the region recorded in the target database, according to different dimensions such as the district, the user, and the application type, and then the DPI device may display the regional statistical results in a management interface corresponding to the DPI device in the form of a drawing, a table, or a text, so that a technician may view the regional statistical results through the management interface, thereby saving time and effort required for manual organization, and having a higher accuracy.
Optionally, the DPI device may further feed back adjustment information for the data channel, and the corresponding processing may be as follows: determining a target data channel to which the target data stream belongs according to the characteristics of the target message; counting target flow information of the target data channel, and calculating the estimated bandwidth capacity of the target data channel based on the target flow information; and if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, generating an adjustment instruction for the target data channel.
In implementation, the service provider's equipment room may receive data traffic from each area through data channels, for example, the equipment room deployed in beijing may receive data traffic from each downtown area through data channels corresponding to each downtown area, such as the eastern city area, the western city area, and the hai lake area, and the service provider may set corresponding bandwidth capacities for each data channel in advance, for example, the bandwidth capacity of the data channel corresponding to the eastern city area may be 10Gbps, and the bandwidth capacity of the data channel corresponding to the western city area may be 12 Gbps. Meanwhile, the service provider may receive an adjustment instruction from the DPI device, and execute the adjustment instruction to adjust the bandwidth capacity of the data channel. Specifically, the DPI device may determine a target data channel to which the target data flow belongs according to the characteristics of the target packet, for example, channel information carried in a specific packet field, and then, the DPI device may calculate target traffic information of the target data channel in combination with the target data flow and other data flows of the target data channel, and further, the DPI device may calculate an estimated bandwidth capacity of the target data channel based on the target traffic information. Then, if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, the DPI device may generate an adjustment instruction for the target data channel, and may further trigger the service provider to adjust the bandwidth capacity of the target data channel through the adjustment instruction.
A modular architecture of the above DPI device is given below, which, as shown in fig. 3, may include a DPI body module and one or more of the following modules: the system comprises an equipment management module, an equipment configuration module, an equipment monitoring module, a log acquisition module, an equipment scheduling module, a primary data acquisition module, a secondary data acquisition module, an information display module and a data diversion module. Wherein:
the device management module may store an installation package and an installation deployment script of the DPI device, and may start or stop the DPI device by sending a start or stop command to the DPI device. Meanwhile, the equipment management module can detect the running state of the DPI equipment, and when the DPI equipment breaks down, the equipment management module can restart the DPI equipment and restore the original configuration of the DPI equipment in a mode of decompressing an installation package of the DPI equipment and running an installation deployment script.
The device monitoring module may be configured to monitor state parameters of the DPI device, such as parameters of a peak flow, a memory usage rate, a CPU usage rate, and a packet loss rate, and may further send the monitored state parameters to the log acquisition module or the device scheduling module in the form of log information.
The log collection module can read log information of the DPI equipment in real time or according to a preset period, the read log information can be stored in a corresponding log file, and the log collection module can also send the log information to other applications, for example, the urgent log information is sent to an email application, so that the urgent log information is sent to technical personnel through the email application, and the technical personnel can conveniently solve the problems in time.
The device scheduling module may determine whether to start the standby device of the DPI device according to the received status parameter of the device monitoring module, for example, when the current peak flow of the DPI device exceeds the warning threshold, the device scheduling module may send a start command to the standby device of the DPI device.
The primary data acquisition module may record the identification result of the DPI device, for example, record a correspondence between each data stream and an application layer protocol, a traffic trend of each application type, and the like, and may store the identification result in a database.
And the secondary data acquisition module can perform big data analysis based on the data recorded by the primary data acquisition module. For example, data traffic of different areas may be classified, and whether the area needs to adjust the bandwidth capacity is determined based on the data traffic of the area; data traffic of application types such as video application, voice application, game application and the like can be distributed to corresponding processing equipment, so that the data traffic can be accelerated to improve the service quality; the market characteristics of a certain area can be analyzed based on the area statistical result of the area; the percentage of each application type may be counted, for example, the percentage of an application type without a record number may be counted, and whether an application type is a malicious application type may be identified by whether the application type has a record number.
And the information display module can display the identification result of the DPI equipment and the statistical result of different databases on a page.
The data diversion module can divert different application types to different processing devices based on the diversion application configuration table.
A device configuration module, as shown in fig. 4, may include one or more of the following configuration units: the device comprises a domain name rule unit, a log configuration unit, an external plug-in unit, a record number unit, a protocol switch unit, a protocol merging unit, an access control unit, a packet filtering unit, a memory configuration unit, an interface configuration unit and the like. The configuration unit can flexibly configure according to the requirements of different application scenarios, and can enable the DPI device to take effect by sending commands such as loading or restarting and the like after configuration is completed, so that the DPI device can process data based on the configured device configuration module. In this way, by performing centralized management on each configuration unit in the device configuration module, the complexity of DPI device configuration can be reduced.
Specifically, the domain name rule unit may record and maintain a corresponding relationship between a domain name or an IP of an application type and the application type; the log configuration unit may configure different log files for each module of the DPI device, for example, configure a storage location, a storage space, whether to divide the log file, a log format, a log level, and the like of the log file; the external plug-in unit can expand the recognition rate of the DPI equipment; the record number unit can record and maintain record number information of a plurality of application types and can continuously maintain record number information of each domain name, so that the DPI equipment can identify malicious application types based on the record number information; the protocol switch unit can be used for starting to identify or not identify a certain protocol; a protocol merging unit that can merge some protocol identification situations into an identification situation for a certain application type; the access control unit may configure filtering rules for source IP, source port, destination IP, destination port, allowing or not allowing some data flows to pass through the DPI device; the packet filtering unit may capture a packet of a certain protocol passing through the DPI device, for example, a capturing rule is configured for a source IP, a destination IP, a source port, a destination port, and a protocol type of the DPI device, so as to capture a packet conforming to the rule; the memory configuration unit can pre-configure the memory space required by the DPI equipment to operate; the interface configuration unit may designate the DPI device to receive data streams from one interface or multiple interfaces, and may analyze traffic conditions of different interfaces of the DPI device, and meanwhile, the interface configuration unit may also identify existing packets, such as packets captured by packet capturing software.
In this embodiment, when a target data stream is received, a target application layer protocol adopted by the target data stream is determined; acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to the target application layer protocol; and distributing the target data stream to target processing equipment corresponding to the target application type. Therefore, the DPI equipment can not only detect and identify the data stream, but also distribute the data stream with the identified application type to the processing equipment for processing the application type, thereby fully exerting the processing capacity of each processing equipment and improving the processing efficiency of the data streams with different application types. Moreover, the DPI equipment can also display the regional statistical result in a management interface corresponding to the DPI equipment in the form of a picture, a table, characters and the like, so that the time and the energy required by manual sorting can be saved, and higher accuracy is achieved. Meanwhile, the module architecture of the DPI equipment provided by the embodiment of the invention can integrate various data processing functions, so that the DPI equipment can be flexibly adapted to various application scenes, and the management complexity and the management cost of the DPI equipment can be effectively reduced.
Based on the same technical concept, an embodiment of the present invention further provides a data processing apparatus applied to a DPI device, as shown in fig. 5, where the apparatus includes:
a determining module 501, configured to determine, when a target data stream is received, a target application layer protocol adopted by the target data stream;
an identifying module 502, configured to obtain a target packet feature of the target data stream, and identify, in combination with a preset identifying rule corresponding to the target application layer protocol, a target application type corresponding to the target packet feature;
a distributing module 503, configured to distribute the target data stream to a target processing device corresponding to the target application type.
Optionally, the apparatus further includes a determining module 504, configured to:
judging whether the target application layer protocol is an application layer protocol to be processed;
and if the target application layer protocol is the application layer protocol to be processed, acquiring the target message characteristics of the target data stream.
Optionally, the apparatus further comprises a statistics module 505, configured to:
recording the target application type as an application type of the target data stream, wherein the target application type is an identified application type or an unidentified application type;
and counting the proportion of the data streams of the identified application type or the unidentified application type in all the data streams according to the application types of all the recorded data streams.
Optionally, the apparatus further comprises a display module 506 for:
determining a target area to which the target data stream belongs according to the target message characteristics, and updating the target message characteristics to a target database corresponding to the target area;
and generating a regional statistical result of the target region based on the target database, and displaying the regional statistical result on a page.
Optionally, the apparatus further includes a generating module 507, configured to:
determining a target data channel to which the target data stream belongs according to the target message characteristics;
counting target flow information of the target data channel, and calculating estimated bandwidth capacity of the target data channel based on the target flow information;
and if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, generating an adjustment instruction for the target data channel.
In this embodiment, when a target data stream is received, a target application layer protocol adopted by the target data stream is determined; acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to the target application layer protocol; and distributing the target data stream to target processing equipment corresponding to the target application type. Therefore, the DPI equipment can not only detect and identify the data stream, but also distribute the data stream with the identified application type to the processing equipment for processing the application type, thereby fully exerting the processing capacity of each processing equipment and improving the processing efficiency of the data streams with different application types. Moreover, the DPI equipment can also display the regional statistical result in a management interface corresponding to the DPI equipment in the form of a picture, a table, characters and the like, so that the time and the energy required by manual sorting can be saved, and higher accuracy is achieved. Meanwhile, the module architecture of the DPI equipment provided by the embodiment of the invention can integrate various data processing functions, so that the DPI equipment can be flexibly adapted to various application scenes, and the management complexity and the management cost of the DPI equipment can be effectively reduced.
It should be noted that: in the data processing apparatus applied to the DPI device, when processing data, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data processing apparatus applied to the DPI device and the data processing method applied to the DPI device provided in the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
Fig. 6 is a schematic structural diagram of a network device according to an embodiment of the present invention. The network device 600 may vary significantly depending on configuration or performance, and may include one or more central processors 622 (e.g., one or more processors) and memory 632, one or more storage media 630 (e.g., one or more mass storage devices) storing applications 642 or data 644. Memory 632 and storage medium 630 may be, among other things, transient or persistent storage. The program stored on the storage medium 630 may include one or more modules (not shown), each of which may include a sequence of instructions operating on the network device. Still further, central processor 622 may be configured to communicate with storage medium 630 to perform a series of instruction operations in storage medium 630 on network device 600.
The network device 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input-output interfaces 658, one or more keyboards 656, and/or one or more operating systems 641, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A data processing method applied to a DPI device is characterized by comprising the following steps:
when a target data stream is received, determining a target application layer protocol adopted by the target data stream in a common port identification or characteristic word identification mode;
acquiring target message characteristics of the target data stream, and identifying a target application type corresponding to the target message characteristics by combining with a preset identification rule corresponding to the target application layer protocol;
distributing the target data stream to target processing equipment corresponding to the target application type;
after identifying the target application type corresponding to the target message feature, the method further includes:
recording the target application type as an application type of the target data stream, wherein the target application type is an identified application type or an unidentified application type;
and counting the proportion of the data streams of the identified application type or the unidentified application type in all the data streams according to the application types of all the recorded data streams.
2. The method of claim 1, wherein after determining a target application layer protocol adopted by the target data stream, further comprising:
judging whether the target application layer protocol is an application layer protocol to be processed;
and if the target application layer protocol is the application layer protocol to be processed, acquiring the target message characteristics of the target data stream.
3. The method of claim 1, further comprising:
determining a target area to which the target data stream belongs according to the target message characteristics, and updating the target message characteristics to a target database corresponding to the target area;
and generating a regional statistical result of the target region based on the target database, and displaying the regional statistical result on a page.
4. The method of claim 1, further comprising:
determining a target data channel to which the target data stream belongs according to the target message characteristics;
counting target flow information of the target data channel, and calculating estimated bandwidth capacity of the target data channel based on the target flow information;
and if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, generating an adjustment instruction for the target data channel.
5. A data processing apparatus for use in a DPI device, the apparatus comprising:
the determining module is used for determining a target application layer protocol adopted by a target data stream in a common port identification or characteristic word identification mode when the target data stream is received;
the identification module is used for acquiring the target message characteristics of the target data stream and identifying the target application type corresponding to the target message characteristics by combining with the preset identification rule corresponding to the target application layer protocol;
the distribution module is used for distributing the target data stream to target processing equipment corresponding to the target application type;
the statistical module is used for recording the target application type as the application type of the target data stream, wherein the target application type is an identified application type or an unidentified application type;
and counting the proportion of the data streams of the identified application type or the unidentified application type in all the data streams according to the application types of all the recorded data streams.
6. The apparatus of claim 5, further comprising a determining module configured to:
judging whether the target application layer protocol is an application layer protocol to be processed;
and if the target application layer protocol is the application layer protocol to be processed, acquiring the target message characteristics of the target data stream.
7. The apparatus of claim 5, further comprising a display module to:
determining a target area to which the target data stream belongs according to the target message characteristics, and updating the target message characteristics to a target database corresponding to the target area;
and generating a regional statistical result of the target region based on the target database, and displaying the regional statistical result on a page.
8. The apparatus of claim 5, further comprising a generation module configured to:
determining a target data channel to which the target data stream belongs according to the target message characteristics;
counting target flow information of the target data channel, and calculating estimated bandwidth capacity of the target data channel based on the target flow information;
and if the estimated bandwidth capacity is higher than the adjustment threshold of the target data channel, generating an adjustment instruction for the target data channel.
9. A network device comprising a processor and a memory, said memory having stored therein at least one instruction, at least one program, set of codes or set of instructions, which is loaded and executed by said processor to implement a data processing method as claimed in any of claims 1 to 4 applied to a DPI device.
10. A computer readable storage medium, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored, loaded and executed by a processor to implement the data processing method as claimed in any of claims 1 to 4, applied to a DPI device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810654859.2A CN108900374B (en) | 2018-06-22 | 2018-06-22 | Data processing method and device applied to DPI equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810654859.2A CN108900374B (en) | 2018-06-22 | 2018-06-22 | Data processing method and device applied to DPI equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108900374A CN108900374A (en) | 2018-11-27 |
| CN108900374B true CN108900374B (en) | 2021-05-25 |
Family
ID=64345538
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810654859.2A Active CN108900374B (en) | 2018-06-22 | 2018-06-22 | Data processing method and device applied to DPI equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108900374B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112187498B (en) * | 2019-07-03 | 2022-09-06 | 中国电信股份有限公司 | Bypass protection method, device and system thereof and Deep Packet Inspection (DPI) system |
| CN111163573A (en) * | 2019-12-27 | 2020-05-15 | 上海力申科学仪器有限公司 | System and method for intelligently identifying lamp holder of multichannel shadowless lamp |
| CN111786985B (en) * | 2020-06-28 | 2023-05-23 | 厦门市美亚柏科信息股份有限公司 | Method, device and storage medium for analyzing TCP and UDP data |
| CN111884876A (en) * | 2020-07-22 | 2020-11-03 | 杭州安恒信息技术股份有限公司 | Method, device, equipment and medium for detecting protocol type of network protocol |
| CN112073335B (en) * | 2020-09-03 | 2021-05-25 | 深圳市掌易文化传播有限公司 | Game data connection card pause processing system and method under big data support |
| CN112583832A (en) * | 2020-12-14 | 2021-03-30 | 北京鼎普科技股份有限公司 | DPI-based application layer protocol identification method and system |
| CN114900350B (en) * | 2022-04-29 | 2024-02-20 | 北京元数智联技术有限公司 | Message transmission method, device, equipment, storage medium and program product |
| CN114978734A (en) * | 2022-05-30 | 2022-08-30 | 新华三信息安全技术有限公司 | Message processing method and device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297270A (en) * | 2013-05-24 | 2013-09-11 | 华为技术有限公司 | Application type recognition method and network equipment |
| WO2014029094A1 (en) * | 2012-08-23 | 2014-02-27 | 华为技术有限公司 | Packet processing method, deep packet inspection requesting network element, and deep packet inspection device |
| CN105357083A (en) * | 2015-12-15 | 2016-02-24 | 福建星网锐捷网络有限公司 | Gateway flow adjusting method and system based on uncertain bandwidth detection |
| CN106027692A (en) * | 2016-05-16 | 2016-10-12 | 北京小米移动软件有限公司 | Information acquisition method and device and server |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101102277B (en) * | 2007-06-20 | 2010-04-14 | 华为技术有限公司 | Recognition control method and system for service data and recognition control device |
| US8358660B2 (en) * | 2009-11-16 | 2013-01-22 | Verizon Patent And Licensing Inc. | Method and system for providing integrated content delivery |
| CN101741744B (en) * | 2009-12-17 | 2011-12-14 | 东南大学 | Network flow identification method |
| CN102045363B (en) * | 2010-12-31 | 2013-10-09 | 华为数字技术（成都）有限公司 | Establishment, identification control method and device for network flow characteristic identification rule |
| US9042252B2 (en) * | 2012-11-13 | 2015-05-26 | Netronome Systems, Incorporated | Inter-packet interval prediction learning algorithm |
| CN103051725B (en) * | 2012-12-31 | 2015-07-29 | 华为技术有限公司 | Application and identification method, data digging method, Apparatus and system |
| CN104348677A (en) * | 2013-08-05 | 2015-02-11 | 华为技术有限公司 | Deep packet inspection method and equipment and coprocessor |
| CN103916294B (en) * | 2014-04-29 | 2018-05-04 | 华为技术有限公司 | The recognition methods of protocol type and device |
| CN104010139B (en) * | 2014-05-23 | 2017-02-22 | 杭州宽云视讯科技有限公司 | Method for achieving video stream seamless switching based on DPI packet inspection technology |
| CN105357071B (en) * | 2015-11-12 | 2018-08-31 | 成都科来软件有限公司 | A kind of network complexity method for recognizing flux and identifying system |
| CN107819646A (en) * | 2017-10-23 | 2018-03-20 | 国网冀北电力有限公司信息通信分公司 | A kind of net flow assorted system and method for distributed transmission |
-
2018
- 2018-06-22 CN CN201810654859.2A patent/CN108900374B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014029094A1 (en) * | 2012-08-23 | 2014-02-27 | 华为技术有限公司 | Packet processing method, deep packet inspection requesting network element, and deep packet inspection device |
| CN103297270A (en) * | 2013-05-24 | 2013-09-11 | 华为技术有限公司 | Application type recognition method and network equipment |
| CN105357083A (en) * | 2015-12-15 | 2016-02-24 | 福建星网锐捷网络有限公司 | Gateway flow adjusting method and system based on uncertain bandwidth detection |
| CN106027692A (en) * | 2016-05-16 | 2016-10-12 | 北京小米移动软件有限公司 | Information acquisition method and device and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108900374A (en) | 2018-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108900374B (en) | Data processing method and device applied to DPI equipment | |
| CN106815112B (en) | Massive data monitoring system and method based on deep packet inspection | |
| CN111752799B (en) | Service link tracking method, device, equipment and storage medium | |
| RU2567235C1 (en) | Radio resource optimisation method, apparatus and system dpi patent | |
| CN112350854B (en) | Flow fault positioning method, device, equipment and storage medium | |
| CN103546343B (en) | The network traffics methods of exhibiting of network traffic analysis system and system | |
| CN112434039A (en) | Data storage method, device, storage medium and electronic device | |
| EP3849136A1 (en) | Service quality monitoring method, apparatus, and system | |
| EP4096226A1 (en) | Fault detection method and apparatus for live broadcast service, electronic device, and readable storage medium | |
| CN106972985B (en) | Method for accelerating data processing and forwarding of DPI (deep packet inspection) equipment and DPI equipment | |
| CN113364804B (en) | Method and device for processing flow data | |
| CN111222547B (en) | Traffic feature extraction method and system for mobile application | |
| CN111258971A (en) | Application state monitoring alarm system and method based on access log | |
| CN111314179A (en) | Network quality detection method, device, equipment and storage medium | |
| CN112672381A (en) | Data association method, device, terminal equipment and medium | |
| CN113206797A (en) | Flow control method and device, electronic equipment and storage medium | |
| CN111177094A (en) | Log data processing method and device, electronic equipment and storage medium | |
| CN108322354B (en) | Method and device for identifying running-stealing flow account | |
| KR102069142B1 (en) | Apparatus and method for automatic extraction of accurate protocol specifications | |
| CN110677327A (en) | Chip-based real-time detection method for RTP flow fault | |
| CN115484047A (en) | Method, device, equipment and storage medium for identifying flooding attack in cloud platform | |
| US20240022507A1 (en) | Information flow recognition method, network chip, and network device | |
| CN113055493A (en) | Data packet processing method, device, system, scheduling device and storage medium | |
| CN110708209B (en) | Virtual machine flow acquisition method and device, electronic equipment and storage medium | |
| CN112688924A (en) | Network protocol analysis system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |