JP7514970B1 - Packet monitoring device, communication system, and program - Google Patents

Abstract

[Problem] To provide a packet monitoring device, communication system, and program for monitoring packet communications by adjusting the balance between accuracy of time stamp data integration and real-timeness by adjusting the survival time. [Solution] The packet monitoring device includes a communication unit that sequentially receives time stamp data including packet header information and time stamps from multiple agents, a distribution unit that performs distribution processing to distribute time stamp data for each packet ID generated from the header information, and a control unit that, based on the time stamp data distributed to the packet ID, passes the time stamp data distributed to the packet ID to post-processing when the survival time related to distribution to the packet ID has elapsed. [Selected Figure] Figure 4

Description

The present invention relates to a packet monitoring device, a communication system, and a program.

Conventionally, when managing the overall communication quality state of an IP network, a flow management unit extracts one packet out of N arriving packets from a packet stream that flows in one direction on a link between nodes such as routers, continues the process of reading, storing and managing the flow information for a certain period of time, and a quality estimation unit uses the flow information to estimate the communication quality of the flow passing through the link to be measured (Patent Document 1).

JP 2011-124750 A

Conventional technology assumes that the end-to-end flow of packets is monitored at a single fixed point, and does not take into account the integration of timestamp data from multiple monitoring points that a single packet passes through.

In addition, in conventional technology, in order to ensure real-time performance, one packet out of every N packets is sampled, reducing the amount of calculation required for monitoring. However, while this method can grasp the trends of network packets, it cannot obtain information from a microscopic perspective, such as momentary increases in delays or packet loss.

The present invention has been made to solve the above problems, and aims to provide a packet monitoring device, communication system, and program that can monitor packet communications by adjusting the balance between the accuracy of timestamp data integration and real-timeness.

To achieve the above object, the packet monitoring device according to the present invention includes a communication unit that sequentially receives timestamp data including packet header information and timestamps from a plurality of agents, a distribution unit that performs a distribution process that distributes timestamp data for each packet ID generated from the header information, and a control unit that, based on the timestamp data distributed for each packet ID, when the survival time for distribution to the packet ID has elapsed, passes the timestamp data distributed to the packet ID to post-processing.

According to this invention, the communication unit sequentially receives timestamp data including packet header information and timestamps from multiple agents. The distribution unit performs a distribution process to distribute the timestamp data for each packet ID generated from the header information.

Then, when the survival time for the allocation to the packet ID has elapsed based on the timestamp data allocated by the control unit to each packet ID, the timestamp data allocated to the packet ID is passed to post-processing.

In this way, a distribution process is performed to distribute the received timestamp data for each packet ID, and when the survival time for distribution to a packet ID has elapsed, the timestamp data distributed to that packet ID is handed over to post-processing. Therefore, by adjusting the survival time, it is possible to monitor packet communications by adjusting the balance between the accuracy of timestamp data integration and real-timeness.

In addition, in the packet monitoring device, as a first mode of the allocation process of the timestamp data, the control unit can transfer the timestamp data allocated to the packet ID to post-processing when the survival time has elapsed since the timestamp data was initially allocated to the packet ID.

In addition, in the packet monitoring device, as a second mode of the allocation process of timestamp data, the control unit rearranges the timestamps according to the size of the timestamps when the timestamp data is allocated to the packet ID based on the timestamp data allocated to the packet ID, and when the difference in the timestamps is equal to or greater than a predetermined threshold, the timestamp data with the oldest timestamp is passed to post-processing, and even if the difference in the timestamps is less than the threshold, the timestamp data allocated to the packet ID can be passed to post-processing when the survival time since the last allocation of timestamp data to the packet ID has elapsed.

In addition, in the packet monitoring device, the timestamp is the time at which the data packet passed through the agent.

The packet monitoring device further includes an input unit that accepts input from a user regarding the survival time, and the control unit can use the survival time according to the accepted input.

The communication system of the present invention also includes a plurality of agents that transmit and receive packets to and from other agents and sequentially transmit timestamp data, including header information and timestamps of the packets, to the packet monitoring device, and the packet monitoring device of the above invention.

The program of the present invention is also a program for causing a computer to function as each part of the packet monitoring device of the present invention.

According to the present invention, by adjusting the survival time, it is possible to monitor packet communications by adjusting the balance between the accuracy of timestamp data integration and real-timeness.

FIG. 2 is a diagram showing how data packets are transmitted and received between agents. 1 is a diagram illustrating an example of a communication system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of an agent according to an embodiment of the present invention. 1 is a block diagram showing a configuration of a packet monitoring device according to an embodiment of the present invention; 4 is a diagram for explaining a distribution process of the packet monitoring device according to the first embodiment; FIG. FIG. 13 is a diagram for explaining post-processing. FIG. 13 is a flowchart showing a time stamp recording processing routine of an agent according to an embodiment of the present invention. 11 is a flowchart showing a distribution processing routine of the packet monitoring device according to the embodiment of the present invention. 11 is a flowchart showing a time to live management processing routine of the packet monitoring device according to the embodiment of the present invention. FIG. 13 is a diagram for explaining a distribution process of a packet monitoring device according to a second embodiment. FIG.

The following describes in detail an embodiment of the present invention with reference to the drawings.

First, we will provide an overview of the embodiment of the present invention.

In an embodiment of the present invention, packet communications are monitored in real time by obtaining timestamps for packets and processing the information.

Here, more importance is being placed on stream data processing, which emphasizes real-time processing and keeps only the data necessary for the purpose from the data collected from the beginning, discarding unnecessary data on the spot as processing continues.

Recently, there has been an increasing emphasis on stream data processing, in which data that is collected with an emphasis on real-time processing is processed while retaining only the data required for the purpose and discarding unnecessary data on the spot.
In order to process various records related to packet communications between multiple devices in real time, the monitoring agent must detect target events that occur within the OS with high accuracy, record the time of detection (obtain a timestamp), and promptly send the data to a backend server where it is processed.The backend server that receives the data must quickly perform processes such as aggregation of related data and chronological sorting of related data, as the timestamp data arrives separately with no guaranteed order.

In an embodiment of the present invention, the backend server performs data merging and chronological sorting at high speed.

In addition, in an embodiment of the present invention, packet communication is monitored using passive delay measurement. Passive delay measurement records the transit time of all or some of the data packets of the actual service.

For example, suppose data packets (e.g., video packets) are being sent and received between agents A and B. When checking the state of the network between agents A and B, typically Ping packets or OWAMP/TWAMP packets are sent and received in addition to the data packets to measure communication and transmission delay time. This method of measuring delay is called active delay measurement, and is a measurement method that places emphasis primarily on checking the normality of the communications infrastructure.

On the other hand, passive latency measurement does not send packets specifically for measurement, but rather measures the transmission latency of all or a sample of the communication packets (data packets) of this service. This monitors not only the health of the infrastructure but also the communication of this service itself.

In addition, TCP/IP communication generally identifies the communication path between agents using a 5-tuple consisting of the TCP and UDP type, source IP address, destination IP address, source port number, and destination port number.

For example, as shown in Figure 1, the passage timestamp of packet n is recorded in three places, agent a, agent m, and agent b, and it is first necessary to link these as belonging to the same packet, i.e., to combine the data. The 5-tuple alone does not provide enough information for packet identification, so other information is added to identify each packet (packet ID). Specifically, in the case of UDP, the packet ID is made up of the 5-tuple, plus the Checksum field in the UDP header and the Identifier field in the IP header. In the case of TCP, the packet ID is made up of the 5-tuple, plus the Sequence Number field in the TCP header.

Therefore, each agent sends a set of packet ID and timestamp information to the backend server as timestamp data, and the backend server calculates the packet ID from the received timestamp data and combines packets with the same packet ID.

The above process allows multiple timestamps related to one packet to be combined into one, but what is done here is merely combining the separate timestamp data into one; it does not yet process the data into delay measurements or other data that has technical or business meaning. Such processing will be performed in post-processing.

[First embodiment]
<System Configuration of First Embodiment>
The configuration of a communication system according to a first embodiment will be described. As shown in Fig. 2, a communication system 100 according to an embodiment of the present invention includes a packet monitoring device 10, which is a back-end server, and a plurality of agents 20. The packet monitoring device 10 and the plurality of agents 20 are connected via a network N, such as a LAN (Local Area Network) such as Ethernet (registered trademark) or the Internet.

The multiple agents 20 can be configured as a computer including a CPU, RAM, and ROM that stores programs and various data for executing various processing routines described below. The agents 20 transmit and receive data, including transferring packets, between the agents 20.

As shown in FIG. 3, the agent 20 has the following functions: a communication unit 30 and a timestamp generation unit 32.

The communication unit 30 transmits and receives packets to and from other agents 20.

The timestamp generation unit 32 generates timestamp data including a packet ID including header information of the packet and a timestamp. Here, the timestamp is the time when the packet passes through the timestamp reference point of the agent 20.

The communication unit 30 sequentially transmits the generated timestamp data to the packet monitoring device 10.

The packet monitoring device 10 in this embodiment can be configured as a computer including a CPU, RAM, and a ROM that stores programs and various data for executing various processing routines described below. As shown in FIG. 4, this packet monitoring device 10 has, as functions, an input unit 38, a communication unit 40, a storage unit 42, a distribution unit 44, a control unit 46, and a processing unit 48.

The input unit 38 accepts input from the user regarding survival time. The input unit 39 accepts, for example, the numerical value of the survival time itself, or an increase or decrease from the default survival time. The communication unit 40 sequentially receives timestamp data from multiple agents 20.

The memory unit 42 has a front queue 42A and a back queue 42B.

The previous queue 42A stores the timestamp data received sequentially in the order in which it was received.

The allocated timestamp data output from the control unit 46 is stored in the post queue 42B in order.

The distribution unit 44 retrieves timestamp data from the previous queue 42A and performs a distribution process to distribute the timestamp data for each packet ID.

When the survival time for the allocation to the packet ID has elapsed based on the timestamp data assigned to the packet ID, the control unit 46 stores the timestamp data assigned to the packet ID in the post-queue 42B to be passed on to post-processing.

Specifically, when the survival time since the timestamp data was first assigned to a packet ID has elapsed, the control unit 46 stores the timestamp data assigned to that packet ID in the post queue 42B.

For example, as shown in FIG. 5, the sorting unit 44 stores the timestamp data Data(n)[1], ..., Data(n)[k] in a temporary data storage box (cache) associated with the key Key(n), which is a packet ID. The timestamp data continues to be stored for the survival time, and when the survival time has elapsed, the control unit 46 stores all the timestamp data in the storage box in the post-queue 42B and erases the storage box.

In this embodiment, the survival time is the time that has elapsed since a storage box corresponding to a certain key was created, and when the survival time has elapsed, the storage box disappears.

When the timestamp data is stored in the post-queue 42B, data processing is performed on the timestamp data. For example, data processing is performed to convert the data into a data type prepared for post-processing.

Figure 5 shows an example in which the survival time of the storage box corresponding to Key(m) has elapsed, and all the timestamp data Data(m)[1], ..., Data(m)[l] are stored in the post queue 42B, and the storage box is erased.

The processing unit 48 retrieves the allocated timestamp data from the post-queue 42B, and performs post-processing such as measuring delay on a packet-by-packet basis, measuring the Packet Delay Variance (PDV) between consecutive packets, performing basic statistical processing on delay and PDV, calculating the average, maximum, minimum, and variance per unit time, detecting packet loss in the communication being measured, detecting changes in the packet transmission path of the communication being measured, or detecting packet congestion in the communication being measured.

For example, if the assigned timestamp data for packet n is P(n), then all the timestamp information is contained within P(n), such as P(n).src (the timestamp for agent a in Figure 1 above) and P(n).dst (the timestamp for agent b in Figure 1 above).

In this case, the simple delay between agents a and b is
Delay(n)=P(n). dst-P(n). src
The PDV is the difference between Delay(n) and Delay(n-1). That is,
PDV(n)=Delay(n)−Delay(n−1)
As can be seen from the calculation of PDV and basic statistics per unit time, P(n) must be correctly arranged in chronological order when processing data.

We will not go into detail about packet loss detection, route change detection, and packet congestion detection, but it is possible to detect the occurrence of an event by comparing packet timestamp data, as shown in Figure 6.

Packet loss and route changes are detected based on the loss of timestamp information between agents a and b. Packet congestion can be detected from the narrowing of the timestamp difference in time series between agents a and b, and between agents m1 and m2. In any case, in the data processing process, the process of sorting the timestamp data in chronological order must be performed as quickly as possible, just like the sorting process.

Although shortening the survival time improves real-time performance, depending on the environment, there is a high possibility that all necessary timestamp data may not be available within the survival time, and data integration may not be performed correctly. For this reason, in an embodiment of the present invention, the survival time can be set or changed by the user. For example, the user takes into consideration the purpose of using the timestamp data and inputs information related to the desired survival time (e.g., the survival time itself, or an increase or decrease from the default survival time) to the packet monitoring device 10 via the input unit 38, and the survival time is determined based on the input.

<Operation of the First Embodiment>
Next, the process performed by each agent 20 according to the embodiment of the present invention will be described with reference to FIG.

When data packets are being transmitted and received between agents 20, each agent 20 executes the timestamp recording process routine shown in FIG. 7.

First, in step S100, it is determined whether or not a target data packet has passed through the communication unit 30 based on the monitoring conditions (hook conditions, for example, 5-tuple) of the monitored packet. For example, when the agent 20 itself sends a target data packet that meets the monitoring conditions, or when a target data packet that meets the monitoring conditions is received from another agent 20, the passage of the target data packet is detected, and the process proceeds to step S102.

In step S102, the timestamp generation unit 32 generates timestamp data including a packet ID including header information of the data packet and a timestamp. Here, the timestamp is the time when the data packet passes through the timestamp reference point of the agent 20.

In step S104, the communication unit 30 transmits the generated timestamp data to the packet monitoring device 10, and the processing routine ends.

Next, the processing performed by the packet monitoring device 10 according to an embodiment of the present invention will be described with reference to Figures 8 and 9.

When the packet monitoring device 10 receives timestamp data from each agent 20, it stores the received data packet in the pre-queue 42A of the storage unit 42. Then, the packet monitoring device 10 executes the distribution processing routine shown in FIG. 8.

First, in step S110, the allocator 44 determines whether timestamp data is stored in the previous queue 42A. If timestamp data is stored in the previous queue 42A, the process proceeds to step S112. On the other hand, if timestamp data is not stored in the previous queue 42A, the process routine is terminated.

In step S112, the distribution unit 44 retrieves the time stamp data from the previous queue 42A, performs a distribution process to distribute the time stamp data for each packet ID, and ends the processing routine. In the distribution process, the time stamp data is stored in a temporary data storage box (cache) linked to a key that is the packet ID.

The packet monitoring device 10 also executes the survival time management processing routine shown in Figure 9.

In step S120, it is determined whether there is a packet ID whose survival time has elapsed. If there is no packet ID whose survival time has elapsed, the processing routine ends. On the other hand, if there is a packet ID whose survival time has elapsed, the process proceeds to step S122.

In step S122, all timestamp data for the storage box of the corresponding packet ID is stored in the post queue 42B, the storage box is deleted, and the processing routine is terminated.

As described above, according to the communication system of the first embodiment, a distribution process is performed to distribute received timestamp data for each packet ID, and when the survival time related to the distribution to the packet ID has elapsed, the timestamp data distributed to the packet ID is handed over to post-processing. This makes it possible to monitor packet communication by adjusting the balance between the accuracy and real-time nature of the integration of timestamp data by adjusting the survival time.

[Second embodiment]
Next, a communication system according to a second embodiment will be described. Since the communication system according to the second embodiment has the same configuration as that of the first embodiment, the same reference numerals are used and the description thereof will be omitted.

In the second embodiment, the allocation process of timestamp data is different from that in the first embodiment.

The distribution unit 44 of the packet monitoring device 10 according to the second embodiment extracts timestamp data from the previous queue 42A and performs a distribution process to distribute the timestamp data for each packet ID.

When timestamp data is assigned to a packet ID, the control unit 46 sorts the data according to the size of the timestamp based on the timestamp data assigned to the packet ID, and when the difference in the timestamps exceeds a predetermined threshold (e.g., 50 ms), the control unit 46 stores the timestamp data with the oldest timestamp in the post-queue 42B to be passed on to post-processing.

The timestamp data stored in the later queue 42B is data that is determined to be no longer available as older timestamp data. The timestamp difference is the timestamp difference between the oldest and newest timestamp data in the storage box corresponding to a certain key.

For example, as shown in FIG. 10, the oldest timestamp data Data(m)[l] among the timestamp data stored in the temporary data storage box (cache) associated with the packet ID key Key(m) is stored in the post queue 42B, and the timestamp data Data(m)[l] is erased from the storage box.

When the survival time (e.g., 100 ms) since the last timestamp data was assigned to a packet ID has elapsed, the control unit 46 determines that no more timestamp data for this packet ID will arrive, and stores the timestamp data assigned to the packet ID in the post-queue 42B to be handed over to post-processing.

For example, as shown in FIG. 10 above, timestamp data Data(n)[1], ..., Data(n)[k] is stored in a temporary data storage box (cache) linked to the packet ID key Key(n) while sorting the timestamps in descending order. The timestamp data continues to be stored for the survival time, and when the survival time has elapsed, all the timestamp data in the storage box is stored in the post-queue 42B and the storage box is erased.

In this embodiment, the survival time is the time from when a change (addition or deletion of data) occurs in a storage box corresponding to a certain key until the storage box disappears without any changes.

Depending on the calculation to be performed in the post-processing, the rule for determining whether the "difference in timestamp" is equal to or greater than the threshold value may be changed. For example, in the case of PDV calculation, the difference in timestamp from the second smallest timestamp may be calculated as the "difference in timestamp".

Specifically, let R1, ..., Ri-1 be the time stamp data in the storage box, and let us assume that the time stamps are sorted in descending order in this order. Then,
Time stamp (R1) - time stamp (Ri-1)>50 [ms]
When this occurs, the "difference in timestamps" between both the timestamp data Ri and Ri-1 is equal to or greater than the threshold. In this case, the two timestamp data Ri and Ri-1 can be considered to be in a definite chronological order, so PDV(i) is calculated using these two pieces of data. Then, only the timestamp data Ri is removed from the storage box.

Note that other configurations and operations of the communication system according to the second embodiment are similar to those of the communication system according to the first embodiment, and therefore will not be described.

As described above, according to the communication system of the second embodiment, a distribution process is performed to distribute received timestamp data for each packet ID, and when the difference in timestamps of the distributed timestamp data exceeds a predetermined threshold, the timestamp data with the oldest timestamp is passed to post-processing, and when the survival time related to the distribution to the packet ID has elapsed, the timestamp data distributed to the packet ID is passed to post-processing. In this way, by adjusting the survival time, it is possible to adjust the balance between the accuracy of the integration of timestamp data and real-timeness, and to monitor packet communication.

The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit of the invention.

10 Packet monitoring device 20 Agent 30 Communication unit 32 Time stamp generation unit 38 Input unit 40 Communication unit 42 Storage unit 42A Pre-queue 42B Post-queue 44 Distribution unit 46 Control unit 48 Processing unit 100 Communication system

Claims (7)

a communication unit that sequentially receives time stamp data including header information and time stamps of packets from a plurality of agents;
a distribution unit that performs a distribution process for distributing time stamp data for each packet ID generated from the header information;
a control unit that transfers the time stamp data assigned to the packet ID to a post-processing when a survival time related to the assignment to the packet ID has elapsed based on the time stamp data assigned to each packet ID;
A packet monitoring device comprising: The packet monitoring device according to claim 1, wherein the control unit passes the timestamp data assigned to the packet ID to post-processing when the survival time has elapsed since the timestamp data was first assigned to the packet ID. The control unit, when the time stamp data is assigned to the packet ID, rearranges the time stamps according to the size of the time stamps based on the time stamp data assigned to the packet ID, and when a difference in time stamps between the oldest time stamp data and the newest time stamp data among the time stamp data assigned to the packet ID becomes equal to or greater than a predetermined threshold, passes the oldest time stamp data to post-processing, and
A packet monitoring device as described in claim 1, wherein even if the difference in timestamps is less than the threshold, if the survival time since the last timestamp data was assigned to the packet ID has elapsed, the timestamp data assigned to the packet ID is handed over to post-processing. The packet monitoring device according to claim 1, wherein the timestamp is the time at which the data packet passed through the agent. an input unit for receiving an input regarding the survival time from a user,
The packet monitoring device according to claim 1 , wherein the control unit uses the time to live according to the received input. a plurality of agents that transmit and receive packets to and from other agents and that sequentially transmit timestamp data including header information and timestamps of the packets to a packet monitoring device;
A packet monitoring device according to any one of claims 1 to 5,
A communication system comprising: A program for causing a computer to function as each part of a packet monitoring device according to any one of claims 1 to 5.