CN113722113A - Traffic statistic method and device - Google Patents
Traffic statistic method and device Download PDFInfo
- Publication number
- CN113722113A CN113722113A CN202111005509.1A CN202111005509A CN113722113A CN 113722113 A CN113722113 A CN 113722113A CN 202111005509 A CN202111005509 A CN 202111005509A CN 113722113 A CN113722113 A CN 113722113A
- Authority
- CN
- China
- Prior art keywords
- client address
- memory table
- flow data
- client
- access information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000007717 exclusion Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 11
- 238000009795 derivation Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000007619 statistical method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 235000019580 granularity Nutrition 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Environmental & Geological Engineering (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method and a device for flow statistics, and relates to the technical field of computers. One embodiment of the method comprises: acquiring access information of a client; the access information comprises a client address and current flow data; judging whether the client address exists in a memory table or not; if so, searching target flow data corresponding to the client address in a memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address. Except that the client address is inserted for the first time and the traffic data needs to be locked, no-lock operation is executed during updating and exporting of the traffic data, the influence of frequent updating and exporting operations of the data on the performance of the proxy server is avoided, and the consumption of system resources is reduced.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for flow statistics.
Background
As a proxy server device, the operation performance is a very important measure for measuring the proxy server device. During the operation period of the proxy server, the traffic usage of each user using the proxy server generally needs to be counted and stored, so that it is very important to accurately and quickly count and record the traffic in the face of a large number of clients and accessing information.
The existing traffic statistical method has high resource consumption, cannot meet the requirement of rapid traffic calculation in a multi-thread proxy server, and can affect the operation performance of the proxy server when traffic data is inserted and exported.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for traffic statistics, where lock-free operation is performed in a thread, and locking is performed only when access information is inserted for the first time, so as to avoid the influence on the performance of a proxy server caused by frequent modification and derivation of subsequent data, and reduce conflict processing of multiple threads in the proxy server on the same access information, thereby reducing system resource consumption and ensuring high performance characteristics of the proxy server.
To achieve the above object, according to a first aspect of the embodiments of the present invention, a method for traffic statistics is provided.
The method for flow statistics of the embodiment of the invention comprises the following steps: acquiring access information of a client; the access information comprises a client address and current flow data; judging whether the client address exists in a memory table or not; if yes, searching target flow data corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address.
Optionally, the accumulating the target traffic data according to the current traffic data includes: and acquiring the position of a pointer pointing to the target flow data in the memory table, and accumulating the number of sending bytes and the number of receiving bytes in the current flow data to the number of sending bytes and the number of receiving bytes in the target flow data by using atomic operation.
Optionally, the method further comprises: and exporting the client address in the memory table and the target flow data corresponding to the client address, and simultaneously returning the number of bytes sent and the number of bytes received in the target flow data in the memory table to zero.
Optionally, the deriving the client address in the memory table and the target traffic data corresponding to the client address includes: and simultaneously exporting the client address in the access information and the target traffic data corresponding to the client address according to a preset period.
Optionally, after the client address in the access information and the new target traffic data corresponding to the client address are derived simultaneously, the method further includes: and counting the flow use condition of the same client in a preset period according to the client address.
Optionally, the obtaining access information of the client includes: receiving an access request sent by the client; the access request comprises a client address; determining a connection handle corresponding to the client address; acquiring current flow data corresponding to the client address through the connection handle; the current traffic data includes: the sending byte number of the client and the receiving byte number of the client.
Optionally, after the current thread performs a locking operation on the mutex lock in the memory table, before inserting the access information into the memory table after the locking is successful, the method further includes: repeating the step of judging whether the client address exists in a memory table or not; if yes, searching a target flow corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, inserting the access information into the memory table after the locking is successfully executed.
Optionally, the memory structure includes one or more memory tables, and before the current thread performs a locking operation on the mutex lock in the memory table, the method further includes: and calculating the abstract value of the client address by using a Hash algorithm, and determining a target memory table corresponding to the access information according to the abstract value.
To achieve the above object, according to a second aspect of the embodiments of the present invention, there is provided an apparatus for traffic statistics.
The device for flow statistics of the embodiment of the invention comprises: the acquisition module is used for acquiring the access information of the client; the access information comprises a client address and current flow data;
the processing module is used for judging whether the client address exists in a memory table or not; if yes, then; searching target flow data corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address.
Optionally, the accumulating the target traffic data according to the current traffic data includes: and acquiring the position of a pointer pointing to the target flow data in the memory table, and accumulating the number of sending bytes and the number of receiving bytes in the current flow data to the number of sending bytes and the number of receiving bytes in the target flow data by using atomic operation.
Optionally, the apparatus further includes a derivation module, configured to derive a client address in the memory table and target traffic data corresponding to the client address, and return the number of bytes sent and the number of bytes received in the target traffic data in the memory table to zero at the same time.
Optionally, the deriving the client address in the memory table and the target traffic data corresponding to the client address includes: and simultaneously exporting the client address in the access information and the target traffic data corresponding to the client address according to a preset period.
Optionally, the deriving module is further configured to, after the client address in the access information and the new target traffic data corresponding to the client address are simultaneously derived, further include: and counting the flow use condition of the same client in a preset period according to the client address.
Optionally, the obtaining access information of the client includes: receiving an access request sent by the client; the access request comprises a client address; determining a connection handle corresponding to the client address; acquiring current flow data corresponding to the client address through the connection handle; the current traffic data includes: the sending byte number of the client and the receiving byte number of the client.
Optionally, the processing module is further configured to, after the current thread performs the locking operation, repeat the step of determining whether the client address exists in the memory table before the access information is inserted into the memory table after the locking is successful; if yes, searching a target flow corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, inserting the access information into the memory table after the locking is successfully executed.
Optionally, the memory structure includes one or more memory tables, and the processing module is further configured to calculate, by using a hash algorithm, a digest value of the client address before the current thread performs a locking operation on the mutex lock in the memory table, and determine, according to the digest value, a target memory table corresponding to the access information.
To achieve the above object, according to a third aspect of the embodiments of the present invention, an electronic device for traffic statistics is provided.
The traffic statistic device of the embodiment of the invention comprises: one or more processors; a storage system for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement the method for traffic statistics of an embodiment of the present invention.
To achieve the above object, according to a fourth aspect of embodiments of the present invention, there is provided a computer-readable medium.
The computer readable medium of the embodiment of the present invention stores thereon a computer program, which when executed by a processor implements the method of traffic statistics of the embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: by performing lock-free operation in the thread and locking only when the access information is inserted for the first time, the influence of frequent updating and exporting of subsequent data on the performance of the proxy server is avoided, and the conflict processing of multiple threads in the proxy server on the same access information is reduced, so that the resource consumption of a system is reduced, and the high-performance characteristic of the proxy server is ensured.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a method for traffic statistics according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a main flow of accumulating target traffic data according to current traffic data according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a main flow of acquiring access information of a client according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a main process of inserting access information into a target memory table according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the main blocks of an apparatus for traffic statistics according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 7 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
According to a first aspect of the embodiments of the present invention, a method for traffic statistics is provided.
Fig. 1 is a schematic diagram of a main flow of a method for traffic statistics according to an embodiment of the present invention. As shown in fig. 1, the method mainly includes:
step S101: acquiring access information of a client; the access information includes a client address and a current traffic.
Step S102: judging whether the client address exists in a memory table or not; if yes, go to step S103; otherwise, step S104 and step S105 are executed.
Step S103: searching target flow data corresponding to the client address in a memory table according to the client address, and accumulating the target flow data according to the current flow data;
step S104: executing locking operation on the mutual exclusion lock in the memory table in the current thread;
step S105: and after the locking is successful, inserting the access information into the memory table, and taking the current flow data as the target flow data corresponding to the client address.
In an alternative embodiment, the current traffic data may include the number of received bytes and the number of sent bytes of the current client address, and the target traffic data is the number of received bytes and the number of sent bytes corresponding to the client address that are stored in the memory table.
For the arrangement of memory tables, in an alternative embodiment, there may be one or more memory tables arranged in the global memory storage structure. Usually, the number of the memory tables is consistent with the number of the working threads of the proxy server. In the execution process, each thread can access the memory table, the number of the memory tables is consistent with the number of the working threads, and the number of the memory tables can be set as small as possible on the premise that the number of lock conflicts is as small as possible.
Accumulating the target traffic data according to the current traffic data, in an alternative embodiment, includes the steps shown in fig. 2:
step S201: acquiring a pointer position pointing to target flow data in a memory table;
step S202: and accumulating the number of sending bytes and the number of receiving bytes in the current flow data to the number of sending bytes and the number of receiving bytes in the target flow data by using atomic operation.
Because the atomic operation is an operation which cannot be interrupted by a thread scheduling mechanism, once the operation is started, the operation is run until the operation is ended, and therefore the accuracy in the flow accumulation process can be ensured.
In an optional embodiment, the method further comprises: and exporting the client address in the memory table and the target flow data corresponding to the client address, and simultaneously resetting the number of transmitted bytes and the number of received bytes in the target flow data in the memory table to zero.
In the actual application process, deriving the client address in the memory table and the target traffic data corresponding to the client address includes: traversing the client address in the memory table, acquiring the pointer position of the client address in the memory table for pointing to the flow data, and replacing the sending byte number and the receiving byte number in the target flow data with a zero value by using atomic operation, namely completing the derivation of the target flow data corresponding to the client address by replacing with the zero value.
In the actual application process, in order to better analyze the traffic data of the proxy server in the processing process, the traffic data needs to be exported according to a certain time interval (frequency), and export time, client address and traffic data need to be persistently stored for subsequent analysis and query.
Illustratively, the data can be exported to a local database of the client, or other storage points with a large amount of storage space, so as to facilitate statistics and calculation of subsequent data.
In an optional embodiment, deriving the client address and the target traffic data corresponding to the client address in the memory table includes: and simultaneously exporting the client address in the access information and the target flow data corresponding to the client address according to a preset period.
The preset period can be set by users according to user needs. For example, the preset period is set to 1 minute, the client address in the memory table and the traffic data corresponding to the client address are derived once every 1 minute, that is, the traffic usage of each client within 1 minute can be obtained in the database, and different derivation frequencies can be realized according to different preset periods, so that statistical analysis of different granularities can be performed.
In an optional embodiment, after the client address in the access information and the traffic data corresponding to the client address are derived simultaneously, the method further includes: and counting the flow use condition of the same client in a preset period according to the address of the client.
In an alternative embodiment, obtaining the access information of the client, as shown in fig. 3, includes:
step S301: receiving an access request sent by a client; the access request includes a client address;
step S302: determining a connection handle corresponding to the client address;
step S303: acquiring current flow data corresponding to the client address through the connection handle; the current traffic data includes: the sending byte number of the client and the receiving byte number of the client.
For the determination of the current traffic data, in an alternative embodiment, the current traffic data includes both the number of received bytes and the number of sent bytes corresponding to the client address.
Illustratively, when a client sends an access request to the proxy server, the proxy server accepts the access request and generates a connection handle that is specific to handling operations associated with the connection. When receiving an access request sent by a client to a proxy server, recording the number of sending bytes corresponding to the address of the client, and when a target server responds to the access request of the client, recording the number of receiving bytes corresponding to the address of the client. Therefore, the current flow data corresponding to the client address contains the number of sending bytes and the number of receiving bytes.
In an actual application process, each client address may receive multiple access requests repeatedly, so that multiple connection handles are generated, and access data of each group is recorded on the multiple connection handles corresponding to the client addresses, where each group of access data includes the number of bytes sent when the client sends an access request to the proxy server and the number of bytes received when the target server responds to the access request of the client, and each group of access data corresponds to a current flow corresponding to one client address.
In an alternative embodiment, after the locking operation is performed on the current thread, before the locked current thread is used to insert the access information into the memory table, the step of determining whether the client address exists in the memory table may be repeatedly performed, as shown in fig. 1, where the step includes: step S104': judging whether the client address exists in a memory table or not; if yes, the step S103 is skipped: searching target flow data corresponding to the client address in a memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, step S105 is performed: and after the locking is successful, inserting the access information into the memory table, and taking the current flow data as the target flow data corresponding to the client address.
In an alternative embodiment, the memory structure includes one or more memory tables, and before the current thread performs the locking operation on the mutex lock in the memory table, the method further includes:
step S401: calculating the abstract value of the client address by using a Hash algorithm;
step S402: and determining a target memory table corresponding to the access information according to the abstract value.
Because the character strings of the client addresses have different IP lengths, the client addresses need to be processed uniformly so as to be conveniently determined and stored in corresponding target memory tables, and different IP addresses can be uniformly distributed to different memory tables as far as possible.
In an optional embodiment, a corresponding relationship between the digest value and the target memory table may be preset, that is, after the digest value is generated, the target memory table is automatically stored corresponding to a certain target memory table.
In another alternative embodiment, a remainder operation may be performed on the client digest value and the memory table quantity value, and different target memory tables may be selected correspondingly according to values of the remainder.
In an alternative embodiment, the locking operation performed on the current thread is a locking operation that performs a mutually exclusive lock on the current thread. Only one mutex lock exists in each memory table, and the addition of the mutex lock can ensure that only one thread is inserted for processing when the client address is stored for the first time, so that interference caused by other threads is avoided. For example, after the first thread performs the locking operation, the second thread also wants to insert data at this time, and performs the locking operation in advance, but since the mutual exclusion lock is occupied by the first thread, the second thread cannot acquire the mutual exclusion lock, that is, is in a blocking state, until the first thread completes the data insertion, and releases the mutual exclusion lock, the second thread may successfully perform the locking operation and perform the subsequent insertion operation.
According to the traffic statistical method, the lock-free operation is carried out in the thread, the locking is carried out only when the access information is inserted for the first time, the influence on the performance of the proxy server caused by frequent modification and derivation of subsequent data is avoided, and the conflict processing of multiple threads in the proxy server on the same access information is reduced, so that the consumption of system resources is reduced, and the high-performance characteristic of the proxy server is ensured.
According to a second aspect of the embodiments of the present invention, an apparatus for traffic statistics applied to a server is provided.
Fig. 5 is a schematic diagram of the main blocks of an apparatus 500 for traffic statistics according to the second aspect of the embodiment of the present invention. As shown in fig. 5, includes:
an obtaining module 501, configured to obtain access information of a client; the access information comprises a client address and current flow data; a processing module 502, configured to determine whether a client address exists in a memory table; if so, searching target flow data corresponding to the client address in a memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address.
Optionally, accumulating the target traffic data according to the current traffic data includes: and acquiring the position of a pointer pointing to target flow data in a memory table, and accumulating the number of sending bytes and the number of receiving bytes in the current flow data to the number of sending bytes and the number of receiving bytes in the target flow data by using atomic operation.
Optionally, the apparatus further includes a derivation module, configured to derive a client address in the memory table and target flow data corresponding to the client address, and return the number of bytes sent and the number of bytes received in the target flow data in the memory table to zero at the same time.
Optionally, deriving the client address and the target traffic data corresponding to the client address in the memory table includes: and simultaneously exporting the client address in the access information and the target flow data corresponding to the client address according to a preset period.
Optionally, the deriving module is further configured to, after deriving the client address in the access information and the new target traffic data corresponding to the client address at the same time, further include: and counting the flow use condition of the same client in a preset period according to the address of the client.
Optionally, the obtaining access information of the client includes: receiving an access request sent by the client, wherein the access request comprises a client address; determining a connection handle corresponding to the client address; acquiring current flow data corresponding to the client address through the connection handle; the current traffic data includes: the sending byte number of the client and the receiving byte number of the client.
Optionally, the processing module 502 is further configured to, after the current thread performs the locking operation, repeat the step of determining whether the client address exists in the memory table before inserting the access information into the memory table after the locking is successful; if so, searching a target flow corresponding to the client address in a memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, inserting the access information into the memory table after the locking is successfully executed.
Optionally, the memory structure includes one or more memory tables, and the processing module 502 is further configured to calculate, by using a hash algorithm, a digest value of the client address before the access information is inserted into the memory table after the locking is successful, and determine, according to the digest value, a target memory table corresponding to the access information.
The flow counting device of the embodiment of the invention can ensure the lock-free operation in the storage process, avoid the influence of frequent updating and exporting of subsequent data on the performance of the proxy server, and reduce the conflict processing of multiple threads in the proxy server on the same access information, thereby reducing the resource consumption of a system and ensuring the high performance characteristic of the proxy server.
The device for flow statistics can execute the method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.
Fig. 6 shows an exemplary system architecture 600 of a traffic statistics method or traffic statistics apparatus to which embodiments of the invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 serves to provide a medium for communication links between the terminal devices 601, 602, 603 and the server 605. Network 604 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 601, 602, 603 to interact with the server 605 via the network 604 to receive or send messages or the like. For example, the terminal devices 601, 602, 603 may send an access request to the server 605 through the network 604, the server 605 may receive the access request through its access interface, and after the server processes the result of the access request, the result of the access request may be fed back to the terminals 601, 602, 603 through the access interface or other interfaces. Various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like, may be installed on the terminal devices 601, 602, and 603.
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 605 may be a server that provides various services, such as a background management server that supports shopping websites browsed by users using the terminal devices 601, 602, and 603. The background management server may analyze and otherwise process the received data such as the access request, and feed back a processing result (e.g., access content) to the terminal device.
It should be noted that the method for traffic statistics provided in the first aspect of the embodiment of the present invention is generally executed by the server 605, and accordingly, the apparatus for traffic statistics provided in the second aspect of the embodiment of the present invention is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks, and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, shown is a block diagram of a computer system 700 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 705 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 707 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 707 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an acquisition module, a processing module, and a derivation module. The names of these modules do not in some cases form a limitation on the modules themselves, and for example, the obtaining module may also be described as a "module for obtaining access information of a client".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
acquiring access information of a client; the access information comprises a client address and current flow data; judging whether the client address exists in a memory table or not; if so, searching target flow data corresponding to the client address in a memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address.
According to the method and the device for flow statistics, lock-free operation is carried out in the thread, locking is carried out only when the access information is inserted for the first time, the influence of frequent modification and derivation of subsequent data on the performance of the proxy server is avoided, and the conflict processing of multiple threads in the proxy server on the same access information is reduced, so that the consumption of system resources is reduced, and the high-performance characteristic of the proxy server is ensured.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (11)
1. A method of traffic statistics, the method comprising:
acquiring access information of a client; the access information comprises a client address and current flow data;
judging whether the client address exists in a memory table or not;
if yes, searching target flow data corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data;
otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address.
2. The method of claim 1, wherein accumulating the target traffic data based on the current traffic data comprises:
and acquiring the position of a pointer pointing to the target flow data in the memory table, and accumulating the number of sending bytes and the number of receiving bytes in the current flow data to the number of sending bytes and the number of receiving bytes in the target flow data by using atomic operation.
3. The method of claim 1, further comprising:
and exporting the client address in the memory table and the target flow data corresponding to the client address, and simultaneously returning the number of bytes sent and the number of bytes received in the target flow data in the memory table to zero.
4. The method of claim 3, wherein the deriving the client address and the target traffic data corresponding to the client address in the memory table comprises:
and simultaneously exporting the client address in the access information and the target traffic data corresponding to the client address according to a preset period.
5. The method of claim 4, further comprising, after deriving the client address in the access information and the new target traffic data corresponding to the client address at the same time, the method further comprising:
and counting the flow use condition of the same client in a preset period according to the client address.
6. The method of claim 1, wherein the obtaining access information of the client comprises:
receiving an access request sent by the client; the access request comprises a client address;
determining a connection handle corresponding to the client address;
acquiring current flow data corresponding to the client address through the connection handle; the current traffic data includes: the sending byte number of the client and the receiving byte number of the client.
7. The method as claimed in claim 1, wherein after the current thread performs the locking operation on the mutex lock in the memory table, before inserting the access information into the memory table after the locking is successful, further comprising:
repeating the step of judging whether the client address exists in a memory table or not; if yes, searching a target flow corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data; otherwise, inserting the access information into the memory table after the locking is successfully executed.
8. The method as claimed in claim 1, wherein the memory structure comprises one or more memory tables, and before the current thread performs the locking operation on the mutex lock in the memory table, the method further comprises:
and calculating the abstract value of the client address by using a Hash algorithm, and determining a target memory table corresponding to the access information according to the abstract value.
9. An apparatus for flow statistics, comprising:
the acquisition module is used for acquiring the access information of the client; the access information comprises a client address and current flow data;
the processing module is used for judging whether the client address exists in a memory table or not;
if yes, searching target flow data corresponding to the client address in the memory table according to the client address, and accumulating the target flow data according to the current flow data;
otherwise, executing locking operation on the mutual exclusion lock in the memory table by the current thread, inserting the access information into the memory table after the locking is successful, and taking the current flow data as the target flow data corresponding to the client address.
10. An electronic device for flow statistics, comprising: one or more processors;
a storage system for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-8.
11. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111005509.1A CN113722113B (en) | 2021-08-30 | 2021-08-30 | Flow statistics method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111005509.1A CN113722113B (en) | 2021-08-30 | 2021-08-30 | Flow statistics method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113722113A true CN113722113A (en) | 2021-11-30 |
| CN113722113B CN113722113B (en) | 2024-07-26 |
Family
ID=78679129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111005509.1A Active CN113722113B (en) | 2021-08-30 | 2021-08-30 | Flow statistics method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113722113B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150171A (en) * | 2022-06-30 | 2022-10-04 | 北京天融信网络安全技术有限公司 | Flow statistical method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007045144A1 (en) * | 2005-10-18 | 2007-04-26 | Huawei Technologies Co., Ltd. | Methods for peer-to-peer application message identifying and operating realization and their corresponding devices |
| CN101316193A (en) * | 2007-05-28 | 2008-12-03 | 北京师范大学珠海分校 | Multi-task scheduling method of measuring probe |
| CN103152281A (en) * | 2013-03-05 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Two-level switch-based load balanced scheduling method |
| CN108259207A (en) * | 2016-12-29 | 2018-07-06 | 北京国双科技有限公司 | Flow statistical method, client, server and system |
| CN113067808A (en) * | 2021-03-15 | 2021-07-02 | 上海哔哩哔哩科技有限公司 | Data processing method, live broadcast method, authentication server and live broadcast data server |
-
2021
- 2021-08-30 CN CN202111005509.1A patent/CN113722113B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007045144A1 (en) * | 2005-10-18 | 2007-04-26 | Huawei Technologies Co., Ltd. | Methods for peer-to-peer application message identifying and operating realization and their corresponding devices |
| CN101316193A (en) * | 2007-05-28 | 2008-12-03 | 北京师范大学珠海分校 | Multi-task scheduling method of measuring probe |
| CN103152281A (en) * | 2013-03-05 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Two-level switch-based load balanced scheduling method |
| CN108259207A (en) * | 2016-12-29 | 2018-07-06 | 北京国双科技有限公司 | Flow statistical method, client, server and system |
| CN113067808A (en) * | 2021-03-15 | 2021-07-02 | 上海哔哩哔哩科技有限公司 | Data processing method, live broadcast method, authentication server and live broadcast data server |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150171A (en) * | 2022-06-30 | 2022-10-04 | 北京天融信网络安全技术有限公司 | Flow statistical method and device, electronic equipment and storage medium |
| CN115150171B (en) * | 2022-06-30 | 2023-11-10 | 北京天融信网络安全技术有限公司 | Flow statistics method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113722113B (en) | 2024-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111858586B (en) | Data processing method and device | |
| CN110689268B (en) | Method and device for extracting indexes | |
| CN111125107A (en) | Data processing method, device, electronic equipment and medium | |
| CN110909022A (en) | Data query method and device | |
| CN112099982B (en) | Crash information positioning method and device, medium and electronic equipment | |
| CN111221869A (en) | Method and device for tracking database transaction time and analyzing database lock | |
| CN107291835B (en) | Search term recommendation method and device | |
| CN113722113B (en) | Flow statistics method and device | |
| CN112579428B (en) | Interface testing method, device, electronic equipment and storage medium | |
| CN113726885B (en) | Flow quota adjusting method and device | |
| CN113220705A (en) | Slow query identification method and device | |
| WO2023273576A1 (en) | Abnormal request processing method and apparatus, electronic device and storage medium | |
| CN112131257B (en) | Data query method and device | |
| CN114281504A (en) | Task scheduling method and device, electronic equipment and computer readable medium | |
| CN114064803A (en) | Data synchronization method and device | |
| CN112948334A (en) | Log processing method and device | |
| CN109656519B (en) | Method and device for automatically accessing service data | |
| CN113254325A (en) | Test case processing method and device | |
| CN113722193A (en) | Method and device for detecting page abnormity | |
| CN113347052A (en) | Method and device for counting user access data through access log | |
| CN113114611A (en) | Method and device for managing blacklist | |
| CN112579615A (en) | Method and device for realizing distributed lock | |
| CN113778909B (en) | Method and device for caching data | |
| CN113821519B (en) | Data processing method and domain drive design architecture | |
| CN114995764A (en) | Data storage method and device based on stream computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |