CN107704357B - Log generation method and device - Google Patents
Log generation method and device Download PDFInfo
- Publication number
- CN107704357B CN107704357B CN201710750744.9A CN201710750744A CN107704357B CN 107704357 B CN107704357 B CN 107704357B CN 201710750744 A CN201710750744 A CN 201710750744A CN 107704357 B CN107704357 B CN 107704357B
- Authority
- CN
- China
- Prior art keywords
- node
- log
- identifier
- target
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 92
- 230000008569 process Effects 0.000 claims abstract description 44
- 238000005215 recombination Methods 0.000 claims abstract description 32
- 230000006798 recombination Effects 0.000 claims abstract description 21
- 230000006870 function Effects 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims description 68
- 230000008521 reorganization Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 7
- 230000002159 abnormal effect Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012790 confirmation Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/875—Monitoring of systems including the internet
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
- Computer And Data Communications (AREA)
Abstract
The embodiment of the application discloses a log generation method and device. One embodiment of the method comprises: the method comprises the steps of collecting a node log generated by a node when a target request is processed, wherein the target request is processed by at least one node, different nodes support different functions, and the node log comprises the following steps: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node; and recombining the node logs of at least one node passed by the target request based on the identifier of the node, the identifier of the target request and the identifier of the adjacent node to generate a recombined log. The embodiment realizes the recombination of the node logs, thereby being convenient for positioning the problem in the recombined logs.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to the field of internet technologies, and in particular, to a log generation method and apparatus.
Background
In the field of internet technology, there are many service requests, such as a take-out order request, a network taxi calling request, etc., and a series of processes or operations for processing the service requests may form a service chain. When an abnormal condition occurs, it is difficult to locate which link in the service chain has a problem. Therefore, how to locate the problem in the service chain is a problem to be solved.
Disclosure of Invention
An object of the embodiments of the present application is to provide an improved log generating method and apparatus, so as to solve the technical problems mentioned in the above background section.
The embodiment of the application provides a1 and a log generation method, where the method includes: the method comprises the steps of collecting a node log generated by a node when a target request is processed, wherein the target request is processed by at least one node, different nodes support different functions, and the node log comprises the following steps: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node; and recombining the node logs of at least one node passed by the target request based on the identifier of the node, the identifier of the target request and the identifier of the adjacent node to generate a recombined log.
A2, the method as A1, the neighboring nodes including a previous node at a previous time point and a subsequent node at a later time point; and generating a reorganization log comprising: determining at least one node that handles the same target request; for each node in the determined at least one node, generating a sub-recombination log according to a node which is a successor of the node, the node and a successor of the node; and merging the generated sub-recombination logs to generate a recombination log.
A3, the method as described above with reference to a1 or a2, the method further comprising: in response to receiving a first query request including a target request identifier, a reorganization log corresponding to the received target request identifier is output.
A4, the method as A1 above, the node log includes a target terminal log; and the above method further comprises: receiving a target terminal log uploaded by a target terminal, wherein the target terminal log further comprises at least one of the following items: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
A5, the method as defined above in a4, said method further comprising: in response to receiving a second query request including query information, outputting a target terminal log corresponding to the query information, wherein the query information includes at least one of: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
A6, the method as in a1, the node log further comprising: processing states of the nodes, the processing states including a failure state; and after generating the reorganization log, the method further comprises: determining whether a node log in which the processing state of the node is a failure state exists in at least one node log forming the recombination log; if so, identifying the node log with the processing state of the node as the failure state in the recombined log in a preset presentation mode.
An embodiment of the present application provides B1, a log generating apparatus, where the apparatus includes: the acquisition unit is configured to acquire a node log generated by a node when the node processes a target request, wherein the target request is processed by at least one node, different nodes support different functions, and the node log includes: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node; and the recombination unit is configured to recombine the node logs of at least one node passed by the target request based on the identifier of the node, the identifier of the target request and the identifier of the adjacent node, and generate a recombined log.
B2, the apparatus as in B1, the neighboring nodes including a previous node at a previous time point and a subsequent node at a later time point; and a recombination unit comprising: a determining module configured to determine at least one node that handles the same target request; the generating module is configured to generate a sub-recombination log for each node in the determined at least one node according to a node which is a successor of the node, the node and a successor of the node; and the merging module is configured to merge the generated sub-recombination logs to generate a recombination log.
B3, the apparatus of any of the above aspects as defined in B1 or B2, further comprising: a first output unit configured to output a reassembly log corresponding to the received target request identifier in response to receiving a first query request including the target request identifier.
B4, the apparatus as in B1, the node log including the target terminal log; and the above apparatus further comprises: the receiving unit is configured to receive a target terminal log uploaded by a target terminal, wherein the target terminal log further includes at least one of the following: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
B5, the apparatus of B4, further comprising: a second output unit configured to output a target terminal log corresponding to query information in response to receiving a second query request including the query information, wherein the query information includes at least one of: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
B6, the apparatus as in B1, the node log further comprising: processing states of the nodes, the processing states including a failure state; and the above apparatus further comprises: a determining unit configured to determine whether a node log in which a processing state of a node is a failure state exists in at least one node log constituting a reassembly log; and the identification unit is configured to respond to the node log corresponding to the node with the failure state in at least one node log forming the recombination log, and identify the node log with the processing state of the node as the failure state in the recombination log in a preset presentation mode.
An embodiment of the present application provides C1, an electronic device, including: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method as described in any of implementations a1 through a 6.
The embodiment of the application provides a computer program product D1, a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of the implementations a1 to a 6.
According to the log generation method and device provided by the embodiment of the application, the node logs generated by the nodes in the process of processing the target request are collected, then, the node logs of at least one node passing through the target request are recombined based on the identifiers of the nodes, the target request identifiers and the identifiers of the adjacent nodes which process the same target request and are adjacent to the nodes in the node logs, so that the recombined logs are generated, the information in the node logs is effectively utilized, the node logs are recombined, and the problem of positioning in the recombined logs is solved conveniently.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a log generation method according to the present application;
FIG. 3 is a schematic diagram of an application scenario of a log generation method according to the present application;
FIG. 4 is a schematic block diagram of one embodiment of a log generation apparatus according to the present application;
FIG. 5 is a block diagram of a computer system suitable for use in implementing a server according to embodiments of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the log generation method and log generation apparatus of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 101, 102, 103 interact with a server 105 via a network 104 to receive or send messages or the like. Various communication client applications, such as a take-away application, a shopping application, an internet taxi service application, an instant messenger, a payment application, etc., may be installed on the terminal devices 101, 102, 103. For example, in a take-away order transaction, the terminal devices 101, 102, 103 may be user terminals, and a user may send a user order placing request to the server 105 by using the user terminals; the terminal devices 101, 102, and 103 may also be merchant terminals, and the merchant terminals may receive a user ordering request sent by the server 105; the terminal devices 101, 102, and 103 may also be distribution terminals, and a distributor may receive an order to be distributed by the server 105 through the distribution terminals; the terminal devices 101, 102, and 103 may also be query terminals, and the user may also query the reorganization log generated by the processing target request using the query terminals.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting information interaction, including but not limited to smartphones, smartwatches, tablets, e-book readers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for the reorganization log displayed on the terminal devices 101, 102, 103. The background server may analyze and perform other processing on the acquired data such as the node logs, and feed back a processing result (e.g., a recombined log) to the terminal device. For example, the background server may first collect a node log generated by a node when processing a target request, then recombine the node log of at least one node through which the target request passes based on an identifier of the node in the node log, a target request identifier, and an identifier of an adjacent node which processes the same target request and is adjacent to the node, generate a recombined log, and may send the recombined log corresponding to the queried target request identifier to the terminal device when a user queries on the terminal device using the target request identifier.
It should be noted that the log generation method provided in the embodiment of the present application is generally executed by the server 105, and accordingly, the log generation device is generally disposed in the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a log generation method according to the present application is shown. The log generation method comprises the following steps:
In this embodiment, an electronic device (e.g., a server shown in fig. 1) on which the log generation method operates may collect a node log generated by a node when processing a target request. A node may be a logical entity in a system that performs computational tasks, such as a process or a thread that performs some tasks, a process is a running activity of a program with independent functions with respect to a certain data set (e.g., a data set included in a target request), and is a basic unit for dynamic execution by an operating system, and a thread is a relatively independent and schedulable execution unit within a process; a node may also be a physical entity such as a terminal device, a server, etc., and a function, a process, or a thread running on the terminal device may be utilized to process a target request to implement a certain function. Different nodes can support different functions, for example, a merchant order taking node supports a merchant order taking function, and an operation activity executed by the merchant order taking node can be that a merchant terminal receives an order placing request submitted by a user, then, the merchant terminal presents an order taking or rejecting icon, after the merchant clicks the order taking icon, a system distributes a distributor for an order submitted by the user, and then, the order placing request is processed by the distributor distributing node. The target request may be various requests received, such as a take-out order request, a network taxi-calling request, a test entry request, and the like. The target request may be processed by at least one node, and a series of processes or operations for processing the target request by the node may form a business chain, such as a take-away order transaction process, a taxi-taking transaction process, and the like. As an example, after the user sends the takeaway order placing request, the takeaway order placing request may be processed by the user payment node, the merchant order taking node, the delivery personnel delivery node, the user confirmation receiving node, and the like, thereby forming a takeaway order transaction business chain.
In this embodiment, for each node of the at least one node processing the target request, the node log generated by the node when processing the target request may include: the electronic device may, for each type of request, sequence the nodes that process the type of request in order of processing the request, and thus may obtain the identifier of the node adjacent to the node according to the identifier of the node. The node log may further include: parameters of the target request, such as parameters of the take-out order placing request, may include merchant address, user address, expected delivery time, commodity name, quantity of commodities, remark information, and the like, as well as process data, such as merchant order taking time, distributor arrival time at the merchant, and the like.
In this embodiment, the electronic device may first perform embedding according to a predefined embedding point specification, and when receiving a request, may generate a log including content required to be generated in the embedding point specification, where the embedding point specification generally requires a target request identifier, an identifier of a current node, and an identifier of a node adjacent to the current node to be generated in a node log, and the embedding point specification may also require process data, upstream and downstream request parameters, and the like to be generated in the node log.
In this embodiment, after the node log is collected, the electronic device may store the node log. Specifically, the electronic device may first monitor whether a log is written into the log disk in real time through the log collection system, and when a log is written into the log disk, the log collection system may transmit the log to the distributed publish-subscribe message system in a form of a message, and then the computing engine may consume the message in the distributed publish-subscribe message system in real time. As an example, the log collection system may be a flash, which is a highly available, highly reliable, distributed system for collecting, aggregating and transmitting mass logs provided by Cloudera, where the flash supports various data senders customized in the log system for collecting data, and at the same time, the flash provides the capability of simply processing and writing data to various data receivers (customizable); the distributed publish-subscribe messaging system may be Kafka, which is a high throughput distributed publish-subscribe messaging system that can handle all the action stream data in a consumer-scale website; the computing engine may be Spark, which is a fast, general-purpose computing engine designed specifically for large-scale data processing. The electronic device may analyze the log in real-time or quasi-real-time (e.g., every minute), may process the log in a quasi-real-time manner using an offline script, may process a service scenario with a high real-time requirement in a real-time manner, and may process a service scenario with fewer problems and fewer nodes required for processing in a quasi-real-time manner.
In this embodiment, after the node logs are collected in step 201, the electronic device may recombine the node logs of at least one node through which the target request passes based on the identifier of the target request, the identifier of the node, and the identifier of the neighboring node, and generate a recombined log. Specifically, the electronic device may generate the reassembly logs by using a depth-first traversal algorithm. Depth-First traversal (Depth-First traversal) may also be referred to as Depth-First Search (Depth-First-Search), and a corresponding topological sorting table of the target graph may be generated using a Depth-First Search algorithm. Assume that the initial state of a given graph G is that all vertices have not been visited. Optionally, a vertex v is selected as the initial starting point (source point) in G, and depth-first traversal can be defined as follows: firstly, visiting a starting point v and marking the starting point v as visited; then, starting from v, each adjacent point w of v is searched. If w has not been visited, then the depth-first traversal continues with w as the new starting point until all vertices in the graph that are in path with the source point v (also referred to as vertices reachable from the source point) have been visited. If the graph still has un-visited vertices, the above process is repeated with another un-visited vertex as a new source point until all vertices in the graph have been visited.
In this embodiment, the process of recombining the node logs by the electronic device may include sequentially sorting the node logs according to a sorting rule, removing duplicate of repeated node logs after sorting, extracting key information of each node log as a title of the node log, sequentially arranging the titles, and presenting the node log corresponding to the clicked title when the title of the node log is clicked, where the electronic device may use a processing result obtained by performing the processing on the node log as the recombined log. The process of removing the duplicate is as follows: if the identifiers of the nodes included in at least two node logs are the same, one node log in the at least two node logs is reserved, and the node logs except the reserved node log are deleted.
In this embodiment, the node log may further include: the processing time point requested by the node processing target may be a time point when the processing is started, a time point when the processing is ended, or a log generation time.
As an example, the electronic device may first access a node B of the at least one node that processes the target request as an initial starting point, and mark the node B as visited; then, starting from the node B, searching the adjacent points of the node B, wherein the searched adjacent points are the node A and the node C, because the node A and the node C are not visited, respectively taking the node A and the node C as new starting points to continue depth-first traversal, taking the node A as a starting point to search the adjacent points of the node A as the node B, but the node B has been visited, taking the node C as a starting point to search the adjacent points of the node C as the node B and the node D, wherein the node B has been visited, the node D has not been visited, and the node D is marked as visited, at the moment, all nodes communicated with the path of the node B have been visited, and no unvisited nodes exist, and the visiting sequence of each node is the node B, the node A, the node C and the node D; finally, the electronic device may sort the node a, the node B, the node C, and the node D in the order from the first to the last of the processing time points.
In some optional implementations of this embodiment, the neighboring nodes of the node may include a previous node at a time point, or may include a subsequent node at a time point that is later than the time point, where the time point may be a time point at which the node processes the target request. The electronic device may first obtain at least one node corresponding to a node log containing the same target request identifier, thereby determining at least one node that processes the same target request; then, for each of the determined at least one node, the electronic device may generate a sub-recombination log according to an identifier of a node that is a successor of the node, the identifier of the node, and an identifier of a successor of the node. Specifically, the electronic device may sort the node log corresponding to the previous node of the node, the node log corresponding to the node, and the node log corresponding to the subsequent node of the node according to the sequence of the identifier of the previous node of the node, the identifier of the node, and the identifier of the subsequent node of the node, and may use a result of the sorting as a sub-reassembly log, which may also be referred to as a partial reassembly log. When the identifier of the previous node does not exist in the node log of the node, the node can be determined as the first node for processing the target request; when the identifier of the successor node does not exist in the node log of the node, the node may be determined to be the last node to process the target request. Finally, the electronic device may merge the generated sub recombination logs to generate a recombination log of the target request processed by the at least one node.
In some optional implementations of this embodiment, when receiving the first query request including the target request identifier, the electronic device may output a reassembly log corresponding to the received target request identifier. The first query request may be a query request sent by a user through a query terminal, and after the electronic device searches the reorganization log corresponding to the target request identifier, the electronic device may return the searched reorganization log to the query terminal so that the query terminal may display the reorganization log.
In some optional implementation manners of this embodiment, the node log may include a target terminal log, the target terminal may be a merchant terminal, and the merchant may receive order information of the user by using the merchant, perform corresponding processing on the order information, and the like; the target terminal can also be a distribution terminal, and a distributor (knight) can receive the order to be distributed by the system, upload the order processing state and the like by using the distribution terminal; the target terminal can also be a user terminal, and a user can browse article information, submit an order, execute order payment and the like by using the user terminal. When the target terminal is a merchant terminal, the target terminal log may be a log generated by the merchant order taking node when processing an order request. Due to the influence of a network environment, a terminal system version and other application processes on the application, some abnormal situations may occur in the order taking function provided by the take-away application for the merchant, for example, situations that the merchant is not reminded when a user order is received, analysis of the abnormal situations generally requires analysis of a merchant log, but the log of the terminal application is not easily obtained, and at this time, manual uploading by the user is required or automatic uploading by the merchant terminal is required when a wireless network (Wi-Fi) is used. The electronic device may receive a target terminal log uploaded by a target terminal, where the target terminal log may further include at least one of the following: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system. The terminal identifier may be an International Mobile Equipment Identity (IMEI), which may also be referred to as an International Mobile Equipment Identity, and each Mobile phone is assigned a group of numbers that is globally unique after being assembled; the user of the target terminal may be a merchant using the target terminal, and the identifier of the user of the target terminal may be an account name registered in a preset application by the merchant.
In some optional implementation manners of this embodiment, when receiving a second query request including query information, the electronic device may output a target terminal log corresponding to the query information, or may output a recombined log in which the target terminal log corresponding to the query information is located, where the query information may include at least one of the following: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system. The second query request may be a query request sent by a user using a query terminal, and after the electronic device searches a target terminal log corresponding to query information, the electronic device may return the searched target terminal log to the query terminal so that the query terminal can display the target terminal log. By way of example, when the user selects the query condition to be: when the terminal identifier is input and inquired by '123456789012345', the electronic equipment can output a target terminal log generated by a terminal with the terminal identifier of '123456789012345'; when the user selects the query conditions to be: when the user identifier of the target terminal is inputted with "18610260000" for inquiry, the electronic device may output a target terminal log generated by a terminal for order processing by a merchant whose identifier of the user of the target terminal is "18610260000"; when the user selects the query conditions to be: when the version number of the application and the version number of the terminal operating system are preset on the target terminal and the version number "3.2.1" of the application and the version number "iOS 10.3.2" of the terminal operating system preset on the target terminal are input for inquiry, the electronic device can output a target terminal log generated by the terminal with the version number "iOS 10.3.2" of the operating system and the version number "3.2.1" of the application of the preset application installed.
In some optional implementation manners of this embodiment, the node log may further include a processing status of the node, where the processing status may include a failure status and a success status. After generating at least one reorganization log, for each reorganization log, the electronic device may first determine at least one node log constituting the reorganization log; then, it may be determined whether a node log generated when the node whose processing state is the failure state processes the target request exists in the at least one node log, and if so, the node log whose processing state is the failure state may be identified in the reassembly log in a preset presentation manner. The electronic device may identify the node log in which the processing state of the node is the failure state by a preset color (e.g., red, yellow, etc.), or identify the node log in which the processing state of the node is the failure state by a processing manner such as bold, underline, slant, etc.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the log generation method according to the present embodiment. In the application scenario of fig. 3, the server 301 first collects node logs 302 generated by processing a user order placing request, where the user order placing request is processed by a user payment node (node a), a merchant order receiving node (node B), a distributor delivery node (node C), and a user confirmation receiving node (node D). The node B log 3021 of the merchant order taking node (node B) includes: identifier "B" of the node, identifier "1101" of the target request, node identifiers "A" and "C" of the neighboring nodes; the node D log 3022 of the user confirmation receiving node (node D) includes: an identifier "D" of a node, an identifier "1101" of a target request, a node identifier "C" of an adjacent node; the node a log 3023 of the user payment node (node a) includes: an identifier of a node "a", an identifier of a target request "1101", a node identifier of a neighboring node "B"; the node C log 3024 of the dispatcher delivery node (node C) includes: identifier "C" of the node, identifier "1101" of the target request, node identifiers "B" and "D" of the neighboring nodes. After that, the server 301 reassembles the node B log 3021, the node D log 3022, the node a log 3023, and the node C log 3024 based on the identifier 303 of the node, the identifier 304 of the target request, and the node identifier 305 of the adjacent node in the respective node logs, and generates a reassembly log 306 arranged in the order of the node a log, the node B log, the node C log, and the node D log.
The method provided by the embodiment of the application realizes the recombination of the node logs by collecting the node logs, recombining the node logs of at least one node for processing the target request based on the collected identifiers of the nodes in the node logs, the identifiers of the target request and the identifiers of the adjacent nodes, and generating the recombined logs, thereby facilitating the positioning problem in the recombined logs.
With further reference to fig. 4, as an implementation of the method shown in the above-mentioned figures, the present application provides an embodiment of a log generating apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which can be applied to various electronic devices.
As shown in fig. 4, the log generation apparatus 400 of the present embodiment includes: an acquisition unit 401 and a recombination unit 402. The acquiring unit 401 is configured to acquire a node log generated when a node processes a target request, where the target request is processed by at least one node, and different nodes support different functions, and the node log includes: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node; the restructuring unit 402 is configured to restructure the node log of at least one node through which the target request passes based on the identifier of the node, the identifier of the target request, and the identifier of the neighboring node, and generate a restructured log.
In this embodiment, the specific processing of the acquisition unit 401 and the reassembly unit 402 of the log generation apparatus 400 may refer to step 201 and step 202 in the corresponding embodiment of fig. 2.
In some optional implementations of this embodiment, the neighboring nodes of the node may include a previous node at a time point, or may include a subsequent node at a time point that is later than the time point, where the time point may be a time point at which the node processes the target request. The reorganizing unit 402 may include a determining module (not shown), a generating module (not shown), and a merging module (not shown). The determining module may first obtain at least one node corresponding to a node log including the same target request identifier, thereby determining at least one node that processes the same target request; then, for each of the determined at least one node, the generating module may generate the sub-recombination log according to an identifier of a node preceding the node, the identifier of the node, and an identifier of a node succeeding the node. Specifically, the generating module may sequence the node log corresponding to the previous node of the node, the node log corresponding to the node, and the node log corresponding to the subsequent node of the node according to the identifier of the previous node of the node, the identifier of the node, and the identifier of the subsequent node of the node, and may use a result of the sequencing as a sub-reassembly log, which may also be referred to as a partial reassembly log. When the identifier of the previous node does not exist in the node log of the node, the node can be determined as the first node for processing the target request; when the identifier of the successor node does not exist in the node log of the node, the node may be determined to be the last node to process the target request. Finally, the merging module may merge the generated sub-recombination logs to generate a recombination log of the target request processed by the at least one node.
In some optional implementations of this embodiment, the log generating apparatus 400 may further include a first output unit 403. When receiving a first query request including a target request identifier, the above-described first output unit 403 may output a reassembly log corresponding to the received target request identifier. The first query request may be a query request sent by a user using a query terminal, and after the first output unit 403 searches for a reassembly log corresponding to the target request identifier, the first query request may return the searched reassembly log to the query terminal so that the query terminal may display the reassembly log.
In some optional implementation manners of this embodiment, the node log may include a target terminal log, the target terminal may be a merchant terminal, and the merchant may receive order information of the user by using the merchant, perform corresponding processing on the order information, and the like; the target terminal can also be a distribution terminal, and a distributor (knight) can receive the order to be distributed by the system, upload the order processing state and the like by using the distribution terminal; the target terminal can also be a user terminal, and a user can browse article information, submit an order, execute a payment order and the like by using the user terminal. When the target terminal is a merchant terminal, the target terminal log may be a log generated by the merchant order taking node when processing an order request. Due to the influence of the network environment, the terminal system version, and other application processes on the application, some abnormal situations may occur in the order taking function provided by the take-away application for the merchant, for example, the situation that the merchant is not reminded when a user order is received, analysis of the abnormal situations generally requires analysis of the merchant log, but the log of the terminal application is not easily obtained, and at this time, the user needs to upload the log manually or the merchant terminal automatically uploads the log when the wireless network is used. The log generating apparatus 400 may further include a receiving unit 404, where the receiving unit 404 may receive a target terminal log uploaded by a target terminal, and the target terminal log may further include at least one of the following: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system. The terminal identifier can be an international mobile equipment identity code or an international mobile equipment identification code, and each mobile phone is endowed with a group of numbers which are unique globally after being assembled; the user of the target terminal may be a merchant using the target terminal, and the identifier of the user of the target terminal may be an account name registered in a preset application by the merchant.
In some optional implementation manners of this embodiment, the log generating apparatus 400 may further include a second output unit 405, and when a second query request including query information is received, the second output unit 405 may output a target terminal log corresponding to the query information, or may output a recomposed log in which the target terminal log corresponding to the query information is located, where the query information may include at least one of the following: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system. The second query request may be a query request sent by a user using a query terminal, and after the second output unit 405 searches for a target terminal log corresponding to query information, the second query request may return the searched target terminal log to the query terminal so that the query terminal can display the target terminal log.
In some optional implementation manners of this embodiment, the node log may further include a processing status of the node, where the processing status may include a failure status and a success status. The log generating apparatus 400 may further include a determining unit 406 and an identifying unit 407, and after generating at least one reorganization log, for each reorganization log, the determining unit 406 may first determine at least one node log constituting the reorganization log; then, the determining unit 406 may determine whether a node log generated when the node with the processing state being the failure state processes the target request exists in the at least one node log, and if so, the identifying unit 407 may identify the node log with the processing state being the failure state in the reassembly log in a preset presentation manner. The identification unit 407 may identify the node log in which the processing state of the node is the failure state by a preset color, or identify the node log in which the processing state of the node is the failure state by a processing manner such as bolding, underlining, or tilting.
Referring now to FIG. 5, a block diagram of a computer system 500 suitable for use as a server in implementing embodiments of the present invention is shown. The server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Liquid Crystal Display (LCD) and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the method of the present application when executed by the Central Processing Unit (CPU) 501. It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit and a recombination unit. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves. For example, a collection unit may also be described as a "unit that collects node logs generated by nodes when processing target requests".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: the method comprises the steps of collecting a node log generated by a node when a target request is processed, wherein the target request is processed by at least one node, different nodes support different functions, and the node log comprises the following steps: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node; and recombining the node logs of at least one node passed by the target request based on the identifier of the node, the identifier of the target request and the identifier of the adjacent node to generate a recombined log.
The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention according to the present invention is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the scope of the invention as defined by the appended claims. For example, the above features and (but not limited to) features having similar functions disclosed in the present invention are mutually replaced to form the technical solution.
Claims (12)
1. A method of log generation, the method comprising:
the method comprises the steps of collecting a node log generated by a node when a target request is processed, wherein the target request is processed by at least one node, different nodes support different functions, and the node log comprises the following steps: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node;
recombining node logs of at least one node passed by the target request based on the identifier of the node, the identifier of the target request and the identifier of the adjacent node to generate a recombined log;
wherein the adjacent nodes comprise a previous node with a previous time point and a subsequent node with a later time point; and
the generating of the reorganization log comprises:
determining at least one node that handles the same target request;
for each node in the determined at least one node, generating a sub-recombination log according to a node which is a successor of the node, the node and a successor of the node;
and merging the generated sub-recombination logs to generate a recombination log.
2. The method of claim 1, further comprising:
in response to receiving a first query request including a target request identifier, a reorganization log corresponding to the received target request identifier is output.
3. The method of claim 1, wherein the node log comprises a target terminal log; and
the method further comprises the following steps:
receiving a target terminal log uploaded by a target terminal, wherein the target terminal log further comprises at least one of the following items: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
4. The method of claim 3, further comprising:
in response to receiving a second query request including query information, outputting a target terminal log corresponding to the query information, wherein the query information includes at least one of: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
5. The method of claim 1, wherein the node log further comprises: a processing state of a node, the processing state comprising a failure state; and
after the generating the reorganization log, the method further comprises:
determining whether a node log in which the processing state of the node is a failure state exists in at least one node log forming the recombination log;
if so, identifying the node log with the processing state of the node as the failure state in the recombined log in a preset presentation mode.
6. An apparatus for log generation, the apparatus comprising:
the acquisition unit is configured to acquire a node log generated by a node when the node processes a target request, wherein the target request is processed by at least one node, different nodes support different functions, and the node log includes: an identifier of a node, a target request identifier, an identifier of a neighboring node that processes the same target request and is adjacent to the node;
the recombination unit is configured to recombine the node logs of at least one node through which the target request passes based on the identifier of the node, the identifier of the target request and the identifier of the adjacent node, and generate a recombined log;
wherein the adjacent nodes comprise a previous node with a previous time point and a subsequent node with a later time point; and
the recombination unit comprises:
a determining module configured to determine at least one node that handles the same target request;
the generating module is configured to generate a sub-recombination log for each node in the determined at least one node according to a node which is a successor of the node, the node and a successor of the node;
and the merging module is configured to merge the generated sub-recombination logs to generate a recombination log.
7. The apparatus of claim 6, further comprising:
a first output unit configured to output a reassembly log corresponding to the received target request identifier in response to receiving a first query request including the target request identifier.
8. The apparatus of claim 6, wherein the node log comprises a target terminal log; and
the device further comprises:
the receiving unit is configured to receive a target terminal log uploaded by a target terminal, wherein the target terminal log further includes at least one of the following: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
9. The apparatus of claim 8, further comprising:
a second output unit configured to output a target terminal log corresponding to query information in response to receiving a second query request including the query information, wherein the query information includes at least one of: the terminal identifier, the identifier of the user of the target terminal, the version number of the preset application on the target terminal and the version number of the terminal operating system.
10. The apparatus of claim 6, wherein the node log further comprises: a processing state of a node, the processing state comprising a failure state; and
the device further comprises:
a determining unit configured to determine whether a node log in which a processing state of a node is a failure state exists in at least one node log constituting a reassembly log;
and the identification unit is configured to respond to the node log corresponding to the node with the failure state in at least one node log forming the recombination log, and identify the node log with the processing state of the node as the failure state in the recombination log in a preset presentation mode.
11. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710750744.9A CN107704357B (en) | 2017-08-28 | 2017-08-28 | Log generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710750744.9A CN107704357B (en) | 2017-08-28 | 2017-08-28 | Log generation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107704357A CN107704357A (en) | 2018-02-16 |
| CN107704357B true CN107704357B (en) | 2020-12-15 |
Family
ID=61171230
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710750744.9A Expired - Fee Related CN107704357B (en) | 2017-08-28 | 2017-08-28 | Log generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107704357B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108572914B (en) * | 2018-03-14 | 2020-12-18 | 口碑(上海)信息技术有限公司 | Method and device for generating log for pressure test |
| CN109117345A (en) * | 2018-09-05 | 2019-01-01 | 深圳市木瓜移动科技有限公司 | Log monitoring method and distributed data system |
| CN110928853A (en) * | 2018-09-14 | 2020-03-27 | 北京京东尚科信息技术有限公司 | Method and device for identifying log |
| CN111402471A (en) * | 2020-03-05 | 2020-07-10 | 慧科(台州)智能系统有限公司 | Personnel processing mode determining method and device and data processing platform |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105635301A (en) * | 2016-01-14 | 2016-06-01 | 郑州悉知信息科技股份有限公司 | Access log merging method and log processing server and system |
| CN105761005A (en) * | 2016-02-23 | 2016-07-13 | 浪潮通用软件有限公司 | Method for tracking progress of business document |
| CN106250424B (en) * | 2016-07-22 | 2019-12-03 | 杭州朗和科技有限公司 | A kind of searching method, the apparatus and system of log context |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9959207B2 (en) * | 2015-06-25 | 2018-05-01 | Vmware, Inc. | Log-structured B-tree for handling random writes |
-
2017
- 2017-08-28 CN CN201710750744.9A patent/CN107704357B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105635301A (en) * | 2016-01-14 | 2016-06-01 | 郑州悉知信息科技股份有限公司 | Access log merging method and log processing server and system |
| CN105761005A (en) * | 2016-02-23 | 2016-07-13 | 浪潮通用软件有限公司 | Method for tracking progress of business document |
| CN106250424B (en) * | 2016-07-22 | 2019-12-03 | 杭州朗和科技有限公司 | A kind of searching method, the apparatus and system of log context |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107704357A (en) | 2018-02-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107704357B (en) | Log generation method and device | |
| US9256686B2 (en) | Using a bloom filter in a web analytics application | |
| CN111428131B (en) | Information pushing method, device and system | |
| CN110928853A (en) | Method and device for identifying log | |
| CN110020162B (en) | User identification method and device | |
| CN109146533B (en) | Information pushing method and device | |
| CN108932640B (en) | Method and device for processing orders | |
| CN110888985A (en) | Alarm information processing method and device, electronic equipment and storage medium | |
| CN110019214A (en) | The method and apparatus that data split result is verified | |
| CN110737655B (en) | Method and device for reporting data | |
| CN113360376A (en) | Buried point testing method and device | |
| CN112749323A (en) | Method and device for constructing user portrait | |
| CN113762674B (en) | Order distribution method and device | |
| CN108985805B (en) | Method and device for selectively executing push task | |
| CN110688295A (en) | Data testing method and device | |
| CN110362583A (en) | A kind of data processing method and device for multi-data source | |
| CN107679096B (en) | Method and device for sharing indexes among data marts | |
| CN113743697A (en) | Risk alarm method and device | |
| CN110110197B (en) | Information acquisition method and device | |
| CN112560938B (en) | Model training method and device and computer equipment | |
| EP3660763A1 (en) | Highly scalable event brokering and audit traceability system | |
| CN111861293B (en) | Method and device for transmitting information | |
| CN114693203A (en) | Information generation method and device, electronic equipment and computer readable medium | |
| CN112070391A (en) | Workshop production task management system, method and device | |
| CN112799797A (en) | Task management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 202, 2 / F, 1-3 / F, No. 11, Shangdi Information Road, Haidian District, Beijing 100084 Applicant after: Beijing Xingxuan Technology Co.,Ltd. Address before: 100085 Beijing, Haidian District on the road to the information on the ground floor of the 1 to the 3 floor of the 2 floor, room 11, 202 Applicant before: Beijing Xiaodu Information Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201215 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |