CN106878064B

CN106878064B - Data monitoring method and device

Info

Publication number: CN106878064B
Application number: CN201710032512.XA
Authority: CN
Inventors: 徐慧; 杨光; 邹辉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2017-01-16
Filing date: 2017-01-16
Publication date: 2021-08-13
Anticipated expiration: 2037-01-16
Also published as: CN106878064A

Abstract

The invention discloses a data monitoring method and a data monitoring device. Wherein, the method comprises the following steps: receiving monitoring data reported by an account; acquiring monitoring parameters of the account configuration, wherein the monitoring parameters at least comprise: monitoring index information and alarm triggering conditions; counting the monitoring data according to the monitoring index information to obtain a counting result; detecting whether the statistical result meets the alarm triggering condition or not; and if the statistical result is detected to accord with the alarm triggering condition, generating alarm information. The invention solves the technical problem of data monitoring index fixation.

Description

Data monitoring method and device

Technical Field

The invention relates to the field of information processing, in particular to a data monitoring method and device.

Background

In the current scheme of monitoring basic data through a cloud host, some common indexes such as cpu usage, memory usage, and disk read-write rate are fixedly collected, so that a user can view a monitoring chart, configure an alarm, and pull monitoring data through a cloud Application Programming Interface (API) in a console.

The existing basic monitoring adopts an Agent end to actively collect and report to a data processing cluster at regular time, processed data is stored in a database DB, a foreground displays monitoring data pulled from the database DB, the data is obtained through a cloud monitoring application programming interface API, and an alarm notice is sent.

As shown in fig. 1, the existing architecture includes: collecting agent: deployed at a mother machine (agent) and a child machine side (baradAgent), executing a preset acquisition logic to acquire a specified index value and reporting barad access; barad access end: receiving data from an acquisition agent and storing the data in a distributed message queue; barad data processing cluster: and pulling data from the message queue and merging and counting the source data according to a preset rule.

The user can configure threshold value alarm of specific index on the front page and specify an alarm receiver, and the alarm triggering module can judge the summary result of the real-time data according to the configuration of the user and then send the alarm.

By the scheme, the reporting Agent only can collect fixed indexes (such as some common indexes) and cannot report other indexes (such as indexes which are not common with other services) related to the user-specified service; in the scheme, through agent end reporting, a user cannot perceive whether reporting is really successful or not, namely cannot know the result of each reporting immediately; in addition, for the alarm part of the basic monitoring, all regions share the same set of alarm configuration, and the situation that a user sets different alarm strategies for objects in different regions cannot be met; and the alarm strategy of basic monitoring, only one alarm strategy can be applied to the same object with the same index, and more flexible strategy configuration cannot be realized.

Aiming at the problem of fixed data monitoring indexes, an effective solution is not provided at present.

Disclosure of Invention

The embodiment of the invention provides a data monitoring method and a data monitoring device, which at least solve the technical problem of data monitoring index fixation.

According to an aspect of an embodiment of the present invention, there is provided a data monitoring method, including: receiving monitoring data reported by an account; acquiring monitoring parameters of the account configuration, wherein the monitoring parameters at least comprise: monitoring index information and alarm triggering conditions; counting the monitoring data according to the monitoring index information to obtain a counting result; detecting whether the statistical result meets the alarm triggering condition or not; and if the statistical result is detected to accord with the alarm triggering condition, generating alarm information.

According to another aspect of the embodiments of the present invention, there is also provided a data monitoring apparatus, including: the first receiving unit is used for receiving monitoring data reported by the account; a first obtaining unit, configured to obtain monitoring parameters of the account configuration, where the monitoring parameters at least include: monitoring index information and alarm triggering conditions; the first statistical unit is used for counting the monitoring data according to the monitoring index information to obtain a statistical result; the first detection unit is used for detecting whether the statistical result meets the alarm triggering condition or not; and the generating unit is used for generating alarm information when the statistical result is detected to accord with the alarm triggering condition.

In the embodiment of the invention, monitoring data are counted through monitoring index information configured for the account, rather than analyzing the data by adopting fixed indexes, so that different indexes can be counted for different accounts, and different counting results can be obtained for different accounts; furthermore, the statistical result is detected by using the alarm triggering condition configured by the account, different alarm triggering conditions can be used for different accounts instead of the same alarm triggering condition for all accounts, the monitoring service can be provided for the user in a targeted manner, the problem of fixed index of data monitoring in the prior art is solved, the data monitoring is performed based on the index configured by the user, and the effect of providing customized monitoring service for the user is achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a schematic diagram of a prior art monitoring architecture;

FIG. 2 is a schematic diagram of a hardware environment for a data monitoring method according to an embodiment of the invention;

FIG. 3 is a schematic diagram of a hardware environment for another data monitoring method according to an embodiment of the invention;

FIG. 4 is a flow diagram of an alternative data monitoring method according to an embodiment of the present invention;

FIG. 5 is a schematic view of an alternative configuration interface for monitoring parameters, according to an embodiment of the invention;

FIG. 6 is a schematic diagram of an interface for displaying an alternative monitoring result according to an embodiment of the present invention;

FIG. 7 is an architecture diagram of a hardware environment for another data monitoring method according to an embodiment of the invention;

FIG. 8 is an architecture diagram of a hardware environment for yet another data monitoring method according to an embodiment of the present invention;

FIG. 9 is an architectural diagram of a hardware environment for yet another data monitoring method according to an embodiment of the invention;

FIG. 10 is a schematic diagram of an alternative data monitoring apparatus according to an embodiment of the present invention; and

fig. 11 is a block diagram of a terminal according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, partial terms or terms appearing in the description of the embodiments of the present invention are applied to the following explanations:

namespace, is a container for the index.

An index, is a monitored variable, and data points represent the value of the variable over time.

The dimension is a structure for uniquely identifying the monitored object, and the dimension determines a dimension value to be a specific monitored object.

And the monitoring object is a specific object of the index and is also a specific value of the dimension.

And dimension aggregation, wherein data statistics operation is performed according to the dimension configured by the user. The original dimension is the dimension initially specified by a user, multiple dimensions are specified to be combined into a polymerization dimension on the basis of the original dimension, and data can be counted according to the polymerization dimension configured by the user only by reporting the data once.

The statistical method refers to a set of statistical methods and statistical periods.

The statistical method refers to a method for calculating data, such as calculating a maximum value, calculating a minimum value, calculating an average value, and summing.

The statistical period refers to the time for calculating the data, and alternatively, the statistical period can be 5 minutes.

The alarm is a function in the monitoring alarm service provided for the user, and the function is used for alarming the abnormal condition of the index and providing alarm information viewing, alarm self-defined threshold and alarm subscription. Optionally, the data is checked at intervals of a plurality of time intervals according to a user-defined threshold, and if an alarm triggering condition is reached, an alarm notification is initiated.

The alarm rules may include general alarm rules and object alarm rules. The general alarm rule is configured on the index, and the alarm rule is used for all monitored objects under the same namespace and dimension under the index; the object alarm rule is an alarm rule configured on a specific monitoring object and used for the object.

The agent is deployed on the master machine and used for collecting and reporting common monitoring indexes.

The slave agent is deployed on the slave machine and used for collecting and reporting common monitoring indexes.

custom monitoring sub-machine data processing cluster.

kafka, a distributed message queue for exchanging data between the data processing layer and the access stratum of a barad cluster.

According to an embodiment of the present invention, an embodiment of a method for data monitoring is provided. Alternatively, in the present embodiment, the data monitoring method described above may be applied to a hardware environment formed by the server 102 and the terminal 104 as shown in fig. 2. As shown in fig. 2, server 102 is connected to terminals 104 via a network, including but not limited to: the terminal 104 is not limited to a PC, a mobile phone, a tablet computer, etc. in a wide area network, a metropolitan area network, or a local area network. The data monitoring method of the embodiment of the present invention may be executed by the server 102, the terminal 104, or both the server 102 and the terminal 104. The data monitoring method of the embodiment of the present invention executed by the terminal 104 may also be executed by a client installed thereon.

According to the embodiment of the application, a user configures monitoring parameters (namely monitoring basic information) on a cloud console, optionally, the monitoring parameters can comprise monitoring index parameters and alarm triggering conditions, wherein the monitoring index parameters can comprise a namespace, a dimension, an index, a polymerization dimension and a statistical mode, the alarm triggering conditions can be alarm configuration, the user calls an Application Programming Interface (API) of a cloud server after completing the configuration of the monitoring basic information, the configured monitoring basic information is reported according to a fixed format, and the cloud server receives the monitoring basic information configured by the user and stores the monitoring basic information in a configuration library.

The user can upload the monitoring data through the data reporting interface (the interface can also be an API interface), the cloud server stores the monitoring data in the database according to the monitoring index information after receiving the monitoring data, and can also store the statistical result in the database after performing statistical analysis on the monitoring data reported through the data reporting interface based on the statistical mode indicated by the monitoring basic information.

Further, the cloud server may present the stored data on a monitoring console, which may be a cloud server webpage for the user.

In an optional embodiment, the cloud server may perform anomaly analysis on the statistical result, for example, detect whether the statistical result meets an alarm trigger condition, and generate alarm information if the statistical result meets the alarm trigger condition.

As shown in fig. 3, a user may log in an account on the terminal 30, and after logging in the account, configure and report monitoring parameters (e.g., monitoring items) on a web page provided by the cloud server, the proxy server 31 reports the monitoring items configured by the user, and the HTTP server 33 reports other HTTP requests. Receiving a monitoring item and monitoring data reported by a user by a reported data receiving cluster 34, caching the monitoring item and the monitoring data received by the reported data receiving cluster 34 by a cached reported data cluster 35, counting the cached data by a processing cluster 36 based on the cached data to obtain statistics, on one hand, storing the processing result by a storage cluster 37, and providing a display cluster 38 for providing page chart display information corresponding to the stored processing result and providing an API (application programming interface) for a terminal or other clusters to pull data; on the other hand, the alarm processing cluster 39 determines whether the alarm triggering condition is satisfied based on the processing result, and may perform operations of basic alarm, custom monitoring alarm, custom message alarm, and automatic setting of a scaling policy in the case that the alarm triggering condition is satisfied. The automatic setting of the Scaling policy may be an Auto Scaling operation, that is, an operation of automatically creating or deleting a virtual machine in a flexible manner, and the policy may be a policy that is customized by a user or implemented according to a plan or based on a system health status check result.

Based on the network architectures provided in fig. 2 and fig. 3, the present application provides a data monitoring method, optionally, fig. 4 is a flowchart of a data monitoring method according to an embodiment of the present invention, as shown in fig. 4, where the flowchart includes the following steps:

step S402: receiving monitoring data reported by an account;

step S404: acquiring monitoring parameters of account configuration, wherein the monitoring parameters at least comprise: monitoring index information and alarm triggering conditions;

step S406: counting the monitoring data according to the monitoring index information to obtain a counting result;

step S408: detecting whether the statistical result meets an alarm triggering condition or not;

step S410: and if the statistical result is detected to accord with the alarm triggering condition, generating alarm information.

Through the embodiment, a user can log in the cloud service platform, after the user logs in, the logged account number is used for reporting monitoring data, the server obtains monitoring parameters configured by the account after receiving the monitoring data reported by the account, statistics is carried out on the monitoring data based on monitoring index information in the monitoring parameters to obtain statistical results, after the statistical results are obtained, whether the statistical results meet alarm triggering conditions or not is detected, and if the statistical results meet the alarm triggering conditions is detected, alarm information is generated. In the embodiment, the monitoring data is counted through the monitoring index information configured for the account, rather than analyzing the data by adopting a fixed index, so that different indexes can be counted for different accounts, and different counting results can be obtained for different accounts; furthermore, the statistical result is detected by using the alarm triggering condition configured by the account, different alarm triggering conditions can be used for different accounts instead of the same alarm triggering condition for all accounts, the monitoring service can be provided for the user in a targeted manner, the problem of fixed index of data monitoring in the prior art is solved, the data monitoring is performed based on the index configured by the user, and the effect of providing customized monitoring service for the user is achieved.

In the technical scheme provided in step S402, a user may enter a cloud platform page on a terminal, optionally, the user may log in the cloud platform page using a verified account, and after logging in the cloud platform page, enter a custom monitoring page through the cloud platform page, where information of one or more application editing interfaces may be recorded in the custom monitoring page, and a corresponding application editing interface may be called through the information, where the information may be a key, and after a predetermined operation (e.g., a click operation) is performed on the key, the cloud server detects the predetermined operation, triggers an event that calls the corresponding interface, and may report monitoring data that needs to be monitored by calling the corresponding interface.

Optionally, the account may directly upload the monitoring data as an attachment to the server, or after writing the code, input the written code in a document entry of a custom monitoring page, and report the code, and the server or the terminal may report the corresponding monitoring data by running the code.

It should be noted that there is no conditional limitation on reporting the monitoring data by the account, for example, the account can upload the monitoring data at any time and any place.

In the technical solution provided in step S404, the server obtains a monitoring parameter configured by the account, where the monitoring parameter may be information configured and reported by the account after reporting the monitoring data, or information configured and reported to the server by the account before reporting the monitoring data. In the case that the monitoring parameters are pre-configured for the account and reported to the server, the monitoring parameters may be stored in the configuration library, and after receiving the monitoring data, the server may read the monitoring parameters configured for the account from the configuration library.

In an alternative embodiment, monitoring the parameters may include at least: monitoring index information and alarm triggering conditions, wherein the monitoring index information can include information for determining a container for storing indexes, information of analysis indexes and dimensions, information of statistical modes and the like, and a user can determine monitoring requirements of the account by configuring the monitoring index information.

Optionally, the monitoring index information includes at least one of: monitoring dimension, monitoring index, aggregation monitoring dimension, statistical period and statistical algorithm.

Further, the monitoring parameters may further include: and the alarm triggering condition can record information such as a threshold value for triggering the alarm, an abnormal state for triggering the alarm and the like. The server may monitor whether the data is abnormal through the alarm triggering condition, for example, when the alarm triggering condition is met, it is determined that the data is abnormal, and when the alarm triggering condition is not met, it is determined that the data is not abnormal.

In the technical solution provided in step S406, the monitoring data is counted according to the monitoring index information to obtain a statistical result. Optionally, the monitoring data is counted according to the monitoring index information configured by the account, instead of counting the monitoring data by using a fixed index, a customized monitoring service may be provided.

In the technical solutions provided in step S408 and step S410, data is counted according to the requirement of the user, and the counting result is monitored. Specifically, whether the statistical result meets an alarm triggering condition is detected; if the statistical result is detected to accord with the alarm triggering condition, generating alarm information; and if the statistical result is detected to be not in accordance with the alarm triggering condition, generating information for indicating that the data is normal.

For example, if the demand of the account is to monitor the browsing volume of a specified webpage, the alarm triggering condition is configured to exceed a predetermined browsing volume, and in the case that the browsing volume of the specified webpage is monitored to exceed the predetermined browsing volume, alarm information may be generated; and if the monitored browsing volume of the specified webpage does not exceed the preset browsing volume, generating information for indicating that the data are normal.

According to the above embodiment of the present invention, the receiving the monitoring data reported by the account may include: and receiving the monitoring data reported by the data reporting interface under the condition that the data reporting interface is detected to be called. In the embodiment, the monitoring data can be reported by calling the data reporting interface, and a collection program does not need to be deployed on a user terminal or a machine for storing data on the cloud server by a user, so that the user can upload the data in a user-defined manner.

In the above embodiment, the server may send the data reported by the data reporting interface to the processing cluster at the back end through the preprocessing cluster, and optionally, the preprocessing cluster determines that the data reporting request is received when detecting an event that invokes the data reporting interface, and the server in the preprocessing cluster performs a frequency limiting operation to limit the data amount processed by the preprocessing cluster and the processing cluster.

Alternatively, the pre-processing cluster may implement a frequency-limited operation by limiting the total amount of requests processed in the cluster and the total amount of requests processed by the cluster for the account.

Specifically, receiving the monitoring data reported through the data reporting interface may include: detecting whether the number of times of the data reporting interface being called exceeds a first number of times (the total amount of processing requests in the cluster) within a first predetermined time period; if the number of times of calling the data reporting interface in the first preset time period is detected not to exceed the first number of times, whether the number of times of calling the data reporting interface by the account in the second preset time period exceeds the second number of times (the total amount of requests of the cluster processing the account) is detected; and if the number of times that the account calls the data reporting interface in the second preset time period is detected not to exceed the second number of times, receiving the monitoring data reported through the data reporting interface.

In an optional embodiment, before obtaining the monitoring parameters of the account configuration, the method may further include: and under the condition that the monitoring management interface is detected to be called, receiving the monitoring parameters reported by the monitoring management interface, and recording the received monitoring index information in a configuration library.

In the embodiment, the monitoring management interface is provided, and the monitoring parameters are reported by calling the monitoring management interface, so that a collecting program does not need to be deployed on a user terminal or a machine for storing data on the cloud server by a user, the monitoring data can be processed according to the monitoring parameters configured by the user, and the effect of customizing the monitoring data is realized.

Further optionally, obtaining the monitoring parameters of the account configuration may include: and reading the monitoring parameters from the configuration library, or reading the monitoring parameters from the cache, wherein the monitoring parameters are synchronized into the cache in a timing mode.

In the above embodiment, the monitoring parameters may be preconfigured in the account, after the server acquires the reported monitoring parameters, the monitoring parameters are stored in the configuration library, and when the monitoring parameters configured by the account need to be acquired, the data may be directly read from the configuration library.

Optionally, the monitoring parameters may be preconfigured in the account, after the server acquires the reported monitoring parameters, the monitoring parameters are stored in the configuration library, and the configured monitoring parameters are stored in a predetermined location at regular time by running the synchronization script, where the predetermined location may be a local cache of the processing cluster, and when the monitoring parameters configured for the account need to be acquired, the data may be directly read from the cache. Through the embodiment, the performance problem caused by frequent reading of the database by the API and storm in the basic alarm is solved by using the cache mechanism of the synchronous script.

According to the above embodiments of the present application, the monitoring index information may at least include: counting period, counting algorithm and monitoring index, wherein, counting the monitoring data according to the monitoring index information, and obtaining the counting result comprises: acquiring index data corresponding to the current statistical period from data which accord with monitoring indexes in the monitoring data; and counting the index data according to a statistical algorithm to obtain a statistical result.

In this example, the statistical period may refer to a period for calculating data, and if the configured statistical period is 5 minutes, the monitoring data is calculated once every five minutes by using a statistical algorithm.

The statistical algorithm is an algorithm for counting index data to be counted in the monitoring data, and the statistical algorithm may include: and algorithms such as summation, maximum, minimum, and average.

Optionally, the monitoring index information configured by the user may include: the method comprises the steps of monitoring indexes and display indexes, wherein the monitoring indexes can be indexes needing to be monitored, such as webpage browsing amount, the display indexes are indexes which cannot be monitored, such as monitoring objects, and the indexes can be returned to an account as part of display information for displaying when being displayed.

According to the above embodiment of the present application, detecting whether the statistical result meets the alarm triggering condition may include: acquiring alarm triggering conditions of index data in the statistical result; under the condition that the index data corresponds to a plurality of alarm triggering conditions, acquiring a single-object alarm condition in the plurality of alarm triggering conditions; and detecting whether the index data accords with the single object alarm condition.

Alternatively, one index object in the index data may be set with one alarm trigger condition, or may be set with a plurality of alarm trigger conditions.

The alarm triggering condition can be a wildcard triggering condition, the wildcard triggering condition can correspond to a plurality of index objects, and the wildcard triggering condition takes effect on all the index objects which have an incidence relation with the wildcard triggering condition; the alarm triggering condition may also be a single object alarm condition, which takes effect on one index object.

If a plurality of alarm triggering conditions are configured on the same index object, a single object alarm condition in the plurality of alarm triggering conditions covers a wildcard triggering condition, that is, the wildcard triggering condition configured on the index object fails.

Further optionally, if a plurality of alarm trigger conditions are configured on one index object, priorities of the plurality of alarm trigger conditions may be set, and an alarm event is triggered according to the priorities of the alarm trigger conditions.

Through the embodiment, one index object can bind a plurality of alarm triggering conditions, so that the monitoring flexibility is improved.

In an optional embodiment, after generating the warning information, the method further includes: acquiring a pre-configured alarm receiving group; and sending the alarm information to an alarm receiving group.

Optionally, an alarm receiving group of each index object or each alarm triggering condition is configured in advance, and when the alarm triggering condition is detected to be met, the corresponding alarm receiving group is obtained, and the alarm information is sent to the alarm receiving group.

Further optionally, a plurality of alarm receiving groups are set for one alarm object or alarm triggering condition, and when the alarm triggering condition is detected to be met, the corresponding alarm receiving group is acquired, and the alarm information is sent to the plurality of alarm receiving groups.

Through the embodiment, one index object can be provided with a plurality of different alarm receiving groups, so that the monitoring flexibility is improved.

According to the above embodiment of the present invention, after obtaining the statistical result, the method may further include: under the condition that a display instruction is received, acquiring a display mode and a display dimension indicated by the display instruction; determining the display information of the statistical result according to the display mode and the display dimension; and sending the display information to the account.

Optionally, the display mode may include displaying in a graph mode, displaying in a curve mode, and the like, and the display dimension may include a dimension of one or more indexes, displaying in an aggregation dimension, displaying in different time intervals, and the like.

Through the embodiment, various service scenes are very numerous and personalized, and a user can find out problems in time when the problems occur in the service operation process. Specifically, in the above embodiment, the process is simplified into the process of reporting data by the user, and the cloud server is responsible for data collection, configuration management, data calculation and storage, view display, and alarm management. Furthermore, in order to ensure that the data reported by the user can be identified and conveniently managed by the back end, the organization structure of the data is defined, the index is the minimum organization structure variable of the reported data, and a plurality of indexes can be put into a name space for classified management, so that the concept of a strategy group in basic alarm is simplified, and the strategy group is directly replaced by an alarm rule and is more flexible; by using a cache mechanism of the synchronous script, the performance problem caused by frequent reading of the database by the API and storm in the basic alarm is solved; the index structure reported by the data is given to the user for self-decision, and the requirement of the user for personalized monitoring is met.

The system architecture of an alternative embodiment of the present application is described in detail below with reference to fig. 5 and 6.

As shown in fig. 5, a user (i.e., an account) may configure a custom monitoring item in a front-end page, and when the account performs basic index configuration, the configurable monitoring item (i.e., the monitoring parameter) includes: the system comprises a namespace, dimensions, indexes (including Chinese and English names and units), statistical modes (including statistical methods and statistical periods), alarm triggering conditions and alarm receiving groups. The user can specify a configuration item in a page of the terminal, after the configuration is completed, an application program editing interface API of the cloud server is called to report the configured monitoring parameter, and after the cloud server receives the configured monitoring item, the data stored in the cloud server can be accurately calculated according to the configuration specified by the user.

Optionally, the customized monitoring operation may include an operation of configuring a monitoring item on the cloud monitoring console, and optionally, the customized monitoring operation may further include an operation of writing a code by a user to report monitoring data. Optionally, after the user writes the code, the written code may be uploaded in a data reporting sample at a document center entry of the cloud server, and the monitoring data may be reported by running the code.

In an optional embodiment, the monitoring data reported by the user can be visualized, so that the user can conveniently monitor the service quality in real time and troubleshoot abnormal problems. Optionally, the cloud server may support a user to conveniently view the index data at the console, for example, the service platform supports single-index multi-object viewing and single-object multi-index viewing, and also supports selection of any time interval to compare the viewed data.

As shown in fig. 6, a user may enter an information presentation page of a monitoring object, in which the user may select an area and a dimension of the monitoring object, in the example shown in fig. 6, an original dimension is selected, after the original dimension is selected, information of the monitoring object in the dimension that conforms to the selected area may be presented, optionally, the monitoring object may be displayed (e.g., aa ═ 2& ss ═ c), monitoring graph information and a statistical period are monitored, further, after information of the monitoring object aa ═ 2& ss ═ c in different time intervals is selected for viewing, information of the monitoring object aa ═ 2& ss ═ c may be presented in a pop-up information box, in which names of the monitoring object and information of the selectable viewing time intervals (e.g., near 1 hour, today, yesterday, near 7 days, near 14 days) are displayed.

Optionally, the configured alarm policy (i.e. the alarm trigger condition described above) supports an indicator-level wildcard rule (e.g. a single-object alarm condition), that is, an alarm policy (e.g. a wildcard trigger condition) that is fully effective for all objects associated with the indicator may be configured. Optionally, the configuration of the alarm rule for the single object level may also be configured, and if the single object alarm rule is configured on the monitored object, the wildcard rule configured on the monitored object is covered, that is, the priority of the single object alarm rule is higher than that of the wildcard rule.

An alternative embodiment of the present application is described in detail below with reference to fig. 7-9.

The system architecture shown in fig. 7 includes: the system comprises a cloud monitoring foreground, a user submachine, a cloud application programming interface, a configuration library, a database, a preprocessing cluster Custom-Nws, a distributed message queue processing cluster Kafka, a data processing cluster Custom-Barad-storm, an alarm management center and an alarm sending module.

In the system architecture, a user reports monitoring data through a data reporting interface on a slave machine of the account (optionally, the slave machine is a data storage machine pre-allocated to the account by a cloud server), wherein a data reporting cluster can be deployed in each region to meet the reporting requirements of the slave machine in each region, optionally, a data processing cluster custom-barad-storm system can be set up in each region to independently process the reported data of each region to avoid mutual conflict, and the configuration of barads is deployed in regions.

The cloud monitoring foreground shown in fig. 7 can provide a visual configuration interface for a user, and the user can configure the reported index information, the alarm policy information, the alarm receiving group, and the like through the interface.

Furthermore, the cloud monitoring front end can also provide a statistical result display page of the self-defined monitoring data reported by the user, a statistical curve with different granularities (such as a statistical curve with 5-minute granularity), and display of alarm information for the user.

The cloud application programming interface may include a data interface and a management interface, wherein the data interface includes a reporting data interface and a pulling data interface. The management interface includes: and providing an external quota metric configuration interface, an alarm management interface and a management interface of namespace.

As shown in fig. 8, after receiving custom monitoring data reported by a user, a preprocessing cluster custom-nws may perform validity check, authentication, dimension strong consistency verification, frequency limitation according to appId differentiation, map the monitoring data to corresponding TOPO (offload) according to appId as a route, and perform multi-path forwarding to multiple TOPO to send the received monitoring data to a Kafka cluster, and optionally, the preprocessing cluster may also count data reception details according to appId Dimensions.

In an alternative embodiment, the pre-processing cluster implements routing by using a pre-configured routing distribution parameter, for example, the pre-configured routing distribution parameter of the distributed storage system CKV is as follows:

“nws_router”:[{“rmin”:1251000000,“rmax”:1251900000,“topo”:“Barad_Comm”},

{“rmin”:1251900001,“rmax”:9999999999,“topo”:“Barad_Device”}]

optionally, after receiving the monitoring data, all the route configuration parameters are traversed, and when the application identification appId is in the range [ rmin, rmax ], the data is sent to the corresponding splitter topo.

For example, the parameters of the configuration are as follows:

[{“rmin”:1251001002,“rmax”:1251001002,“topo”:“Barad_Device”},

{ "rmin": 1251000000, "rmax": 9999999999, "topo": Barad _ Comm "} ], for the configuration, if the appid is 1251001002, the monitoring data corresponding to the appid is simultaneously sent to the Device Barad _ Device and the Device Barad _ Comm.

Further, after determining the configuration parameters, the configuration information may be timed to synchronize, for example, each minute of timing synchronization configuration, for example, the configuration synchronization of CKV is performed again when the last synchronization time exceeds 1 minute.

In an alternative embodiment, frequency limiting, timestamp validation and staining of the NWS may be accomplished by preconfigured CKV parameters.

Optionally, each NWS service maintains a total request count (i.e. the first number of times) in the local memory, defaults to an upper limit of 100W per minute, and rejects all services when the upper limit is exceeded; alternatively, each NWS service maintains a count of the number of requests for the secretId for each account in local memory (i.e. the second number of times described above), defaults to 10000 per minute, and rejects the secretId service when the upper limit is exceeded.

For example, the parameters that can be preconfigured are: "nws # 1251001002" { "request": 1000, "metric": 10000, "tsdown": 1800, "tsup": 600, "colour": 1234567890 }.

Specifically, with appId as a key, obtaining ckv a request and a threshold value metric field in the configuration, and if no configuration parameter exists, obtaining a default value in the configuration file nws.conf of the pre-processing cluster; judging whether the current request number and the index number exceed the upper limit of the sending frequency, if not, forwarding and updating the CKV count; if the upper limit is exceeded, the request is denied.

Alternatively, the frequency limitation may be implemented by using a full frequency-limited method to accurately control the reporting frequency of one app, and specifically, the CKV maintains the request number and the report index data count at the current time:

“1251001002#request”:99，

“1251001002#metric”:999。

it should be noted that, the full-spectrum-limited is performed only by the request of authentication, and in order to improve throughput, CKVs for counting may be added horizontally (configured in zookeeper), and the NWS is mapped to different CKVs by appId. Wherein zookeeper is a distributed, open source distributed application coordination service.

Optionally, the authentication processing is performed by: the common parameters are sorted in ascending order of dictionary according to key, and the request character strings are spliced, for example, the common parameters are formatted into k ═ v, and connected by "&", so that the following request character strings are obtained:

Action＝PutMonitorData&Nonce＝345122&Region＝gz&SecretId＝xxxxxxx&Timestamp＝1408704141。

signing the obtained request character string by using an HMAC-SHA1 algorithm, encoding the generated signature string by using Base64 to obtain a final signature string, verifying the signature by using a cloud security authentication interface, and obtaining the appId of the user when the verification is successful; and when the verification fails, directly discarding the data without performing data statistics of the appId dimension.

Wherein, the common parameters can be input together when configuring the monitoring parameters. Optionally, the monitoring data carries an application identifier appId (or account identifier).

Optionally, performing timestamp verification on the request after receiving the request, specifically, taking ckv the tsdown and tsup fields (unit s) in the configuration with appId as a key, and taking the default value in the configuration file nws. conf if there are no configuration parameters; and setting the current timestamp as now, and discarding if the timestamp in the data is not in the range of [ now-tsdown, now + tsup ].

Alternatively, using appId as a key, obtain ckv the colour field (timestamp) in the configuration, if there is no configuration, take the default value of 0; and if the current time is less than the value of colour, generating seqId and adding the seqId into the current data.

According to the above embodiments of the present application, the namespace related configuration can be configured in advance and synchronized to storm (a real-time computing service); the time granularity supports 1 minute and 5 minutes of calculation; the method comprises the steps of storing uniform hbase (if the future pulling performance is insufficient, CKV or redis clusters can be considered to store data for 2 days to improve the real-time data pulling performance, and the hbase should provide C1 equipment, so that the equipment number is increased, and the data storage amount of a single machine is reduced); the alarm supports namespace global threshold configuration (low priority) and object-specific index threshold configuration (high priority); and dyeing data of a certain apple, and referring to a barad implementation mode, adding sequence selectable items to nws, and logging when storm finds that the data is recorded.

As shown in FIG. 9, the barad frame includes: the system comprises a data reading module kafkarawspoot, a configuration information reading module policyscspout, a receiving module statbolt, a storage module hbasebolt and an alarm module alarmbolt, wherein optionally, each module in the framework uses data flow to perform data processing.

Wherein kafkawspout is used to read user data from kafka, optionally a startup parameter is added to the module to support the function of one storm cluster reading multiple kafka clusters, optionally the timestamp can be filtered when reading data, as the user may calculate historical data.

The policysyncspout is used for regularly reading alarm policy configuration from a configuration library (optionally, an appid policy needing to be processed can be synchronized), and routing to a specified alarmbolt (consistent with a data stream, distributed according to namespace).

The statbot is used for receiving data of kafkarawspoout, carrying out hash according to namespace, establishing a statistical node for calculation, sending a statistical result to a storage and alarm bolt when the time limit is exceeded, and controlling the number of monitoring objects and the number of timestamps below the namespace to prevent insufficient memory and a memory data storage structure caused by abnormal data: amespace- > object- > metric- > logTime- > data.

The hbasebolt is used for receiving and storing the statistical result, and the row format is as follows: md5 value + dimGroupid + period + stabtype; according to Appid ten-day tables (2 tables in one month, 1-15/16-last), each Appid is an object table and a real-time table, the real-time table records the latest values and timestamps of all indexes of the object, and the object table is in a rowkey format: namespace + md 5; meanwhile, one copy of data is stored in a plurality of hbases, and the data reliability is improved.

Alarmbolt is used for receiving the statistical result to judge the abnormity, and informing the abnormal information to an alarm management center, wherein the specific flow is consistent with barad.

And the log bolt, namely a log bolt, is used for receiving the operation information reported by each component, and the granularity is refined to the index.

The alarm management center shown in fig. 7 may configure a general alarm policy (without binding an object) for a specific index (dimGroupId), and may configure an interval (not push, only push once, push at a certain interval time) for pushing an alarm to an alarm sending module in an alarm policy dimension; and (4) configuring a callback URL (Uniform resource locator) in an alarm strategy dimension (continuous alarm, configurable push interval and URL parameter support template).

It should be further noted that a synchronization script can be configured in advance, and the script reads the configuration required by the custom monitoring of each component from the CDB and synchronizes to ckv in a timed manner, and since many configurations need to be frequently pulled, a performance bottleneck will occur if the CDB (mysql) is directly read. Therefore, a cache mechanism similar to the principle of memcache is adopted, and the performance bottleneck is eliminated.

Through the embodiment, the concept of the strategy group in the basic alarm is simplified, the alarm rule is directly used for replacing the strategy group, the strategy group is more flexible, one object can bind a plurality of alarm rules, and a plurality of different receiving groups can be set; by using a cache mechanism of the synchronous script, the performance problem caused by frequent reading of the database by the API and storm in the basic alarm is solved; the index structure reported by the data is given to the user for self-decision, and the requirement of the user for personalized monitoring is met.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

According to the embodiment of the invention, the invention also provides a data monitoring device for implementing the data monitoring method. Fig. 10 is a schematic diagram of an alternative data monitoring apparatus according to an embodiment of the present invention, as shown in fig. 10, the apparatus may include:

a first receiving unit 1001, configured to receive monitoring data reported by an account;

a first obtaining unit 1003, configured to obtain monitoring parameters of account configuration, where the monitoring parameters at least include: monitoring index information and alarm triggering conditions;

a first statistical unit 1005, configured to count the monitoring data according to the monitoring index information to obtain a statistical result;

a first detecting unit 1007, configured to detect whether a statistical result meets an alarm triggering condition;

a generating unit 1009, configured to generate alarm information when it is detected that the statistical result meets the alarm triggering condition.

According to the above embodiment, the first receiving unit may include: and the second receiving unit is used for receiving the monitoring data reported by the data reporting interface when the condition that the data reporting interface is called is detected.

Specifically, the second receiving unit may include: the first detection subunit is configured to detect whether the number of times that the data reporting interface is called exceeds a first number of times within a first predetermined time period; the second detection subunit is used for detecting whether the number of times of calling the data reporting interface by the account exceeds a second number of times within a second preset time period when the number of times of calling the data reporting interface within the first preset time period is detected to be not more than the first number of times; and the third detection subunit is used for receiving the monitoring data reported by the data reporting interface when detecting that the number of times of calling the data reporting interface by the account in a second preset time period does not exceed the second number of times.

Optionally, the first receiving unit is further configured to receive the monitoring parameter reported by the monitoring management interface and record the received monitoring index information in the configuration library, when it is detected that the monitoring management interface is called before the monitoring parameter configured for the account is acquired. The first acquisition unit further includes: the first reading module is used for reading the monitoring parameters from the configuration library, or the second reading module is used for reading the monitoring parameters from the cache, wherein the monitoring parameters are synchronized to the cache in a timing mode.

Optionally, the monitoring index information at least includes: statistics cycle, statistical algorithm and monitoring index, wherein, first statistics unit still includes: the second acquisition unit is used for acquiring index data corresponding to the current statistical period from the data which accord with the monitoring index in the monitoring data; and the second statistical unit is used for carrying out statistics on the index data according to a statistical algorithm to obtain a statistical result.

Further, the first detection unit further includes: the third acquisition unit is used for acquiring the alarm triggering condition of the index data in the statistical result; the fourth obtaining unit is used for obtaining a single-object alarm condition in the plurality of alarm trigger conditions under the condition that the index data corresponds to the plurality of alarm trigger conditions; and the second detection unit is used for detecting whether the index data accords with the single-object alarm condition.

In an optional embodiment, the apparatus further comprises: the first acquisition module is used for acquiring the display mode and the display dimension indicated by the display instruction under the condition of receiving the display instruction after the statistical result is obtained; the determining module is used for determining the display information of the statistical result according to the display mode and the display dimension; and the first sending module is used for sending the display information to the account.

In another optional embodiment, the apparatus further comprises: the first acquisition module is used for acquiring a pre-configured alarm receiving group after the alarm information is generated; and the first sending module is used for sending the alarm information to the alarm receiving group.

It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may operate in a hardware environment as shown in fig. 2, and may be implemented by software or hardware.

According to the embodiment of the invention, the invention also provides a server or a terminal for implementing the data monitoring method.

Fig. 11 is a block diagram of a terminal according to an embodiment of the present invention, and as shown in fig. 11, the terminal may include: one or more processors 201 (only one of which is shown), a memory 203, and a transmission device 205 (such as the transmission device in the above embodiment), as shown in fig. 11, the terminal may further include an input/output device 207.

The memory 203 may be used to store software programs and modules, such as program instructions/modules corresponding to the data monitoring method and apparatus in the embodiments of the present invention, and the processor 201 executes various functional applications and data processing by running the software programs and modules stored in the memory 203, that is, implements the data monitoring method described above. The memory 203 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 203 may further include memory located remotely from the processor 201, which may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 205 is used for receiving or sending data via a network, and can also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 205 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 205 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

Wherein the memory 203 is specifically used for storing application programs.

The processor 201 may call the application stored in the memory 203 via the transmission means 205 to perform the following steps: receiving monitoring data reported by an account; acquiring monitoring parameters of account configuration, wherein the monitoring parameters at least comprise: monitoring index information and alarm triggering conditions; counting the monitoring data according to the monitoring index information to obtain a counting result; detecting whether the statistical result meets an alarm triggering condition or not; and if the statistical result is detected to accord with the alarm triggering condition, generating alarm information.

The processor 201 is further configured to perform the following steps: and under the condition that the data reporting interface is detected to be called, receiving the monitoring data reported by the data reporting interface so as to receive the monitoring data reported by the account.

The processor 201 is further configured to perform the following steps: detecting whether the called times of a data reporting interface in a first preset time period exceed a first time; if the number of times of calling the data reporting interface in the first preset time period is detected to be not more than the first number of times, whether the number of times of calling the data reporting interface by the account in the second preset time period is more than the second number of times is detected; and if the number of times that the account calls the data reporting interface in the second preset time period is detected not to exceed the second number of times, receiving the monitoring data reported by the data reporting interface so as to receive the monitoring data reported by the data reporting interface.

The processor 201 is further configured to perform the following steps: before acquiring the monitoring parameters configured for the account, under the condition that the monitoring management interface is detected to be called, receiving the monitoring parameters reported by the monitoring management interface, and recording the received monitoring index information in a configuration library.

The processor 201 is further configured to perform the following steps: and reading the monitoring parameters from the configuration library, or reading the monitoring parameters from the cache, wherein the monitoring parameters are synchronized to the cache in a timing mode to obtain the monitoring parameters of the account configuration.

Optionally, the monitoring index information at least includes: statistical period, statistical algorithm and monitoring index, wherein the processor 201 is further configured to perform the following steps: acquiring index data corresponding to the current statistical period from data which accord with monitoring indexes in the monitoring data; and counting the index data according to a statistical algorithm to obtain a statistical result, and counting the monitoring data according to the monitoring index information to obtain a statistical result.

The processor 201 is further configured to perform the following steps: acquiring alarm triggering conditions of index data in the statistical result; under the condition that the index data corresponds to a plurality of alarm triggering conditions, acquiring a single-object alarm condition in the plurality of alarm triggering conditions; and detecting whether the index data accords with the single-object alarm condition or not so as to detect whether the statistical result accords with the alarm triggering condition or not.

The processor 201 is further configured to perform the following steps: after the statistical result is obtained, under the condition that a display instruction is received, acquiring a display mode and a display dimension indicated by the display instruction; determining the display information of the statistical result according to the display mode and the display dimension; and sending the display information to the account.

The processor 201 is further configured to perform the following steps: after generating the alarm information, acquiring a pre-configured alarm receiving group; and sending the alarm information to an alarm receiving group.

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.

It can be understood by those skilled in the art that the structure shown in fig. 11 is only an illustration, and the terminal may be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 11 is a diagram illustrating a structure of the electronic device. For example, the terminal may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 11, or have a different configuration than shown in FIG. 11.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The embodiment of the invention also provides a storage medium. Alternatively, in this embodiment, the storage medium may be a program code for executing the data monitoring method.

Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

receiving monitoring data reported by an account; acquiring monitoring parameters of account configuration, wherein the monitoring parameters at least comprise: monitoring index information and alarm triggering conditions; counting the monitoring data according to the monitoring index information to obtain a counting result; detecting whether the statistical result meets an alarm triggering condition or not; and if the statistical result is detected to accord with the alarm triggering condition, generating alarm information.

Optionally, the storage medium is further arranged to store program code for performing the steps of: and under the condition that the data reporting interface is detected to be called, receiving the monitoring data reported by the data reporting interface so as to receive the monitoring data reported by the account.

Optionally, the storage medium is further arranged to store program code for performing the steps of: detecting whether the called times of a data reporting interface in a first preset time period exceed a first time; if the number of times of calling the data reporting interface in the first preset time period is detected to be not more than the first number of times, whether the number of times of calling the data reporting interface by the account in the second preset time period is more than the second number of times is detected; and if the number of times that the account calls the data reporting interface in the second preset time period is detected not to exceed the second number of times, receiving the monitoring data reported by the data reporting interface so as to receive the monitoring data reported by the data reporting interface.

Optionally, the storage medium is further arranged to store program code for performing the steps of: before acquiring the monitoring parameters configured for the account, under the condition that the monitoring management interface is detected to be called, receiving the monitoring parameters reported by the monitoring management interface, and recording the received monitoring index information in a configuration library.

Optionally, the storage medium is further arranged to store program code for performing the steps of: and reading the monitoring parameters from the configuration library, or reading the monitoring parameters from the cache, wherein the monitoring parameters are synchronized to the cache in a timing mode to obtain the monitoring parameters of the account configuration.

Optionally, the monitoring index information at least includes: statistical cycles, statistical algorithms and monitoring metrics, wherein optionally the storage medium is further arranged to store program code for performing the steps of: acquiring index data corresponding to the current statistical period from data which accord with monitoring indexes in the monitoring data; and counting the index data according to a statistical algorithm to obtain a statistical result, and counting the monitoring data according to the monitoring index information to obtain a statistical result.

Optionally, the storage medium is further arranged to store program code for performing the steps of: acquiring alarm triggering conditions of index data in the statistical result; under the condition that the index data corresponds to a plurality of alarm triggering conditions, acquiring a single-object alarm condition in the plurality of alarm triggering conditions; and detecting whether the index data accords with the single-object alarm condition or not so as to detect whether the statistical result accords with the alarm triggering condition or not.

Optionally, the storage medium is further arranged to store program code for performing the steps of: after the statistical result is obtained, under the condition that a display instruction is received, acquiring a display mode and a display dimension indicated by the display instruction; determining the display information of the statistical result according to the display mode and the display dimension; and sending the display information to the account.

Optionally, the storage medium is further arranged to store program code for performing the steps of: after generating the alarm information, acquiring a pre-configured alarm receiving group; and sending the alarm information to an alarm receiving group.

Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A method for monitoring data, comprising:

the front-end page is displayed for a user to configure a self-defined monitoring item, and the configurable monitoring item in the front-end page comprises: a statistical mode, an alarm triggering condition and an alarm receiving group;

receiving monitoring data reported by an account, storing monitoring parameters in a configuration library, running a synchronous script, and storing the monitoring parameters in a local cache of a processing cluster at regular time;

receiving monitoring parameters reported by a monitoring management interface under the condition that the monitoring management interface is detected to be called, so that an acquisition program does not need to be deployed on a user terminal or a machine for storing data on a cloud server by a user; recording the received monitoring index information in the configuration library; reading the monitoring parameters from the configuration library, or,

reading the monitoring parameters from the cache, wherein the monitoring parameters at least comprise: monitoring index information and alarm triggering conditions, wherein the monitoring parameters are synchronized to the cache in a timing mode, and the monitoring index information comprises monitoring indexes needing to be monitored and display indexes which cannot be monitored;

counting the monitoring data according to the monitoring index information to obtain a counting result;

under the condition that a display instruction is received, acquiring a display mode and a display dimension indicated by the display instruction;

determining display information of the statistical result according to the display mode and the display dimension, wherein the display information further comprises the display indexes which cannot be monitored;

sending the display information to the account;

acquiring an alarm triggering condition of index data in the statistical result;

when the index data corresponds to a plurality of alarm triggering conditions, and the plurality of alarm triggering conditions include wildcard triggering conditions and single-object alarm conditions, the single-object alarm conditions in the plurality of alarm triggering conditions cover the wildcard triggering conditions, indicate that the wildcard triggering conditions are invalid, and acquire the single-object alarm conditions in the plurality of alarm triggering conditions;

detecting whether the index data meets the single object alarm condition;

if the statistical result is detected to accord with the single object alarm condition, generating alarm information;

acquiring a plurality of different alarm receiving groups corresponding to the preset single-object alarm condition;

sending the alarm information to the plurality of different alarm receiving groups;

displaying an information display page of the monitored object;

after a user selects a target dimension in the information display page, displaying information of a monitoring object which accords with a selected area in the target dimension, wherein the information of the monitoring object comprises the monitoring object, monitoring chart information and a statistical period;

after a user selects to view information of a monitored object in different time intervals, displaying the information of the monitored object in a popped information frame, wherein the name of the monitored object and the information of the selectable viewing time intervals are displayed in the popped information frame.

2. The method of claim 1, wherein receiving monitoring data reported by an account comprises:

and receiving the monitoring data reported by the data reporting interface under the condition that the data reporting interface is detected to be called.

3. The method of claim 2, wherein receiving the monitoring data reported through the data reporting interface comprises:

detecting whether the called times of the data reporting interface exceed a first time within a first preset time period;

if the number of times of calling the data reporting interface in a first preset time period is detected to be not more than the first number of times, detecting whether the number of times of calling the data reporting interface by the account in a second preset time period is more than the second number of times;

and if the number of times that the account calls the data reporting interface in a second preset time period is detected not to exceed a second number of times, receiving the monitoring data reported through the data reporting interface.

4. The method according to any one of claims 1 to 3, wherein the monitoring index information at least includes: counting period, counting algorithm and monitoring index, wherein the monitoring data is counted according to the monitoring index information, and the obtained counting result comprises the following steps:

acquiring index data corresponding to the current statistical period from the data which accord with the monitoring index in the monitoring data;

and counting the index data according to the statistical algorithm to obtain a statistical result.

5. The method of claim 1, wherein the monitoring metric information includes at least one of: monitoring dimension, monitoring index, aggregation monitoring dimension, statistical period and statistical algorithm.

6. A data monitoring device, wherein the device displays a front-end page for a user to configure a custom monitoring item, and the configurable monitoring item in the front-end page comprises: a statistical mode, an alarm triggering condition and an alarm receiving group;

the device comprises:

the first receiving unit is used for receiving monitoring data reported by an account, storing monitoring parameters in a configuration library, running a synchronous script and storing the monitoring parameters in a local cache of a processing cluster at regular time;

the first receiving unit is further configured to receive the monitoring parameters reported through the monitoring management interface under the condition that the monitoring management interface is detected to be called before the monitoring parameters of the account configuration are acquired, so that a user terminal or a user does not need to deploy an acquisition program on a machine storing data on a cloud server; recording the received monitoring index information in the configuration library;

a first obtaining unit, configured to obtain monitoring parameters of the account configuration, where the monitoring parameters at least include: monitoring index information and alarm triggering conditions, the first obtaining unit further comprises: the first reading module is used for reading the monitoring parameters from the configuration library, or the second reading module is used for reading the monitoring parameters from the cache, wherein the monitoring parameters are synchronized to the cache in a timing mode, and the monitoring index information comprises monitoring indexes needing to be monitored and display indexes which cannot be monitored;

the first statistical unit is used for counting the monitoring data according to the monitoring index information to obtain a statistical result;

a first detecting unit, configured to detect whether the statistical result meets the alarm triggering condition, where the first detecting unit further includes: the third acquisition unit is used for acquiring the alarm triggering condition of the index data in the statistical result; a fourth obtaining unit, configured to, when the index data corresponds to multiple alarm trigger conditions and the multiple alarm trigger conditions include a wildcard trigger condition and a single-object alarm condition, cover the wildcard trigger condition with the single-object alarm condition in the multiple alarm trigger conditions, indicate that the wildcard trigger condition is invalid, and obtain the single-object alarm condition in the multiple alarm trigger conditions; the second detection unit is used for detecting whether the index data meets the single-object alarm condition;

the generating unit is used for generating alarm information when the statistical result is detected to accord with the alarm triggering condition;

the device further comprises:

the first acquisition module is used for acquiring the display mode and the display dimension indicated by the display instruction under the condition of receiving the display instruction after the statistical result is obtained;

the determining module is used for determining the display information of the statistical result according to the display mode and the display dimension, wherein the display information further comprises the display indexes which cannot be monitored;

the first sending module is used for sending the display information to the account;

the device further comprises:

the first acquisition module is used for acquiring a plurality of different alarm receiving groups corresponding to the preset single-object alarm condition after the alarm information is generated;

the first sending module is used for sending the alarm information to the alarm receiving group;

the apparatus is further configured to:

displaying an information display page of the monitored object;

7. The apparatus of claim 6, wherein the first receiving unit comprises:

and the second receiving unit is used for receiving the monitoring data reported by the data reporting interface when the condition that the data reporting interface is called is detected.

8. The apparatus of claim 7, wherein the second receiving unit comprises:

the first detection subunit is configured to detect whether the number of times that the data reporting interface is called exceeds a first number of times within a first predetermined time period;

the second detection subunit is configured to, when it is detected that the number of times that the data reporting interface is called in a first predetermined time period does not exceed a first number of times, detect whether the number of times that the account calls the data reporting interface in a second predetermined time period exceeds a second number of times;

and the third detection subunit is configured to receive the monitoring data reported by the data reporting interface when it is detected that the number of times that the account calls the data reporting interface within a second predetermined time period does not exceed a second number of times.

9. The apparatus according to any one of claims 6 to 8, wherein the monitoring index information at least includes: statistics cycle, statistical algorithm and monitoring index, wherein, first statistics unit still includes:

the second acquisition unit is used for acquiring index data corresponding to the current statistical period from the data which accords with the monitoring index in the monitoring data;

and the second statistical unit is used for carrying out statistics on the index data according to the statistical algorithm to obtain a statistical result.

10. The apparatus of claim 6, wherein the monitoring indicator information comprises at least one of: monitoring dimension, monitoring index, aggregation monitoring dimension, statistical period and statistical algorithm.