CN117454295A - Dynamic threshold abnormality detection method, system, storage medium and intelligent device - Google Patents
Dynamic threshold abnormality detection method, system, storage medium and intelligent device Download PDFInfo
- Publication number
- CN117454295A CN117454295A CN202311535202.1A CN202311535202A CN117454295A CN 117454295 A CN117454295 A CN 117454295A CN 202311535202 A CN202311535202 A CN 202311535202A CN 117454295 A CN117454295 A CN 117454295A
- Authority
- CN
- China
- Prior art keywords
- index
- maintenance
- time stamp
- abnormal
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 56
- 230000005856 abnormality Effects 0.000 title claims abstract description 15
- 238000012423 maintenance Methods 0.000 claims abstract description 136
- 230000002159 abnormal effect Effects 0.000 claims abstract description 82
- 239000013598 vector Substances 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 23
- 230000008859 change Effects 0.000 claims description 29
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims 2
- 238000012549 training Methods 0.000 abstract description 6
- 238000010801 machine learning Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A dynamic threshold abnormality detection method, a system, a storage medium and an intelligent device, comprising: constructing an index prediction model, inputting a time stamp feature vector, outputting an operation and maintenance index prediction value corresponding to a time stamp, and extracting sample data training of a historical detection period; the method comprises the steps of collecting actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous time stamp feature vectors in a time interval into an index prediction model to obtain operation and maintenance index predicted values of each input time stamp, determining an operation and maintenance index threshold range of each time stamp according to the operation and maintenance index predicted values of all the continuous time stamps, and marking the operation and maintenance index value of one time stamp as an abnormal index and alarming when the actual operation and maintenance index value collected by the time stamp is not in the operation and maintenance index threshold range. The invention relates to the field of computers, which can dynamically and automatically set the threshold range of operation and maintenance indexes at each time point and effectively realize real-time abnormality detection of operation and maintenance key indexes.
Description
Technical Field
The invention relates to a dynamic threshold abnormality detection method, a dynamic threshold abnormality detection system, a storage medium and intelligent equipment, and relates to the field of computers.
Background
In the field of operation and maintenance, index anomaly detection refers to detecting and identifying anomalies that deviate significantly from normal behavior or expected patterns by real-time monitoring and analyzing key indexes of a system or device. These key indicators may be various performance indicators, operational status indicators, resource utilization indicators, network traffic indicators, etc., depending on the different systems and applications.
The objective of the index anomaly detection is to discover and identify potential problems or anomalies in advance so as to take corresponding measures in time to solve or prevent possible faults, performance degradation, security threats, and the like. By monitoring key indexes in real time and detecting abnormality, an operation and maintenance team can timely acquire the state and health condition of the system and can alarm, record, analyze and respond to abnormal conditions.
Most of index anomaly detection algorithms in the prior industry are based on fixed thresholds, but the fixed thresholds are often required to be set and updated empirically, so that the proper thresholds are difficult to determine, the adaptability to different indexes is poor, and the complexity of maintenance and management is increased.
Patent application CN 202211569513.5 (application name: a method and apparatus for detecting system index, electronic device and storage medium thereof, applicant: medium mail information technology (Beijing) limited, application date: 2022.12.08) discloses a method and apparatus for detecting system index, electronic device and storage medium thereof. The method comprises the steps of carrying out prediction processing on index items in a target system based on a prediction model to obtain a prediction result of the index items, wherein the prediction result comprises index prediction values of the index items at all time points in a prediction time period; determining a dynamic threshold value of the index item at each time point based on the index predicted value of the index item at each time point in the predicted time period; and acquiring actual index data of the target system at each time point, and performing anomaly detection on the actual index data based on the dynamic threshold value of the corresponding time point. The specific implementation method of the technical scheme when determining the dynamic threshold value of the index item at each time point is as follows: drawing a prediction base line based on index prediction values of the index items at all time points in a prediction time period; forming an upper interval line based on peak points in the predicted baseline, and determining an upper threshold value of each time point based on the upper interval line; forming a lower interval line based on the valley points in the prediction base line, and determining a lower limit threshold value of each time point based on the lower interval line; or, for the index predicted value at any time point, determining a corresponding upper limit threshold and lower limit threshold based on a preset positive and negative error. That is, the technical scheme determines the dynamic threshold according to the peak point and the valley point of the predicted baseline or based on a preset positive and negative error, wherein the threshold of each time point of the index item in the predicted time period is uniform according to the peak point and the valley point of the predicted baseline, and a threshold range cannot be dynamically set for each time point independently; based on the preset positive and negative errors, an appropriate error value needs to be preset manually, and cannot be automatically and dynamically calculated and adjusted according to actual services.
Therefore, how to dynamically and automatically set the threshold range of the operation and maintenance index at each time point, so as to effectively realize the real-time anomaly detection of the operation and maintenance key index, and the operation and maintenance key index is a technical problem which is focused by technicians.
Disclosure of Invention
Accordingly, the present invention aims to provide a method, a system, a storage medium and an intelligent device for detecting dynamic threshold abnormality, which can dynamically and automatically set a threshold range of an operation and maintenance index at each time point, thereby effectively realizing real-time abnormality detection of the operation and maintenance key index.
In order to achieve the above object, the present invention provides a method for detecting dynamic threshold abnormality, comprising:
firstly, constructing an index prediction model, wherein the input of the index prediction model is a time stamp feature vector, the output of the index prediction model is an operation and maintenance index prediction value corresponding to an input time stamp, and the time stamp feature vector and an actual operation and maintenance index value in a historical detection period are extracted to serve as sample data to train the index prediction model;
and secondly, acquiring actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous time stamp feature vectors acquired in a certain time interval into a trained index prediction model to obtain operation and maintenance index predicted values of each input time stamp, determining an operation and maintenance index threshold range corresponding to each time stamp according to the operation and maintenance index predicted values of all continuous time stamps, and marking the operation and maintenance index values of one time stamp as abnormal indexes and alarming when the actual operation and maintenance index value acquired by the time stamp is not in the operation and maintenance index threshold range.
In order to achieve the above object, the present invention further provides a dynamic threshold abnormality detection system, including:
the index prediction model construction device is used for constructing an index prediction model, wherein the input of the index prediction model is a time stamp feature vector, the output of the index prediction model is an operation and maintenance index prediction value corresponding to the input time stamp, and the time stamp feature vector and the actual operation and maintenance index value in a historical detection period are extracted to serve as sample data to train the index prediction model;
the index anomaly prediction device is used for acquiring actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous timestamp feature vectors acquired in a certain time interval into a trained index prediction model to obtain an operation and maintenance index prediction value of each input timestamp, determining an operation and maintenance index threshold range corresponding to each timestamp according to the operation and maintenance index prediction values of all continuous timestamps, and marking the operation and maintenance index value of one timestamp as an anomaly index and alarming when the actual operation and maintenance index value acquired by the timestamp is not in the operation and maintenance index threshold range.
In order to achieve the above object, the present invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the above method.
In order to achieve the above object, the present invention further provides an intelligent device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the above method when executing the program.
Compared with the prior art, the invention has the beneficial effects that: the invention carries out characteristic construction and model training on the acquired service or host performance index data by adopting a machine learning algorithm, and dynamically and automatically calculates and obtains the operation and maintenance index threshold range of each time point according to the operation and maintenance index predicted values of all continuous time stamps, the technical scheme is simple and effective, and in the application of the existing network, whether the acquired operation and maintenance index is abnormal or not can be detected and alarmed in a very short time (such as 5 seconds) or not, thereby effectively meeting the real-time and high-efficiency requirements of the existing network on operation and maintenance key index detection and saving system resources; the invention further dynamically adjusts the operation and maintenance index threshold range of each time stamp according to the change trend of the operation and maintenance index, thereby truly realizing automatic threshold adjustment without manually configuring parameters.
Drawings
Fig. 1 is a flowchart of a dynamic threshold anomaly detection method according to an embodiment of the present invention.
Fig. 2 is a flowchart of a specific step of updating an abnormal index in sample data when the index prediction model is trained by extracting a time stamp feature vector and an actual operation and maintenance index value in a history detection period as sample data in step one of fig. 1.
Fig. 3 is a schematic structural diagram of a dynamic threshold abnormality detection system according to a second embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent.
Example 1
Referring to fig. 1, fig. 1 is a flowchart of a dynamic threshold anomaly detection method according to an embodiment of the present invention, including:
firstly, constructing an index prediction model, wherein the input of the index prediction model is a time stamp feature vector, the output of the index prediction model is an operation and maintenance index prediction value corresponding to an input time stamp, and the time stamp feature vector and an actual operation and maintenance index value in a historical detection period are extracted to serve as sample data to train the index prediction model;
and secondly, acquiring actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous time stamp feature vectors acquired in a period of time (for example, 5 seconds) into a trained index prediction model to obtain an operation and maintenance index predicted value of each input time stamp, determining an operation and maintenance index threshold range corresponding to each time stamp according to the operation and maintenance index predicted values of all continuous time stamps, and marking the operation and maintenance index value of one time stamp as an abnormal index and alarming when the actual operation and maintenance index value acquired by the time stamp is not in the operation and maintenance index threshold range.
The detection period in the present application may be a day or a week, and the time stamp feature vector may include: whether holiday, busy or idle, working day, timestamp ID, etc., wherein, whether holiday, busy or idle can be judged by calendar, if the hour value in the timestamp is 23:00-06:00, when busy and the rest are idle, the timestamp ID is a value obtained by sorting all timestamps in each detection period according to time sequence and normalizing each sorted sequence number, for example: when the time stamp feature vector is [1, 0, 0.3], the time stamp is holiday, busy hour and non-working day, and the time stamp ID is 0.3, and the time stamp feature vector can also contain a plurality of other normalized operation and maintenance index values according to the actual service requirement.
And step one, a machine learning algorithm can be selected to construct an index prediction model according to actual service requirements. Preferably, the invention can specifically adopt a multi-element linear fitting algorithm to construct an index prediction model, namely, the calculation formula of the index prediction model is as follows: f (x) 1 ,x 2 ,…x n )＝α 1 ×x 1 +α 2 ×x 2 +…+α n ×x n +β, where x 1 、x 2 、…、x n Respectively time stamp feature vectorsEach element value of a, alpha 1 、α 2 、…、α n Respectively, coefficient to be fitted, beta is constant to be fitted, and alpha is obtained through calculation by continuous training of sample data in each historical detection period 1 、α 2 、…、α n 、β。
In the second step, the interval of time can be set to be 5 seconds, so that the abnormality detection can be carried out on the actual operation and maintenance index values acquired in 5 seconds every 5 seconds, and the real-time performance of the method is effectively ensured. The number N of consecutive time stamp feature vectors acquired over a period of time may be 1, 2 or 3. Determining the operation and maintenance index threshold range corresponding to each time stamp according to the operation and maintenance index predicted values of all the continuous time stamps may further include:
step 21, calculating the average value Y of the operation and maintenance index predicted values of all the continuous time stamps av ：Then calculate the index reference dynamic value σ for all consecutive timestamps: />Y z 、Y z-1 The operation and maintenance index predicted values of the z < th > and the z < 1 > time stamps respectively, and N is the total number of all continuous time stamps;
step 22, calculating the operation and maintenance index change rate of each time stamp:ct z the operation and maintenance index change rate of the z-th time stamp is calculated, and then the absolute ratio of the change trend of each time stamp is calculated: />ct_abs_normal z Is the absolute ratio of the change trend of the z-th time stamp, |ct z I is ct z Absolute value of Y max Is the maximum value of the operation and maintenance index predicted values of all continuous time stamps, and finally according to the operation and maintenance index change rate and change of each time stampCalculating the absolute ratio of chemical trend to obtain dynamic value parameters of each time stamp: />
Step 23, determining an operation and maintenance index threshold range of each timestamp: [ S ] z-min ,S z-max ]Wherein, [ S ] z-min ,S z-max ]Is the operation and maintenance index threshold range of the z-th time stamp, S z-min ＝Y z -(1-a z )*σ，S z-max ＝Y z +(1-a z )*σ。
Through the steps, the threshold range can be dynamically adjusted according to the change trend of the operation and maintenance index, specifically, the change trend is obvious, the threshold range can be automatically expanded, the change trend is not obvious, and the threshold range can be automatically reduced; and the adjustment direction of the threshold range can be automatically determined according to the arithmetic symbols, the positive sign represents the upward fine adjustment, the negative sign represents the downward fine adjustment, and the whole dynamic threshold range is adjusted upward or downward, so that the automatic threshold adjustment is truly realized, and the manual parameter configuration is not needed.
It should be noted that, in order to effectively improve the detection accuracy after the training of the index prediction model, preferably, the historical detection period extracted in the step one may be the last detection period before the current detection period, so that the index prediction model is continuously adaptively adjusted and trained according to the change of the actual service data in the last detection period at the beginning of each detection period, thereby detecting and alarming the actual operation and maintenance index value acquired in the current detection period in real time.
Referring to fig. 2, fig. 2 is a flowchart of a specific step of updating an abnormal index in sample data when the index prediction model is trained by extracting a time stamp feature vector and an actual operation and maintenance index value in a history detection period as sample data in the first step of fig. 1, so that training of the index prediction model is more accurate, and the method includes:
a1, screening actual operation and maintenance index values in a history detection period by adopting a DBSCAN algorithm, and marking abnormal indexes;
a2, extracting all marked abnormal indexes, and forming an abnormal index sequence by a plurality of continuous abnormal indexes with time stamps;
the abnormal indexes in the step A2 not only comprise the abnormal indexes in the step A1, but also comprise the abnormal indexes marked by the method in the history detection period, a plurality of abnormal index sequences can be obtained by the step A2, and the time stamps of all the abnormal indexes in each abnormal index sequence are continuous;
step A3, updating all abnormal indexes in each abnormal index sequence one by one: firstly, selecting the earliest timestamp and the latest timestamp from each abnormal index sequence, and then acquiring the actual operation and maintenance index values respectively acquired by the two timestamps before the earliest timestamp and after the latest timestamp from a database: r is R i1 、R i2 ，R i1 、R i2 The actual operation and maintenance index values of the two time stamps before the earliest time stamp and after the latest time stamp in the ith abnormal index sequence are respectively calculated, and then the index mean value of each abnormal index sequence is calculated:R i-av is the index mean value of the ith abnormal index sequence, and calculates the filling threshold value of each abnormal index sequence: />Wherein, len i Is the number of abnormal indexes in the ith abnormal index sequence, and finally, each abnormal index in each abnormal index sequence is updated: />Wherein R is ij Is the operation and maintenance index value after the j abnormal index in the i abnormal index sequence is updated.
Example two
Referring to fig. 3, fig. 3 is a schematic structural diagram of a dynamic threshold abnormality detection system according to a second embodiment of the present invention, including:
the index prediction model construction device is used for constructing an index prediction model, wherein the input of the index prediction model is a time stamp feature vector, the output of the index prediction model is an operation and maintenance index prediction value corresponding to the input time stamp, and the time stamp feature vector and the actual operation and maintenance index value in a historical detection period are extracted to serve as sample data to train the index prediction model;
the index anomaly prediction device is used for acquiring actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous timestamp feature vectors acquired in a certain time interval into a trained index prediction model to obtain an operation and maintenance index prediction value of each input timestamp, determining an operation and maintenance index threshold range corresponding to each timestamp according to the operation and maintenance index prediction values of all continuous timestamps, and marking the operation and maintenance index value of one timestamp as an anomaly index and alarming when the actual operation and maintenance index value acquired by the timestamp is not in the operation and maintenance index threshold range.
The detection period in the present application may be a day or a week, and the time stamp feature vector may include: whether holiday, busy or idle, working day, timestamp ID, etc., wherein, whether holiday, busy or idle can be judged by calendar, if the hour value in the timestamp is 23:00-06:00, when busy and the rest are idle, the timestamp ID is a value obtained by sorting all timestamps in each detection period in time sequence and normalizing each sorted sequence number, and the timestamp feature vector can also contain a plurality of other normalized operation and maintenance index values according to actual service requirements.
The index prediction model construction device can select a machine learning algorithm to construct an index prediction model, and specifically adopts a multi-element linear fitting algorithm to construct the index prediction model, namely, the calculation formula of the index prediction model is as follows: f (x) 1 ,x 2 ,…x n )＝α 1 ×x 1 +α 2 ×x 2 +…+α n ×x n +β, where x 1 、x 2 、…、x n Each element value, alpha, in the time stamp feature vector 1 、α 2 、…、α n Respectively, coefficients to be fitted, beta being coefficients to be fittedThe constant of the combination is calculated and obtained by continuous training of sample data in each historical detection period 1 、α 2 、…、α n 、β。
The index anomaly prediction device further comprises:
an index threshold calculating unit for calculating the average value Y of the operation and maintenance index predicted values of all the collected continuous time stamps av ：And calculates the index reference dynamic value sigma of all consecutive time stamps: />Y z 、Y z-1 The operation and maintenance index predicted values of the z < th > and the z < 1 > are respectively calculated, N is the total number of all continuous time stamps, and then the operation and maintenance index change rate of each time stamp is calculated: />ct z The operation and maintenance index change rate of the z-th time stamp is calculated, and the absolute change trend ratio of each time stamp is calculated: />ct_abs_normal z Is the absolute ratio of the change trend of the z-th time stamp, |ct z I is ct z Absolute value of Y max The operation and maintenance index prediction value is the maximum value of all continuous time stamps, and finally, the dynamic value parameter of each time stamp is calculated and obtained according to the operation and maintenance index change rate and the change trend absolute ratio of each time stamp: />a z Is the dynamic value parameter of the z-th timestamp, thereby determining the operation and maintenance index threshold range of each timestamp: [ S ] z-min ,S z-max ]Wherein, [ S ] z-min ,S z-max ]Is the operation and maintenance index threshold range of the z-th time stamp, S z-min ＝Y z -(1-a z )*σ，S z-max ＝Y z +(1-a z )*σ。
The index prediction model construction device further comprises:
the abnormal index screening unit is used for screening the actual operation and maintenance index values in the historical detection period by adopting a DBSCAN algorithm and marking the abnormal index;
the model sample processing unit extracts all marked abnormal indexes, forms a plurality of abnormal indexes with continuous time stamps into an abnormal index sequence, and then updates all abnormal indexes in each abnormal index sequence one by one: firstly, selecting the earliest timestamp and the latest timestamp from each abnormal index sequence, and then acquiring the actual operation and maintenance index values respectively acquired by the two timestamps before the earliest timestamp and after the latest timestamp from a database: r is R i1 、R i2 ，R i1 、R i2 The actual operation and maintenance index values of two time stamps before the earliest time stamp and after the latest time stamp in the ith abnormal index sequence are respectively calculated, and the index mean value of each abnormal index sequence is calculated:R i-av is the index mean value of the ith abnormal index sequence, and calculates the filling threshold value of each abnormal index sequence: />Wherein, len i Is the number of abnormal indexes in the ith abnormal index sequence, and finally, each abnormal index in each abnormal index sequence is updated: />Wherein R is ij Is the operation and maintenance index value after the j abnormal index in the i abnormal index sequence is updated.
Furthermore, an embodiment of the present invention proposes a storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described method.
In addition, the embodiment of the invention also provides intelligent equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the method when executing the program.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (12)
1. The dynamic threshold abnormality detection method is characterized by comprising the following steps:
firstly, constructing an index prediction model, wherein the input of the index prediction model is a time stamp feature vector, the output of the index prediction model is an operation and maintenance index prediction value corresponding to an input time stamp, and the time stamp feature vector and an actual operation and maintenance index value in a historical detection period are extracted to serve as sample data to train the index prediction model;
and secondly, acquiring actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous time stamp feature vectors acquired in a certain time interval into a trained index prediction model to obtain operation and maintenance index predicted values of each input time stamp, determining an operation and maintenance index threshold range corresponding to each time stamp according to the operation and maintenance index predicted values of all continuous time stamps, and marking the operation and maintenance index values of one time stamp as abnormal indexes and alarming when the actual operation and maintenance index value acquired by the time stamp is not in the operation and maintenance index threshold range.
2. The method of claim 1, wherein the time stamp feature vector comprises: whether the time stamp ID is a numerical value obtained by sequencing all time stamps in each detection period according to time sequence and normalizing each sequenced serial number, and the time stamp feature vector also comprises a plurality of other normalized operation and maintenance index values.
3. The method of claim 1, wherein the first step of constructing the index prediction model by using a multiple linear fitting algorithm is performed by the following calculation formula: f (x) 1 ,x 2 ,…x n )＝α 1 ×x 1 +α 2 ×x 2 +…+α n ×x n +β, where x 1 、x 2 、…、x n Each element value, alpha, in the time stamp feature vector 1 、α 2 、…、α n Respectively the coefficients to be fitted, and beta is the constant to be fitted.
4. The method according to claim 1, wherein in the second step, the operation index threshold range corresponding to each timestamp is determined according to the operation index predicted values of all the continuous timestamps, including:
step 21, calculating the average value Y of the operation and maintenance index predicted values of all the continuous time stamps av ：Then calculate the index reference dynamic value σ for all consecutive timestamps: />Y z 、Y z-1 The operation and maintenance index predicted values of the z < th > and the z < 1 > time stamps respectively, and N is the total number of all continuous time stamps;
step 22, calculating the operation and maintenance index change rate of each time stamp:ct z the operation and maintenance index change rate of the z-th time stamp is calculated, and then the absolute ratio of the change trend of each time stamp is calculated: />ct_abs_normal z Is the absolute ratio of the change trend of the z-th time stamp, |ct z I is ct z Absolute value of Y max The operation and maintenance index prediction value is the maximum value of all continuous time stamps, and finally, the dynamic value parameter of each time stamp is calculated and obtained according to the operation and maintenance index change rate and the change trend absolute ratio of each time stamp: />a z Is the dynamic value parameter of the z-th timestamp;
step 23, determining an operation and maintenance index threshold range of each timestamp: [ S ] z-min ,S 2-max ]Wherein, [ S ] 2-min ,S z-max ]Is the operation and maintenance index threshold range of the z-th time stamp, S z-min ＝Y z -(1-a z )*σ，S z-max ＝Y z +(1-a z )*σ。
5. The method according to claim 1, wherein in the first step, when the index prediction model is trained by extracting the time stamp feature vector and the actual operation and maintenance index value in the history detection period as sample data, the method further updates the abnormal index in the sample data, including:
a1, extracting all marked abnormal indexes, and forming an abnormal index sequence by a plurality of continuous abnormal indexes with time stamps;
step A2, updating all abnormal indexes in each abnormal index sequence one by one: firstly, selecting the earliest timestamp and the latest timestamp from each abnormal index sequence, and then acquiring the actual operation and maintenance index values respectively acquired by the two timestamps before the earliest timestamp and after the latest timestamp from a database: r is R i1 、R i2 ，R i1 、R i2 The actual operation and maintenance index values of the two time stamps before the earliest time stamp and after the latest time stamp in the ith abnormal index sequence are respectively calculated, and then the index mean value of each abnormal index sequence is calculated:R i-av is the index mean value of the ith abnormal index sequence, and calculates the filling threshold value of each abnormal index sequence: />Wherein, len i Is the number of abnormal indexes in the ith abnormal index sequence, and finally, each abnormal index in each abnormal index sequence is updated: />Wherein R is ij Is in the ith abnormal index sequenceThe operation and maintenance index value after the j-th abnormal index is updated.
6. A dynamic threshold anomaly detection system, comprising:
the index prediction model construction device is used for constructing an index prediction model, wherein the input of the index prediction model is a time stamp feature vector, the output of the index prediction model is an operation and maintenance index prediction value corresponding to the input time stamp, and the time stamp feature vector and the actual operation and maintenance index value in a historical detection period are extracted to serve as sample data to train the index prediction model;
the index anomaly prediction device is used for acquiring actual operation and maintenance index values in a current detection period in real time, inputting a plurality of continuous timestamp feature vectors acquired in a certain time interval into a trained index prediction model to obtain an operation and maintenance index prediction value of each input timestamp, determining an operation and maintenance index threshold range corresponding to each timestamp according to the operation and maintenance index prediction values of all continuous timestamps, and marking the operation and maintenance index value of one timestamp as an anomaly index and alarming when the actual operation and maintenance index value acquired by the timestamp is not in the operation and maintenance index threshold range.
7. The system of claim 6, wherein the timestamp feature vector comprises: whether the time stamp ID is a numerical value obtained by sequencing all time stamps in each detection period according to time sequence and normalizing each sequenced serial number, and the time stamp feature vector also comprises a plurality of other normalized operation and maintenance index values.
8. The system according to claim 6, wherein the index prediction model constructing means adopts a multiple linear fitting algorithm to construct the index prediction model, that is, the calculation formula of the index prediction model is: f (x) 1 ,x 2 ,…x n )＝α 1 ×x 1 +α 2 ×x 2 +…+α n ×x n +β, where x 1 、x 2 、…、x n Each element value, alpha, in the time stamp feature vector 1 、α 2 、…、α n Respectively the coefficients to be fitted, and beta is the constant to be fitted.
9. The system according to claim 6, wherein the index anomaly prediction means includes:
an index threshold calculating unit for calculating the average value Y of the operation and maintenance index predicted values of all the collected continuous time stamps av ：And calculates the index reference dynamic value sigma of all consecutive time stamps: />Y z 、Y z-1 The operation and maintenance index predicted values of the z < th > and the z < 1 > are respectively calculated, N is the total number of all continuous time stamps, and then the operation and maintenance index change rate of each time stamp is calculated: />ct z The operation and maintenance index change rate of the z-th time stamp is calculated, and the absolute change trend ratio of each time stamp is calculated: />ct_abs_normal z Is the absolute ratio of the change trend of the z-th time stamp, |ct z I is ct z Absolute value of Y max The operation and maintenance index prediction value is the maximum value of all continuous time stamps, and finally, the dynamic value parameter of each time stamp is calculated and obtained according to the operation and maintenance index change rate and the change trend absolute ratio of each time stamp: />a z Is the dynamic value parameter of the z-th timestamp, thereby determining the operation and maintenance index of each timestampThreshold range: [ S ] z-min ,S z-max ]Wherein, [ S ] z-min ,S z-max ]Is the operation and maintenance index threshold range of the z-th time stamp, S z-min ＝Y z -(1-a z )*σ，S z-max ＝Y z +(1-a z )*σ。
10. The system according to claim 6, wherein the index prediction model constructing means includes:
the model sample processing unit extracts all marked abnormal indexes, forms a plurality of abnormal indexes with continuous time stamps into an abnormal index sequence, and then updates all abnormal indexes in each abnormal index sequence one by one: firstly, selecting the earliest timestamp and the latest timestamp from each abnormal index sequence, and then acquiring the actual operation and maintenance index values respectively acquired by the two timestamps before the earliest timestamp and after the latest timestamp from a database: r is R i1 、R i2 ，R i1 、R i2 The actual operation and maintenance index values of two time stamps before the earliest time stamp and after the latest time stamp in the ith abnormal index sequence are respectively calculated, and the index mean value of each abnormal index sequence is calculated:R i-av is the index mean value of the ith abnormal index sequence, and calculates the filling threshold value of each abnormal index sequence: />Wherein, len i Is the number of abnormal indexes in the ith abnormal index sequence, and finally, each abnormal index in each abnormal index sequence is updated: />Wherein R is ij Is the operation and maintenance index value after the j abnormal index in the i abnormal index sequence is updated.
11. A storage medium having stored thereon a computer program, which when executed by a processor, implements a method according to any of claims 1 to 5.
12. A smart device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 5 when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311535202.1A CN117454295A (en) | 2023-11-17 | 2023-11-17 | Dynamic threshold abnormality detection method, system, storage medium and intelligent device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311535202.1A CN117454295A (en) | 2023-11-17 | 2023-11-17 | Dynamic threshold abnormality detection method, system, storage medium and intelligent device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117454295A true CN117454295A (en) | 2024-01-26 |
Family
ID=89585354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311535202.1A Pending CN117454295A (en) | 2023-11-17 | 2023-11-17 | Dynamic threshold abnormality detection method, system, storage medium and intelligent device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117454295A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118536046A (en) * | 2024-07-18 | 2024-08-23 | 青岛鼎信通讯股份有限公司 | Method, device, electric energy meter and medium for identifying abnormal jump of electric energy meter sampling signal |
-
2023
- 2023-11-17 CN CN202311535202.1A patent/CN117454295A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118536046A (en) * | 2024-07-18 | 2024-08-23 | 青岛鼎信通讯股份有限公司 | Method, device, electric energy meter and medium for identifying abnormal jump of electric energy meter sampling signal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111475804B (en) | Alarm prediction method and system | |
| CN110807024B (en) | Dynamic threshold anomaly detection method and system, storage medium and intelligent device | |
| CN110888788B (en) | Abnormality detection method, abnormality detection device, computer device, and storage medium | |
| US7779467B2 (en) | N grouping of traffic and pattern-free internet worm response system and method using N grouping of traffic | |
| CN111641519B (en) | Abnormal root cause positioning method, device and storage medium | |
| CN111506478A (en) | Method for realizing alarm management control based on artificial intelligence | |
| CN112183758A (en) | Method and device for realizing model training and computer storage medium | |
| CN111177485B (en) | Parameter rule matching based equipment fault prediction method, equipment and medium | |
| Pacella et al. | Using recurrent neural networks to detect changes in autocorrelated processes for quality monitoring | |
| CN112769605B (en) | Heterogeneous multi-cloud operation and maintenance management method and hybrid cloud platform | |
| CN110738255A (en) | device state monitoring method based on clustering algorithm | |
| CN106953766B (en) | Alarm method and device | |
| CN113574480B (en) | Device for predicting equipment damage | |
| CN113516174B (en) | Call chain abnormality detection method, computer device, and readable storage medium | |
| CN111368980A (en) | State detection method, device, equipment and storage medium | |
| CN111753875A (en) | Power information system operation trend analysis method and device and storage medium | |
| CN117454295A (en) | Dynamic threshold abnormality detection method, system, storage medium and intelligent device | |
| CN113705726A (en) | Traffic classification method and device, electronic equipment and computer readable medium | |
| JP2023520066A (en) | Data processing for industrial machine learning | |
| CN114338351B (en) | Network anomaly root cause determination method and device, computer equipment and storage medium | |
| KR101960755B1 (en) | Method and apparatus of generating unacquired power data | |
| CN114202009A (en) | Medical equipment performance index abnormity detection method and device based on PU learning | |
| CN114138601A (en) | Service alarm method, device, equipment and storage medium | |
| CN117193088B (en) | Industrial equipment monitoring method and device and server | |
| CN113656452A (en) | Method and device for detecting abnormal index of call chain, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |