CN117854014B - Automatic capturing and analyzing method for comprehensive abnormal phenomenon - Google Patents
Automatic capturing and analyzing method for comprehensive abnormal phenomenon Download PDFInfo
- Publication number
- CN117854014B CN117854014B CN202410264045.3A CN202410264045A CN117854014B CN 117854014 B CN117854014 B CN 117854014B CN 202410264045 A CN202410264045 A CN 202410264045A CN 117854014 B CN117854014 B CN 117854014B
- Authority
- CN
- China
- Prior art keywords
- data
- image
- abnormal
- time
- temperature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 151
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 121
- 230000033001 locomotion Effects 0.000 claims abstract description 80
- 230000005856 abnormality Effects 0.000 claims abstract description 71
- 230000008859 change Effects 0.000 claims abstract description 35
- 238000007781 pre-processing Methods 0.000 claims abstract description 34
- 238000004458 analytical method Methods 0.000 claims abstract description 32
- 238000000605 extraction Methods 0.000 claims abstract description 25
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 18
- 230000011218 segmentation Effects 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims description 54
- 238000001514 detection method Methods 0.000 claims description 24
- 238000012360 testing method Methods 0.000 claims description 24
- 238000012544 monitoring process Methods 0.000 claims description 20
- 238000009826 distribution Methods 0.000 claims description 19
- 239000013598 vector Substances 0.000 claims description 19
- 238000004836 empirical method Methods 0.000 claims description 14
- 239000000203 mixture Substances 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 7
- 208000012661 Dyskinesia Diseases 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 238000012706 support-vector machine Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000009825 accumulation Methods 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 230000000737 periodic effect Effects 0.000 claims description 3
- 230000003595 spectral effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 8
- 230000007613 environmental effect Effects 0.000 abstract description 7
- 238000005286 illumination Methods 0.000 abstract description 6
- 230000004044 response Effects 0.000 abstract description 5
- 238000010892 electric spark Methods 0.000 description 26
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/478—Contour-based spectral representations or scale-space representations, e.g. by Fourier analysis, wavelet analysis or curvature scale-space [CSS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an automatic capturing and analyzing method for comprehensive abnormal phenomena, which adjusts the updating speed of a background model through a dynamic learning rate, adapts to environmental illumination and dynamic background change, integrates various background modeling methods and dynamic weight coefficients, and realizes accurate background modeling and foreground extraction; through feature segmentation and multi-level network structure feature extraction, the fine structure and the large-scale structure of the image are comprehensively described, and the feature description accuracy is improved; by combining acoustic data preprocessing and characteristic parameter extraction, the image technology is assisted to detect the abnormality, so that the capturing accuracy of the abnormality is improved; the abnormal motion and state of the equipment are timely found out by analyzing the equipment motion data in real time, and the response speed and instantaneity of the system are improved; the abnormal electric energy and temperature parameters of the equipment are captured and analyzed through a real-time abnormality index analysis algorithm, so that comprehensive equipment state analysis is realized.
Description
Technical Field
The invention relates to the technical field of capturing and data analysis, in particular to an automatic capturing and analyzing method for comprehensive abnormal phenomena.
Background
An anomaly refers to a deviation of the behavior of a system, device, or software from expected or normal behavior under a particular environment or condition; traditional anomaly capture methods rely primarily on manual observation and detection. For example, a system administrator or maintenance personnel captures anomalies by viewing system logs, monitoring dashboards, or directly observing system behavior; traditional anomaly capturing and analyzing methods mainly rely on manual work, and have the problems of low efficiency, poor accuracy, poor timeliness, high complexity and strong subjectivity, and with the development of computer technology and data analysis technology, more and more automatic tools and methods are applied to capturing and analyzing anomalies, and the technologies provide higher efficiency, better accuracy and stronger automation capability.
For example, KR102595182B1 "image abnormality detection method" provides a "method of detecting an abnormality in an object using an image of the object generated in a factory", and an image modeling unit receives an input image of the object and generates an output image. A modeling generating step of calculating a plurality of parameters from the image modeling generating step and generating a multiple distribution composed of the plurality of parameters, and an abnormality detecting unit, the multiple distribution and the multiple distribution of the multiple distribution step may include an abnormality determining step to detect an abnormality "in an object or an image using a threshold, but the method mainly performs abnormality detection based on image data without considering other data sources, which means that in some cases, contributions of other perception data such as sound, vibration, etc. to abnormality detection may be ignored, resulting in a decrease in detection accuracy; and the method uses only image modeling and multiple distributions to determine anomalies, in complex scenarios, various anomaly types may not be fully captured and distinguished.
Disclosure of Invention
In order to solve the problems existing in the prior art, the application provides an automatic capturing and analyzing method for comprehensive abnormal phenomena.
The technical scheme of the application is as follows:
An automatic capture and analysis method for comprehensive anomalies, the method comprising:
Selecting monitoring equipment to monitor test equipment in a test process in real time to obtain real-time data of the monitoring equipment, and preprocessing the real-time data of the monitoring equipment, wherein the real-time data of the monitoring equipment comprises image data, sound wave data, motion data, electrical parameters and temperature data;
Performing acoustic anomaly detection according to the acoustic data after data preprocessing, if acoustic anomalies, performing background modeling on the image data after data preprocessing, extracting a foreground, performing feature extraction by using the foreground, performing anomaly identification according to the extracted features, and recording the acoustic data and the image data when anomalies exist;
Detecting abnormal movement according to the movement data after the data preprocessing, capturing the abnormal parameters according to the electric parameters and the temperature data after the data preprocessing, and recording the movement data, the electric parameters and the temperature data when the abnormality exists;
Judging whether the existing abnormality has regularity according to the sound wave data, the image data, the motion data, the electric parameters and the temperature data when the abnormality exists, and generating corresponding analysis and evaluation.
Preferably, the acoustic wave abnormality detection according to the acoustic wave data after the data preprocessing specifically includes:
converting the sound wave data after data preprocessing into a frequency domain by utilizing Fourier transformation, and extracting frequency components and amplitude values from frequency domain signals to serve as characteristic parameters of sound waves;
calculating and comparing Euclidean distance between the extracted acoustic wave characteristic parameters and acoustic wave characteristic parameters under the normal working state of test equipment; and presetting an acoustic wave threshold value, and judging that the acoustic wave is abnormal when the calculated Euclidean distance exceeds the acoustic wave threshold value.
Preferably, background modeling of the data-preprocessed image data is specifically:
Carrying out graying treatment on the image data subjected to data pretreatment to obtain a gray value of an image, initializing a background model by using the gray value of the image, and expressing as follows:
；
In the method, in the process of the invention, Is the gray value of the ith pixel point in the first image and is taken as the initial value/>, of the background modelThe first image is derived from a sequence of image data;
calculating dynamic learning rate according to gray value of image Expressed as:
；
In the method, in the process of the invention, At time/>, for the ith pixelDynamic learning rate of (2); /(I)Is the basic learning rate; /(I)To adjust parameters; At time/>, for the ith pixel Gray values of the image of (a); /(I)At time/>, for the ith pixelGray values of the image of (a);
respectively calculating the ith pixel point in the image window The average gray value and the median gray value in the range are expressed as:
；
；
In the method, in the process of the invention, Is the image window/>, of an image in an image sequenceAverage gray value of the ith pixel point in the image; Is the image window/>, of an image in an image sequence The median gray value of the ith pixel point in the image;
Establishing a Gaussian mixture model to express the probability distribution of gray values of the ith pixel point, wherein the probability distribution is expressed as the formula:
；
In the method, in the process of the invention, Is the probability distribution of the gray value of the ith pixel point; /(I)Is the number of gaussian components; /(I)IsThe weights, means and variances of the individual gaussian components.
Preferably, the extraction prospect is specifically:
Combining the dynamic learning rate, the mean value, the median value and the Gaussian mixture model, comprehensively updating the background model of each pixel point in the image data, and expressing as follows by a formula:
；
In the method, in the process of the invention, Is a weight coefficient;
comparing the gray value of the pixel point with the comprehensively updated background model for each image in the image sequence, judging the image as a foreground if the difference value is larger than a threshold value, and expressing the gray value as:
；
In the method, in the process of the invention, To be at timeWhen the ith pixel point is a binary representation of the foreground, 1 is the foreground, and 0 is the background;
repeating the steps of calculating dynamic learning rate, modeling the mean value and the median background, modeling the Gaussian mixture model, comprehensively updating the background model and extracting the foreground, and updating the background model and extracting the foreground in real time.
Preferably, the feature extraction by using the foreground specifically includes feature segmentation of the extracted foreground image, including labeling a communication region of the foreground image after binarization processing of the foreground image, and feature extraction of a micro-level and a macro-level of the labeled communication region by using a multi-level network structure, wherein the micro-level features include local density and local heterogeneity, the macro-level features include region contrast and region uniformity, and the features of the micro-level and the macro-level are combined into a comprehensive feature vector.
Preferably, the abnormality recognition based on the extracted features specifically includes:
Calculating the similarity of the comprehensive feature vector and the abnormal phenomenon feature vector by using the cosine similarity, if the similarity exceeds a preset threshold, marking that the abnormal phenomenon exists in the image area corresponding to the comprehensive feature vector, constructing an abnormal phenomenon image data set, and finishing preliminary screening;
and carrying out positioning marking on the abnormal phenomenon image data set by using threshold segmentation and morphological operation, extracting image features from a target area of each positioning marking, constructing an abnormal feature library, and carrying out feature matching on the image features and the abnormal feature library to confirm whether the corresponding abnormal phenomenon exists in the target area.
Preferably, the feature matching specifically includes:
Calculating similarity of color histograms of image features and abnormal feature library by using Papanicolaou distance And comparing the differences in color moment;
calculating the frequency difference between the image features and the abnormal occurrence in the abnormal feature library by using a difference method ；
Calculating similarity of image features and abnormal time-varying features in abnormal feature library by using dynamic time warping algorithm；
Defining a weighting factorSatisfy；
Using similarity of color histogramsFrequency differenceSimilarity to time-varying characteristicsThe comprehensive similarity is calculated and expressed as:
；
In the method, in the process of the invention, To synthesize similarity,Weights of color, frequency and time-varying characteristics, respectively;
presetting an image similarity threshold IfJudging that the corresponding abnormal phenomenon exists;
and confirming whether the target area has an abnormal phenomenon or not through matching the image features with the abnormal feature library.
Preferably, the abnormal motion detection according to the motion data after the data preprocessing specifically includes:
According to three-dimensional space positioning historical data of normal movement of test equipment, an abnormal movement detection model is established, a normal movement range is calculated based on statistical characteristics of the normal movement data, and a threshold value of the normal movement range is set;
Training an abnormal motion detection model by using a support vector machine algorithm according to the historical data of the normal motion data and the abnormal motion data of the test equipment and the set normal motion range threshold value to obtain a trained abnormal motion detection model;
inputting the real-time motion data after data preprocessing into an abnormal motion detection model after training, and when the abnormal motion detection model judges that the motion data of the test equipment exceeds a normal range, generating abnormal motion of the test equipment.
Preferably, capturing the abnormal parameters according to the electrical parameters and the temperature data after the data preprocessing specifically comprises:
Initial thresholds for setting current, voltage and temperature ；
Initializing an abnormality indexExpressed as:
；
Acquiring electric energy parameter data and temperature data after data preprocessing, including current VoltageAnd temperature；
Calculating real-time deviations of current, voltage and temperature、、Expressed as:
；
；
；
In the method, in the process of the invention, 、、Respectively representing the deviation of the current, the voltage and the temperature from the corresponding threshold values;
The sum of squares of the deviations and the rate of change of each parameter are calculated as:
In the method, in the process of the invention, Is the sum of squares of the deviations; /(I)、、Representing the rates of change of current, voltage and temperature, respectively;
calculating a correlation coefficient among the current, the voltage and the temperature, and expressing the correlation coefficient as:
；
；
；
In the method, in the process of the invention, 、、Respectively representing the correlation coefficients between the current and the voltage, the current and the temperature, and the voltage and the temperature;
frequency domain characteristics of current, voltage and temperature obtained by Fourier transform and energy spectral density calculation ；
Calculating an abnormality index based on the deviation, the rate of change, the correlation coefficient, and the frequency domain characteristicsExpressed as:
；
In the method, in the process of the invention, Is a weight coefficient; /(I)Is an abnormality index;
If abnormality index And if the warning value exceeds the warning value preset according to an empirical method, judging that the corresponding electric energy parameter data and temperature data are abnormal parameters.
Preferably, judging whether the existing abnormality has regularity according to the sound wave data, the image data, the motion data, the electrical parameters and the temperature data when the abnormality exists, and generating a corresponding analysis evaluation report specifically includes:
determining whether there is an abnormal concentration within a specific time period or time interval by the time interval and duration of the acoustic wave data, the image data, the motion data, the electrical parameters and the temperature data when the abnormality exists;
analyzing the distribution of the abnormal data in space through the position of pixels in the image data when the abnormality exists or the position of a sensor corresponding to the acoustic wave data when the abnormality exists, and determining whether the abnormal accumulation of a specific area or position exists or not;
analyzing corresponding variation trend through the motion data, the electrical parameters and the temperature data when the abnormality exists, and judging whether the motion data, the electrical parameters and the temperature data when the abnormality exists show gradually increasing or decreasing trend or whether periodic fluctuation exists;
And generating a corresponding analysis evaluation report according to the analysis of the corresponding time, space and change trend.
Compared with the prior art, the invention has the beneficial effects that:
1) The invention provides a comprehensive anomaly automatic capturing and analyzing method, in the background modeling process of image data, a dynamic learning rate, a mean value, a median value and a Gaussian mixture model are used for comprehensively updating a background model, the updating speed of the background model can be adaptively adjusted by an algorithm through the introduction of the dynamic learning rate, so that the method is better suitable for environmental illumination change and dynamic background change, and by integrating various background modeling methods and combining dynamic weight coefficients, the method can realize accurate background modeling and foreground extraction in various scenes;
2) The invention provides an automatic capturing and analyzing method for comprehensive abnormal phenomena, which realizes comprehensive description of fine structures and large-scale structures in images by carrying out feature segmentation and multi-level network structure feature extraction on a foreground, adopts various feature extraction methods, such as local density, local heterogeneity, regional contrast, regional uniformity and the like, can accurately reveal the structures and changes in the images, and improves the description accuracy of the features;
3) The invention provides a comprehensive abnormal phenomenon automatic capturing and analyzing method, wherein an abnormal recognition part marks an image area with similarity exceeding a threshold value as possible to have an abnormal phenomenon by calculating cosine similarity of a feature vector and an abnormal phenomenon feature vector, and further accurately judges the abnormal phenomenon by matching a feature library;
The invention provides an automatic capturing and analyzing method for comprehensive abnormal phenomena, which is characterized in that abnormal motion detection is carried out through three-dimensional space position data, and real-time analysis of motion data is carried out through establishing a model so as to detect the abnormality of equipment; by utilizing analysis of the electrical parameters and the temperature data, a real-time abnormality index analysis algorithm is introduced to calculate the abnormality index of the equipment, so that the real-time capturing and analysis of the equipment abnormality parameters are realized, the working state of the equipment is evaluated, potential abnormality problems are found, and the response speed and the real-time performance of the method are improved.
Drawings
FIG. 1 is a flow chart of a method of an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
The invention provides the following technical scheme: an automatic capturing and analyzing method for comprehensive abnormal phenomena.
Example 1
As shown in fig. 1, the embodiment provides a method for automatically capturing and analyzing comprehensive abnormal phenomena, which specifically includes the following steps:
s1, selecting monitoring equipment to monitor test equipment in a test process in real time, obtaining real-time data of the monitoring equipment, and preprocessing the real-time data of the monitoring equipment, wherein the real-time data of the monitoring equipment comprises image data, sound wave data, motion data, electrical parameters and temperature data;
The monitoring equipment comprises a camera, an acoustic wave sensor, a three-dimensional sensor, an electric parameter and a temperature sensor;
S11, selecting cameras with high resolution and high response speed, determining the number and the installation positions of the cameras according to the size and the shape of a monitoring area, adjusting the angles and the focal lengths of the cameras to ensure that all areas needing to be monitored can be covered, and further capturing image data of the test equipment in a running process in real time by the cameras;
S12, an acoustic wave sensor with high selection sensitivity and high anti-interference capability is arranged at a key part of test equipment, and further captures acoustic wave data generated when the equipment operates in real time;
S13, positioning a three-dimensional sensor with high selection precision and good stability, determining the installation position of the three-dimensional sensor according to the size and shape of test equipment, and adjusting parameters of the three-dimensional sensor to ensure that the three-dimensional movement of the test equipment can be accurately captured, and further capturing the position and movement data of the equipment in a three-dimensional space in real time by the three-dimensional sensor;
S14, selecting installation positions of the electric parameters and the temperature sensors, and ensuring that the electric parameters and the temperature data of the equipment can be accurately captured, and further, capturing the electric parameters (current and voltage) and the temperature data of the equipment in real time by the electric parameters and the temperature sensors;
After the monitoring equipment is selected and installed, the test environment is optimized, the position and the gray level of the light source are adjusted, shadows and light reflection are reduced, and the definition of the image is ensured; cleaning a monitoring area, ensuring that no irrelevant object enters the field of view of the camera, marking the boundary of the equipment, and avoiding the equipment from entering the blind area of the camera in the moving process; according to the size of the environmental noise, the sensitivity of the acoustic wave sensor is adjusted, so that the acoustic wave sensor can accurately capture acoustic wave signals of the equipment under various environmental conditions;
performing data preprocessing on real-time data of monitoring equipment, including:
Performing simple denoising treatment on the image data to obtain image data after data preprocessing; converting the sound wave data from an analog signal to a digital signal, filtering to remove noise interference, and obtaining sound wave data after data preprocessing; calibrating the motion data by a contrast calibration method and performing filtering treatment to obtain motion data after data preprocessing; performing analog-to-digital conversion on the electric parameters and the temperature data, and performing data calibration through a comparison calibration method to obtain the electric parameters and the temperature data after data preprocessing;
Data basis is provided for capturing and analyzing the subsequent abnormal phenomenon through data preprocessing of the data;
s2, detecting acoustic anomalies according to the acoustic data after data preprocessing, and recording the acoustic data when anomalies exist;
Converting the sound wave data after data preprocessing from a time domain to a frequency domain by utilizing Fourier transformation, and extracting main frequency components and corresponding amplitudes thereof from frequency domain signals to serve as characteristic parameters of sound waves; calculating and comparing Euclidean distance between the extracted acoustic wave characteristic parameters and acoustic wave characteristic parameters in a normal working state; setting a threshold according to an empirical method, judging as abnormal sound waves when the calculated Euclidean distance exceeds the threshold, and assisting an image technology to detect the abnormality;
S3, if the acoustic wave data are abnormal, carrying out background modeling on the image data after the data preprocessing, extracting a foreground, and carrying out feature extraction by utilizing the foreground to carry out abnormal recognition;
s31, background modeling;
S311, initializing a background model;
Carrying out graying treatment on the image data subjected to data pretreatment to obtain an image data format suitable for subsequent treatment, namely pixel data of an image;
Initializing a background model by using gray values of the first image, and expressing the background model as:
；
In the method, in the process of the invention, Is the gray value of the ith pixel point in the first image and is taken as the initial value/>, of the background modelThe first image is derived from an image data sequence, and the initialization background model lays a foundation for updating a subsequent background model and extracting a foreground;
Furthermore, in order to better adapt to the change of ambient light and the change of dynamic background, the accuracy of foreground extraction is improved, and for the ith pixel point in different images, the dynamic learning rate is calculated based on the gray level difference value between the ith pixel point and the previous background model in the image data sequence Expressed as:
；
In the method, in the process of the invention, At time/>, for the ith pixelThe dynamic learning rate of the pixel point is used for controlling the updating speed of the background model and is derived from the gray level change condition of the pixel point; /(I)The basic learning rate is a constant preset according to an empirical method and is used for controlling the basic value of the learning rate; /(I)For adjusting parameters, the learning rate is controlled to be a constant preset according to an empirical method so as to adapt to the change of different scenes; /(I)At time/>, for the ith pixelGray values of the image of (a); /(I)At time/>, for the ith pixelGray values of the image of (a);
s312, modeling a mean value and a median background;
respectively calculating the ith pixel point in the image window Average and median gray values within and as part of a background model, expressed as:
；
；
In the method, in the process of the invention, Is the average gray value of the ith pixel, derived from the image windowA sequence of images within; Is the median gray value of the ith pixel point and is derived from the image window/> A sequence of images within; the two methods can eliminate the influence of temporary noise and illumination variation in the background model;
s313, modeling a Gaussian mixture model;
Establishing a Gaussian mixture model to express the probability distribution of gray values of the ith pixel point, wherein the probability distribution is expressed as the formula:
In the method, in the process of the invention, The probability distribution of gray values of the ith pixel point is represented by a Gaussian mixture model; /(I)The number of Gaussian components is a constant preset according to an empirical method; /(I)IsThe weight, the mean value and the variance of the Gaussian components are updated in real time through an EM algorithm so as to adapt to the dynamic change of the background;
s32, extracting a prospect;
S321, comprehensively updating a background model;
Combining the dynamic learning rate, the mean value, the median value and the Gaussian mixture model, comprehensively updating the background model of each pixel point in the image data, and expressing as follows by a formula:
；
In the method, in the process of the invention, The method is characterized in that the method comprises the steps of adjusting contribution degrees of different models according to a constant preset by an empirical method to realize more accurate background modeling;
S322, comparing the gray value of the pixel point with the comprehensively updated background model for each image, judging the image to be foreground if the difference value is larger than a threshold value, and expressing the foreground as follows by a formula:
In the method, in the process of the invention, To be at timeWhen the ith pixel point is a binary representation of the foreground, 1 is the foreground, and 0 is the background; through the steps, the dynamic foreground in the image sequence can be extracted in real time;
Further, each image in the image sequence is subjected to comprehensive cyclic updating, and the steps of calculating dynamic learning rate, mean value and median background modeling, gaussian mixture model modeling, comprehensively updating background models and extracting the foreground are repeatedly executed, so that the background models are updated in real time and the foreground is extracted, and the abnormal phenomenon can be accurately identified in various scenes;
According to the embodiment, the update speed of the background model can be adaptively adjusted by introducing the dynamic learning rate, so that the method is better suitable for environmental illumination change and dynamic background change, and accurate background modeling and foreground extraction can be realized in various scenes by integrating various background modeling methods and combining dynamic weight coefficients.
S33, extracting features;
Further, the foreground obtained in the steps is subjected to feature segmentation, and the specific process is as follows:
S331, performing binarization processing on a foreground image;
S3311, threshold selection;
In the first step of image binarization processing, a suitable thresholding method needs to be selected, and this embodiment provides two thresholding methods, including a global thresholding method and an adaptive thresholding method, in which:
the global threshold method is suitable for the situation that the difference between the background gray level and the foreground gray level is large, and a proper global threshold T is selected by calculating the gray level histogram of the whole image;
The self-adaptive threshold algorithm is suitable for the situation that the gray level difference between the background and the foreground is inconsistent, and the self-adaptive threshold algorithm can calculate the gray level average value or the median value of the local neighborhood of each pixel in the image as a threshold value;
selecting a proper threshold method according to the specific condition of the image;
S3312, threshold application;
After the threshold value is available, the threshold value is applied, and each pixel in the image is determined whether the pixel belongs to the foreground or the background according to the gray value and the corresponding threshold value, so that the binarization processing of the image is completed;
S332, analyzing and marking the communication area;
Initializing a marking matrix of the image, ensuring that the marks of all pixel points are initially 0, and laying a foundation for the subsequent communication area marks;
Starting from the upper left corner of the image, scanning each pixel row by row and column by column, checking the marking condition of the pixel points in the neighborhood of each foreground pixel, and carrying out corresponding region marking and merging;
preferably, in the whole scanning process, all the region merging information needs to be recorded, so that each connected region is ensured to have a unique mark;
Further, in order to accurately and comprehensively extract the features of all the connected areas, for each marked connected area, a multi-level network structure is constructed according to the embodiment, and the steps for constructing the multi-level network structure specifically include:
at the microscopic level, defining the pixel point of each communication area Is a node of the network, whereinFor the row and column coordinates of the pixel points in the image, calculating the gray difference weight/>, of the adjacent pixel points in spaceExpressed as:
；
In the method, in the process of the invention, Respectively gray values of two adjacent pixel points in space; /(I)AndThe weight adjustment parameters are obtained according to an empirical method; /(I)The pixel point acquisition time difference; the microstructure of the image multi-level network structure is constructed in the process, the similarity and the difference between pixel points are revealed through the gray level difference weight, and a foundation is laid for subsequent feature extraction;
At the macro level, each sub-region is defined Is a sub-network in whichFor the row and column indexes of the subareas in the image, calculating the gray difference weight/>, of the adjacent subareasExpressed as:
In the method, in the process of the invention, Is the gray average value of two adjacent sub-areas; /(I)AndThe weight adjustment parameters are obtained according to an empirical method; /(I)The sub-region acquisition time difference aims at constructing a macro structure of an image multi-level network structure, and the correlation among the regions is revealed through the gray level difference weight among the sub-regions, so that a foundation is provided for extracting macro features;
further, micro-level features and macro-level features of the multi-level network structure features are calculated:
At the microscopic level, local density and local heterogeneity are calculated, wherein:
Local density representation node Surrounding average connection weightExpressed as:
；
local heterogeneity representation node Variability of surrounding connection weightsExpressed as:
；
In the method, in the process of the invention, Is the average of the weights around the node; /(I)Is the spatial distance between nodes; /(I)AndIs an adjustment parameter, and is determined according to an empirical method; /(I)Representing nodesIs the number of neighbor nodes, i.e. AND nodeThe total number of other nodes directly connected; the process extracts the local features of the image from the microscopic level, reveals the fine structure and change in the image, and lays a foundation for comprehensive feature analysis;
at the macro level, the area contrast and area uniformity are calculated, wherein:
the area contrast represents the average gray scale difference of a sub-network and its neighboring sub-network Expressed as:
；
Region uniformity represents variability in gray-level mean across all sub-networks Expressed as:
；
In the method, in the process of the invention, Is the average value of the gray level average value of all the subnetworks; /(I)Is the spatial distance between networks; /(I)AndIs an adjustment parameter; /(I)Is a subnetworkIs the number of adjacent sub-areas, i.e. with sub-networkThe total number of adjacent other subregions; the process extracts the regional characteristics of the image from the macroscopic level, reveals the large-scale structure and change of the image, and provides more comprehensive information for comprehensive characteristic analysis;
further, combining features of micro-level and macro-level into a comprehensive feature vector Expressed as:
；
The process integrates the characteristics of different layers to form a comprehensive characteristic vector so as to comprehensively describe the multi-layer structure characteristics of the image;
finally, to ensure that each feature is in the same dimension, the feature vector Each featureNormalization was performed and expressed as:
；
finally, outputting the image characteristics of each normalized multi-level network structure Component eigenvectors; The normalization processing eliminates the dimensional difference among the features, so that the contribution of each feature in the model is balanced, and the stability and accuracy of the model are improved;
The application realizes the comprehensive description of the internal microstructure and the large-scale structure of the image by carrying out feature segmentation and multi-level network structure feature extraction on the foreground, and can accurately reveal the internal structure and change of the image by adopting various feature extraction methods such as local density, local heterogeneity, regional contrast, regional uniformity and the like, thereby improving the description accuracy of the features.
S34, performing anomaly identification by using the features extracted in the step S33, and recording image data when anomalies exist;
S341, preliminary screening;
Firstly, obtaining the feature vector of each image area by a multi-level network structure method based on the step S33 For each image region, its eigenvector/>, is calculated using a cosine similarity formulaAnd anomaly feature vectorIf the similarity exceeds a threshold set according to an empirical method, marking the image area as possibly having an abnormal phenomenon, and constructing an abnormal phenomenon image data set to complete preliminary screening; /(I)
Preferably, the anomaly characteristic vectorObtained by the following means:
Collecting a certain number of image samples with known abnormal phenomena, and extracting the characteristics of each abnormal sample by adopting a multi-level network structure method to obtain the abnormal phenomenon characteristic vector of each sample ; New abnormal samples are continuously collected along with the time and the development of technology, the characteristic vectors of the samples are extracted, and the characteristic vector library is updated;
s342, abnormality identification;
Recording information such as the position, time and the like of an image area marked as possibly having an abnormal phenomenon in the abnormal phenomenon image data set;
s3421, constructing an abnormal feature library, performing feature matching on the image features and the abnormal feature library to confirm whether a corresponding abnormal phenomenon exists in a target area, and particularly, in the embodiment, establishing an electric spark feature library to identify the electric spark abnormal phenomenon, wherein the establishing process specifically comprises the following steps:
s34211, extracting visual characteristics, namely extracting main shape characteristics of electric sparks by using a contour approximation method and calculation of geometric moments, and quantifying size and regularity characteristics by calculating the area, perimeter and shape compactness of an electric spark region;
S34212, applying a Canny algorithm and an edge direction histogram to accurately describe edge information and edge direction distribution;
S34213, analyzing the color characteristics, calculating and normalizing a color histogram under an RGB color space to eliminate the influence of illumination, and simultaneously calculating first-order, second-order and third-order color moments to comprehensively describe the color mean, variance and skewness of electric sparks;
s34214, measuring the flicker frequency of the electric spark, counting the occurrence times of the electric spark in a window by setting a fixed time window, such as 1 second, so as to calculate the flicker frequency, further recording the change of the frequency, analyzing the stability and the periodicity of the frequency, and providing a basis for subsequent feature matching;
S34215, analyzing the brightness time-varying characteristics of the electric spark, calculating the average brightness of the electric spark area in each frame of image, constructing a time sequence of the brightness, applying Fourier transformation, and deeply analyzing the frequency components of the brightness change to reveal the time-varying rule of the electric spark brightness;
through the operation, all the characteristics of the electric spark are comprehensively extracted and stored in the characteristic library, so that a solid foundation is laid for subsequent electric spark identification and anomaly detection;
S3422, in order to further confirm whether the electric spark abnormal phenomenon exists, extracting deeper features of the initially screened image area which is likely to be the abnormal phenomenon, and matching the extracted features with features in an electric spark feature library to accurately judge whether the foreground objects are actually abnormal phenomena such as electric sparks or not, wherein the method comprises the following steps of:
S34221, performing positioning marking on the initially screened image areas which are likely to be abnormal phenomena by applying threshold segmentation and morphological operation, and extracting image features of each marked target area by adopting the same method as a corresponding abnormal feature library, wherein the embodiment adopts the same method as an electric spark feature library;
S34222, performing feature matching, firstly calculating the similarity between the image features and the color histogram of the electric spark feature library by using the Pasteur distance Bhattacharyya distance Comparing the differences in color moment;
calculating flicker frequency difference between image features and electric sparks in electric spark feature library by using difference method ；
Calculating similarity of brightness time-varying characteristics of electric sparks in image features and electric spark feature library by using dynamic time warping algorithm；
Further, the weighting factors are defined according to an empirical methodSatisfy；
The comprehensive similarity is calculated and expressed as:
；
In the method, in the process of the invention, To synthesize similarity,Weights of color, flicker frequency, and luminance time-varying characteristics, respectively;
setting a proper threshold according to historical data and experience IfJudging that the electric spark abnormality exists;
through matching with the electric spark feature library, whether the electric spark abnormality exists in the target area is confirmed;
according to the application, through preprocessing of sound wave data and characteristic parameter extraction, the image technology is assisted to detect the abnormality, so that the capturing accuracy of the abnormal electric spark phenomenon is further improved.
S4, detecting abnormal movement according to the movement data after the data preprocessing, and recording the movement data when the abnormality exists;
According to the three-dimensional space positioning historical data of normal movement of the test equipment, an abnormal movement detection model is established by utilizing the existing neural network, wherein the existing neural network comprises a multi-layer perceptron, a long-short-term memory network, a convolution neural network or a self-encoder, and based on the statistical characteristics of the normal movement data, the normal movement range is calculated, and the threshold value of the normal movement range is set; training an abnormal motion detection model by using a Support Vector Machine (SVM) algorithm according to historical data of normal motion data and abnormal motion data of the equipment and a set normal motion range threshold;
in order to analyze equipment motion data in real time, timely finding and confirming abnormal motion of equipment, inputting the preprocessed real-time motion data into an abnormal motion detection model, and when the model judges that the equipment motion data exceeds a normal range, testing the equipment to perform abnormal motion;
S5, capturing abnormal parameters according to the electrical parameters and the temperature data after the data preprocessing, and recording the electrical parameters and the temperature data when the abnormality exists;
In order to more accurately realize instantaneous electric parameter and temperature monitoring, the embodiment introduces a real-time abnormality index analysis algorithm, wherein the core idea of the algorithm is to calculate the abnormality index of equipment in real time and dynamically evaluate the working state of the equipment, thereby timely finding out potential abnormality problems, and the specific implementation process is as follows:
S51, initial setting, namely setting an initial threshold and a weight coefficient according to equipment specifications and safety standards;
initial thresholds are set for current, voltage and temperature: ，，；
Initializing an abnormality index ：；
S52, acquiring the preprocessed electric energy parameter data and temperature data; current of the device after time t pre-treatmentVoltageAnd temperature；
S53, calculating an expansion deviation, and calculating a real-time deviation, a deviation square sum and a change rate;
calculating real-time deviations of current, voltage and temperature 、、Expressed as:
；
；
；
In the method, in the process of the invention, 、、Respectively representing the deviation of the current, the voltage and the temperature from the threshold values thereof, and is used for evaluating the real-time working state of the equipment; /(I)
The sum of squares of the deviations and the rate of change of each parameter are calculated as:
In the method, in the process of the invention, Is the sum of squares of the deviations, used to quantify the overall degree of abnormality of the device; /(I)、、Representing the rates of change of current, voltage and temperature, respectively, for monitoring the instantaneous change of the state of the device;
S54, correlation and characteristic analysis, namely calculating correlation coefficients among parameters and analyzing frequency domain energy characteristics;
calculating a correlation coefficient among the current, the voltage and the temperature, and expressing the correlation coefficient as:
；
；
；
In the method, in the process of the invention, 、、The correlation coefficients between the current and the voltage, between the current and the temperature and between the voltage and the temperature are respectively represented and used for evaluating the mutual influence among the parameters, and a Pearson correlation coefficient calculation method is adopted;
analysis of frequency domain characteristics of current, voltage and temperature using fourier transform and energy spectral density ；
S55, calculating an abnormality index, integrating the characteristics, and calculating the abnormality index;
Calculating an abnormality index based on the deviation, the rate of change, the correlation coefficient, and the frequency domain characteristics Expressed as:
；
In the method, in the process of the invention, The weight coefficient is obtained according to an empirical method; /(I)Is an abnormality index, comprehensively considers deviation, change rate, correlation and frequency domain characteristics of each parameter, and quantifies the abnormality degree of equipment;
S56, abnormality judgment, namely judging the equipment state according to the abnormality index, and dynamically adjusting a threshold value and a weight coefficient;
If abnormality index If the warning value exceeds the warning value preset according to an empirical method, judging that the vehicle is abnormal;
Particularly, the threshold value, the weight coefficient and the warning value can be dynamically adjusted according to the real-time working state and the environmental change so as to adapt to the actual running condition of the equipment;
S6, judging whether the existing abnormality has regularity according to the sound wave data, the image data, the motion data, the electrical parameters and the temperature data when the abnormality exists, and generating a corresponding analysis evaluation report;
The regularity analysis comprises time pattern analysis of abnormal data, spatial distribution analysis of the abnormal data and change trend analysis of the abnormal data, wherein:
determining whether there is an abnormal concentration within a specific time period or time interval by the time interval and duration of the acoustic wave data, the image data, the motion data, the electrical parameters and the temperature data when the abnormality exists;
analyzing the distribution of the abnormal data in space through the position of pixels in the image data when the abnormality exists or the position of a sensor corresponding to the acoustic wave data when the abnormality exists, and determining whether the abnormal accumulation of a specific area or position exists or not;
analyzing corresponding variation trend through the motion data, the electrical parameters and the temperature data when the abnormality exists, and judging whether the motion data, the electrical parameters and the temperature data when the abnormality exists show gradually increasing or decreasing trend or whether periodic fluctuation exists;
And generating a corresponding analysis evaluation report according to the analysis of the corresponding time, space and change trend.
According to the application, through analyzing the motion data and the electric energy parameter data of the equipment in real time, the abnormal motion and the abnormal state of the equipment can be found and confirmed in time, the response speed and the instantaneity of the system are improved, the real-time capturing and analysis of the abnormal parameters of the equipment are realized through a real-time abnormal index analysis algorithm, and the comprehensive and comprehensive analysis of the state of the equipment is realized.
The technical scheme of the embodiment can effectively solve the technical problems that abnormal phenomena in various test processes are not captured fully and accurately, and the method is subjected to a series of effect investigation, and the algorithm can adaptively adjust the updating speed of a background model through the introduction of a dynamic learning rate, so that the method is better suitable for environmental illumination change and dynamic background change, and can realize accurate background modeling and foreground extraction in various scenes by integrating various background modeling methods and combining dynamic weight coefficients; the comprehensive description of the internal fine structure and the large-scale structure of the image is realized by carrying out feature segmentation and multi-level network structure feature extraction on the foreground, and the internal structure and the change of the image can be accurately revealed by adopting various feature extraction methods such as local density, local heterogeneity, regional contrast, regional uniformity and the like, so that the description accuracy of the features is improved; the method has the advantages that through pretreatment of sound wave data and extraction of characteristic parameters, the image technology is assisted to detect the abnormality, and the capturing accuracy of abnormal electric spark phenomenon is further improved; by analyzing the equipment motion data and the electric energy parameter data in real time, the abnormal motion and abnormal state of the equipment can be found and confirmed in time, the response speed and the instantaneity of the system are improved, the real-time capturing and analysis of the equipment abnormal parameters are realized through a real-time abnormal index analysis algorithm, and the comprehensive and comprehensive analysis of the equipment state is realized.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present invention.
Claims (8)
1. An automatic capturing and analyzing method for comprehensive abnormal phenomena, which is characterized by comprising the following steps:
Step 1: selecting monitoring equipment to monitor test equipment in a test process in real time to obtain real-time data of the monitoring equipment, and preprocessing the real-time data of the monitoring equipment, wherein the real-time data of the monitoring equipment comprises image data, sound wave data, motion data, electrical parameters and temperature data;
step 2: performing acoustic anomaly detection according to acoustic data after data preprocessing, if acoustic anomalies, performing background modeling on the image data after data preprocessing, extracting a foreground, performing feature extraction by using the foreground, performing anomaly identification according to the extracted features, and recording the acoustic data and the image data when anomalies exist, wherein the background modeling on the image data after data preprocessing specifically comprises the following steps:
Carrying out graying treatment on the image data subjected to data pretreatment to obtain a gray value of an image, initializing a background model by using the gray value of the image, and expressing as follows:
；
In the method, in the process of the invention, Is the gray value of the ith pixel point in the first image and is taken as the initial value/>, of the background modelThe first image is derived from a sequence of image data;
calculating dynamic learning rate according to gray value of image Expressed as:
；
In the method, in the process of the invention, At time/>, for the ith pixelDynamic learning rate of (2); /(I)Is the basic learning rate; /(I)To adjust parameters; /(I)At time/>, for the ith pixelGray values of the image of (a); /(I)At time/>, for the ith pixelGray values of the image of (a);
respectively calculating the ith pixel point in the image window The average gray value and the median gray value in the range are expressed as:
；
；
In the method, in the process of the invention, Is the image window/>, of an image in an image sequenceAverage gray value of the ith pixel point in the image; Is the image window/>, of an image in an image sequence The median gray value of the ith pixel point in the image;
Establishing a Gaussian mixture model to express the probability distribution of gray values of the ith pixel point, wherein the probability distribution is expressed as the formula:
；
In the method, in the process of the invention, Is the probability distribution of the gray value of the ith pixel point; /(I)Is the number of gaussian components; /(I)IsThe weights, means and variances of the individual gaussian components;
The extraction prospect is specifically as follows:
Combining the dynamic learning rate, the mean value, the median value and the Gaussian mixture model, comprehensively updating the background model of each pixel point in the image data, and expressing as follows by a formula:
；
In the method, in the process of the invention, Is a weight coefficient;
comparing the gray value of the pixel point with the comprehensively updated background model for each image in the image sequence, judging the image as a foreground if the difference value is larger than a threshold value, and expressing the gray value as:
；
In the method, in the process of the invention, To be at timeWhen the ith pixel point is a binary representation of the foreground, 1 is the foreground, and 0 is the background;
Repeatedly executing the steps of calculating dynamic learning rate, modeling the mean value and the median background, modeling the Gaussian mixture model, comprehensively updating the background model and extracting the foreground, and updating the background model and extracting the foreground in real time;
step 3: detecting abnormal movement according to the movement data after the data preprocessing, capturing the abnormal parameters according to the electric parameters and the temperature data after the data preprocessing, and recording the movement data, the electric parameters and the temperature data when the abnormality exists;
step 4: judging whether the existing abnormality has regularity according to the sound wave data, the image data, the motion data, the electric parameters and the temperature data when the abnormality exists, and generating a corresponding analysis evaluation report.
2. The method for automatically capturing and analyzing comprehensive anomalies according to claim 1, wherein the acoustic anomalies are detected according to the acoustic data pre-processed by the data, specifically:
converting the sound wave data after data preprocessing into a frequency domain by utilizing Fourier transformation, and extracting frequency components and amplitude values from frequency domain signals to serve as characteristic parameters of sound waves;
calculating and comparing Euclidean distance between the extracted acoustic wave characteristic parameters and acoustic wave characteristic parameters under the normal working state of test equipment; and presetting an acoustic wave threshold value, and judging that the acoustic wave is abnormal when the calculated Euclidean distance exceeds the acoustic wave threshold value.
3. The method for automatically capturing and analyzing the comprehensive abnormal phenomenon according to claim 1, wherein the feature extraction by utilizing the foreground is specifically feature segmentation of an extracted foreground image, the method comprises the steps of marking a communication area of the foreground image after binarization processing of the foreground image, and feature extraction of a micro level and a macro level of the marked communication area by utilizing a multi-level network structure, wherein the micro level features comprise local density and local heterogeneity, the macro level features comprise area contrast and area uniformity, and the micro level features and the macro level features are combined into a comprehensive feature vector.
4. The method for automatically capturing and analyzing comprehensive anomalies according to claim 3, wherein the anomaly identification based on the extracted features is specifically:
Calculating the similarity of the comprehensive feature vector and the abnormal phenomenon feature vector by using the cosine similarity, if the similarity exceeds a preset threshold, marking that the abnormal phenomenon exists in the image area corresponding to the comprehensive feature vector, constructing an abnormal phenomenon image data set, and finishing preliminary screening;
and carrying out positioning marking on the abnormal phenomenon image data set by using threshold segmentation and morphological operation, extracting image features from a target area of each positioning marking, constructing an abnormal feature library, and carrying out feature matching on the image features and the abnormal feature library to confirm whether the corresponding abnormal phenomenon exists in the target area.
5. The method for automatically capturing and analyzing comprehensive anomalies according to claim 4, wherein the feature matching is specifically:
Calculating similarity of color histograms of image features and abnormal feature library by using Papanicolaou distance And comparing the differences in color moment;
calculating the frequency difference between the image features and the abnormal occurrence in the abnormal feature library by using a difference method ；
Calculating similarity of image features and abnormal time-varying features in abnormal feature library by using dynamic time warping algorithm；
Defining a weighting factorSatisfy；
Using similarity of color histogramsFrequency differenceSimilarity to time-varying characteristicsThe comprehensive similarity is calculated and expressed as:
；
In the method, in the process of the invention, To synthesize similarity,Weights of color, frequency and time-varying characteristics, respectively;
presetting an image similarity threshold IfJudging that the corresponding abnormal phenomenon exists;
and confirming whether the target area has an abnormal phenomenon or not through matching the image features with the abnormal feature library.
6. The method for automatically capturing and analyzing comprehensive anomalies according to claim 5, wherein the detecting of anomalies according to the data-preprocessed motion data is specifically:
According to three-dimensional space positioning historical data of normal movement of test equipment, an abnormal movement detection model is established, a normal movement range is calculated based on statistical characteristics of the normal movement data, and a threshold value of the normal movement range is set;
Training an abnormal motion detection model by using a support vector machine algorithm according to the historical data of the normal motion data and the abnormal motion data of the test equipment and the set normal motion range threshold value to obtain a trained abnormal motion detection model;
inputting the real-time motion data after data preprocessing into an abnormal motion detection model after training, and when the abnormal motion detection model judges that the motion data of the test equipment exceeds a normal range, generating abnormal motion of the test equipment.
7. The method for automatically capturing and analyzing comprehensive anomalies according to claim 6, wherein capturing anomalies according to the data-preprocessed electrical parameters and temperature data is specifically:
Initial thresholds for setting current, voltage and temperature ；
Initializing an abnormality indexExpressed as:
；
Acquiring electric energy parameter data and temperature data after data preprocessing, including current VoltageAnd temperature；
Calculating real-time deviations of current, voltage and temperature、、Expressed as:
；
；
；
In the method, in the process of the invention, 、、Respectively representing the deviation of the current, the voltage and the temperature from the corresponding threshold values;
The sum of squares of the deviations and the rate of change of each parameter are calculated as:
In the method, in the process of the invention, Is the sum of squares of the deviations; /(I)、、Representing the rates of change of current, voltage and temperature, respectively;
calculating a correlation coefficient among the current, the voltage and the temperature, and expressing the correlation coefficient as:
；
；
；
In the method, in the process of the invention, 、、Respectively representing the correlation coefficients between the current and the voltage, the current and the temperature, and the voltage and the temperature;
frequency domain characteristics of current, voltage and temperature obtained by Fourier transform and energy spectral density calculation ；
Calculating an abnormality index based on the deviation, the rate of change, the correlation coefficient, and the frequency domain characteristicsExpressed as:
；
In the method, in the process of the invention, Is a weight coefficient; /(I)Is an abnormality index;
If abnormality index And if the warning value exceeds the warning value preset according to an empirical method, judging that the corresponding electric energy parameter data and temperature data are abnormal parameters.
8. The method for automatically capturing and analyzing comprehensive anomalies according to claim 7, wherein the step of determining whether the anomalies are regular according to the sound wave data, the image data, the motion data, the electrical parameters and the temperature data when the anomalies are present, and the step of generating a corresponding analysis evaluation report is specifically:
determining whether there is an abnormal concentration within a specific time period or time interval by the time interval and duration of the acoustic wave data, the image data, the motion data, the electrical parameters and the temperature data when the abnormality exists;
analyzing the distribution of the abnormal data in space through the position of pixels in the image data when the abnormality exists or the position of a sensor corresponding to the acoustic wave data when the abnormality exists, and determining whether the abnormal accumulation of a specific area or position exists or not;
analyzing corresponding variation trend through the motion data, the electrical parameters and the temperature data when the abnormality exists, and judging whether the motion data, the electrical parameters and the temperature data when the abnormality exists show gradually increasing or decreasing trend or whether periodic fluctuation exists;
And generating a corresponding analysis evaluation report according to the analysis of the corresponding time, space and change trend.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410264045.3A CN117854014B (en) | 2024-03-08 | 2024-03-08 | Automatic capturing and analyzing method for comprehensive abnormal phenomenon |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410264045.3A CN117854014B (en) | 2024-03-08 | 2024-03-08 | Automatic capturing and analyzing method for comprehensive abnormal phenomenon |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117854014A CN117854014A (en) | 2024-04-09 |
| CN117854014B true CN117854014B (en) | 2024-05-31 |
Family
ID=90543809
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410264045.3A Active CN117854014B (en) | 2024-03-08 | 2024-03-08 | Automatic capturing and analyzing method for comprehensive abnormal phenomenon |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117854014B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105184818A (en) * | 2015-09-06 | 2015-12-23 | 山东华宇航天空间技术有限公司 | Video monitoring abnormal behavior detection method and detections system thereof |
| CN107463898A (en) * | 2017-08-01 | 2017-12-12 | 闽江学院 | The stage performance abnormal behavior monitoring method of view-based access control model sensing network |
| KR20210158037A (en) * | 2020-06-23 | 2021-12-30 | 동명대학교산학협력단 | Method for tracking multi target in traffic image-monitoring-system |
| CN116269738A (en) * | 2023-05-25 | 2023-06-23 | 深圳市科医仁科技发展有限公司 | Intelligent control method, device, equipment and storage medium of radio frequency therapeutic apparatus |
| CN116776202A (en) * | 2023-06-20 | 2023-09-19 | 山东交通学院 | Hump shunting band-type brake abnormality monitoring system based on multisource data fusion algorithm |
| CN117115169A (en) * | 2023-10-25 | 2023-11-24 | 宁波吉烨汽配模具有限公司 | Intelligent recognition method for abnormal deformation of surface of die-casting die of automobile part |
| CN117197713A (en) * | 2023-09-08 | 2023-12-08 | 南京敏锐科技有限公司 | Extraction method based on digital video monitoring system |
-
2024
- 2024-03-08 CN CN202410264045.3A patent/CN117854014B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105184818A (en) * | 2015-09-06 | 2015-12-23 | 山东华宇航天空间技术有限公司 | Video monitoring abnormal behavior detection method and detections system thereof |
| CN107463898A (en) * | 2017-08-01 | 2017-12-12 | 闽江学院 | The stage performance abnormal behavior monitoring method of view-based access control model sensing network |
| KR20210158037A (en) * | 2020-06-23 | 2021-12-30 | 동명대학교산학협력단 | Method for tracking multi target in traffic image-monitoring-system |
| CN116269738A (en) * | 2023-05-25 | 2023-06-23 | 深圳市科医仁科技发展有限公司 | Intelligent control method, device, equipment and storage medium of radio frequency therapeutic apparatus |
| CN116776202A (en) * | 2023-06-20 | 2023-09-19 | 山东交通学院 | Hump shunting band-type brake abnormality monitoring system based on multisource data fusion algorithm |
| CN117197713A (en) * | 2023-09-08 | 2023-12-08 | 南京敏锐科技有限公司 | Extraction method based on digital video monitoring system |
| CN117115169A (en) * | 2023-10-25 | 2023-11-24 | 宁波吉烨汽配模具有限公司 | Intelligent recognition method for abnormal deformation of surface of die-casting die of automobile part |
Non-Patent Citations (1)
| Title |
|---|
| 《视频监控中异常场景检测与分析研究》;杨之杰;《优秀硕士论文》;20140101;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117854014A (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902633B (en) | Abnormal event detection method and device based on fixed-position camera monitoring video | |
| Schmugge et al. | Detection of cracks in nuclear power plant using spatial-temporal grouping of local patches | |
| CN116758491B (en) | Printing monitoring image analysis method and system applied to 3D printing | |
| CN117668518B (en) | Discrete intelligent manufacturing method and system | |
| CN111507426A (en) | No-reference image quality grading evaluation method and device based on visual fusion characteristics | |
| CN117576093B (en) | EL image-based solar cell piece hidden crack detection method | |
| CN111402249B (en) | Image evolution analysis method based on deep learning | |
| CN112668754A (en) | Power equipment defect diagnosis method based on multi-source characteristic information fusion | |
| CN106682604B (en) | Blurred image detection method based on deep learning | |
| TWI779957B (en) | Image analysis model establishment method and image analysis apparatus | |
| CN117854014B (en) | Automatic capturing and analyzing method for comprehensive abnormal phenomenon | |
| CN117994236B (en) | Vortex spun broken yarn intelligent detection method and system based on image analysis | |
| CN117147107B (en) | Communication module detection system and detection method | |
| CN113077002B (en) | Machine olfaction visual sensing data analysis method based on space heterodyne Raman spectrum | |
| CN116309493A (en) | Method and system for detecting defects of textile products | |
| KR102292602B1 (en) | Deep learning and image processing based bolt loosening detection method | |
| CN111795982A (en) | High-voltage cable joint internal defect detection method based on X-ray imaging technology | |
| CN112508946A (en) | Cable tunnel abnormity detection method based on antagonistic neural network | |
| CN118470873B (en) | Thermal forming component pose monitoring and early warning method | |
| CN110781758A (en) | Dynamic video monitoring method and device for abnormal pantograph structure | |
| Su et al. | Bridge Pavement Crack Detection Under Uneven Illumination Using Improved Pcnn Algorithm | |
| CN117649387B (en) | Defect detection method suitable for object with complex surface texture | |
| CN118570211B (en) | Water quality abnormality detection analysis method and system based on deep learning | |
| CN118508438B (en) | Photovoltaic module power loss prediction method, system, equipment and storage medium | |
| CN118037715B (en) | Intelligent welding spot analysis optimization method and system for PCBA board |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |