CN117854014B - A comprehensive method for automatically capturing and analyzing abnormal phenomena - Google Patents

Abstract

The invention relates to an automatic capturing and analyzing method for comprehensive abnormal phenomena, which adjusts the updating speed of a background model through a dynamic learning rate, adapts to environmental illumination and dynamic background change, integrates various background modeling methods and dynamic weight coefficients, and realizes accurate background modeling and foreground extraction; through feature segmentation and multi-level network structure feature extraction, the fine structure and the large-scale structure of the image are comprehensively described, and the feature description accuracy is improved; by combining acoustic data preprocessing and characteristic parameter extraction, the image technology is assisted to detect the abnormality, so that the capturing accuracy of the abnormality is improved; the abnormal motion and state of the equipment are timely found out by analyzing the equipment motion data in real time, and the response speed and instantaneity of the system are improved; the abnormal electric energy and temperature parameters of the equipment are captured and analyzed through a real-time abnormality index analysis algorithm, so that comprehensive equipment state analysis is realized.

Description

A comprehensive method for automatically capturing and analyzing abnormal phenomena

Technical Field

The invention relates to the technical field of capture and data analysis, and mainly to a method for automatically capturing and analyzing comprehensive abnormal phenomena.

Background Art

Anomalies refer to the behavior of a system, device, or software that deviates from the expected or normal behavior under a specific environment or condition. Traditional methods of capturing anomalies mainly rely on manual observation and detection. For example, system administrators or maintenance personnel capture anomalies by viewing system logs, monitoring dashboards, or directly observing system behavior. Traditional methods of capturing and analyzing anomalies mainly rely on manual labor, which has problems such as low efficiency, poor accuracy, poor timeliness, high complexity, and strong subjectivity. With the development of computer technology and data analysis technology, more and more automated tools and methods are being applied to the capture and analysis of anomalies. These technologies provide higher efficiency, better accuracy, and stronger automation capabilities.

For example, KR102595182B1 "Image Anomaly Detection Method" provides a "method for detecting anomalies in an object using an image of an object generated in a factory, and an image modeling unit receives an input image of the object and generates an output image. A modeling generation step, a multiple distribution step, for calculating multiple parameters according to the image modeling generation step, and generating a multiple distribution composed of multiple parameters, and an anomaly detection unit, a multiple distribution of the multiple distribution step and an anomaly detection unit The multiple distribution may include an anomaly determination step to use a threshold to detect anomalies in an object or image", but this method mainly performs anomaly detection based on image data without considering other data sources, which means that in some cases, the contribution of other perceptual data such as sound, vibration, etc. to anomaly detection may be ignored, resulting in a decrease in detection accuracy; and this method only uses image modeling and multiple distributions to determine anomalies, and in complex scenarios, it may not be able to fully capture and distinguish various types of anomalies.

Summary of the invention

In order to solve the above problems existing in the prior art, the present application provides a method for automatically capturing and analyzing comprehensive abnormal phenomena.

The technical solution of this application is as follows:

A method for automatically capturing and analyzing comprehensive abnormal phenomena, the method comprising:

Selecting a monitoring device to monitor the test device in real time during the test, obtaining real-time data of the monitoring device, and performing data preprocessing on the real-time data of the monitoring device, wherein the real-time data of the monitoring device includes image data, sound wave data, motion data, and electrical parameters and temperature data;

Performing acoustic anomaly detection based on the acoustic wave data after data preprocessing, if the acoustic wave is abnormal, performing background modeling on the image data after data preprocessing and extracting foreground, extracting features using the foreground, and identifying anomalies based on the extracted features, and recording the acoustic wave data and image data when the anomaly exists;

Perform abnormal motion detection based on the motion data after data preprocessing, capture abnormal parameters based on the electrical parameters and temperature data after data preprocessing, and record the motion data, electrical parameters and temperature data when the abnormality exists;

Based on the sound wave data, image data, motion data, electrical parameters and temperature data when the abnormality occurs, it is determined whether the existing abnormality has regularity, and a corresponding analysis and evaluation is generated.

Preferably, the acoustic wave anomaly detection is performed according to the acoustic wave data after data preprocessing as follows:

The pre-processed acoustic wave data is converted into the frequency domain using Fourier transform, and the frequency components and amplitudes are extracted from the frequency domain signals as characteristic parameters of the acoustic wave;

The Euclidean distance between the extracted acoustic wave characteristic parameters and the acoustic wave characteristic parameters under normal working conditions of the test equipment is calculated and compared; a sound wave threshold is preset, and when the calculated Euclidean distance exceeds the sound wave threshold, it is determined to be an abnormal sound wave.

Preferably, background modeling is performed on the image data after data preprocessing as follows:

The image data after data preprocessing is grayed to obtain the gray value of the image, and the background model is initialized using the gray value of the image, which is expressed as:

;

In the formula, is the gray value of the i-th pixel in the first image, which serves as the initial value of the background model , the first image is derived from an image data sequence;

Calculate dynamic learning rate based on the grayscale value of the image , expressed as:

;

In the formula, is the i-th pixel at time Dynamic learning rate; is the basic learning rate; To adjust the parameters; is the i-th pixel at time The gray value of the image; is the i-th pixel at time The gray value of the image;

Calculate the i-th pixel in the image window respectively The average grayscale value and median grayscale value within are expressed as follows:

;

;

In the formula, is the image window of an image in the image sequence The average gray value of the i-th pixel in ; is the image window of an image in the image sequence The median gray value of the i-th pixel in ;

A Gaussian mixture model is established to represent the probability distribution of the gray value of the i-th pixel, which is expressed as:

;

In the formula, is the probability distribution of the gray value of the i-th pixel; is the number of Gaussian components; It is The weights, means and variances of the Gaussian components.

Preferably, the extraction prospects are specifically:

Combining the dynamic learning rate, mean, median and Gaussian mixture model, the background model of each pixel in the image data is comprehensively updated, which can be expressed as:

;

In the formula, is the weight coefficient;

For each image in the image sequence, the grayscale value of the pixel is compared with the comprehensively updated background model. If the difference is greater than the threshold, it is determined to be the foreground, which can be expressed as:

;

In the formula, For in time When , the binary value indicates whether the i-th pixel is the foreground, 1 indicates it is the foreground, and 0 indicates it is the background;

The steps of calculating a dynamic learning rate, mean and median background modeling, Gaussian mixture model modeling, comprehensively updating the background model and extracting the foreground are repeatedly executed to update the background model and extract the foreground in real time.

Preferably, the feature extraction using the foreground is specifically to perform feature segmentation on the extracted foreground image, including marking the connected areas of the foreground image after binarization of the foreground image, and performing micro-level and macro-level feature extraction on the marked connected areas using a multi-level network structure, wherein the micro-level features include local density and local heterogeneity, and the macro-level features include regional contrast and regional uniformity, and the micro-level and macro-level features are combined into a comprehensive feature vector.

Preferably, the abnormality identification is performed according to the extracted features as follows:

The cosine similarity is used to calculate the similarity between the comprehensive feature vector and the abnormal phenomenon feature vector. If the similarity exceeds the preset threshold, the image area corresponding to the comprehensive feature vector is marked as having an abnormal phenomenon, and an abnormal phenomenon image dataset is constructed to complete the preliminary screening.

The abnormal phenomenon image dataset is located and marked using threshold segmentation and morphological operations, image features are extracted from the target area of each location mark, an abnormal feature library is constructed, and feature matching is performed between the image features and the abnormal feature library to confirm whether there is a corresponding abnormal phenomenon in the target area.

Preferably, the feature matching is specifically:

The similarity between the image features and the color histogram of the abnormal feature library is calculated using the Bhattacharyya distance. , and compare the differences in color moments;

The difference method is used to calculate the frequency difference between the image features and the abnormality feature library when the abnormality occurs ;

The dynamic time warping algorithm is used to calculate the similarity between image features and abnormal time-varying characteristics in the abnormal feature library. ;

Defining weight factors ,satisfy ;

Using the similarity of color histograms , frequency difference Similarity to time-varying characteristics Calculate the comprehensive similarity, expressed as:

;

In the formula, is the comprehensive similarity, are the weights of color, frequency, and time-varying characteristics, respectively;

Preset image similarity threshold ,if , it is determined that the corresponding abnormal phenomenon exists;

By matching the image features with the abnormal feature library, it is confirmed whether there are abnormal phenomena in the target area.

Preferably, abnormal motion detection is performed based on the motion data after data preprocessing as follows:

According to the historical data of the three-dimensional spatial positioning of the normal movement of the test equipment, an abnormal movement detection model is established, and based on the statistical characteristics of the normal movement data, the normal movement range is calculated and the threshold of the normal movement range is set;

According to the historical data of normal motion data and abnormal motion data of the test equipment and the set normal motion range threshold, the abnormal motion detection model is trained by using the support vector machine algorithm to obtain the trained abnormal motion detection model;

The real-time motion data after data preprocessing is input into the trained abnormal motion detection model. When the abnormal motion detection model determines that the motion data of the test equipment exceeds the normal range, the test equipment has abnormal motion.

Preferably, the abnormal parameters are captured according to the electrical parameters and temperature data after data preprocessing as follows:

Set initial thresholds for current, voltage, and temperature ;

Initialize exception index , expressed as:

;

Obtain the electrical energy parameter data and temperature data after data preprocessing, including current ,Voltage and temperature ;

Calculates real-time deviations in current, voltage, and temperature , , , expressed as:

;

;

;

In the formula, , , Represent the deviation of current, voltage and temperature from the corresponding threshold respectively;

Calculate the sum of squares of the deviations and the rate of change of each parameter, expressed as:

In the formula, is the sum of squares of deviations; , , Represent the rate of change of current, voltage and temperature respectively;

Calculate the correlation coefficient between current, voltage and temperature, expressed as:

;

;

;

In the formula, , , Respectively represent the correlation coefficients between current and voltage, current and temperature, and voltage and temperature;

The frequency domain characteristics of current, voltage and temperature are obtained by Fourier transform and energy spectral density calculation. ;

Calculate anomaly index based on deviation, rate of change, correlation coefficient and frequency domain characteristics , expressed as:

;

In the formula, is the weight coefficient; is the anomaly index;

If the abnormal index If the warning value preset according to the empirical method is exceeded, the corresponding electric energy parameter data and temperature data are determined to be abnormal parameters.

Preferably, the acoustic wave data, image data, motion data, electrical parameters and temperature data when the abnormality exists are used to determine whether the abnormality exists with regularity, and the corresponding analysis and evaluation report is generated specifically as follows:

Determine whether there is a concentration of abnormalities within a specific time period or time interval by using the time interval and duration of the acoustic wave data, image data, motion data, electrical parameters, and temperature data when the abnormality exists;

By analyzing the position of the sensor corresponding to the pixel in the image data when the abnormality exists or the acoustic wave data when the abnormality exists, the spatial distribution of the abnormal data is analyzed to determine whether there is an abnormal cluster in a specific area or position;

By analyzing the corresponding change trends of the motion data, electrical parameters and temperature data when the abnormality exists, whether the motion data, electrical parameters and temperature data when the abnormality exists show a trend of gradual increase or decrease, or whether there is periodic fluctuation;

Generate a corresponding analysis and evaluation report based on the analysis of time, space and change trends.

Compared with the prior art, the present invention has the following beneficial effects:

1) The present invention provides a comprehensive method for automatically capturing and analyzing abnormal phenomena. In the process of background modeling of image data, a dynamic learning rate, mean, median and Gaussian mixture model are used to comprehensively update the background model. By introducing a dynamic learning rate, the algorithm can adaptively adjust the update speed of the background model, thereby better adapting to changes in ambient lighting and dynamic background changes. By integrating multiple background modeling methods and combining dynamic weight coefficients, the present invention can achieve accurate background modeling and foreground extraction in various scenarios.

2) The present invention provides a comprehensive method for automatically capturing and analyzing abnormal phenomena. By performing feature segmentation on the foreground and extracting multi-level network structure features, a comprehensive description of the subtle structure and large-scale structure inside the image is achieved. A variety of feature extraction methods, such as local density, local heterogeneity, regional contrast, and regional uniformity, can be used to accurately reveal the structure and changes inside the image and improve the accuracy of feature description.

3) The present invention provides a comprehensive method for automatically capturing and analyzing abnormal phenomena. The abnormality recognition part calculates the cosine similarity between the feature vector and the feature vector of the abnormal phenomenon, marks the image area where the similarity exceeds the threshold as a possible abnormal phenomenon, and further accurately judges the abnormal phenomenon by matching the feature library;

The present invention provides a comprehensive abnormal phenomenon automatic capture and analysis method, which performs abnormal motion detection through three-dimensional spatial position data, and analyzes the motion data in real time by building a model to detect the abnormality of the equipment; utilizes the analysis of electrical parameters and temperature data, introduces a real-time abnormal index analysis algorithm to calculate the abnormal index of the equipment, realizes the real-time capture and analysis of the abnormal parameters of the equipment, evaluates the working status of the equipment, discovers potential abnormal problems, and improves the response speed and real-time performance of the method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method according to an embodiment of the present invention.

DETAILED DESCRIPTION

The specific implementation modes of the present invention are described below to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

The present invention provides the following technical solution: a method for automatically capturing and analyzing comprehensive abnormal phenomena.

Example 1

As shown in FIG1 , this embodiment provides a method for automatically capturing and analyzing comprehensive abnormal phenomena, and the specific steps include:

S1. Select a monitoring device to monitor the test device in real time during the test, obtain real-time data of the monitoring device, and perform data preprocessing on the real-time data of the monitoring device, wherein the real-time data of the monitoring device includes image data, sound wave data, motion data, and electrical parameters and temperature data;

The monitoring equipment includes a camera, an acoustic wave sensor, a three-dimensional sensor, an electrical parameter and a temperature sensor;

S11. Select cameras with high resolution and fast response speed, determine the number and installation location of cameras according to the size and shape of the monitoring area, and adjust the angle and focal length of the cameras to ensure that all areas that need to be monitored can be covered. Furthermore, the cameras capture image data in real time during the operation of the test equipment;

S12. Select an acoustic wave sensor with high sensitivity and strong anti-interference ability and install it in the key parts of the test equipment. Furthermore, the acoustic wave sensor captures the acoustic wave data generated by the equipment in real time during operation;

S13, selecting a three-dimensional sensor with high precision and good stability for positioning, determining the installation position of the three-dimensional sensor according to the size and shape of the test equipment, adjusting the parameters of the three-dimensional sensor to ensure that the three-dimensional movement of the test equipment can be accurately captured, and further, the three-dimensional sensor captures the position and movement data of the equipment in three-dimensional space in real time;

S14, selecting an installation position of an electrical parameter and temperature sensor to ensure that the electrical parameter and temperature data of the device can be accurately captured. Furthermore, the electrical parameter and temperature sensor captures the electrical parameters (current, voltage) and temperature data of the device in real time;

After selecting and installing the above monitoring equipment, optimize the test environment, adjust the position and grayscale of the light source, reduce shadows and reflections, and ensure image clarity; clean the monitoring area to ensure that no irrelevant objects enter the camera's field of view, mark the boundaries of the equipment to prevent the equipment from entering the camera's blind spot during movement; adjust the sensitivity of the acoustic sensor according to the size of the environmental noise to ensure that the acoustic sensor can accurately capture the equipment's acoustic signal under various environmental conditions;

Perform data preprocessing on real-time data of monitoring equipment, including:

Perform simple denoising on the image data to obtain image data after data preprocessing; convert the acoustic wave data from analog signals to digital signals, and perform filtering to remove noise interference to obtain acoustic wave data after data preprocessing; calibrate the motion data by using the comparative calibration method and perform filtering to obtain motion data after data preprocessing; perform analog-to-digital conversion on the electrical parameter and temperature data and perform data calibration by using the comparative calibration method to obtain electrical parameter and temperature data after data preprocessing;

Preprocessing the above data provides data basis for subsequent abnormal phenomenon capture and analysis;

S2. Performing acoustic wave anomaly detection based on the acoustic wave data after data preprocessing, and recording the acoustic wave data when anomalies exist;

The pre-processed acoustic wave data is converted from the time domain to the frequency domain using Fourier transform, and the main frequency components and their corresponding amplitudes are extracted from the frequency domain signal as characteristic parameters of the acoustic wave; the Euclidean distance between the extracted acoustic wave characteristic parameters and the acoustic wave characteristic parameters under normal working conditions is calculated and compared; a threshold is set according to the empirical method, and when the calculated Euclidean distance exceeds the threshold, it is determined to be an abnormal acoustic wave, and the auxiliary image technology is used for abnormality detection;

S3. If the sound wave data is abnormal, background modeling is performed on the image data after data preprocessing and foreground is extracted, and feature extraction is performed using the foreground to identify abnormalities;

S31, background modeling;

S311, initializing the background model;

Grayscale processing is performed on the image data after data preprocessing to obtain an image data format suitable for subsequent processing, that is, pixel data of the image;

The background model is initialized using the grayscale value of the first image, expressed as:

;

In the formula, is the gray value of the i-th pixel in the first image, which serves as the initial value of the background model , the first image is derived from an image data sequence, and the above-mentioned initialized background model lays the foundation for subsequent background model updating and foreground extraction;

Furthermore, in order to better adapt to changes in ambient lighting and dynamic background changes and improve the accuracy of foreground extraction, for the i-th pixel in different images, the dynamic learning rate is calculated based on the grayscale difference between it and the previous background model in the image data sequence. , expressed as:

;

In the formula, is the i-th pixel at time The dynamic learning rate is used to control the update speed of the background model, which is derived from the grayscale change of the pixel points; is the basic learning rate, which is a constant preset according to the empirical method and is used to control the basic value of the learning rate; It is a constant preset by empirical method to adjust the parameter, which controls the speed of change of learning rate to adapt to the changes of different scenarios. is the i-th pixel at time The gray value of the image; is the i-th pixel at time The gray value of the image;

S312, performing mean and median background modeling;

Calculate the i-th pixel in the image window respectively The average grayscale value and median grayscale value in the background are expressed as follows:

;

;

In the formula, is the average gray value of the i-th pixel, derived from the image window The image sequence within; is the median gray value of the i-th pixel, derived from the image window The above two methods can eliminate the influence of temporary noise and illumination changes in the background model;

S313, performing Gaussian mixture modeling;

A Gaussian mixture model is established to represent the probability distribution of the gray value of the i-th pixel, which is expressed as:

In the formula, is the probability distribution of the gray value of the i-th pixel, represented by a Gaussian mixture model; is the number of Gaussian components, which is a constant preset by empirical method; It is The weights, means and variances of the Gaussian components are updated in real time through the EM algorithm to adapt to the dynamic changes of the background;

S32, extracting foreground;

S321, comprehensively update the background model;

Combining the dynamic learning rate, mean, median and Gaussian mixture model, the background model of each pixel in the image data is comprehensively updated, which can be expressed as:

;

In the formula, is the weight coefficient, which adjusts the contribution of different models according to the constants preset by the empirical method to achieve more accurate background modeling;

S322. For each image, the grayscale value of the pixel is compared with the comprehensively updated background model. If the difference is greater than the threshold, it is determined to be the foreground, which is expressed as:

In the formula, For in time The binary representation of whether the i-th pixel is the foreground, 1 indicates that it is the foreground, and 0 indicates that it is the background. Through the above steps, the dynamic foreground in the image sequence can be extracted in real time;

Furthermore, a comprehensive cyclic update is performed for each image in the image sequence, and the steps of calculating a dynamic learning rate, mean and median background modeling, Gaussian mixture modeling, comprehensive updating of the background model, and foreground extraction are repeatedly performed to update the background model and extract the foreground in real time, thereby ensuring that abnormal phenomena can be accurately identified in various scenarios;

By introducing a dynamic learning rate, this embodiment can adaptively adjust the update speed of the background model to better adapt to changes in ambient lighting and dynamic background changes. By integrating multiple background modeling methods and combining dynamic weight coefficients, this embodiment can achieve accurate background modeling and foreground extraction in various scenarios.

S33, feature extraction;

Furthermore, feature segmentation is performed on the foreground obtained in the above steps. The specific process is as follows:

S331, performing binarization processing on the foreground image;

S3311, threshold selection;

In the first step of the image binarization process, it is necessary to select a suitable threshold method. This embodiment provides two threshold methods, including a global threshold method and an adaptive threshold method, where:

The global threshold method is suitable for situations where the grayscale difference between the background and foreground is large. An appropriate global threshold T is selected by calculating the grayscale histogram of the entire image.

The adaptive thresholding method is applicable to the situation where the grayscale difference between the background and foreground is inconsistent. For each pixel in the image, the adaptive thresholding method calculates the grayscale mean or median of its local neighborhood as the threshold;

Choose the appropriate threshold method according to the specific situation of the image;

S3312, threshold application;

After having the threshold, the next step is to apply this threshold to each pixel in the image, according to its gray value and the corresponding threshold, to determine whether the pixel belongs to the foreground or the background, thus completing the binarization of the image;

S332, analysis and marking of connected regions;

Initialize the label matrix of the image to ensure that the labels of all pixels are initially 0, laying the foundation for subsequent connected area labeling;

Starting from the upper left corner of the image, scan each pixel row by row and column by column. For each foreground pixel, check the marking status of the pixels in its neighborhood, and perform corresponding region marking and merging.

Preferably, during the entire scanning process, all region merging information needs to be recorded to ensure that each connected region has a unique label;

Furthermore, in order to accurately and comprehensively extract the features of all connected regions, for each marked connected region, this embodiment constructs a multi-level network structure, and the steps of constructing the multi-level network structure are specifically as follows:

At the micro level, define the pixels of each connected area is a node in the network, where For the row and column coordinates of the pixel in the image, calculate the grayscale difference weight of adjacent pixels in space , expressed as:

;

In the formula, are the grayscale values of two adjacent pixels in space; and is the weight adjustment parameter, obtained by empirical method; is the time difference of pixel acquisition; the above process constructs the microstructure of the multi-level network structure of the image, reveals the similarities and differences between pixels through the grayscale difference weight, and lays the foundation for subsequent feature extraction;

At the macro level, define each sub-area is a subnetwork, where is the row and column index of the sub-region in the image, and calculates the grayscale difference weight of adjacent sub-regions , expressed as:

In the formula, is the grayscale mean of two adjacent sub-regions; and is the weight adjustment parameter, obtained by empirical method; is the time difference of sub-region acquisition. The above process aims to construct the macro structure of the multi-level network structure of the image, reveal the correlation between regions through the grayscale difference weights between sub-regions, and provide a basis for macro feature extraction;

Furthermore, the micro-level features and macro-level features of the multi-level network structure features are calculated:

At the microscopic level, local density and local heterogeneity are calculated, where:

Local density represents nodes The average connection weight around , expressed as:

;

Local heterogeneity represents nodes Variability of surrounding connection weights , expressed as:

;

In the formula, is the average value of weights around the node; is the spatial distance between nodes; and is the adjustment parameter, determined by empirical method; Representation Node The number of neighbor nodes of the node The total number of other directly connected nodes; The above process extracts the local features of the image from the micro level, reveals the subtle structure and changes inside the image, and lays the foundation for comprehensive feature analysis;

At the macroscopic level, regional contrast and regional uniformity are calculated, where:

Regional contrast represents the average grayscale difference between a subnetwork and its adjacent subnetworks , expressed as:

;

Regional uniformity represents the variability of the grayscale mean of all sub-networks , expressed as:

;

In the formula, is the average of the grayscale means of all sub-networks; is the spatial distance between networks; and is the adjustment parameter; It is a subnetwork The number of neighboring sub-regions, that is, the number of neighboring sub-regions of the sub-network The total number of other adjacent sub-regions; the above process extracts the regional features of the image from the macro level, reveals the large-scale structure and changes of the image, and provides more comprehensive information for comprehensive feature analysis;

Furthermore, the features at the micro and macro levels are combined into a comprehensive feature vector , expressed as:

;

The above process is to integrate features at different levels to form a comprehensive feature vector to fully describe the multi-level structural characteristics of the image;

Finally, to ensure that each feature is in the same dimension, the feature vector Each feature in Normalized, expressed as:

;

Finally, the output is the normalized multi-level network structure image features The feature vector composed of ; The above normalization process eliminates the dimensional differences between features, making the contribution of each feature in the model balanced and improving the stability and accuracy of the model;

This application achieves a comprehensive description of the subtle and large-scale structures inside the image by performing feature segmentation on the foreground and extracting multi-level network structure features. It uses a variety of feature extraction methods, such as local density, local heterogeneity, regional contrast, and regional uniformity, which can accurately reveal the structure and changes inside the image and improve the accuracy of feature description.

S34, using the features extracted in step S33 to identify abnormalities, and recording image data when abnormalities exist;

S341, preliminary screening;

First, based on step S33, a feature vector of each image region is obtained by a multi-level network structure method. , for each image region, use the cosine similarity formula to calculate its feature vector and the anomaly feature vector If the similarity exceeds the threshold set by the empirical method, the image area is marked as a possible abnormality, and an abnormality image dataset is constructed to complete the preliminary screening;

Preferably, the abnormal phenomenon feature vector , obtained by the following means:

Collect a certain number of image samples with known abnormal phenomena, use a multi-level network structure method to extract features from each abnormal sample, and obtain the abnormal phenomenon feature vector of each sample. ; As time goes by and technology develops, new abnormal samples are continuously collected, their feature vectors are extracted, and the feature vector library is updated;

S342, abnormality identification;

For image areas marked as possibly having abnormal phenomena in the abnormal phenomenon image dataset, record their location, time and other information;

S3421, construct an abnormal feature library, perform feature matching between the image features and the abnormal feature library to confirm whether there is a corresponding abnormal phenomenon in the target area. In particular, this embodiment establishes an electric spark feature library to identify electric spark abnormal phenomena, and the establishment process is specifically as follows:

S34211. Extract visual characteristics, extract the main shape features of the EDM by using the contour approximation method and the calculation of geometric moments, and quantify the size and regularity characteristics by calculating the area, perimeter and shape compactness of the EDM area;

S34212. Apply the Canny algorithm and edge direction histogram to accurately describe edge information and edge direction distribution;

S34213, focus on analyzing color characteristics, calculate and normalize the color histogram in RGB color space to eliminate the influence of illumination, and calculate the first-order, second-order and third-order color moments to fully describe the color mean, variance and skewness of the EDM;

S34214, determining the flicker frequency of the electric spark, by setting a fixed time window, such as 1 second, counting the number of times the electric spark appears in the window, thereby calculating the flicker frequency, further recording the change of the frequency, analyzing its stability and periodicity, and providing a basis for subsequent feature matching;

S34215. Analyze the time-varying characteristics of spark brightness, calculate the average brightness of the spark area in each frame of the image, and construct a time series of brightness. Apply Fourier transform to deeply analyze the frequency components of brightness changes and reveal the time-varying law of spark brightness.

Through the above operations, the various features of the EDM are fully extracted and stored in the feature library, laying a solid foundation for subsequent EDM recognition and anomaly detection;

S3422. In order to further confirm whether there is an abnormal phenomenon such as electric spark, a more in-depth feature extraction is performed on the image areas that may be abnormal phenomena after preliminary screening, and the extracted features are matched with the features in the electric spark feature library to accurately determine whether these foreground objects are really abnormal phenomena such as electric spark, as follows:

S34221. Apply threshold segmentation and morphological operation to the image areas that may be abnormal phenomena after preliminary screening to perform positioning and marking. For each marked target area, use the same method as the corresponding abnormal feature library to extract image features. This embodiment uses the same method as the electric spark feature library to extract;

S34222, perform feature matching, first use Bhattacharyya distance to calculate the similarity between the image feature and the color histogram of the EDM feature library , compare the differences in color moments;

The difference between the image features and the spark frequency in the spark feature library is calculated using the difference method ;

The dynamic time warping algorithm is used to calculate the similarity between the image features and the time-varying characteristics of the brightness of the spark in the spark feature library. ;

Furthermore, the weight factor is defined according to the empirical method ,satisfy ;

Calculate the comprehensive similarity, expressed as:

;

In the formula, is the comprehensive similarity, are the weights of the time-varying characteristics of color, flicker frequency, and brightness, respectively;

Set an appropriate threshold based on historical data and experience ,if , it is determined that there is an abnormal spark phenomenon;

By matching with the spark feature library, it is possible to confirm whether there is spark anomaly in the target area.

This application further improves the accuracy of capturing abnormal electric spark phenomena by preprocessing acoustic wave data and extracting characteristic parameters, and performing abnormality detection with auxiliary image technology.

S4, performing abnormal motion detection according to the motion data after data preprocessing, and recording the motion data when abnormality exists;

According to the historical data of the three-dimensional spatial positioning of the normal movement of the test equipment, an abnormal movement detection model is established by using an existing neural network, wherein the existing neural network includes a multi-layer perceptron, a long short-term memory network, a convolutional neural network or an autoencoder, and based on the statistical characteristics of the normal movement data, a normal movement range is calculated, and a threshold of the normal movement range is set; according to the historical data of the normal movement data and the abnormal movement data of the equipment and the set normal movement range threshold, a support vector machine (SVM) algorithm is used to train the abnormal movement detection model;

In order to analyze the equipment motion data in real time and promptly discover and confirm the abnormal motion of the equipment, the pre-processed real-time motion data is input into the abnormal motion detection model. When the model determines that the equipment motion data exceeds the normal range, the test equipment has abnormal motion;

S5. Capture abnormal parameters according to the electrical parameters and temperature data after data preprocessing, and record the electrical parameters and temperature data when the abnormality exists;

In order to more accurately realize instantaneous electrical parameter and temperature monitoring, this embodiment introduces a real-time abnormal index analysis algorithm. The core idea of the algorithm is to calculate the abnormal index of the device in real time and dynamically evaluate the working status of the device, so as to timely discover potential abnormal problems. The specific implementation process is as follows:

S51, initial setting, setting the initial threshold and weight coefficient according to the equipment specifications and safety standards;

Set initial thresholds for current, voltage, and temperature: , , ;

Initialize exception index : ;

S52, obtaining pre-processed electric energy parameter data and temperature data; the current of the device after time t pre-processing ,Voltage and temperature ;

S53, extended deviation calculation, calculating the real-time deviation, the square sum of the deviation and the rate of change;

Calculates real-time deviations in current, voltage, and temperature , , , expressed as:

;

;

;

In the formula, , , Respectively represent the deviation of current, voltage and temperature from their thresholds, and are used to evaluate the real-time working status of the device;

Calculate the sum of squares of the deviations and the rate of change of each parameter, expressed as:

In the formula, It is the sum of squares of deviations, used to quantify the overall abnormality of the equipment; , , Represent the rate of change of current, voltage and temperature respectively, and are used to monitor instantaneous changes in equipment status;

S54, correlation and characteristic analysis, calculating the correlation coefficient between parameters and analyzing the frequency domain energy characteristics;

Calculate the correlation coefficient between current, voltage and temperature, expressed as:

;

;

;

In the formula, , , They represent the correlation coefficients between current and voltage, current and temperature, and voltage and temperature, respectively, and are used to evaluate the mutual influence between various parameters, using the Pearson correlation coefficient calculation method;

Analyze the frequency domain characteristics of current, voltage and temperature using Fourier transform and energy spectral density ;

S55, abnormal index calculation, combining various characteristics to calculate the abnormal index;

Calculate anomaly index based on deviation, rate of change, correlation coefficient and frequency domain characteristics , expressed as:

;

In the formula, is the weight coefficient, obtained by empirical method; It is an abnormality index, which comprehensively considers the deviation, rate of change, correlation and frequency domain characteristics of each parameter to quantify the abnormality of the equipment;

S56, abnormality determination, determining the device status according to the abnormality index, and dynamically adjusting the threshold and weight coefficient;

If the abnormal index If it exceeds the warning value preset based on experience, it is judged as abnormal;

In particular, the threshold, weight coefficient and warning value can be dynamically adjusted according to the real-time working status and environmental changes to adapt to the actual operation of the equipment;

S6. Determine whether the abnormality exists in a regular pattern based on the acoustic wave data, image data, motion data, electrical parameters and temperature data when the abnormality exists, and generate a corresponding analysis and evaluation report;

Regularity analysis includes time pattern analysis of abnormal data, spatial distribution analysis of abnormal data, and change trend analysis of abnormal data, among which:

Determine whether there is a concentration of abnormalities within a specific time period or time interval by using the time interval and duration of the acoustic wave data, image data, motion data, electrical parameters, and temperature data when the abnormality exists;

By analyzing the position of the sensor corresponding to the pixel in the image data when the abnormality exists or the acoustic wave data when the abnormality exists, the spatial distribution of the abnormal data is analyzed to determine whether there is an abnormal cluster in a specific area or position;

By analyzing the corresponding change trends of the motion data, electrical parameters and temperature data when the abnormality exists, whether the motion data, electrical parameters and temperature data when the abnormality exists show a trend of gradual increase or decrease, or whether there is periodic fluctuation;

Generate a corresponding analysis and evaluation report based on the analysis of time, space and change trends.

By real-time analysis of equipment motion data and power parameter data, this application can promptly detect and confirm abnormal movement and abnormal status of equipment, thereby improving the response speed and real-time performance of the system. Through the real-time abnormal index analysis algorithm, real-time capture and analysis of abnormal equipment parameters are achieved, thus realizing a comprehensive and integrated analysis of the equipment status.

The technical solution of this embodiment can effectively solve the technical problem of insufficient and inaccurate capture of abnormal phenomena in various test processes. In addition, the above method has undergone a series of effect surveys. By introducing a dynamic learning rate, the algorithm can adaptively adjust the update speed of the background model, so as to better adapt to changes in ambient light and dynamic background changes. By integrating multiple background modeling methods and combining dynamic weight coefficients, the solution can achieve accurate background modeling and foreground extraction in various scenarios. By performing feature segmentation on the foreground and extracting multi-level network structure features, a comprehensive description of the fine structure and large-scale structure inside the image is achieved. By using multiple feature extraction methods, such as local density, local heterogeneity, regional contrast and regional uniformity, the structure and changes inside the image can be accurately revealed, and the accuracy of feature description can be improved. By preprocessing the acoustic wave data and extracting characteristic parameters, the auxiliary image technology is used for abnormal detection, which further improves the accuracy of capturing abnormal electric spark phenomena. By real-time analysis of equipment motion data and electric energy parameter data, the abnormal motion and abnormal state of the equipment can be discovered and confirmed in time, which improves the response speed and real-time performance of the system. The real-time abnormal index analysis algorithm realizes real-time capture and analysis of abnormal parameters of the equipment, and realizes a comprehensive and integrated analysis of the equipment status.

The present invention is described with reference to the flowcharts and/or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the processes and/or boxes in the flowchart and/or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims (8)

1. A method for automatically capturing and analyzing comprehensive abnormal phenomena, characterized in that the method comprises: Step 1: Select a monitoring device to monitor the test device in real time during the test, obtain real-time data of the monitoring device, and perform data preprocessing on the real-time data of the monitoring device, wherein the real-time data of the monitoring device includes image data, sound wave data, motion data, electrical parameters and temperature data; Step 2: Perform acoustic anomaly detection based on the acoustic wave data after data preprocessing. If the acoustic wave is abnormal, perform background modeling on the image data after data preprocessing and extract the foreground, use the foreground to extract features, perform abnormality identification based on the extracted features, and record the acoustic wave data and image data when the abnormality exists. Specifically, the background modeling of the image data after data preprocessing is as follows: The image data after data preprocessing is grayed to obtain the gray value of the image, and the background model is initialized using the gray value of the image, which is expressed as: ; In the formula, is the gray value of the i-th pixel in the first image, which serves as the initial value of the background model , the first image is derived from an image data sequence; Calculate dynamic learning rate based on the grayscale value of the image , expressed as: ; In the formula, is the i-th pixel at time Dynamic learning rate; is the basic learning rate; To adjust the parameters; is the i-th pixel at time The gray value of the image; is the i-th pixel at time The gray value of the image; Calculate the i-th pixel in the image window respectively The average grayscale value and median grayscale value within are expressed as follows: ; ; In the formula, is the image window of an image in the image sequence The average gray value of the i-th pixel in ; is the image window of an image in the image sequence The median gray value of the i-th pixel in ; A Gaussian mixture model is established to represent the probability distribution of the gray value of the i-th pixel, which is expressed as: ; In the formula, is the probability distribution of the gray value of the i-th pixel; is the number of Gaussian components; It is The weights, means and variances of the Gaussian components; The extraction prospects are as follows: Combining the dynamic learning rate, mean, median and Gaussian mixture model, the background model of each pixel in the image data is comprehensively updated, which can be expressed as: ; In the formula, is the weight coefficient; For each image in the image sequence, the grayscale value of the pixel is compared with the comprehensively updated background model. If the difference is greater than the threshold, it is determined to be the foreground, which can be expressed as: ; In the formula, For in time When , the binary value indicates whether the i-th pixel is the foreground, 1 indicates it is the foreground, and 0 indicates it is the background; Repeat the steps of calculating a dynamic learning rate, mean and median background modeling, Gaussian mixture modeling, comprehensively updating the background model and extracting the foreground, and update the background model and extract the foreground in real time; Step 3: Perform abnormal motion detection based on the motion data after data preprocessing, capture abnormal parameters based on the electrical parameters and temperature data after data preprocessing, and record the motion data, electrical parameters and temperature data when the abnormality exists; Step 4: Determine whether the abnormality is regular based on the acoustic wave data, image data, motion data, electrical parameters and temperature data when the abnormality exists, and generate a corresponding analysis and evaluation report. 2. According to the method for automatically capturing and analyzing comprehensive abnormal phenomena in claim 1, it is characterized in that the acoustic wave abnormality detection is performed according to the acoustic wave data after data preprocessing, specifically: The pre-processed acoustic wave data is converted into the frequency domain using Fourier transform, and the frequency components and amplitudes are extracted from the frequency domain signals as characteristic parameters of the acoustic wave; The Euclidean distance between the extracted acoustic wave characteristic parameters and the acoustic wave characteristic parameters under normal working conditions of the test equipment is calculated and compared; a sound wave threshold is preset, and when the calculated Euclidean distance exceeds the sound wave threshold, it is determined to be an abnormal sound wave. 3. According to the method for automatic capture and analysis of comprehensive abnormal phenomena described in claim 1, it is characterized in that the feature extraction using the foreground is specifically to perform feature segmentation on the extracted foreground image, including marking the connected areas of the foreground image after binarization of the foreground image, and using a multi-level network structure to perform micro-level and macro-level feature extraction on the marked connected areas, wherein the micro-level features include local density and local heterogeneity, and the macro-level features include regional contrast and regional uniformity, and the micro-level and macro-level features are combined into a comprehensive feature vector. 4. The method for automatically capturing and analyzing comprehensive abnormal phenomena according to claim 3 is characterized in that the abnormality identification based on the extracted features is specifically: The cosine similarity is used to calculate the similarity between the comprehensive feature vector and the abnormal phenomenon feature vector. If the similarity exceeds the preset threshold, the image area corresponding to the comprehensive feature vector is marked as having an abnormal phenomenon, and an abnormal phenomenon image dataset is constructed to complete the preliminary screening. The abnormal phenomenon image dataset is located and marked using threshold segmentation and morphological operations, image features are extracted from the target area of each location mark, an abnormal feature library is constructed, and feature matching is performed between the image features and the abnormal feature library to confirm whether there is a corresponding abnormal phenomenon in the target area. 5. A comprehensive abnormal phenomenon automatic capture and analysis method according to claim 4, characterized in that the feature matching is specifically: The similarity between the image features and the color histogram of the abnormal feature library is calculated using the Bhattacharyya distance. , and compare the differences in color moments; The difference method is used to calculate the frequency difference between the image features and the abnormality feature library when the abnormality occurs ; The dynamic time warping algorithm is used to calculate the similarity between image features and abnormal time-varying characteristics in the abnormal feature library. ; Defining weight factors ,satisfy ; Using the similarity of color histograms , frequency difference Similarity to time-varying characteristics Calculate the comprehensive similarity, expressed as: ; In the formula, is the comprehensive similarity, are the weights of color, frequency, and time-varying characteristics, respectively; Preset image similarity threshold ,if , it is determined that the corresponding abnormal phenomenon exists; By matching the image features with the abnormal feature library, it is confirmed whether there are abnormal phenomena in the target area. 6. A comprehensive abnormal phenomenon automatic capture and analysis method according to claim 5, characterized in that abnormal motion detection is performed based on the motion data after data preprocessing: According to the historical data of the three-dimensional spatial positioning of the normal movement of the test equipment, an abnormal movement detection model is established, and based on the statistical characteristics of the normal movement data, the normal movement range is calculated and the threshold of the normal movement range is set; According to the historical data of normal motion data and abnormal motion data of the test equipment and the set normal motion range threshold, the abnormal motion detection model is trained by using the support vector machine algorithm to obtain the trained abnormal motion detection model; The real-time motion data after data preprocessing is input into the trained abnormal motion detection model. When the abnormal motion detection model determines that the motion data of the test equipment exceeds the normal range, the test equipment has abnormal motion. 7. A comprehensive abnormal phenomenon automatic capture and analysis method according to claim 6, characterized in that the abnormal parameters are captured according to the electrical parameters and temperature data after data preprocessing, specifically: Set initial thresholds for current, voltage, and temperature ; Initialize exception index , expressed as: ; Obtain the electrical energy parameter data and temperature data after data preprocessing, including current ,Voltage and temperature ; Calculates real-time deviations in current, voltage, and temperature , , , expressed as: ; ; ; In the formula, , , Represent the deviation of current, voltage and temperature from the corresponding threshold respectively; Calculate the sum of squares of the deviations and the rate of change of each parameter, expressed as: In the formula, is the sum of squares of deviations; , , Represent the rate of change of current, voltage and temperature respectively; Calculate the correlation coefficient between current, voltage and temperature, expressed as: ; ; ; In the formula, , , Respectively represent the correlation coefficients between current and voltage, current and temperature, and voltage and temperature; The frequency domain characteristics of current, voltage and temperature are obtained by Fourier transform and energy spectral density calculation. ; Calculate anomaly index based on deviation, rate of change, correlation coefficient and frequency domain characteristics , expressed as: ; In the formula, is the weight coefficient; is the anomaly index; If the abnormal index If the warning value preset according to the empirical method is exceeded, the corresponding electric energy parameter data and temperature data are determined to be abnormal parameters. 8. A comprehensive abnormal phenomenon automatic capture and analysis method according to claim 7, characterized in that the existing abnormality is judged to be regular according to the sound wave data, image data, motion data, electrical parameters and temperature data when the abnormality exists, and the corresponding analysis and evaluation report is generated specifically as follows: Determine whether there is a concentration of abnormalities within a specific time period or time interval by using the time interval and duration of the acoustic wave data, image data, motion data, electrical parameters, and temperature data when the abnormality exists; By analyzing the position of the sensor corresponding to the pixel in the image data when the abnormality exists or the acoustic wave data when the abnormality exists, the spatial distribution of the abnormal data is analyzed to determine whether there is an abnormal cluster in a specific area or position; By analyzing the corresponding change trends of the motion data, electrical parameters and temperature data when the abnormality exists, whether the motion data, electrical parameters and temperature data when the abnormality exists show a trend of gradual increase or decrease, or whether there is periodic fluctuation; Generate a corresponding analysis and evaluation report based on the analysis of time, space and change trends.