CN118741088B - A method and system for identifying abnormal image signals in a video conferencing system - Google Patents
A method and system for identifying abnormal image signals in a video conferencing system Download PDFInfo
- Publication number
- CN118741088B CN118741088B CN202411223508.8A CN202411223508A CN118741088B CN 118741088 B CN118741088 B CN 118741088B CN 202411223508 A CN202411223508 A CN 202411223508A CN 118741088 B CN118741088 B CN 118741088B
- Authority
- CN
- China
- Prior art keywords
- scale analysis
- analysis mode
- abnormal
- video
- threshold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 118
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000004458 analytical method Methods 0.000 claims abstract description 186
- 230000005856 abnormality Effects 0.000 claims abstract description 48
- 230000008569 process Effects 0.000 claims abstract description 30
- 238000011217 control strategy Methods 0.000 claims abstract description 18
- 238000012423 maintenance Methods 0.000 claims description 76
- 239000013598 vector Substances 0.000 claims description 35
- 238000004364 calculation method Methods 0.000 claims description 25
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000000977 initiatory effect Effects 0.000 claims 1
- 230000000717 retained effect Effects 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 9
- 230000004044 response Effects 0.000 abstract description 4
- 230000003044 adaptive effect Effects 0.000 abstract description 3
- 238000010606 normalization Methods 0.000 description 10
- 230000008859 change Effects 0.000 description 5
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000002547 anomalous effect Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000003997 social interaction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
本发明公开了一种视频会议系统图像画面信号异常识别方法与系统，涉及画面异常识别技术领域，识别系统对当前视频会议采取多尺度分析模式，通过分析多项参数的变化为当前尺度分析模式生成异常系数，依据调整指数动态调整当前尺度分析模式的异常阈值，并将异常系数与调整后异常阈值进行对比，识别当前尺度分析模式是否存在异常，根据异常识别结果生成相应的控制策略。该识别系统不仅能够实时处理大量视频数据，还能显著提高视频会议系统的响应速度和资源利用率，引入自适应阈值检测，并通过在不同时间和空间尺度上检测视频信号异常，能够更全面地捕捉各种类型的异常，有效提高图像画面信号异常检测的准确性。
The present invention discloses a method and system for identifying abnormalities in image screen signals of a video conference system, and relates to the technical field of screen abnormality identification. The identification system adopts a multi-scale analysis mode for the current video conference, generates an abnormality coefficient for the current scale analysis mode by analyzing the changes of multiple parameters, dynamically adjusts the abnormality threshold of the current scale analysis mode according to the adjustment index, and compares the abnormality coefficient with the adjusted abnormality threshold to identify whether the current scale analysis mode is abnormal, and generates a corresponding control strategy according to the abnormality identification result. The identification system can not only process a large amount of video data in real time, but also significantly improve the response speed and resource utilization of the video conference system, introduce adaptive threshold detection, and detect video signal abnormalities at different time and space scales, so as to more comprehensively capture various types of abnormalities and effectively improve the accuracy of image screen signal abnormality detection.
Description
Technical Field
The invention relates to the technical field of image anomaly identification, in particular to a method and a system for identifying image and image signal anomalies of a video conference system.
Background
With the widespread use of video teleconferencing in everyday life, such as enterprise conferences, remote education, medical collaboration, nationwide collaboration, social interaction, etc., video conference systems generally employ digital signal transmission techniques and use compression algorithms (such as h.264 or h.265) to reduce bandwidth requirements, and in video conference systems, the identification of image picture signal anomalies is critical because these systems rely on high quality audio-video transmissions to ensure the effectiveness of remote communication.
The prior art has the following defects:
1. The existing identification method generally needs to decode and process the video in full frames, which has higher requirement on computational resources, and particularly occupies computational resources in a real-time video conference, thereby causing delay increase and processing efficiency reduction of the video conference system;
2. the identification method generally uses a fixed threshold to judge the abnormality of the image signal, which is easy to cause false alarm or missing report under different network environments or video contents, for example, when the network condition is not good, the fixed threshold may cause frequent false alarm;
3. The recognition method only analyzes on a single scale (such as a single frame or a single image block), and easily ignores anomalies in a small range or anomalies accumulated for a long time, resulting in insufficient detection of anomalies in image picture signals.
Disclosure of Invention
The invention aims to provide a video conference system image picture signal abnormality identification method and system, which can not only process a large amount of video data in real time but also remarkably improve the response speed and the resource utilization rate of a video conference system by directly extracting features in a compression domain, and introduce self-adaptive threshold detection, and can more comprehensively capture various types of abnormalities by detecting video signal abnormalities on different time and space scales, thereby effectively improving the accuracy of image picture signal abnormality detection and solving the defects in the background technology.
In order to achieve the above object, the present invention provides the following technical solutions: a method for identifying anomalies in image picture signals of a video conference system, said method comprising the steps of:
The recognition system adopts a multi-scale analysis mode for the current video conference, compresses video signals into a plurality of parameters in the video image coding process of each scale analysis mode, and generates abnormal coefficients by analyzing the change of the plurality of parameters into the current scale analysis mode, wherein the plurality of parameters comprise discrete cosine transform coefficients and motion vectors;
Acquiring network parameters and video parameters of a current scale analysis mode, generating an adjustment index by combining the network parameters and the video parameters, and dynamically adjusting an abnormal threshold value of the current scale analysis mode according to the adjustment index;
Comparing the anomaly coefficient with the adjusted anomaly threshold value, identifying whether the current scale analysis mode is abnormal or not, identifying whether other scale analysis modes are abnormal or not, and generating a corresponding control strategy according to an anomaly identification result;
And after the video conference is finished, carrying out weighted average calculation on the abnormal coefficients of all the scale analysis modes to obtain a maintenance value, and judging whether maintenance suggestions need to be generated for the video conference system according to the analysis result of the maintenance value.
In a preferred embodiment, the recognition system adopts a multi-scale analysis mode for the current video conference, wherein the multi-scale analysis mode comprises a short-time full-picture analysis mode, a short-time partial-picture analysis mode, a long-time full-picture analysis mode and a long-time partial-picture analysis mode.
In a preferred embodiment, in the video image encoding process of each scale analysis mode, the video signal is compressed into a plurality of parameters, comprising the steps of:
The plurality of parameters includes discrete cosine transform coefficients and motion vectors;
in the current scale analysis mode, dividing a video stream into multiple frames of image data, and dividing each frame of image data into a plurality of image blocks;
performing discrete cosine transform on each image block, converting pixel values in a space domain into discrete cosine transform coefficients in a frequency domain, wherein the discrete cosine transform coefficients describe frequency characteristics of the image blocks, reserving the image blocks with the frequency characteristics smaller than or equal to a frequency threshold, and deleting the image blocks with the frequency characteristics larger than the frequency threshold;
The region most similar to a certain block in the current frame is searched in the previous frame through a block matching algorithm to generate a motion vector for describing the motion of the image block between frames.
In a preferred embodiment, generating anomaly coefficients for a current scale analysis mode by analyzing changes in a plurality of parameters comprises the steps of:
Calculating the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector of each scale analysis mode, and comprehensively calculating the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector to obtain the abnormal coefficient of the current scale analysis mode, wherein the expression is as follows:
In which, in the process, As the coefficient of anomaly it is,For the amplitude of the fluctuation of the discrete cosine transform coefficient,For the magnitude of the motion vector fluctuation,、Weight coefficients of the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector are respectively, and、Are all greater than 0.
In a preferred embodiment, network parameters and video parameters of the current scale analysis mode are obtained, and an adjustment index is generated by combining the network parameters and the video parameters, comprising the following steps:
Acquiring network parameters and video parameters of a current scale analysis mode, wherein the network parameters comprise network delay and network packet loss rate, carrying out normalization processing on the network delay and the network packet loss rate, mapping a value range of the network delay and the network packet loss rate between [0,1], acquiring a network delay normalization value and a network packet loss rate normalization value, adding the network delay normalization value and the network packet loss rate normalization value to acquire a network instability coefficient, wherein the video parameters comprise scene complexity, comprehensively calculating the network instability coefficient and the scene complexity to acquire an adjustment index, and the expression is as follows:
In which, in the process, In order to adjust the index of the light,As a coefficient of network instability (co-efficient),In order to be a scene complexity,、Proportional coefficients of the network instability coefficient and the scene complexity, respectively, and、Are all greater than 0.
In a preferred embodiment, dynamically adjusting the anomaly threshold value of the current scale analysis mode according to the adjustment index comprises the steps of:
Dynamically adjusting an abnormal threshold value of the current scale analysis mode through the acquired adjustment index, wherein the expression is as follows: In which, in the process, In order to adjust the post-abnormality threshold,In order to adjust the pre-anomaly threshold value,To adjust the index.
In a preferred embodiment, comparing the anomaly coefficient with the adjusted anomaly threshold value, identifying whether an anomaly exists in the current dimension analysis mode, comprising the steps of:
Comparing the abnormal coefficient with an adjusted abnormal threshold, wherein the adjusted abnormal threshold is used for identifying whether the current scale analysis mode is abnormal or not, if the abnormal coefficient is smaller than or equal to the adjusted abnormal threshold, identifying that the current scale analysis mode is not abnormal, if the abnormal coefficient is larger than the adjusted abnormal threshold, identifying that the current scale analysis mode is abnormal, repeatedly obtaining the abnormal coefficient of other scale analysis modes, comparing the abnormal coefficient of the other scale analysis modes with the adjusted abnormal threshold, and identifying whether the other scale analysis modes are abnormal or not.
In a preferred embodiment, the generation of the corresponding control strategy according to the anomaly recognition result comprises the following steps:
When the abnormality of all the scale analysis modes is identified, the video conference is not controlled, and when the abnormality of any one of the scale analysis modes is identified, a corresponding control strategy is generated, wherein the control strategy comprises that the identification system automatically initiates a frame retransmission request or restarts the video conference and sends an alarm signal prompt.
In a preferred embodiment, the maintenance value is obtained by performing weighted average calculation on the anomaly coefficients of all scale analysis modes, and whether maintenance advice needs to be generated for the video conference system is judged according to the analysis result of the maintenance value, including the following steps:
and carrying out weighted average calculation on the abnormal coefficients of all the scale analysis modes to obtain maintenance values, wherein the expression is as follows: In which, in the process, In order to maintain the value of the value,Is the firstThe anomaly coefficients of the individual scale analysis patterns,Is the firstWeight coefficient of individual scale analysis mode, and，Representing the number of scale analysis modes;
comparing the acquired maintenance value with a preset maintenance threshold, judging that maintenance advice is not required to be generated for the video conference system if the maintenance value is smaller than or equal to the maintenance threshold, and judging that maintenance advice is required to be generated for the video conference system if the maintenance value is larger than the maintenance threshold.
The video conference system image picture signal abnormality recognition system comprises a calculation module, a threshold dynamic adjustment module, an abnormality analysis module and a maintenance module;
The calculation module: adopting a multi-scale analysis mode for the current video conference, compressing video signals into a plurality of parameters in the video image coding process of each scale analysis mode, wherein the plurality of parameters comprise discrete cosine transform coefficients and motion vectors, and generating abnormal coefficients for the current scale analysis mode by analyzing the change of the plurality of parameters;
Threshold dynamic adjustment module: acquiring network parameters and video parameters of a current scale analysis mode, generating an adjustment index by combining the network parameters and the video parameters, and dynamically adjusting an abnormal threshold value of the current scale analysis mode according to the adjustment index;
an anomaly analysis module: comparing the anomaly coefficient with the adjusted anomaly threshold value, identifying whether the current scale analysis mode is abnormal or not, identifying whether other scale analysis modes are abnormal or not, and generating a corresponding control strategy according to an anomaly identification result;
and a maintenance module: and after the video conference is finished, carrying out weighted average calculation on the abnormal coefficients of all the scale analysis modes to obtain a maintenance value, and judging whether maintenance suggestions need to be generated for the video conference system according to the analysis result of the maintenance value.
In the technical scheme, the invention has the technical effects and advantages that:
1. According to the invention, a multi-scale analysis mode is adopted for a current video conference through an identification system, an abnormal coefficient is generated for the current scale analysis mode through analyzing the change of a plurality of parameters, network parameters and video parameters of the current scale analysis mode are obtained, an adjustment index is generated by combining the network parameters and the video parameters, an abnormal threshold value of the current scale analysis mode is dynamically adjusted according to the adjustment index, the abnormal coefficient is compared with the adjusted abnormal threshold value, whether the current scale analysis mode is abnormal or not is identified, and a corresponding control strategy is generated according to an abnormal identification result. The recognition system directly extracts features in a compressed domain, avoids complete decoding of videos, can process a large amount of video data in real time, can obviously improve the response speed and the resource utilization rate of a video conference system, introduces adaptive threshold detection, can more comprehensively capture various types of anomalies by detecting video signal anomalies on different time and space scales, and effectively improves the accuracy of image picture signal anomaly detection.
2. After the video conference is finished, the method carries out weighted average calculation on the abnormal coefficients of all scale analysis modes to obtain the maintenance value, and judges whether maintenance suggestions need to be generated for the video conference system according to the analysis result of the maintenance value, thereby improving the management efficiency of the video conference system;
3. In the invention, the maintenance value is used for reflecting the overall abnormal state of the video conference, when the video conference is finished, even if all the scale analysis modes are detected to have no abnormality (the abnormal coefficient is smaller than or equal to the adjusted abnormal threshold value) in the video conference process, when all the scale analysis modes are developed towards the bad trend (namely, the abnormal coefficient is close to the abnormal threshold value), the maintenance value still indicates that the whole video conference system is possibly abnormal, the accuracy of the abnormal analysis of the video conference system is effectively improved by comprehensively analyzing all the scale analysis modes, and the maintenance proposal is conveniently judged to be generated for the video conference system or not, and the maintenance efficiency of the video conference system is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1: referring to fig. 1, the method for identifying abnormal image signals of a video conference system according to the embodiment includes the following steps:
the method comprises the steps that an identification system adopts a multi-scale analysis mode for a current video conference, video signals are compressed into a plurality of parameters in the video image encoding process of each scale analysis mode, the plurality of parameters comprise discrete cosine transform coefficients and motion vectors, abnormal coefficients are generated for the current scale analysis mode through analysis of the change of the plurality of parameters, network parameters and video parameters of the current scale analysis mode are obtained, an adjustment index is generated by combining the network parameters and the video parameters, an abnormal threshold value of the current scale analysis mode is dynamically adjusted according to the adjustment index, the abnormal coefficients are compared with the adjusted abnormal threshold value, whether the current scale analysis mode is abnormal or not is identified, whether the other scale analysis modes are abnormal or not is then identified, a corresponding control strategy is generated according to an abnormal identification result, after the video conference is finished, a maintenance value is obtained through weighted average calculation of the abnormal coefficients of all the scale analysis modes, and whether maintenance suggestions are needed to be generated for the video conference system is judged according to the analysis result of the maintenance value.
According to the application, a multi-scale analysis mode is adopted for a current video conference through an identification system, an abnormal coefficient is generated for the current scale analysis mode through analyzing the change of a plurality of parameters, network parameters and video parameters of the current scale analysis mode are obtained, an adjustment index is generated by combining the network parameters and the video parameters, an abnormal threshold value of the current scale analysis mode is dynamically adjusted according to the adjustment index, the abnormal coefficient is compared with the adjusted abnormal threshold value, whether the current scale analysis mode is abnormal or not is identified, and a corresponding control strategy is generated according to an abnormal identification result. The recognition system directly extracts features in a compressed domain, avoids complete decoding of videos, can process a large amount of video data in real time, can obviously improve the response speed and the resource utilization rate of a video conference system, introduces adaptive threshold detection, can more comprehensively capture various types of anomalies by detecting video signal anomalies on different time and space scales, and effectively improves the accuracy of image picture signal anomaly detection.
After the video conference is finished, the method carries out weighted average calculation on the abnormal coefficients of all scale analysis modes to obtain the maintenance value, and judges whether maintenance suggestions need to be generated for the video conference system according to the analysis result of the maintenance value, thereby improving the management efficiency of the video conference system.
Example 2: the recognition system adopts a multi-scale analysis mode for the current video conference, wherein the multi-scale analysis mode comprises a short-time full-picture analysis mode, a short-time local-picture analysis mode, a long-time full-picture analysis mode and a long-time local-picture analysis mode;
Specifically, in the application, the short-time value is 5s, and the long-time value is 15s.
In the video image encoding process of each scale analysis mode, compressing a video signal into a plurality of parameters including discrete cosine transform coefficients and motion vectors, generating abnormal coefficients for the current scale analysis mode by analyzing changes of the plurality of parameters, comprising the steps of:
In the current scale analysis mode, the video stream is divided into multiple frames of image data, each frame of image data being divided into several image blocks (typically 8x8 or 16x16 pixel blocks) so as to encode a local area of the image data, and the block processing helps to compress the image data more efficiently.
Performing a Discrete Cosine Transform (DCT) on each image block, converting the pixel values in the spatial domain into discrete cosine transform coefficients in the frequency domain, the discrete cosine transform coefficients describing the frequency characteristics of the image block, typically comprising low frequency and high frequency components:
The pixel values in the spatial domain are converted into discrete cosine transform coefficients in the frequency domain, expressed as:
，
In the formula, Is the discrete cosine transform coefficient of the block,Is the pixel value in the spatial domain, corresponding to the position in the image block，Is the size of the image block (typically 8x8 or 16x 16),: A cosine function representing a cosine wave of a particular frequency for each discrete cosine transform coefficient, representing the intensity of the frequency in the image block,AndIs a normalized coefficient defined as:
， And Is a normalized coefficient for correcting the transformed energy so that the energy before and after transformation is consistent,Representing the lateral frequency components of the image block in the frequency domain,Representing the longitudinal frequency components of the image block in the frequency domain.
Low frequency component: representing the overall brightness and color variation of the image;
High frequency component: details and edge information of the image are captured.
And reserving the image blocks with the frequency characteristics smaller than or equal to the frequency threshold value, deleting the image blocks with the frequency characteristics larger than the frequency threshold value, so that the low-frequency components are reserved, and the high-frequency components are discarded, thereby realizing the compression of the image data.
Searching a region most similar to a certain block in the current frame in the previous frame through a block matching algorithm to generate a motion vector for describing the motion of the image block between frames, predicting the image block of the current frame based on the motion vector, and only encoding a prediction error, thereby further reducing the inter-frame redundancy and improving the compression efficiency;
searching a region most similar to a certain block in the current frame in the previous frame through a block matching algorithm to generate a motion vector, wherein the motion vector acquisition logic comprises the following steps: in the appointed searching range, calculating the sum of absolute differences between a certain block in the current frame and each candidate block in the reference frame, and selecting the position with the smallest sum of absolute differences as the best matching block, wherein the offset between the position and the original position is the motion vector, and the expression is as follows: In which, in the process, In order for the motion vector to be a motion vector,A position offset that minimizes the sum of absolute differences;
the absolute difference sum is calculated as:
，
In the formula, Is the pixel value of the block to be matched in the current frame, and the upper left corner position of the block is，Is the pixel value of the block to be tested in the reference frame, the upper left corner position of the block is，Is the size of the image block (typically 8x8 or 16x 16),Is the offset within the search window, which represents the positional offset of the test block relative to the current block in the reference frame, the smaller the sum of absolute differences, the more similar the block of the current frame and the block of the reference frame.
The discrete cosine transform coefficients and motion vectors of each image block are entropy encoded (e.g., huffman encoded or arithmetic encoded) to further compress the data amount. These encoded parameters will be used as a compressed representation of the video signal.
Calculating the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector of each scale analysis mode, and comprehensively calculating the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector to obtain the abnormal coefficient of the current scale analysis mode, wherein the expression is as follows:
In which, in the process, As the coefficient of anomaly it is,For the amplitude of the fluctuation of the discrete cosine transform coefficient,For the magnitude of the motion vector fluctuation,、Weight coefficients of the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector are respectively, and、Are all greater than 0.
The fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector are obtained through a general parameter standard deviation calculation formula, and the general parameter standard deviation calculation formula is as follows: In which, in the process, As the standard deviation of the parameters,Is the firstThe original parameters of the individual image blocks are,As an average value of the parameters,Dividing the image blocks in the current scale analysis mode;
The calculation method of the fluctuation amplitude of the discrete cosine transform coefficient and the fluctuation amplitude of the motion vector are the same as the calculation of the standard deviation of the general parameter, and are not described in detail herein;
The larger the fluctuation amplitude of the discrete cosine transform coefficient is, the larger the discrete cosine transform coefficient value difference among a plurality of image blocks is, namely the more abnormal video conference images are likely to exist, such as image blurring, artifacts or mosaic phenomena, the larger the fluctuation amplitude of the motion vector is, the larger the motion vector value difference among a plurality of image blocks is, namely the more abnormal video conference images are likely to exist, such as jitter, clamping or frame loss phenomena.
Acquiring network parameters and video parameters of a current scale analysis mode, and generating an adjustment index by combining the network parameters and the video parameters, wherein the adjustment index comprises the following steps:
The method comprises the steps of obtaining network parameters and video parameters of a current scale analysis mode, wherein the network parameters comprise network delay and network packet loss rate, carrying out normalization processing on the network delay and the network packet loss rate, mapping a value range of the network delay and the network packet loss rate between [0,1], obtaining a network delay normalization value and a network packet loss rate normalization value, adding the network delay normalization value and the network packet loss rate normalization value to obtain a network instability coefficient, wherein the larger the network instability coefficient is, the worse the network state is, the identification system needs to properly increase an abnormal threshold value at the moment, frequent false alarm caused by slight fluctuation of the network state is avoided, the video parameters comprise scene complexity, the larger the scene complexity is, the identification system needs to properly reduce the abnormal threshold value, the monitoring strength of a complex scene is improved, and the network instability coefficient and the scene complexity are comprehensively calculated to obtain an adjustment index, and the expression is that:
In which, in the process, In order to adjust the index of the light,As a coefficient of network instability (co-efficient),In order to be a scene complexity,、Proportional coefficients of the network instability coefficient and the scene complexity, respectively, and、Are all greater than 0.
The calculation logic of the scene complexity is as follows: calculating gradient assignment of a current scale analysis mode by using a Sobel operator, wherein the expression is as follows: In which, in the process, At pixel point for imageThe magnitude of the gradient at this point,Representing an image at a pixel pointA horizontal gradient at the point of the flow,Representing an image at a pixel pointA vertical gradient thereat;
calculating scene complexity of the image according to the gradient amplitude, wherein the expression is as follows:
，
In the formula, In order to be a scene complexity,For image size, i.e. representing that the image hasRow of linesThe pixel points of the column are arranged,To indicate the function, i.e. when the gradient amplitudeGreater than the amplitude thresholdWhen the value of the indication function is 1, otherwise, the value is 0, and the greater the scene complexity is, the more the abnormal threshold is required to be reduced, so that the monitoring strength of the complex scene is improved.
Dynamically adjusting an abnormal threshold of the current scale analysis mode according to the adjustment index, comprising the following steps:
The larger the adjustment index is, the more the abnormal threshold value of the current scale analysis mode is required to be increased and adjusted, the dynamic adjustment is carried out on the abnormal threshold value of the current scale analysis mode through the obtained adjustment index, and the expression is as follows: In which, in the process, In order to adjust the post-abnormality threshold,In order to adjust the pre-anomaly threshold value,To adjust the index.
Comparing the anomaly coefficient with an adjusted anomaly threshold value, and identifying whether the current scale analysis mode is anomalous or not, and identifying whether other scale analysis modes are anomalous or not after identifying whether the current scale analysis mode is anomalous or not, wherein the method comprises the following steps:
Comparing the abnormal coefficient with an adjusted abnormal threshold, wherein the adjusted abnormal threshold is used for identifying whether the current scale analysis mode is abnormal or not, if the abnormal coefficient is smaller than or equal to the adjusted abnormal threshold, identifying that the current scale analysis mode is not abnormal, if the abnormal coefficient is larger than the adjusted abnormal threshold, identifying that the current scale analysis mode is abnormal, repeatedly obtaining the abnormal coefficient of other scale analysis modes, comparing the abnormal coefficient of the other scale analysis modes with the adjusted abnormal threshold, and identifying whether the other scale analysis modes are abnormal or not.
Generating a corresponding control strategy according to the abnormal recognition result, comprising the following steps:
When the abnormality of all the scale analysis modes is identified, the video conference is not controlled, and when the abnormality of any one of the scale analysis modes is identified, a corresponding control strategy is generated, wherein the control strategy comprises that the identification system automatically initiates a frame retransmission request or restarts the video conference and sends an alarm signal prompt.
After the video conference is finished, carrying out weighted average calculation on the abnormal coefficients of all scale analysis modes to obtain maintenance values, and judging whether maintenance suggestions need to be generated for the video conference system according to the analysis results of the maintenance values, wherein the method comprises the following steps:
and carrying out weighted average calculation on the abnormal coefficients of all the scale analysis modes to obtain maintenance values, wherein the expression is as follows: In which, in the process, In order to maintain the value of the value,Is the firstThe anomaly coefficients of the individual scale analysis patterns,Is the firstWeight coefficient of individual scale analysis mode, and，Representing the number of scale analysis modes;
The maintenance value is used for reflecting the overall abnormal state of the video conference, when the video conference is finished, even if all the scale analysis modes are detected to have no abnormality (the abnormality coefficient is smaller than or equal to the adjusted abnormality threshold value) in the video conference process, when all the scale analysis modes are developed towards the bad trend (namely the abnormality coefficient is close to the abnormality threshold value), the maintenance value still indicates that the whole video conference system is possibly abnormal, the accuracy of the abnormality analysis of the video conference system is effectively improved by comprehensively analyzing all the scale analysis modes, whether maintenance suggestions are needed to be generated for the video conference system or not is conveniently judged, and the maintenance efficiency of the video conference system is improved;
comparing the acquired maintenance value with a preset maintenance threshold, judging that maintenance advice is not required to be generated for the video conference system if the maintenance value is smaller than or equal to the maintenance threshold, and judging that maintenance advice is required to be generated for the video conference system if the maintenance value is larger than the maintenance threshold.
Example 3: the embodiment of the system for identifying the image picture signal abnormality of the video conference system comprises a calculation module, a threshold dynamic adjustment module, an abnormality analysis module and a maintenance module;
The calculation module: the method comprises the steps that a multi-scale analysis mode is adopted for a current video conference, video signals are compressed into multiple parameters in the video image coding process of each scale analysis mode, the multiple parameters comprise discrete cosine transform coefficients and motion vectors, abnormal coefficients are generated for the current scale analysis mode through analysis of the changes of the multiple parameters, the abnormal coefficients are sent to an abnormal analysis module and a maintenance module, and the multi-scale analysis mode is sent to the maintenance module;
threshold dynamic adjustment module: acquiring network parameters and video parameters of the current scale analysis mode, generating an adjustment index by combining the network parameters and the video parameters, dynamically adjusting an abnormal threshold of the current scale analysis mode according to the adjustment index, and transmitting the adjusted abnormal threshold to an abnormal analysis module;
An anomaly analysis module: comparing the anomaly coefficient with the adjusted anomaly threshold value, identifying whether the current scale analysis mode is abnormal or not, identifying whether other scale analysis modes are abnormal or not, and generating a corresponding control strategy according to an anomaly identification result;
and a maintenance module: and after the video conference is finished, carrying out weighted average calculation on the abnormal coefficients of all the scale analysis modes to obtain a maintenance value, and judging whether maintenance suggestions need to be generated for the video conference system according to the analysis result of the maintenance value.
The above formulas are all formulas with dimensions removed and numerical values calculated, the formulas are formulas with a large amount of data collected for software simulation to obtain the latest real situation, and preset parameters in the formulas are set by those skilled in the art according to the actual situation.
It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural. In addition, the character "/" herein generally indicates that the associated object is an "or" relationship, but may also indicate an "and/or" relationship, and may be understood by referring to the context.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411223508.8A CN118741088B (en) | 2024-09-03 | 2024-09-03 | A method and system for identifying abnormal image signals in a video conferencing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411223508.8A CN118741088B (en) | 2024-09-03 | 2024-09-03 | A method and system for identifying abnormal image signals in a video conferencing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118741088A CN118741088A (en) | 2024-10-01 |
| CN118741088B true CN118741088B (en) | 2024-11-12 |
Family
ID=92847885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411223508.8A Active CN118741088B (en) | 2024-09-03 | 2024-09-03 | A method and system for identifying abnormal image signals in a video conferencing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118741088B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509827A (en) * | 2017-02-27 | 2018-09-07 | 阿里巴巴集团控股有限公司 | The recognition methods of anomalous content and video flow processing system and method in video flowing |
| CN117523285A (en) * | 2023-11-10 | 2024-02-06 | 中国工商银行股份有限公司 | Financial image anomaly detection method and device based on statistical characteristics |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004128962A (en) * | 2002-10-03 | 2004-04-22 | Canon Inc | Surveillance camera and controlling device |
| JP2007180970A (en) * | 2005-12-28 | 2007-07-12 | Matsushita Electric Ind Co Ltd | Video processor and monitoring camera system |
| CN110795996B (en) * | 2019-09-18 | 2024-03-12 | 平安科技（深圳）有限公司 | Method, device, equipment and storage medium for classifying heart sound signals |
| KR20240076459A (en) * | 2022-11-22 | 2024-05-30 | 연세대학교 산학협력단 | Video Anomaly Detection Apparatus and Method using Relational Embedding |
| CN117854014B (en) * | 2024-03-08 | 2024-05-31 | 国网福建省电力有限公司 | A comprehensive method for automatically capturing and analyzing abnormal phenomena |
| CN118171195A (en) * | 2024-03-12 | 2024-06-11 | 国网安徽省电力有限公司宣城供电公司 | Transmission monitoring system for intelligent substation relay protection overhaul test |
| CN117896552B (en) * | 2024-03-14 | 2024-07-12 | 浙江华创视讯科技有限公司 | Video conference processing method, video conference system and related device |
| CN118433071B (en) * | 2024-07-03 | 2024-09-10 | 江苏小牛电动科技有限公司 | Real-time signal quality monitoring method and system for cloud interaction of electric vehicle |
-
2024
- 2024-09-03 CN CN202411223508.8A patent/CN118741088B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509827A (en) * | 2017-02-27 | 2018-09-07 | 阿里巴巴集团控股有限公司 | The recognition methods of anomalous content and video flow processing system and method in video flowing |
| CN117523285A (en) * | 2023-11-10 | 2024-02-06 | 中国工商银行股份有限公司 | Financial image anomaly detection method and device based on statistical characteristics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118741088A (en) | 2024-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | A convolutional neural network-based approach to rate control in HEVC intra coding | |
| JP2006507775A (en) | Method and apparatus for measuring the quality of a compressed video sequence without criteria | |
| JP2020508010A (en) | Image processing and video compression method | |
| EP3952307A1 (en) | Video processing apparatus and processing method of video stream | |
| Martini et al. | Image quality assessment based on edge preservation | |
| CN109167997B (en) | Video quality diagnosis system and method | |
| CN108989802B (en) | A method and system for quality estimation of HEVC video stream using inter-frame relationship | |
| Lin et al. | PEA265: Perceptual assessment of video compression artifacts | |
| CN106412572B (en) | A kind of video flowing coding quality evaluation method based on kinetic characteristic | |
| CN112383777A (en) | Video coding method and device, electronic equipment and storage medium | |
| Moorthy et al. | Efficient video quality assessment along temporal trajectories | |
| CN115802038A (en) | Quantization parameter determination method and device and video coding method and device | |
| Wang et al. | A two-stage h. 264 based video compression method for automotive cameras | |
| CN112887587B (en) | Self-adaptive image data fast transmission method capable of carrying out wireless connection | |
| CN118741088B (en) | A method and system for identifying abnormal image signals in a video conferencing system | |
| TWI586175B (en) | Method and system for managing bandwidth of video conference | |
| CN111145219B (en) | Efficient video moving target detection method based on Codebook principle | |
| Qi et al. | Non-reference image quality assessment based on super-pixel segmentation and information entropy | |
| Nami et al. | Lightweight Multitask Learning for Robust JND Prediction using Latent Space and Reconstructed Frames | |
| Choi et al. | Perceptual flicker visibility prediction model | |
| JP5567413B2 (en) | Outline extraction system, decoding apparatus and outline extraction program | |
| Hossain et al. | No reference prediction of quality metrics for H. 264 compressed infrared image sequences for UAV applications | |
| CN113362315B (en) | Image quality evaluation method and evaluation model based on multi-algorithm fusion | |
| Lin et al. | Toward Efficient Video Compression Artifact Detection and Removal: A Benchmark Dataset | |
| CN118474353B (en) | Information transmission method and system in digital enterprise management process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |