CN116311080B - Monitoring image detection method and device - Google Patents
Monitoring image detection method and device Download PDFInfo
- Publication number
- CN116311080B CN116311080B CN202310531083.6A CN202310531083A CN116311080B CN 116311080 B CN116311080 B CN 116311080B CN 202310531083 A CN202310531083 A CN 202310531083A CN 116311080 B CN116311080 B CN 116311080B
- Authority
- CN
- China
- Prior art keywords
- image
- newly added
- recognition model
- monitoring
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the technical field of image processing, and provides a monitoring image detection method and device. The method comprises the following steps: obtaining each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time; performing image recognition on at least one newly added image according to the pre-trained first image recognition model to obtain a detection result of the newly added image; performing image recognition on at least one non-newly added image according to the pre-trained second image recognition model to obtain a detection result of the non-newly added image; the newly added image is a monitoring image with the accumulated time length which is stored in the database for the first time being smaller than the preset time length, and the non-newly added image is a monitoring image with the accumulated time length which is stored in the database for the first time not smaller than the preset time length and is identified as non-abnormal by the first image identification model; the second image recognition model is larger than the first image recognition model. The monitoring image detection method provided by the embodiment of the application can improve the efficiency of abnormality detection on multiple monitoring images.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a monitoring image detection method and device.
Background
In order to ensure the safety of the reservoir, monitoring equipment is usually required to be arranged in each area of the reservoir to monitor, and whether an abnormality occurs in a certain area or not is detected in time. And judging whether an abnormality occurs in a certain area or not, and judging by using the monitoring image of the area. The monitoring image of the area is detected in an image recognition mode to detect whether the monitoring image is abnormal or not. If the monitoring image is abnormal, the abnormal occurrence of the area can be determined.
In order to improve the accuracy of the monitoring image detection, in the related art, the monitoring image may be initially detected by a lightweight image recognition model. If the monitoring image is identified as an abnormal monitoring image through the preliminary detection, the detection is ended, and if the monitoring image is identified as a non-abnormal monitoring image through the preliminary detection, the secondary detection can be performed through a more accurate image identification model, so that whether the monitoring image is abnormal or not can be finally determined. However, since the monitored image detected secondarily is the monitored image detected primarily, the detection time of the monitored image is the sum of the detection time of the two times, which results in poor scheduling efficiency of computer resources when detecting the multi-monitored image and low efficiency when detecting the multi-monitored image abnormally.
Disclosure of Invention
The present application is directed to solving at least one of the technical problems existing in the related art. Therefore, the application provides a monitoring image detection method which can improve the efficiency of abnormality detection on multiple monitoring images.
The application also provides a monitoring image detection device.
The application further provides electronic equipment.
The application also proposes a computer readable storage medium.
According to an embodiment of the first aspect of the present application, a method for detecting a surveillance image includes:
obtaining each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time;
performing image recognition on at least one newly-added image according to a pre-trained first image recognition model to obtain a detection result of the newly-added image; the method comprises the steps of,
performing image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the new image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-new image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length and identified as non-abnormal by the first image identification model;
the second image recognition model is larger than the first image recognition model.
The method has the advantages that the newly added image is detected through the lightweight first image recognition model, the non-newly added image is detected through the non-lightweight second image recognition model, so that the newly added image with real-time requirements can be detected rapidly, the non-newly added image which has no real-time requirements and is detected rapidly is detected secondarily, the scheduling of computer resources is more reasonable, and the efficiency of anomaly detection on multiple monitoring images is improved.
According to an embodiment of the present application, the performing image recognition on at least one of the added images according to the pre-trained first image recognition model to obtain a detection result of the added image includes:
inputting each first image frame of the newly added image into the first image recognition model to obtain an image label of each first image frame;
and obtaining a detection result of the newly added image according to the image label of each first image frame.
According to an embodiment of the present application, the detecting result of the added image is obtained according to the image tag of each first image frame, including:
acquiring each target image frame with the image tag being an abnormal image tag from each first image frame;
determining the newly added image as an abnormal image in the case that abnormal image areas among a plurality of target image frames are matched;
wherein the abnormal image region is determined by the first image recognition model.
According to one embodiment of the present application, acquiring each target image frame whose image tag is an abnormal image tag from each of the first image frames includes:
acquiring an alternative image frame with an image tag being an abnormal image tag from each first image frame;
and determining the position information of the abnormal image area of the alternative image frame, which is in a preset area corresponding to the monitoring image to which the alternative image frame belongs, and determining the alternative image frame as the target image frame.
According to an embodiment of the present application, the performing image recognition on at least one of the non-newly added images according to the pre-trained second image recognition model to obtain a detection result of the non-newly added image includes:
acquiring the non-newly added images with the accumulated times smaller than the preset times from the non-newly added images according to the accumulated times of image identification of the non-newly added images;
and carrying out image recognition on the non-newly added images with the accumulated times smaller than the preset times according to the second image recognition model to obtain detection results of the non-newly added images with the accumulated times smaller than the preset times.
According to one embodiment of the present application, the cumulative number of times is determined by the number of times the non-newly added image is subjected to image recognition by the second image recognition model.
According to an embodiment of the present application, the performing image recognition on at least one of the non-newly added images according to the pre-trained second image recognition model to obtain a detection result of the non-newly added image includes:
inputting each second image frame of the non-newly added image into the second image recognition model to obtain an image tag of the second image frame;
and obtaining a detection result of the non-newly added image according to the image label of the second image frame.
According to a second aspect of the present application, a monitoring image detecting apparatus includes:
the monitoring image acquisition module is used for acquiring each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time;
the newly added image detection module is used for carrying out image recognition on at least one newly added image according to a pre-trained first image recognition model to obtain a detection result of the newly added image; the method comprises the steps of,
the non-newly added image detection module is used for carrying out image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the added image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-added image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length;
the second image recognition model is larger than the first image recognition model.
An electronic device according to an embodiment of a third aspect of the present application includes a processor and a memory storing a computer program, where the processor implements the surveillance image detection method according to any of the above embodiments when executing the computer program.
A computer readable storage medium according to a fourth aspect of the present application stores thereon a computer program which, when executed by a processor, implements the surveillance image detection method according to any of the above embodiments.
The above technical solutions in the embodiments of the present application have at least one of the following technical effects:
the method has the advantages that the newly added image is detected through the lightweight first image recognition model, the non-newly added image is detected through the non-lightweight second image recognition model, so that the newly added image with real-time requirements can be detected rapidly, the non-newly added image which has no real-time requirements and is detected rapidly is detected secondarily, the scheduling of computer resources is more reasonable, and the efficiency of anomaly detection on multiple monitoring images is improved.
Drawings
In order to more clearly illustrate the application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a first flowchart of a surveillance image detection method according to some embodiments of the application;
FIG. 2 is a second flowchart of a surveillance image detection method according to some embodiments of the application;
FIG. 3 is a third flowchart of a surveillance image detection method according to some embodiments of the present application;
FIG. 4 is a fourth flowchart of a surveillance image detection method according to some embodiments of the application;
FIG. 5 is a fifth flowchart of a surveillance image detection method according to some embodiments of the application;
fig. 6 is a schematic structural diagram of a monitoring image detecting device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The following describes and illustrates the method and apparatus for detecting a surveillance image according to the embodiments of the present application in detail by using several specific embodiments.
In an embodiment, a method for detecting a surveillance image is provided, and the method is applied to a server for detecting the surveillance image. The server may be an independent server or a server cluster formed by a plurality of servers, and may also be a cloud server for providing basic cloud computing services such as cloud services, cloud message databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, large message data, artificial intelligent sampling point devices, and the like.
As shown in fig. 1, the method for detecting a surveillance image provided in this embodiment includes:
step 101, obtaining each newly added image and each non-newly added image from each monitored image according to the moment when each monitored image is stored in a database for the first time;
102, performing image recognition on at least one of the newly added images according to a pre-trained first image recognition model to obtain a detection result of the newly added image; the method comprises the steps of,
step 103, performing image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the added image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-added image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length;
the second image recognition model is larger than the first image recognition model.
In some embodiments, the monitoring images collected by the monitoring devices disposed in each area of the reservoir are stored in the database after collection. The monitoring images of different areas can be divided into different image types, namely, the image types of each monitoring image can be divided according to the monitored areas of the monitoring images in the database. If the monitoring image is aimed at an ornamental area for tourists in the reservoir, the image type of the monitoring image can be divided into ornamental types; the monitoring images of patrol areas which are only allowed to be accessed by workers and are provided with matched equipment such as distribution boxes, control equipment and monitoring equipment in the reservoir can be classified into patrol types; the image type of the monitoring image for monitoring the water surface area in the reservoir can be divided into a water surface area monitoring type and the like. The duration of each monitoring image can be the same, such as 1 minute, or different, and the specific duration can be set according to actual requirements.
When each monitoring image is stored in the database for the first time, the moment when the monitoring image is stored in the database for the first time is recorded. According to the moment when each monitoring image is first stored in the database, each monitoring image can be divided into a newly added image and a non-newly added image. As a possible implementation manner, if the accumulated time length of the time when the monitoring image is first stored in the database from the current time is less than the preset time length, for example, less than 5 minutes, the monitoring image may be determined to be a new monitoring image. If the accumulated time length of the moment of the monitoring image stored in the database for the first time from the current moment is not less than the preset time length, that is, the accumulated time length is greater than or equal to the preset time length, the monitoring image can be determined to be a non-newly added monitoring image.
In addition, when it is detected that N monitoring images are first stored in the database at a certain time, the N monitoring images may be determined as newly added monitoring images, and the rest of monitoring images in the database may be determined as non-newly added monitoring images.
And for the newly added images, performing image recognition on each newly added image through a pre-trained first image recognition model so as to screen out abnormal images from each newly added image. The first image recognition model is obtained by training a negative sample set consisting of a plurality of abnormal images and a positive sample set consisting of a plurality of normal images. The first image recognition model is a lightweight image recognition model such as MobileNetV 3. The recognition speed of the lightweight image recognition model is faster, but the accuracy is slightly poorer.
And when the image recognition is carried out on the newly added images, the image recognition can be carried out on each non-newly added image by adopting a second image recognition model so as to recognize the abnormal image from each non-newly added image. The non-newly added image is a monitoring image which is determined to be non-abnormal by the first image recognition model. The second image recognition model can also be trained from a negative sample set consisting of a plurality of abnormal images and a positive sample set consisting of a plurality of normal images. The second image recognition model is a larger image recognition model than the first image recognition model, e.g., it may be a yolo3+crnn trained image recognition model. The image recognition mode of the first image model and the second image model can be performed in parallel, so that screening of various monitoring images can be performed simultaneously, computer resources are scheduled more efficiently, and the abnormality detection efficiency is further improved.
The method has the advantages that the newly added image is detected through the lightweight first image recognition model, the non-newly added image is detected through the non-lightweight second image recognition model, so that the newly added image with real-time requirements can be detected rapidly, the non-newly added image which has no real-time requirements and is detected rapidly is detected secondarily, the scheduling of computer resources is more reasonable, and the efficiency of anomaly detection on multiple monitoring images is improved.
In view of the fact that the monitor image is generally composed of a plurality of image frames, in order to enable more accurate detection of the monitor image, as shown in fig. 2, in some embodiments, performing image recognition on at least one of the added images according to a pre-trained first image recognition model to obtain a detection result of the added image, including:
step 201, inputting each first image frame of the newly added image into the first image recognition model to obtain an image tag of each first image frame;
step 202, obtaining a detection result of the new image according to the image label of each first image frame.
In some embodiments, for the newly added image, image frames may be extracted first, so as to obtain each first image frame of the newly added image. After each first image frame of the newly added image is obtained, each first image frame is sequentially input into a first image recognition model for image recognition, so that the image label of each first image frame is output through the first image recognition model. The image label is an abnormal image label indicating that the first image frame is abnormal, or a normal image label indicating that the first image frame is normal. If the obtained image label is an abnormal image label after a certain first image frame is input into the first image recognition model, the first image frame can be indicated to be abnormal; similarly, if the obtained image tag is a normal image tag, it may indicate that the first image frame is normal.
After the image tag of each first image frame is obtained through the first image recognition model, if the image tag of a certain first image frame in each first image frame is an abnormal image tag, the new image can be determined to be an abnormal image, and the new image is deleted from the database; and if the image labels of the first image frames are all normal image labels, determining the newly added image as a normal image.
Considering that the first image recognition model may have a false recognition condition on the first image frame due to the performance of the model itself or due to noise existing in a certain first image frame in the newly added image, if the abnormal first image frame is recognized as normal, the repeated verification can be performed through the second image recognition model. However, if the normal first image frame is identified as abnormal, the new image may be deleted and cannot be secondarily verified through the second image identification model, which affects the detection accuracy of the monitored image. For this purpose, in some embodiments, as shown in fig. 3, according to the image tag of each of the first image frames, a detection result of the added image is obtained, including:
step 301, obtaining each target image frame with an image tag being an abnormal image tag from each first image frame;
step 302, determining the newly added image as an abnormal image when abnormal image areas among a plurality of target image frames are matched;
wherein the abnormal image region is determined by the first image recognition model.
In some embodiments, after the first image frame is input into the first image recognition model, if the first image recognition model determines that the first image frame is abnormal, the abnormal image label of the first image frame is output, and at the same time, the abnormal image region of the first image frame is also output. If a certain first image frame is judged to be abnormal by the first image recognition model, namely, the image label output by the first image recognition model is an abnormal image label, the first image frame is determined to be a target image frame.
For each target image frame, the similarity matching may be performed on the abnormal image areas of each target image frame to determine whether the similarity of the abnormal image areas between two or more target image frames reaches a preset value, for example, the similarity reaches 90%. If there is an abnormal image area match between the plurality of target image frames, then the newly added image including each target image frame is determined as an abnormal image. If the abnormal image areas among the target image frames are not matched, or only one target image frame exists, the detection result of the target image frames is possibly misidentification caused by the performance of the first image identification model and/or noise of the image frames, at the moment, the newly added image comprising the target image frames is determined to be a normal image and is stored in a database, so that the image is further checked through the second image identification module.
By acquiring each target image frame with the image label being the abnormal image label from each first image frame and determining the newly-added image as the abnormal image when abnormal image areas among a plurality of target image frames are matched, the situation that the newly-added image cannot be repeatedly checked due to the fact that the performance of a model and/or noise exists in a certain first image frame in the newly-added image is reduced, and the detection accuracy of the monitoring image is improved.
Because of the monitoring images of the reservoir, there are specific areas of interest. For example, a monitoring image for monitoring the water surface of a reservoir, although the monitoring image also comprises other images except for the water surface area, the specific area of interest is the water surface, and the images of other areas are all negligible background images. Thus, to reduce misrecognition due to the influence of the background image, in some embodiments, as shown in fig. 4, acquiring each target image frame in which the image tag is an abnormal image tag in each of the first image frames includes:
step 401, obtaining an alternative image frame with an image tag being an abnormal image tag from each first image frame;
step 402, determining location information of an abnormal image area of the candidate image frame, where the location information is in a preset area corresponding to the monitored image to which the candidate image frame belongs, and determining the candidate image frame as the target image frame.
In some embodiments, each of the first image frames may be labeled as an alternative image frame. Then, for any alternative image frame, the corresponding preset area can be determined according to the type of the monitoring image to which the image frame belongs. As a possible implementation manner, a one-to-one mapping relationship between each image type and each preset area is preset for each image type of each monitoring image. If the image type of the monitoring image is ornamental, the preset area can be an area A with the center point of the monitoring image as the center and the radius R. The preset areas corresponding to different image types can be divided according to actual conditions. When any alternative image frame is acquired, a corresponding preset area can be found from the mapping relation according to the image type of the monitoring image to which the image frame belongs.
After a preset area corresponding to the monitored image to which the alternative image frame belongs is obtained, matching the position information of the abnormal image area of the alternative image frame with the preset area, and judging whether the position information of the abnormal image area of the alternative image frame is located in the preset area or not. If the position information of the abnormal image area of the alternative image frame is located outside the corresponding preset area, the abnormal background which can be ignored is indicated to appear in the alternative image frame, and the alternative image frame is ignored at the moment. If the position information of the abnormal image area of the alternative image frame is located in the corresponding preset area, the situation that the abnormality occurs in the alternative image frame is a foreground needing to be focused is indicated, and the alternative image frame is determined to be the target image frame at the moment, so that the false recognition caused by the influence of the background image is reduced, and the accuracy of the detection result of the non-newly added image is further improved.
In order to further improve the efficiency of anomaly detection, as shown in fig. 5, in some embodiments, performing image recognition on at least one of the non-newly added images according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image, including:
step 501, according to the accumulated times of image recognition of each non-newly added image, acquiring the non-newly added image with the accumulated times smaller than a preset times from each non-newly added image;
step 502, performing image recognition on the non-newly added images with the accumulated times smaller than the preset times according to the second image recognition model, so as to obtain a detection result of the non-newly added images with the accumulated times smaller than the preset times.
The accumulated times can be determined by the times of image recognition of the non-newly added images through the second image recognition model.
As a possible implementation manner, for each non-newly added image, the cumulative number of times of image recognition performed by the second image recognition model may be obtained from each non-newly added image to be smaller than a preset number of times, for example, 1 time of non-newly added image is subjected to image recognition by using the second image recognition model. Because the accuracy of the monitored image subjected to image recognition by the second image recognition model is high enough, when the non-newly added image is subjected to image recognition by the second image recognition model, the monitored image subjected to image recognition by the second image recognition model can be removed, the processing amount of the second image recognition model is reduced, and the detection accuracy is ensured and the abnormality detection efficiency is improved.
Further, to enable more accurate detection of the monitored image, in some embodiments, performing image recognition on at least one of the non-newly added images according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image includes: inputting each second image frame of the non-newly added image into the second image recognition model to obtain an image tag of the second image frame; and obtaining a detection result of the non-newly added image according to the image label of the second image frame.
In some embodiments, for the non-newly added image, the extraction of the image frame may be performed first, so as to obtain each second image frame of the non-newly added image. After each second image frame of the non-newly added image is obtained, each second image frame is sequentially input into a second image recognition model for image recognition, so that the image label of each second image frame is output through the second image recognition model. The image label is an abnormal image label indicating that the second image frame is abnormal, or a normal image label indicating that the second image frame is normal. If the obtained image label is an abnormal image label after a certain second image frame is input into the second image recognition model, the second image frame can be indicated to be abnormal; similarly, if the obtained image tag is a normal image tag, it may indicate that the second image frame is normal.
After the image tag of each second image frame is obtained through the second image recognition model, if the image tag of a certain second image frame in each second image frame is an abnormal image tag, the non-newly added image can be determined to be an abnormal image, and the non-newly added image is deleted from the database; and if the image labels of the second image frames are all normal image labels, determining the non-newly added image as a normal image.
In some embodiments, in the same manner as the processing of the newly added image, for the non-newly added image, each target image frame whose image tag is an abnormal image tag may be obtained from each second image frame, and the non-newly added image may be determined as the abnormal image when the abnormal image areas among the plurality of target image frames match. Wherein the abnormal image region is determined by the second image recognition model.
Similarly, in some embodiments, the same processing manner as that of the newly added image may also be used to obtain, for the non-newly added image, an alternative image frame whose image tag is an abnormal image tag from each second image frame, and when determining that the position information of the abnormal image area of the alternative image frame is in the preset area corresponding to the monitored image to which the alternative image frame belongs, determine the alternative image frame as the target image frame.
In some embodiments, to further improve the accuracy of anomaly detection, after each anomaly image is screened from each monitored image by using a first image recognition model and a second image recognition model, comparing each image frame or image label of any anomaly image with each preset anomaly image stored in advance to determine whether the anomaly image has an image frame matched with each preset anomaly image; or whether the image label is an image frame of the abnormal image and is matched with any preset abnormal image. If yes, determining that the abnormal monitoring image is abnormal, and marking the file name of the abnormal image as red; otherwise, the abnormal image is marked to be confirmed, for example, the file name of the abnormal image is marked yellow, so that the staff can manually confirm the abnormal image. After the manual determination is carried out by the staff, the abnormal image is input into the first image recognition model and the second image recognition model as a training set for training. If the abnormal image is determined to be abnormal, training the abnormal image as a negative sample set; otherwise, training the abnormal image as a positive sample.
The following describes the monitoring image detection device provided by the present application, and the monitoring image detection device described below and the monitoring image detection method described above can be referred to correspondingly.
In one embodiment, as shown in fig. 6, there is provided a monitoring image detecting apparatus, including:
the monitor image obtaining module 210 is configured to obtain each newly added image and each non-newly added image from each monitor image according to the moment when each monitor image is first stored in the database;
the newly added image detection module 220 is configured to perform image recognition on at least one newly added image according to a pre-trained first image recognition model, so as to obtain a detection result of the newly added image; the method comprises the steps of,
the non-newly added image detection module 230 is configured to perform image recognition on at least one non-newly added image according to a pre-trained second image recognition model, so as to obtain a detection result of the non-newly added image;
the added image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-added image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length;
the second image recognition model is larger than the first image recognition model.
The method has the advantages that the newly added image is detected through the lightweight first image recognition model, the non-newly added image is detected through the non-lightweight second image recognition model, so that the newly added image with real-time requirements can be detected rapidly, the non-newly added image which has no real-time requirements and is detected rapidly is detected secondarily, the scheduling of computer resources is more reasonable, and the efficiency of anomaly detection on multiple monitoring images is improved.
In one embodiment, the environment detection module 220 is specifically configured to:
determining the comparison result as a first comparison result, and carrying out water quality detection on the reservoir in real time by taking the moment of determining the first comparison result as an initial moment so as to determine the water quality detection result of the reservoir in real time according to the current water quality of the reservoir until the detection duration reaches a preset duration;
wherein the preset time period is longer than the duration of the first comparison result.
In one embodiment, the additional image detection module 220 is specifically configured to:
inputting each first image frame of the newly added image into the first image recognition model to obtain an image label of each first image frame;
and obtaining a detection result of the newly added image according to the image label of each first image frame.
In one embodiment, the additional image detection module 220 is specifically configured to:
acquiring each target image frame with the image tag being an abnormal image tag from each first image frame;
determining the newly added image as an abnormal image in the case that abnormal image areas among a plurality of target image frames are matched;
wherein the abnormal image region is determined by the first image recognition model.
In one embodiment, the additional image detection module 220 is specifically configured to:
acquiring an alternative image frame with an image tag being an abnormal image tag from each first image frame;
and determining the position information of the abnormal image area of the alternative image frame, which is in a preset area corresponding to the monitoring image to which the alternative image frame belongs, and determining the alternative image frame as the target image frame.
In one embodiment, the non-added image detection module 230 is specifically configured to:
acquiring the non-newly added images with the accumulated times smaller than the preset times from the non-newly added images according to the accumulated times of image identification of the non-newly added images;
and carrying out image recognition on the non-newly added images with the accumulated times smaller than the preset times according to the second image recognition model to obtain detection results of the non-newly added images with the accumulated times smaller than the preset times.
In an embodiment, the cumulative number of times is determined by the number of times the non-newly added image is subjected to image recognition by the second image recognition model.
In one embodiment, the non-added image detection module 230 is specifically configured to:
inputting each second image frame of the non-newly added image into the second image recognition model to obtain an image tag of the second image frame;
and obtaining a detection result of the non-newly added image according to the image label of the second image frame.
Fig. 7 illustrates a physical schematic diagram of an electronic device, as shown in fig. 7, which may include: processor 810, communication interface (Communication Interface) 820, memory 830, and communication bus 840, wherein processor 810, communication interface 820, memory 830 accomplish communication with each other through communication bus 840. The processor 810 may call a computer program in the memory 830 to perform a surveillance image detection method, for example, including:
obtaining each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time;
performing image recognition on at least one newly-added image according to a pre-trained first image recognition model to obtain a detection result of the newly-added image; the method comprises the steps of,
performing image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the new image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-new image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length and identified as non-abnormal by the first image identification model;
the second image recognition model is larger than the first image recognition model.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, an embodiment of the present application further provides a storage medium, where the storage medium includes a computer program, where the computer program may be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer is capable of executing the monitoring image detection method provided in the foregoing embodiments, for example, including:
obtaining each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time;
performing image recognition on at least one newly-added image according to a pre-trained first image recognition model to obtain a detection result of the newly-added image; the method comprises the steps of,
performing image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the new image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-new image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length and identified as non-abnormal by the first image identification model;
the second image recognition model is larger than the first image recognition model.
The apparatus embodiments described above are merely illustrative, wherein elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present application without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (8)
1. A method for monitoring an image, comprising:
obtaining each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time;
performing image recognition on at least one newly-added image according to a pre-trained first image recognition model to obtain a detection result of the newly-added image; the method comprises the steps of,
performing image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the new image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-new image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length and identified as non-abnormal by the first image identification model;
the second image recognition model is larger than the first image recognition model;
the step of performing image recognition on at least one of the added images according to the pre-trained first image recognition model to obtain a detection result of the added image comprises the following steps:
inputting each first image frame of the newly added image into the first image recognition model to obtain an image label of each first image frame;
acquiring each target image frame with the image tag being an abnormal image tag from each first image frame;
determining the newly added image as an abnormal image in the case that abnormal image areas among a plurality of target image frames are matched;
wherein the abnormal image region is determined by the first image recognition model.
2. The monitored image detection method according to claim 1, wherein acquiring each target image frame whose image tag is an abnormal image tag from each of the first image frames, comprises:
acquiring an alternative image frame with an image tag being an abnormal image tag from each first image frame;
and determining the position information of the abnormal image area of the alternative image frame, which is in a preset area corresponding to the monitoring image to which the alternative image frame belongs, and determining the alternative image frame as the target image frame.
3. The monitored image detection method according to claim 1, wherein said performing image recognition on at least one of the non-newly added images according to the pre-trained second image recognition model to obtain a detection result of the non-newly added image comprises:
acquiring the non-newly added images with the accumulated times smaller than the preset times from the non-newly added images according to the accumulated times of image identification of the non-newly added images;
and carrying out image recognition on the non-newly added images with the accumulated times smaller than the preset times according to the second image recognition model to obtain detection results of the non-newly added images with the accumulated times smaller than the preset times.
4. The monitored image detection method according to claim 3, wherein said cumulative number of times is determined by the number of times said non-newly added image is subjected to image recognition by said second image recognition model.
5. The method for detecting a monitored image according to claim 1, 3 or 4, wherein the performing image recognition on at least one of the non-newly added images according to the pre-trained second image recognition model to obtain a detection result of the non-newly added image comprises:
inputting each second image frame of the non-newly added image into the second image recognition model to obtain an image tag of the second image frame;
and obtaining a detection result of the non-newly added image according to the image label of the second image frame.
6. A monitoring image detecting apparatus, comprising:
the monitoring image acquisition module is used for acquiring each newly added image and each non-newly added image from each monitoring image according to the moment when each monitoring image is stored in the database for the first time;
the newly added image detection module is used for carrying out image recognition on at least one newly added image according to a pre-trained first image recognition model to obtain a detection result of the newly added image; the method comprises the steps of,
the non-newly added image detection module is used for carrying out image recognition on at least one non-newly added image according to a pre-trained second image recognition model to obtain a detection result of the non-newly added image;
the added image is the monitoring image with the accumulated time length stored in the database for the first time being smaller than the preset time length, and the non-added image is the monitoring image with the accumulated time length stored in the database for the first time being not smaller than the preset time length;
the second image recognition model is larger than the first image recognition model;
the newly added image detection module is specifically configured to:
inputting each first image frame of the newly added image into the first image recognition model to obtain an image label of each first image frame;
acquiring each target image frame with the image tag being an abnormal image tag from each first image frame;
determining the newly added image as an abnormal image in the case that abnormal image areas among a plurality of target image frames are matched;
wherein the abnormal image region is determined by the first image recognition model.
7. An electronic device comprising a processor and a memory storing a computer program, wherein the processor implements the surveillance image detection method of any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the surveillance image detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310531083.6A CN116311080B (en) | 2023-05-12 | 2023-05-12 | Monitoring image detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310531083.6A CN116311080B (en) | 2023-05-12 | 2023-05-12 | Monitoring image detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116311080A CN116311080A (en) | 2023-06-23 |
| CN116311080B true CN116311080B (en) | 2023-09-12 |
Family
ID=86788989
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310531083.6A Active CN116311080B (en) | 2023-05-12 | 2023-05-12 | Monitoring image detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116311080B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027397A (en) * | 2019-11-14 | 2020-04-17 | 上海交通大学 | Method, system, medium and device for detecting comprehensive characteristic target in intelligent monitoring network |
| CN114120023A (en) * | 2020-09-01 | 2022-03-01 | 顺丰科技有限公司 | Method and device for identifying copied image and computer readable storage medium |
| CN114283378A (en) * | 2021-12-22 | 2022-04-05 | 京东方科技集团股份有限公司 | Monitoring method, monitoring device, storage medium and electronic equipment |
| WO2022116720A1 (en) * | 2020-12-02 | 2022-06-09 | 歌尔股份有限公司 | Target detection method and apparatus, and electronic device |
| CN115116004A (en) * | 2022-06-27 | 2022-09-27 | 禾麦科技开发（深圳）有限公司 | Office area abnormal behavior detection system and method based on deep learning |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102592076B1 (en) * | 2015-12-14 | 2023-10-19 | 삼성전자주식회사 | Appartus and method for Object detection based on Deep leaning, apparatus for Learning thereof |
-
2023
- 2023-05-12 CN CN202310531083.6A patent/CN116311080B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027397A (en) * | 2019-11-14 | 2020-04-17 | 上海交通大学 | Method, system, medium and device for detecting comprehensive characteristic target in intelligent monitoring network |
| CN114120023A (en) * | 2020-09-01 | 2022-03-01 | 顺丰科技有限公司 | Method and device for identifying copied image and computer readable storage medium |
| WO2022116720A1 (en) * | 2020-12-02 | 2022-06-09 | 歌尔股份有限公司 | Target detection method and apparatus, and electronic device |
| CN114283378A (en) * | 2021-12-22 | 2022-04-05 | 京东方科技集团股份有限公司 | Monitoring method, monitoring device, storage medium and electronic equipment |
| CN115116004A (en) * | 2022-06-27 | 2022-09-27 | 禾麦科技开发（深圳）有限公司 | Office area abnormal behavior detection system and method based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 彭彪等."运动目标检测与特征提取算法的多层并行优化".《电视技术》.2014,第173-177页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116311080A (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107528832B (en) | Baseline construction and unknown abnormal behavior detection method for system logs | |
| CN111382623A (en) | Live broadcast auditing method, device, server and storage medium | |
| CN112434178B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN113095132A (en) | Neural network-based gas field target detection and behavior identification method, system, terminal and storage medium | |
| CN111523362A (en) | Data analysis method and device based on electronic purse net and electronic equipment | |
| CN113269042B (en) | Intelligent traffic management method and system based on driving vehicle violation identification | |
| CN112800975A (en) | Behavior identification method in security check channel based on image processing | |
| CN106571040B (en) | Suspicious vehicle confirmation method and equipment | |
| CN108629310B (en) | Engineering management supervision method and device | |
| CN115410112A (en) | Elevator abnormal stay recognition method and device and storage medium | |
| CN112949456B (en) | Video feature extraction model training and video feature extraction method and device | |
| CN116311080B (en) | Monitoring image detection method and device | |
| CN117114420B (en) | Image recognition-based industrial and trade safety accident risk management and control system and method | |
| CN112347889B (en) | Substation operation behavior identification method and device | |
| CN116884041B (en) | Forestry insect disaster prediction method and system based on regional historical data | |
| CN115083004B (en) | Identity recognition method and device and computer readable storage medium | |
| CN115424253A (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN115761842A (en) | Automatic updating method and device for human face base | |
| CN111313541A (en) | Transformer equipment exception handling method and system | |
| CN111222370A (en) | Case studying and judging method, system and device | |
| WO2022262013A1 (en) | Method and system for collecting and monitoring vehicle state information | |
| CN111709340B (en) | Umbrella use behavior detection method and system | |
| CN114241363A (en) | Process identification method, process identification device, electronic device, and storage medium | |
| CN113723698A (en) | Pig farm live pig monitoring and predicting method and system, electronic device and storage medium | |
| CN113569879A (en) | Training method of abnormal recognition model, abnormal account recognition method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |