WO2016172870A1

WO2016172870A1 - Video monitoring method, video monitoring system and computer program product

Info

Publication number: WO2016172870A1
Application number: PCT/CN2015/077783
Authority: WO
Inventors: 俞刚; 李超; 尚泽远; 何奇正
Original assignee: 北京旷视科技有限公司; 北京小孔科技有限公司
Priority date: 2015-04-29
Filing date: 2015-04-29
Publication date: 2016-11-03
Also published as: CN105493502A; US9875405B2; US20160379049A1; CN105493502B

Abstract

The present disclosure relates to a video monitoring method, a video monitoring system and a computer program product. The video monitoring method comprises: acquiring video data collected by a video data collection apparatus; and based on pre-set scenario information and the video data, executing the determination and statistics of a monitored object in the scenario.

Description

Video monitoring method, video monitoring system, and computer program product

Technical field

The present disclosure relates to the field of video surveillance, and more particularly, to video surveillance methods, video surveillance systems, and computer program products.

Background technique

Current video surveillance often relies on manual detection and processing by staff. Therefore, although many scenes have camera coverage, since there is no intelligent video monitoring method, a large amount of manpower is required for processing and monitoring. Moreover, when the coverage of the surveillance camera becomes large, it is difficult to efficiently process and react to some emergencies.

Intelligent video surveillance can automatically estimate the density of people in the scene according to the video data of the camera, and monitor the flow of people and traffic in the video scene in real time. Intelligent monitoring has a very wide application environment. For example, from the perspective of urban security, by monitoring the density of the crowd, when the crowd density is too large, an alarm can be given and processed in time to avoid some possible emergencies. Statistics on the number of cars on urban roads can give crowded status in real time and make recommendations for drivers' travel. For the traffic statistics of the mall, you can intelligently provide some consumer suggestions and analysis.

There are two main approaches to the known intelligent flow monitoring methods. The first one relies on the pedestrian detection algorithm, but its statistical data is not accurate enough for high-density flow areas or occlusions. In addition, the pedestrian detection algorithm itself is time-consuming and difficult to achieve real-time monitoring. Another way is to extract some features (such as edges, textures, etc.) from the current image independently of the pedestrians, and perform regression analysis based on these features and some training data to obtain the number of pedestrians in the current region. The accuracy of such methods tends to be low. When the pedestrian density is relatively large, the effect is acceptable, but when the number of pedestrians decreases, the error will be large. In addition, the single regression method is difficult to distinguish between pedestrians and cars, so the application scenario is very limited.

Summary of the invention

The present disclosure has been made in view of the above problems. The present disclosure provides a video monitoring method based on depth video, a video monitoring system, and a computer program product, which can accurately calculate and analyze the population density, the traffic volume, and the number of people flowing in the current scene in real time.

According to an embodiment of the present disclosure, a video monitoring method is provided, including: obtaining via The video data collected by the video data collecting device; and based on the preset scene information and the video data, performing determination and statistics of the monitoring object in the scene for monitoring corresponding to the scene information.

Further, a video monitoring method according to an embodiment of the present disclosure, wherein the setting of the scene information in advance includes configuring a scene for monitoring, and setting a monitoring area and a monitoring line in a scene for monitoring.

Further, the video monitoring method according to an embodiment of the present disclosure, wherein the setting the scene information in advance further comprises: performing probability estimation on each pixel point in the monitoring area based on a predetermined model to acquire a probability of each pixel point a model for determining foreground data and background data in the video data.

In addition, the video monitoring method according to an embodiment of the present disclosure, wherein the setting the scene information in advance further includes: marking a location of the plurality of first type monitoring objects in the monitoring area, and determining a first type of monitoring of each location The weight of the object relative to the area of the monitored area.

Further, the video monitoring method according to an embodiment of the present disclosure further includes determining a foreground area and a background area in the monitoring area based on the probability model, and removing a shadow area in the foreground area to obtain Monitor the foreground area of the target.

In addition, the video monitoring method according to an embodiment of the present disclosure further includes dividing the monitoring foreground area into a plurality of monitoring foreground sub-areas, wherein the plurality of monitoring foreground sub-areas are not connected to each other.

Further, a video monitoring method according to an embodiment of the present disclosure, wherein the determining and counting of the monitoring object in the scene for monitoring corresponding to the scene information is performed based on the scene information and the video data, including: Performing edge detection on each of the plurality of monitoring foreground sub-regions, acquiring contour information of the monitoring foreground sub-region, and if the contour information satisfies a predetermined contour condition, determining that the monitoring object corresponding to the contour information is The second type of monitoring object.

Further, a video monitoring method according to an embodiment of the present disclosure, wherein the determining and counting of the monitoring object in the scene for monitoring corresponding to the scene information is further performed based on the scene information and the video data, further including Removing a second region corresponding to the second type of monitoring object from each of the plurality of monitoring foreground sub-regions to obtain a first region corresponding to the first type of monitoring object, based on the first region The area and the weight corresponding to the first type of monitoring object determine a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-areas.

Moreover, the video monitoring method according to an embodiment of the present disclosure further includes generating warning information when the first number is greater than a predetermined threshold.

Further, a video monitoring method according to an embodiment of the present disclosure, wherein the determining and counting of the monitoring object in the scene for monitoring corresponding to the scene information is further performed based on the scene information and the video data, further including Determining, in a first direction perpendicular to the monitoring line, based on a moving speed of the first type of monitoring object on the monitoring line, passing the monitoring line in the first direction during a predetermined time interval An area of the first area, and based on a weight of the first type of monitoring object on the monitoring line, determining the first pass of the monitoring line in the first direction during the predetermined time interval The number of class monitoring objects.

In accordance with another embodiment of the present disclosure, a video surveillance system is provided, comprising: a processor; a memory; and computer program instructions stored in the memory, executed when the computer program instructions are executed by the processor The following steps: acquiring video data collected via the video data collecting device; and performing determination and statistics of the monitoring object in the scene for monitoring corresponding to the scene information based on the preset scene information and the video data.

Further, a video monitoring system according to another embodiment of the present disclosure, wherein the step of presetting the scene information is performed when the computer program instructions are executed by the processor, the step of presetting the scene information includes : Configure the scenario for monitoring and set the monitoring area and monitoring line in the scenario for monitoring.

Further, a video monitoring system according to another embodiment of the present disclosure, wherein the step of performing the setting of the scene information in advance when the computer program instructions are executed by the processor further comprises: pairing the monitoring area based on a predetermined model Each pixel is subjected to probability estimation to obtain a probability model of each pixel for determining foreground data and background data in the video data.

Further, a video monitoring system according to another embodiment of the present disclosure, wherein the step of performing configuration presetting the scene information when the computer program instructions are executed by the processor further comprises: marking a plurality of the monitoring areas The positions of the first type of monitoring objects determine the weight of the first type of monitoring objects of each location relative to the area of the monitoring area.

Further, a video surveillance system according to another embodiment of the present disclosure, wherein when the computer program instructions are executed by the processor, the step of: determining a foreground region in the monitored area and based on the probability model A background area, and the shaded area in the foreground area is removed to obtain a monitoring foreground area as a monitoring target.

Further, a video surveillance system according to another embodiment of the present disclosure, wherein when the computer program instructions are executed by the processor, the step of: dividing the monitoring foreground area into a plurality of monitoring foreground sub-areas, The plurality of monitoring foreground sub-areas are not connected to each other.

Further, a video surveillance system according to another embodiment of the present disclosure, wherein when the computer program instructions are executed by the processor, performing monitoring based on the scene information and the video data, performing monitoring objects in the scene The determining and counting step includes: performing edge detection on each of the plurality of monitoring foreground sub-regions, acquiring contour information of the monitoring foreground sub-region, and determining, if the contour information meets a predetermined contour condition, corresponding to the The monitoring object of the contour information is the second type of monitoring object.

Further, a video surveillance system according to another embodiment of the present disclosure, wherein when the computer program instructions are executed by the processor, performing monitoring based on the scene information and the video data, performing monitoring objects in the scene The determining and counting step further includes: removing a second region corresponding to the second type of monitoring object from each of the plurality of monitoring foreground sub-regions to obtain a first region corresponding to the first type of monitoring object Determining, according to an area of the first area and a weight corresponding to the first type of monitoring object, a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-areas.

Moreover, a video surveillance system in accordance with another embodiment of the present disclosure, wherein the computer program instructions, when executed by the processor, further perform the step of generating warning information when the first number is greater than a predetermined threshold.

Further, a video monitoring system according to another embodiment of the present disclosure, wherein when the computer program instructions are executed by the processor, performing execution based on the scene information and the video data, corresponding to the scene information The determining and counting of the monitoring object in the monitored scenario further includes: determining, in the first direction perpendicular to the monitoring line, based on a moving speed of the first type of monitoring object on the monitoring line Determining an area of the first area of the monitoring line in the first direction during a predetermined time interval, and determining, during the predetermined time interval, based on a weight of the first type of monitoring object on the monitoring line The number of the first type of monitored objects passing the monitoring line in the first direction.

According to still another embodiment of the present disclosure, a computer program product is provided, comprising a computer readable storage medium on which computer program instructions are stored, the computer program instructions being executed while being executed by a computer The following steps: acquiring video data collected via the video data collection device; and performing determination and statistics of the monitoring object in the scenario based on the preset scene information and the video data.

It is to be understood that both the foregoing general description

DRAWINGS

The above as well as other objects, features and advantages of the present invention will become more apparent from the embodiments of the invention. The drawings are intended to provide a further understanding of the embodiments of the invention, In the figures, the same reference numerals generally refer to the same parts or steps.

FIG. 1 is a flow chart illustrating a video monitoring method according to an embodiment of the present invention.

2 is a functional block diagram illustrating a video surveillance system in accordance with an embodiment of the present invention.

FIG. 3 is a flowchart further illustrating setting scene information in a video monitoring method according to an embodiment of the present invention.

4 is a flow chart further illustrating pre-processing of video data in a video surveillance method in accordance with an embodiment of the present invention.

5A to 5C are diagrams illustrating preprocessing of video data in a video monitoring method according to an embodiment of the present invention.

FIG. 6 is a flow chart further illustrating the statistics of the number of monitored objects in the video monitoring method according to an embodiment of the present invention.

FIG. 7 is a schematic diagram illustrating type determination of a monitoring object in a video monitoring method according to an embodiment of the present invention.

FIG. 8 is a flowchart further illustrating flow statistics of a monitoring object in a video monitoring method according to an embodiment of the present invention.

9A and 9B are diagrams illustrating traffic statistics of a monitoring object in a video monitoring method according to an embodiment of the present invention.

FIG. 10 is a schematic block diagram illustrating a video monitoring system according to an embodiment of the present invention.

detailed description

In order to make the objects, the technical solutions and the advantages of the present invention more apparent, the exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is apparent that the described embodiments are only a part of the embodiments of the present invention, and are not to be construed as limiting the embodiments of the invention. Based on the embodiments of the invention described in this disclosure, those skilled in the art are All other embodiments obtained without creative efforts are intended to fall within the scope of the invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a flow chart illustrating a video monitoring method according to an embodiment of the present invention. As shown in FIG. 1, a video monitoring method according to an embodiment of the present invention includes the following steps.

In step S101, video data acquired via the video capture device is acquired. In one embodiment of the invention, the video capture device is a camera (such as an RGB camera or a depth camera) capable of acquiring video data of a subject. The video capture device is mounted and fixed in a plan view. Acquiring video data collected via the video capture device includes, but is not limited to, receiving video data transmitted from the video capture device via a wired or wireless method after the video capture device configured by the physical location is separately configured to acquire video data. Alternatively, the video capture device may be physically co-located with other modules or components in the video surveillance system or even within the same chassis, and other modules or components in the video surveillance system are received from the video capture device via the internal bus. Video data. Thereafter, the processing proceeds to step S102.

In step S102, based on the preset scene information and the video data, determination and statistics of monitoring objects in the scene for monitoring corresponding to the scene information are performed. In an embodiment of the present invention, the preset scene information includes, but is not limited to, a region of interest (ROI) and a line of interest (LOI) in a scene to be monitored. The area of interest (ROI) is the area in the scene where the number of people (number of vehicles) needs to be counted. The line of interest (LOI) is a virtual line in the scene that requires human traffic statistics. Hereinafter, the processing of setting the scene information in advance, and the processing of the determination and statistics of the monitoring object in the scene will be described in further detail with reference to the accompanying drawings.

The video monitoring method according to the embodiment of the present invention can accurately perform statistics and analysis on the population density, the traffic volume, and the number of people flowing in the current scene based on the preset scene information and the video data in real time.

Hereinafter, a video monitoring system that performs the above video monitoring method will be further described with reference to FIG.

2 is a functional block diagram illustrating a video surveillance system in accordance with an embodiment of the present invention. As shown in FIG. 2, the video surveillance system 20 according to the embodiment of the present invention includes a video data acquisition module 21, a monitoring scenario configuration module 22, a video data preprocessing module 23, and a monitoring object statistics module 24. The video data obtaining module 21, the monitoring scene configuration module 22, the video data pre-processing module 23, and the monitoring object statistics module 24 may be configured, for example, by hardware (server, dedicated computer, etc.), software, firmware, and any feasible combination thereof. .

Specifically, the video data acquiring module 21 is configured to acquire video data. In one embodiment of the present invention, the video data acquisition module 21 may be a video capture device including an RGB camera capable of acquiring video data of a subject. In another embodiment of the present invention, the video data acquisition module 21 may include a video capture device of a depth camera (depth camera) capable of acquiring depth information of a subject. The video data obtaining module 21 can be physically separated from the monitoring scene configuration module 22, the video data pre-processing module 23, and the monitoring object statistics module 24, or physically located at the same location or even inside the same chassis. In a case where the video data acquiring module 21 is physically separated from the monitoring scene configuration module 22, the video data pre-processing module 23, and the monitoring object statistics module 24, the video data acquiring module 21 further performs the wired or wireless manner. The depth video data acquired by the video capture device is sent to a subsequent module. In the case where the video data acquiring module 21 and the subsequent monitoring scene configuration module 22, the video data pre-processing module 23, and the monitoring object statistics module 24 are physically located at the same position or even inside the same casing, the video data acquiring module The depth video data acquired by the video capture device is sent to the subsequent module via the internal bus. The video data may be RGB color video data or RGBD video data including depth information. Before the video data is transmitted via wired or wireless means or via the internal bus, its predetermined format can be encoded and compressed into video data packets to reduce the amount of traffic and bandwidth required for transmission.

The monitoring scenario configuration module 22 configures a scenario for monitoring, and sets a monitoring area and a monitoring line in the scenario for monitoring. Specifically, after the operator of the video monitoring system 20 according to the embodiment of the present invention is fixed as the video data acquiring module 21 in a plan view, the monitoring area and the monitoring line in the monitoring scene captured by the video data acquiring module 21 are set. . In an embodiment of the present invention, the monitoring area is an area of interest (ROI) in a scene to be monitored as described above, that is, an area in the scene where the number of people (number of vehicles) is required to be counted. In an embodiment of the present invention, the monitoring area may be set to all areas in the entire monitoring scene captured by the video data acquiring module 21, or may be set to monitor a part of the specific area in the scene. The monitoring line is a line of interest (LOI) in a scene to be monitored as described above, and the line of interest (LOI) is a virtual line in the scene that requires human traffic statistics, which may be parallel to the ground in the monitoring scene. flat. In an embodiment of the present invention, the monitoring line is a straight line parallel to the ground plane in a plane such as an entrance and exit in a monitoring scene, and statistics of the flow of people passing through the monitoring line to obtain the flow of people entering and leaving the entrance and exit.

Further, the monitoring scenario configuration module 22 is configured in the monitoring area based on a predetermined model. Probability estimation is performed for each pixel to obtain a probability model for each pixel. In an embodiment of the present invention, the monitoring scene configuration module 22 performs probability estimation on a single pixel by using a multi-Gaussian model to obtain a probability model of each pixel point, and uses the model to determine foreground data and background for the acquired video data. data. After configuring the scene for monitoring, the probability model is the probability that the video data is background data. Since the background is relatively fixed and the foreground is arbitrary, then the foreground data is different from the background data.

Further, the monitoring scenario configuration module 22 marks the locations of the plurality of first type of monitoring objects in the monitoring area, and determines the weight of the first type of monitoring objects of each location relative to the area of the monitoring area. In an embodiment of the present invention, the first type of monitoring object may be a pedestrian, and the second type of monitoring object to be described below may be a vehicle. In an embodiment of the present invention, the locations of the plurality of pedestrians (the first type of monitoring objects) are manually marked according to the acquired video data in the monitoring area, and the weights of the pedestrians in each of the above locations are estimated by a regression algorithm. That is, the weight is the area occupied by a pedestrian in the video image at that location. It is easy to understand that the weight is related to the relative position of the camera and the scene. Actually, the farther away from the camera, the smaller the size occupied by the pedestrian on the image, so the weight given to the position is larger.

The video data pre-processing module 23 performs pre-processing of the video data on the video data acquired by the video data acquiring module 21. In an embodiment of the present invention, the video data pre-processing module 23 determines the foreground area and the background area in the monitoring area based on the probability model determined by the monitoring scene configuration module 22, and removes the foreground. A shaded area in the area to obtain a monitored foreground area as a monitoring target. Further, the video data pre-processing module 23 divides the monitoring foreground area into a plurality of monitoring foreground sub-areas, and the plurality of monitoring foreground sub-areas are not connected to each other.

The monitoring object statistic module 24 performs the determination and statistics of the monitoring object in the scenario based on the scenario information set by the monitoring scenario configuration module 22 and the video data acquired by the video data acquiring module 21. In an embodiment of the present invention, the monitoring object statistic module 24 performs edge detection on each of the plurality of monitoring foreground sub-regions obtained by the video data pre-processing module 23, and acquires the monitoring foreground sub-region. Contour information. If the profile information satisfies a predetermined profile condition, it is determined that the monitored object corresponding to the profile information is a second type of monitoring object (vehicle). Further, the monitoring object statistic module 24 removes a second region corresponding to the second type of monitoring object from each of the plurality of monitoring foreground sub-regions to obtain a monitoring object corresponding to the first category (pedestrian) a first region based on the area of the first region and corresponding to The weight determined by the monitoring scenario configuration module 22 of the first type of monitoring object determines a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-regions. Further, the monitoring object statistics module 24 generates warning information when the first number is greater than a predetermined threshold. In another embodiment of the present invention, the monitoring object statistics module 24 performs statistics on human traffic for the monitoring line set by the monitoring scenario configuration module 22. The monitoring object statistic module 24 determines, in a first direction perpendicular to the monitoring line, based on a moving speed of the first type of monitoring object on the monitoring line, in the first direction during a predetermined time interval. Determining, by the monitoring, the area of the first area of the monitoring line, and based on the weight of the first type of monitoring object on the monitoring line, passing the monitoring in the first direction during the predetermined time interval The number of said first type of monitored objects of the line.

Hereinafter, various specific steps of the video monitoring method according to the embodiment of the present invention executed by the respective modules of the video monitoring system according to the embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 3 is a flowchart further illustrating setting scene information in a video monitoring method according to an embodiment of the present invention. As shown in FIG. 3, the process of setting scene information in the video monitoring method according to the embodiment of the present invention includes the following steps.

In step S301, a video data collecting device is configured. A camera as the video capture device is installed in a scene to be monitored. Typically, the camera is mounted at a height of 2-3.5 meters and its viewing angle is looking down on the ground. Thereafter, the processing proceeds to step S302.

In step S302, a scenario for monitoring is configured, and a monitoring area and a monitoring line in the scene for monitoring are set. As described above, the monitoring area is an area in the scene to be monitored in which the number of people (number of vehicles) needs to be counted. In an embodiment of the present invention, the monitoring area may be set to all areas in the entire monitoring scene captured by the video data acquiring module 21, or may be set to monitor a part of the specific area in the scene. The monitoring line is a virtual line in the scenario where human traffic statistics are required, which may be parallel to a ground plane in the monitoring scenario. In an embodiment of the present invention, the monitoring line is a straight line parallel to the ground plane in a plane such as an entrance and exit in a monitoring scene, and statistics of the flow of people passing through the monitoring line to obtain the flow of people entering and leaving the entrance and exit. Thereafter, the processing proceeds to step S303.

In step S303, probability estimation is performed on each pixel point in the monitoring area based on the predetermined model to acquire a probability model of each pixel point. In one embodiment of the present invention, a multi-Gaussian model is used to estimate a probability of a single pixel, and a probability model of each pixel is obtained, and the foreground data and the background data are determined for the acquired video data. After configuring the scenario for monitoring, The probability model is the probability that the video data is background data. Thereafter, the processing proceeds to step S304.

In step S304, the locations of the plurality of first type monitoring objects in the monitoring area are marked, and the weight of the area of the first type of monitoring objects of each location relative to the area of the monitoring area is determined. In an embodiment of the present invention, the locations of the plurality of pedestrians (the first type of monitoring objects) are manually marked according to the acquired video data in the monitoring area, and the weights of the pedestrians in each of the above locations are estimated by a regression algorithm. That is, the weight is the area occupied by a pedestrian in the video image at that location. It is easy to understand that the weight is related to the relative position of the camera and the scene. Actually, the farther away from the camera, the smaller the size occupied by the pedestrian on the image, so the weight given to the position is larger.

After the video data collection device is installed by the process of setting the scene information in steps S301 to S304 as described above, the monitoring area and the monitoring line for monitoring, the probability model for determining the foreground and the background, and the first type of monitoring are determined. The area weight of the object in the monitoring area. The above parameters will be used for real-time monitoring based on video data thereafter.

4 is a flow chart further illustrating pre-processing of video data in a video surveillance method in accordance with an embodiment of the present invention. As shown in FIG. 4, the flow of preprocessing of video data in the video monitoring method according to an embodiment of the present invention includes the following steps.

In step S401, a foreground area and a background area in the monitoring area are determined based on the probability model. After acquiring the video data for monitoring, the foreground data and the background data in the video data, that is, the foreground area and the background area in the monitoring area are determined using the probability model determined in the above step S303. Thereafter, the processing proceeds to step S402.

In step S402, the shaded area in the foreground area is removed to obtain a monitored foreground area as a monitoring target. In one embodiment of the invention, if there is a shadow in the monitored area, the shaded portion in the foreground area is cleared by the shadow detection method. The shadow detection method can be based on two physical principles. One physical principle is that the brightness of the light in the shaded portion is darker than the brightness of the light in the non-shaded portion. Another physical principle is that the shaded portion preserves the texture information of the ground itself. The shadow detection method first finds the shadow portion and then culls it from the foreground region. Thereafter, the processing proceeds to step S403.

In step S403, the monitoring foreground area is divided into a plurality of monitoring foreground sub-areas, and the plurality of monitoring foreground sub-areas are not connected to each other. By dividing the monitoring foreground area into multiple monitoring foreground sub-areas, multiple monitoring foreground sub-areas are not connected to each other, and errors caused by estimating the number of people in the large foreground area can be avoided.

Obtained by the flow of preprocessing of the video data of steps S401 to S403 as described above The video data for monitoring is pre-processed for further statistical analysis. Hereinafter, a schematic effect of video data pre-processing will be described with reference to the drawings.

As shown in FIG. 5A, based on the probability model, the background area 51 in the monitoring area is determined.

Further, extraction of the foreground region is performed based on the determined background region 51 and the corresponding shaded portion removal. As shown in Figure 5B, foreground area 52 for further video surveillance statistical analysis is extracted.

Further, a blocking operation is performed on the foreground area 52. As shown in FIG. 5C, the extracted foreground area 52 is divided into a plurality of monitoring foreground sub-areas 52 ₁ , 52 ₂ and 52 ₃ .

Through the pre-processing of the video data as shown in Figures 5A through 5C, the plurality of monitored foreground sub-regions 52 ₁ , 52 ₂ and 52 _{3 obtained} will be used for statistical analysis of the monitored objects.

FIG. 6 is a flow chart further illustrating the statistics of the number of monitored objects in the video monitoring method according to an embodiment of the present invention. As shown in FIG. 6, the flow of monitoring the number of objects in the video monitoring method according to the embodiment of the present invention includes the following steps.

In step S601, edge detection is performed for each of the plurality of monitoring foreground sub-regions, and contour information of the monitoring foreground sub-region is acquired. In an embodiment of the present invention, processing such as Hough transform may be applied to obtain contour information in each of a plurality of monitored foreground sub-regions 52 ₁ , 52 _{2 ,} and 52 ₃ , that is, to find a monitoring foreground. The line in the area. Thereafter, the processing proceeds to step S602.

In step S602, the contour information satisfies the predetermined contour condition? In an embodiment of the invention, the predetermined contour condition may be a horizontal line ratio of the monitored object in the foreground sub-area. Assume that the pedestrians in the surveillance scene (the first type of monitoring objects) are standing, so the line on the pedestrian is mainly a straight line perpendicular to the ground, and the vehicle (the second type of monitoring object) has a longer line because of the line above the body. Parallel to the horizontal line of the ground, so that the monitoring object in the monitoring foreground sub-area can be classified as a pedestrian (a first type of monitoring object) or a vehicle (a second type of monitoring object) according to the horizontal line ratio of the monitoring object. If an affirmative result is obtained in step S602, the outline information satisfies a predetermined contour condition, that is, the outline information display outline has a longer horizontal line parallel to the ground, the processing proceeds to step S603.

In step S603, it is determined that the monitoring object corresponding to the contour information is the vehicle (the second type of monitoring object). Further, the number of the second type of monitoring objects in the statistical count of the monitoring object may be correspondingly increased by one. Thereafter, the process proceeds to step S604.

Conversely, if a negative result is obtained in step S602, the outline information does not satisfy the predetermined contour condition, the process proceeds directly to step S604.

In step S604, the second region corresponding to the second type of monitoring object is removed from each of the plurality of monitoring foreground sub-regions to obtain a first region corresponding to the pedestrian (the first type of monitoring object). Thereafter, the process proceeds to step S605.

In step S605, a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-areas is determined based on the area of the first area and the weight corresponding to the first type of monitoring object. As described above, based on the position of each of the plurality of monitoring foreground sub-areas, the size of the area occupied by each pedestrian (the first type of monitoring object) at the position is estimated based on the respective weights determined in step S304. The area of the first area obtained in step S604 is larger than the area occupied by each pedestrian (the first type of monitoring object), that is, the pedestrian in each of the plurality of monitoring foreground sub-areas can be obtained (the first type of monitoring object) The first number. Thereafter, the processing proceeds to step S606.

In step S606, it is determined whether the first number obtained in step S605 is greater than a predetermined threshold. The predetermined threshold can be understood as the population density in the presence of a safety hazard. If a positive result is obtained in step S606, that is, the first number is greater than the predetermined threshold, the process proceeds to step S607.

In step S607, warning information is generated. The operator of the video monitoring system according to the embodiment of the present invention can discover the existing security risks in real time according to the warning information, so as to take corresponding measures in time.

Conversely, if a negative result is obtained in step S606, that is, the first number is not greater than the predetermined threshold, the process returns to step S601 to continue the execution of the real-time video monitoring process.

As shown in FIG. 7, among the plurality of monitoring foreground sub-regions 52 ₁ , 52 ₂ and 52 ₃ , only the contour information of the plurality of monitoring foreground sub-regions 52 ₂ shows that they have a longer horizontal line L ₂ parallel to the ground. The monitoring foreground sub-area 52 ₂ can be determined as the presence vehicle (the second type of monitoring object). The contour information of the monitoring foreground sub-areas 52 ₁ and 52 ₃ are both displayed as straight lines H ₁ and H ₃ which are mainly perpendicular to the ground, and the monitoring foreground sub-areas 52 ₁ and 52 _{3 can be} determined to be pedestrians (first type of monitoring) Object).

In addition to being able to monitor the number of pedestrians and vehicles in the foreground area, the video monitoring method according to an embodiment of the present invention can also implement traffic statistics of the monitored object.

FIG. 8 is a flowchart further illustrating flow statistics of a monitoring object in a video monitoring method according to an embodiment of the present invention. As shown in FIG. 8, the monitoring pair in the video monitoring method according to the embodiment of the present invention The flow of the flow statistics includes the following steps.

In step S801, for the first direction perpendicular to the monitoring line, the moving speed of the first type of monitoring object on the monitoring line is determined. Thereafter, the processing proceeds to step S802.

In step S802, based on the moving speed, an area of the first area that passes the monitoring line in the first direction during the predetermined time interval is determined. Thereafter, the processing proceeds to step S803.

In step S803, based on the weights of the first type of monitoring objects on the monitoring line, the number of first type of monitoring objects that pass the monitoring line in the first direction during the predetermined time interval is determined. Specifically, according to the row human rights determined on the monitoring line in the monitoring scenario, the average size of the pedestrians on the monitoring line can be estimated. Then, the flow of people passing through the monitoring line from the first frame to the current frame (that is, the area of the first area passing through the monitoring line) is added up, divided by the average occupied area of the pedestrian above the monitoring line, that is, up to the current time. The number of pedestrians passing through the monitoring line during the scheduled time interval.

As shown in FIG. 9A, the thick solid line in the intermediate frame is the monitor line 91. The pedestrian (the first type of monitoring object) represented by the elliptical circle passes through the monitoring line 91 in two directions (from right to left and from left to right) at respective times.

As shown in FIG. 9B, for example, only the first direction from right to left is considered, and only one pedestrian is schematically shown. When the pedestrian passes the monitoring line 91, there is a certain speed. When the speed of passing through the monitoring line 91 is faster, a wider rectangle is generated, and the length of the rectangle is related to the width of the pedestrian body. As shown in Figure 9B, for time t-4 to t, the area of all rectangles is summed to be the total flow area that passes during that time period. In conjunction with the line of human rights at the monitoring line 91, the pedestrian flow passed during the time period can be determined.

FIG. 10 is a schematic block diagram illustrating a video monitoring system according to an embodiment of the present invention. As shown in FIG. 10, a video surveillance system 10 in accordance with an embodiment of the present invention includes a processor 101, a memory 102, and computer program instructions 103 stored in the memory 102.

The computer program instructions 103 may, when the processor 101 is running, implement the functions of the various functional modules of the video surveillance system in accordance with an embodiment of the present invention, and/or may perform various steps of the video surveillance method in accordance with an embodiment of the present invention.

Specifically, when the computer program instructions 103 are executed by the processor 101, the following steps are performed: acquiring video data collected via the video data collecting device; and based on a preset field The scene information and the video data perform determination and statistics of monitoring objects in the scene for monitoring corresponding to the scene information.

Further, the step of setting the scene information in advance when the computer program instructions 103 are executed by the processor 101, the step of setting the scene information in advance includes: configuring a scene for monitoring, and setting for Monitoring area and monitoring line in the monitored scene.

Further, the step of performing the setting of the scene information in advance when the computer program instructions 103 are executed by the processor 101 further comprises: performing probability estimation on each pixel point in the monitoring area based on a predetermined model to acquire each A probability model of pixels for determining foreground data and background data in the video data.

Further, the step of performing configuration to pre-set the scene information when the computer program instructions 103 are executed by the processor 101 further includes: marking locations of the plurality of first-class monitoring objects in the monitoring area, determining each The weight of the first type of monitored object of the location relative to the area of the monitored area.

Further, when the computer program instructions 103 are executed by the processor 101, the following steps are further performed: determining a foreground area and a background area in the monitoring area based on the probability model, and removing a shadow in the foreground area Area to obtain a monitoring foreground area as a monitoring target.

Further, when the computer program instructions 103 are executed by the processor 101, the following steps are further performed: dividing the monitoring foreground area into a plurality of monitoring foreground sub-areas, the plurality of monitoring foreground sub-areas not communicating with each other.

Further, when the computer program instructions 103 are executed by the processor 101, performing the determining and counting of the monitored objects in the scene based on the scene information and the video data includes: for the plurality of Monitoring each of the foreground sub-areas to perform edge detection, acquiring contour information of the monitoring foreground sub-area, and if the contour information satisfies a predetermined contour condition, determining that the monitoring object corresponding to the contour information is a second type of monitoring object.

Further, when the computer program instructions 103 are executed by the processor 101, the step of performing determination and statistics of monitoring objects in the scene based on the scene information and the video data further includes: from the plurality of Each of the monitoring foreground sub-areas removes a second area corresponding to the second type of monitoring object to obtain a first area corresponding to the first type of monitoring object, based on an area of the first area and corresponding to The weight of the first type of monitoring object determines a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-areas.

Furthermore, the following is also performed when the computer program instructions 103 are executed by the processor 101 Step: Generate warning information when the first number is greater than a predetermined threshold.

Further, when the computer program instructions 103 are executed by the processor 101, performing the step of performing determination and counting of monitoring objects in the scene based on the scene information and the video data further includes: Determining, in a first direction of the monitoring line, determining, by the moving speed of the first type of monitoring object on the monitoring line, the first area passing through the monitoring line in the first direction during a predetermined time interval Area of the first type, and based on the weight of the first type of monitoring object on the monitoring line, determining the number of the first type of monitoring objects passing the monitoring line in the first direction during the predetermined time interval .

Modules in a video surveillance system in accordance with embodiments of the present invention may be implemented by computer programs stored in a memory in a video surveillance system in accordance with an embodiment of the present invention, or may be in a computer in accordance with an embodiment of the present invention The computer instructions stored in the computer readable storage medium of the program product are implemented by the computer when executed.

The computer readable storage medium can be any combination of one or more computer readable storage media, for example, a computer readable storage medium includes computer readable program code for randomly generating a sequence of action instructions, and another computer can The read storage medium contains computer readable program code for performing face activity recognition.

The computer readable storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet, a hard disk of a personal computer, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory. (EPROM), Portable Compact Disk Read Only Memory (CD-ROM), USB memory, or any combination of the above storage media.

The exemplary embodiments of the invention are described above in detail and are not intended to be limiting. It will be understood by those skilled in the art that various modifications, combinations or sub-combinations of the embodiments may be made without departing from the spirit and scope of the invention.

Claims

A video monitoring method includes:

Obtaining video data collected via a video data collection device;

The determination and statistics of the monitoring object in the scene for monitoring corresponding to the scene information are performed based on the preset scene information and the video data.
The video monitoring method according to claim 1, wherein the setting the scene information in advance comprises:

Configure the scenario for monitoring and set the monitoring area and monitoring line in the scenario for monitoring.
The video monitoring method of claim 2, wherein the setting the scene information in advance further comprises:

Probabilistic estimation is performed on each pixel in the monitoring area based on a predetermined model to obtain a probability model for each pixel for determining foreground data and background data in the video data.
The video monitoring method of claim 3, wherein the setting the scene information in advance further comprises:

Marking the locations of the plurality of first type of monitoring objects in the monitoring area, determining the weight of the first type of monitoring objects of each location relative to the area of the monitoring area.
The video monitoring method of claim 3, further comprising:

Based on the probability model, a foreground area and a background area in the monitoring area are determined, and a shadow area in the foreground area is removed to obtain a monitoring foreground area as a monitoring target.
The video monitoring method of claim 5, further comprising:

The monitoring foreground area is divided into a plurality of monitoring foreground sub-areas, and the plurality of monitoring foreground sub-areas are not connected to each other.
The video monitoring method according to claim 6, wherein the determining and counting of the monitoring object in the scene for monitoring corresponding to the scene information is performed based on the scene information and the video data, including:

Performing edge detection on each of the plurality of monitoring foreground sub-regions, acquiring contour information of the monitoring foreground sub-region, and if the contour information satisfies a predetermined contour condition, determining that the monitoring object corresponding to the contour information is The second type of monitoring object.
The video monitoring method according to claim 7, wherein said scene information and said said The video data, the determining and counting of the monitoring object in the scenario for monitoring corresponding to the scenario information further includes:

Removing a second region corresponding to the second type of monitoring object from each of the plurality of monitoring foreground sub-regions to obtain a first region corresponding to the first type of monitoring object, based on the first region Determining, by the area and the weight corresponding to the first type of monitoring object, a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-areas.
The video monitoring method of claim 8 further comprising:

A warning message is generated when the first number is greater than a predetermined threshold.
The video monitoring method according to claim 8, wherein the determining and counting the monitoring object in the scene for monitoring corresponding to the scene information is further performed based on the scene information and the video data, further comprising:

For a first direction perpendicular to the monitoring line, based on a moving speed of the first type of monitoring object on the monitoring line, determining a location through the monitoring line in the first direction during a predetermined time interval Determining an area of the first area, and determining, according to the weight of the first type of monitoring object on the monitoring line, the first type that passes the monitoring line in the first direction during the predetermined time interval The number of monitored objects.
A video surveillance system comprising:

processor;

Memory; and

Computer program instructions stored in the memory perform the following steps when the computer program instructions are executed by the processor:

Obtaining video data collected via a video data collection device;

The determination and statistics of the monitoring object in the scene for monitoring corresponding to the scene information are performed based on the preset scene information and the video data.
A video surveillance system according to claim 11, wherein said step of presetting said scene information is performed when said computer program instructions are executed by said processor, said step of presetting said scene information comprising:

Configure the scenario for monitoring and set the monitoring area and monitoring line in the scenario for monitoring.
The video surveillance system of claim 12, wherein the step of performing the pre-setting of the scene information when the computer program instructions are executed by the processor further comprises:

Probabilistic estimation is performed on each pixel in the monitoring area based on a predetermined model to obtain a probability model for each pixel for determining foreground data and background data in the video data.
The video surveillance system of claim 13 wherein the step of performing configuration to pre-set the scene information when the computer program instructions are executed by the processor further comprises:

Marking the locations of the plurality of first type of monitoring objects in the monitoring area, determining the weight of the first type of monitoring objects of each location relative to the area of the monitoring area.
The video surveillance system of claim 13 wherein said computer program instructions are further operative when said processor is executed by said processor:

Based on the probability model, a foreground area and a background area in the monitoring area are determined, and a shadow area in the foreground area is removed to obtain a monitoring foreground area as a monitoring target.
The video surveillance system of claim 15 wherein said computer program instructions are further operative when said processor is executed by said processor:

The monitoring foreground area is divided into a plurality of monitoring foreground sub-areas, and the plurality of monitoring foreground sub-areas are not connected to each other.
A video surveillance system according to claim 16 wherein when said computer program instructions are executed by said processor, performing execution based on said scene information and said video data for monitoring The steps of determining and counting the monitored objects in the scenario include:

Performing edge detection on each of the plurality of monitoring foreground sub-regions, acquiring contour information of the monitoring foreground sub-region, and if the contour information satisfies a predetermined contour condition, determining that the monitoring object corresponding to the contour information is The second type of monitoring object.
A video surveillance system according to claim 17, wherein when said computer program instructions are executed by said processor, performing execution based on said scene information and said video data for monitoring The steps of determining and counting the monitored objects in the scenario also include:

Removing a second region corresponding to the second type of monitoring object from each of the plurality of monitoring foreground sub-regions to obtain a first region corresponding to the first type of monitoring object, based on the first region Determining, by the area and the weight corresponding to the first type of monitoring object, a first number of the first type of monitoring objects in each of the plurality of monitoring foreground sub-areas.
The video surveillance system of claim 18 wherein said computer program instructions are further operative when said processor is executed by said processor:

A warning message is generated when the first number is greater than a predetermined threshold.
The video surveillance system of claim 18 wherein said computer program instructions The step of performing the determining and counting of the monitoring object in the scenario for monitoring corresponding to the scenario information based on the scenario information and the video data when the processor is executed further includes:

For a first direction perpendicular to the monitoring line, based on a moving speed of the first type of monitoring object on the monitoring line, determining a location through the monitoring line in the first direction during a predetermined time interval Determining an area of the first area, and determining, according to the weight of the first type of monitoring object on the monitoring line, the first type that passes the monitoring line in the first direction during the predetermined time interval The number of monitored objects.
A computer program product comprising a computer readable storage medium having stored thereon computer program instructions, the computer program instructions, when executed by a computer, perform the following steps:

Obtaining video data collected via a video data collection device;

The determination and statistics of the monitoring object in the scene for monitoring corresponding to the scene information are performed based on the preset scene information and the video data.