KR101205800B1 - Wireless video surveillance system for hierarchically detecting event and method thereof - Google Patents

Abstract

PURPOSE: A wireless image monitoring system and a method thereof are provided to increase the accuracy of event detection by determining generation of an image through a plurality of algorithms. CONSTITUTION: If an event is detected, a wireless image monitoring system obtains an image with the driving of an image sensor(S320). The system detects the event by using a background removing algorithm(S330). The system detects the event by using a body recognition algorithm(S340). The system performs storage and transmission of an image. [Reference numerals] (AA) Start; (BB,DD,FF) No; (CC,EE,GG) Yes; (HH) End; (S310) Event detection by using a infrared ray sensor?; (S320) Obtaining image through an image sensor; (S330) Event detection by using background-removing algorithm?; (S340) Event detection by using a human body-recognizing algorithm?; (S350) Storing and transmitting images

Description

Wireless video surveillance system and method for detecting incidents hierarchically {WIRELESS VIDEO SURVEILLANCE SYSTEM FOR HIERARCHICALLY DETECTING EVENT AND METHOD THEREOF}

The present invention relates to a video surveillance system, and more particularly, in a case where it is determined that an incident has occurred using an infrared sensor, a hierarchy for sequentially determining whether an incident of an image acquired using the image sensor is generated using a plurality of algorithms. The present invention relates to a wireless video surveillance system for detecting an event and a method thereof.

In general, the video surveillance system installs a plurality of cameras around the monitoring target area of various buildings, airports, roads, railways, military units, etc., stores the image information obtained from them, and displays them separately on a remote display device. And it refers to a system that can recognize and express the situation immediately when an event such as an intrusion or accident occurs.

The rapid development of low-power and low-cost technologies in the semiconductor field requires a wireless video surveillance system in which sensors are deployed in a wide range for image-based applications such as environmental monitoring and emergency response.

In a wireless video surveillance system, the battery in the function of each sensor node is the most limiting factor for the overall system performance. Huge amounts of image data captured from CMOS image sensors require a high degree of compression before they can be stored and transmitted. In addition, wireless video surveillance systems must accurately and efficiently detect and filter out non-events.

An energy aware approach is therefore required to extend the lifetime of such a system that can meet application requirements. The lifetime of a system can be months or years, depending on the application.

Accordingly, to solve the problems of the prior art, an object of the present invention is to sequentially determine whether an event occurs in an image acquired using an image sensor using a plurality of algorithms when it is determined that an event has occurred using an infrared sensor. The present invention provides a wireless video surveillance system and method for detecting an event hierarchically.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above objects, a method for detecting an event hierarchically according to an aspect of the present invention comprises the steps of: (a) detecting an event indicating the presence of a moving person in a specific area using infrared; (b) if the event is detected, obtaining an image for the specific area; And (c) primarily determining whether an event occurs in the acquired image using a background removal algorithm; And (d) if an event on the image is generated as a result of the determination, secondly determining whether an event occurs on the image using a human body recognition algorithm, and the step (c) includes the image. Sampling a plurality of background model candidate images at regular intervals, selecting one background model candidate image among the obtained plurality of background model candidate images, and using the selected background model candidate image from the image. The background image may be removed and the occurrence of an event on the image may be determined according to the removed result.

Preferably, the number of the background model candidate images may decrease when the probability of error occurrence increases as a result of event detection for the image and increase when the probability of error occurrence decreases.

Preferably, the step (d) may recognize the human body while moving a window having a predetermined size in the image at a predetermined interval, and determine whether or not an event occurs in the image according to the recognition result.

Preferably, the interval at which the window is moved may decrease when the probability of error occurrence increases as a result of event detection for the image and increase when the error probability decreases.

Through this, the present invention can improve the accuracy of event detection by sequentially determining the occurrence of the event of the image acquired using the image sensor using a plurality of algorithms when it is determined that the event occurred using the infrared sensor. It can be effective.

In addition, in the present invention, when it is determined that an event has occurred by using an infrared sensor, by sequentially determining whether an event occurs in an image acquired by using an image sensor by using a plurality of algorithms, energy consumption for event detection is reduced. It can be effected.

1 is an exemplary view showing a wireless video surveillance system according to an embodiment of the present invention.
2 is an exemplary view for explaining an operation principle of detecting an event according to an embodiment of the present invention.
3 is an exemplary view showing a wireless video surveillance method according to an embodiment of the present invention.
4 is an exemplary view showing a simulation result to which a background removal algorithm according to an embodiment of the present invention is applied.
5 is an exemplary view showing a simulation result to which a human body recognition algorithm according to an embodiment of the present invention is applied.

Hereinafter, a wireless video surveillance system and method for detecting an event hierarchically according to an embodiment of the present invention will be described with reference to FIGS. 1 to 5. It will be described in detail focusing on the parts necessary to understand the operation and action according to the present invention. Like reference numerals in the drawings denote like elements throughout the specification.

That is, in the present invention, when it is determined that an incident has occurred by using an infrared sensor, an event detection method for sequentially determining whether occurrence of an event of an image acquired by using an image sensor using a plurality of algorithms is proposed. As a plurality of algorithms, a background removal algorithm and a human body recognition algorithm may be used.

1 is an exemplary view showing a wireless video surveillance system according to an embodiment of the present invention.

As shown in FIG. 1, the wireless video surveillance system according to the present invention includes an infrared sensor 110, an event detector 120, a system controller 130, an image sensor 140, and a video encoder. 150), memory 160, transceiver 170, and the like.

The infrared sensor 110 may detect an event representing a situation in which a person moving in a specific area exists. For example, the infrared sensor 110 may be a passive infrared (PIR) sensor.

In this case, the infrared sensor 110 may be repeatedly turned on and off at a specific cycle.

When an event is detected by the infrared sensor 110, the image sensor 140 may be driven to acquire an image of a specific area. That is, the present invention intends to acquire an image through the image sensor only when an event is detected by the infrared sensor.

The event detector 120 may receive an image obtained by driving the image sensor 140 when the infrared sensor 110 detects an event. The event detector 120 determines whether an event has occurred for the received image, and applies a plurality of algorithms.

That is, the event detector 120 primarily determines whether an event occurs on an image by using a background subtraction algorithm. If an event occurs as a result of the primary judgment, the event detector 120 determines whether an event occurs on the image by using a human body detection algorithm.

2 is an exemplary view for explaining an operation principle of detecting an event according to an embodiment of the present invention.

As shown in FIG. 2, the event detector 120 according to the present invention may play a role of determining whether or not an image acquired from an image sensor is valuable in a system. That is, the event detector hierarchically determines the existence of an event.

Figure (a) shows the hierarchical decision structure for event and non-event decision of the event detector 120. In other words, the event detector consists of three stages: L1, L2, and L3.

First, the event detector determines whether or not an event has occurred at L1. If it is determined that no event has occurred, the image is discarded without going to L2. On the other hand, the event detector goes to L2 if it is determined that an event has occurred. Likewise, if the event detector is determined to have occurred in L2, the event detector goes to the final stage, L3. When the event is detected in L3, the event is finally considered as an actual event and transmitted to the backend module.

In this case, an infrared sensor, an image sensor, an event detector, a system controller, etc. may be classified as a front end module, and a video encoder, a memory, a transceiver, etc. may be classified as a backend module.

In other words, the event detector considers the image as a valuable image only when all three stages of L1, L2, and L3 are considered to have occurred, and discards the image when it is determined that no event has occurred in one of the three stages. You lose. In this case, an event is defined as one or more people appearing in an image acquired from an image sensor, and a non-event defines all cases that are not events.

Figure (b) shows the relationship between error probability and energy consumption in each stage. First of all, based on the stages, the energy consumption decreases as the error probability of not accurately determining an event increases. On the contrary, the energy consumption increases as the error probability decreases. In addition, as a whole, as the stage increases, the error probability decreases, but the energy consumption increases.

Meanwhile, in order to design such a hierarchical event detector, probability modeling is performed using the hypothesis testing method used in statistics. There are four types of hypothesis testing: true positive (tp), false positive (fp), false negative (fn), and true negative (tn). Event occurrence probabilities for each of these cases in step i of the hierarchical event detector may be defined as in Equation 1 below.

[Equation 1]

는 i단에서의 실제 사건의 발생 유무에 대한 사실값을 나타내고,

는 i단에서의 사건 탐지기의 판단에 의한 사건 발생 유무의 추정값을 나타낼 수 있다.here,

Represents the fact value for the actual occurrence of event at stage i,

May represent an estimated value of occurrence of an event by the event detector in stage i.

Also, since the event detector has a hierarchical structure, the result of the lower stage affects the upper stage. Accordingly, each i stage has a cumulative probability of occurrence of four cases from 1 stage to i stage, which may be defined as in Equation 2 below.

&Quot; (2) "

Here, p (e ₁ ) represents the probability of occurrence of an event, p (e ₀ ) may represent the probability of occurrence of a non-event, and has a relationship of p (e ₁ ) = 1-p (e ₀ ).

Accordingly, the probability of a positive decision judged to have a person in stage i and a negative decision judged to have no person may be defined as in Equation 3 below.

&Quot; (3) "

Therefore, each i-stage transmits data to i + 1 stage with the probability of P _i ^PD which is the sum of positive elements, and discards the data with the probability of P _i ^ND . P _i ^PD and P _i ^ND have a complementary relationship.

The video encoder 150 may encode an image provided from the image sensor 140 and store the encoded image in the memory 160 or provide it to another node through the transceiver 170.

The system controller 130 may control functions of various devices in the system.

3 is an exemplary view showing a wireless video surveillance method according to an embodiment of the present invention.

As shown in FIG. 3, the wireless video surveillance system according to the present invention checks whether an event is detected using an infrared sensor in a specific area (S310), and if an event is detected as a result of the checking, drives an image sensor to drive the image. An image may be acquired through the image sensor (S320).

Next, the wireless video surveillance system may determine whether or not an event occurs for the acquired image using a plurality of algorithms. That is, the wireless video surveillance system may primarily detect an event using a background removal algorithm (S330), and the operation principle thereof will be described.

The background model B ^t (p) defined at the position p and the time ^t is selected among the candidates of the n background models that satisfies a specific condition optimally. These n background model candidates are obtained by subsampling at intervals of ts in the original image, and can be stored in a set called S. The background model B ^t (p) can be defined as in Equation 4 below.

&Quot; (4) "

B ^t (p) = x _k

x _k selects an optimal candidate among the candidates of the background model in S that satisfies Equation 5 below.

[Equation 5]

Here, as the number n of background model candidates increases, the probability of obtaining an accurate background model with respect to the current image increases. This is because the basic images are constantly moving, so when comparing several frames for a specific position, the background is more likely than the object.

Therefore, as the number of background model candidates increases, the probability of error decreases. At the same time, however, the amount of computation to be processed increases, resulting in an increase in the energy consumed. Especially in environments where the background changes frequently, a large amount of samples are required to obtain an accurate background model. Conversely, as n decreases, the probability of obtaining an accurate model decreases, but also reduces the energy consumed in the event searcher.

4 is an exemplary view showing a simulation result to which a background removal algorithm according to an embodiment of the present invention is applied.

As shown in FIG. 4, the relationship between the number n of background model candidates and the error probability is shown in a performance evaluation of tracking and surveillance (PETS) image.

Figure (a) shows two people on the screen with true foreground, Figure (b) shows the background model when n is 3, and Figure (c) shows the background model when n is 10. have. Figure (b) shows an inaccurate background model because of an insufficient number of background model candidates, but in Figure (c) it can be seen that an accurate background model has been created with a sufficient number of background model candidates.

Figures (d) and (e) show the difference frames created using the generated background models of these figures (b) and (c), respectively. Here, the difference frame may indicate a frame obtained by subtracting the background model frame from the current frame. In the case of Figure (d), we can see that not only two true foregrounds but also two false foregrounds are caused by the incorrect background model. However, in Figure (e), we can see that only two true foregrounds occur because of the exact background model.

As such, the relationship between the number n of background model candidates and the error probability can be confirmed. That is, as the number n of background model candidates increases, only a true true foreground can be obtained. On the contrary, as n decreases, the probability of generating a false foreground increases. Of course, as the n changes, the amount of energy consumed in the embedded system also changes.

Next, the wireless video surveillance system may detect an incident secondly using a human body recognition algorithm (S340), and the operation principle thereof will be described. Here, the human body recognition algorithm can recognize a human body using a histogram of an oriented gradient descriptor.

When you scan a frame or image with a window of a certain size, you find the human body. The width of the moving window is called stride. Therefore, scanning with a smaller stride increases the probability of finding a person accurately, but increases the amount of computation, which also increases the energy consumed by the event detector. On the other hand, if you scan with a large stride, you will find a person in a frame, which reduces the probability of finding a person, but consumes less energy.

5 is an exemplary view showing a simulation result to which a human body recognition algorithm according to an embodiment of the present invention is applied.

As shown in Fig. 5, the relationship between the strider and the error probability is shown. You can see that there are seven children in this picture, which provide for human body recognition.

Figures (a), (b), and (c) show that the strides are 16, 8, and 4, respectively. As the stride decreases, the probability of recognizing a person increases.

Next, the wireless video surveillance system may store and transmit an image that is determined to have occurred (S350).

In addition, since the event detector is hierarchically designed, accordingly, since L _i operates, a positive decision must occur at L _{i -1} . Accordingly, the average energy consumed by the event detector E _ed can be defined as shown in Equation 6 below.

&Quot; (6) "

Here, E _i means energy consumed at stage _i , and may be calculated as P _i × t _i ^com . P _i and t _i ^com can represent the power consumption at stage _i and the computation time spent on running the algorithm. More specifically, the power consumption in the i stage means energy consumed during the process of repeatedly turning on and off the sensor in the infrared sensor, and processes one frame or image in the case of a background removal algorithm or a human body recognition algorithm. It may mean energy consumed.

을 만족하면서 시스템에서 소모되는 에너지 Esys를 최소화하게 되는데 이를 다음의 [수학식 7]과 같이 정의할 수 있다.Event detectors should operate in the direction of minimizing the energy of the whole system, and several models and constraints are needed to find the conditions that minimize the energy of the system. The first is the probabilistic model of the hierarchical event detector, the second is the characteristic model of events occurring around the system, the third is the power model of the front-end and back-end modules, and finally the system constraints. Given this input, error probability constraints in the system

To minimize the energy Esys consumed in the system while satisfying this can be defined as [Equation 7].

[Equation 7]

Here, the probability P ^inaccs that Esys and the hierarchical event detector make a wrong decision may be defined as in Equation [8].

[Equation 8]

E _sys = E _ed + E _back

= E _ed + E _{back (1)} + E _{back (0)}

= E _ed + p ₃ ^PD × (E _enc + E _{tx / mem} )

= E ₁ + p ₁ ^PD × E ₂ × p ₂ ^PD × E ₃ + p ₃ ^PD × (P _enc × t _enc + P _{tx / mem} × t _{tx / mem} )

의 false 확률로 구성될 수 있다.The energy E _back consumed by the backend modules may be divided into E _{back (1)} and E _{back (0)} . E _{back (1)} is the energy consumed when the actual event is processed in the backend modules, and E _{back (0)} may mean energy consumed when the actual non-event is processed in the backend modules. For an ideal event detector, E _{back (0)} would be zero. In addition, in [Equation 8], t _enc , t _tx , and t _mem may mean a time taken for the video encoder, the transceiver, and the memory to process data. In addition, P ^inaccs and p ₃ ^FP

It may consist of a false probability of.

Similarly, the probability P ^acc of the hierarchical event detector to make an accurate judgment is composed of a true probability and may be defined as in Equation 9 below.

&Quot; (9) "

On the other hand, between the energy consumed E _{back (1)} and E _{back (0)} there is a tradeoff according to the calculation amount of the event detector. If the event detector handles a large amount of computation, E _{back (1)} increases because the probability of the actual event being considered as an actual event and transmitted to the backend module increases. Similarly, E _{back (0)} is reduced because it determines that the actual non-event is a real non-event and does not transmit it to the backend module. On the contrary, if the event detector handles a small amount of computation, E _{back (1)} decreases because the probability of failing to judge a real event as a real event increases and the amount of data processed in the backend module is reduced. E _{back (0)} also increases because the actual non-event is misinterpreted as an event, increasing the probability that it will be processed in the backend module.

Those skilled in the art will be able to make various modifications and variations without departing from the essential features of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

110: infrared sensor
120: event detector
130: system controller
140: image sensor
150: video encoder
160: memory
170: transceiver

Claims (4)

(a) detecting an event indicating a situation in which a moving person exists in a specific area by using infrared rays;
(b) if the event is detected, obtaining an image for the specific area; And
(c) primarily determining whether an event occurs in the acquired image using a background removal algorithm; And
(d) if an event on the image is generated as a result of the determination, secondly determining whether an event occurs on the image by using a human body recognition algorithm;
To include, the step (c),
Sampling a plurality of background model candidate images from the image at predetermined intervals, selecting one of the background model candidate images among the obtained plurality of background model candidate images, and using the selected background model candidate image. Removing a background image from the image and judging whether an event has occurred for the image based on the result of the removal;
In the step (d), the human body is recognized while moving a window having a predetermined size in the image at a predetermined interval, and the event is hierarchically characterized by determining whether an event occurs in the image according to the recognized result. Method for Detecting The method according to claim 1,
The number of candidate background model images is
And decreasing the probability of error occurrence as a result of the event detection on the image and increasing the probability of error occurrence. delete The method according to claim 1,
The interval at which the window moves,
And decreasing the probability of error occurrence as a result of the event detection on the image and increasing the probability of error occurrence.

Images