KR101679148B1

KR101679148B1 - Detection System of Smoke and Flame using Depth Camera

Info

Publication number: KR101679148B1
Application number: KR1020150083913A
Authority: KR
Inventors: 권순각
Original assignee: 동의대학교 산학협력단
Priority date: 2015-06-15
Filing date: 2015-06-15
Publication date: 2016-12-06

Abstract

The present invention relates to a method of monitoring fire using a depth camera. More specifically, the depth camera is installed at a fire monitoring area, changes of a depth value photographed by the depth camera are continuously detected and analyzed, thus fire occurrence is warned. The present invention relates to a technology to provide a highly reliable fire monitoring system for minimizing errors on detection and operation, because of environmental factors and variables, which often occur at a fire monitoring sensor such as a smoke sensor, a flame sensor, and a heat sensor which are generally used, and detecting a symptom of the fire such as smoke and flame in advance. According to the method of monitoring fire using a depth camera of the present invention, a fire monitoring target, which is an area of interest of a depth image data analysis change, is selected, a depth camera module is installed, object recognition with respect to the selected fire monitoring target is performed, and a background is detected. Next, depth image is photographed in real time, and every depth image data is stored in a DB module. At this time, an algorithm is sequentially applied, in which changes on the depth image data are analyzed in real time while photographing the image in real time to compare with background data, thereby detecting a depth image frame which is suspicious of fire, and transmitting the data to a control console and to a speaker module for outputting a fire alarm sound at the same time. The separate modules as above are integrated into one system.

Description

Technical Field [0001] The present invention relates to a fire detection system using a depth camera,

The present invention relates to a fire detection system for detecting a depth of a fire in a fire detection zone and continuously detecting and analyzing changes in depth values photographed by a depth camera, which is frequently caused by environmental factors and variables in fire detection sensors such as a sensor, a flame detection sensor and a heat detection sensor, To a technology for providing a highly reliable fire surveillance system capable of early detection of the same fire.

Fire related accidents are the most frequent occurrences of safety accidents involving various buildings and the greatest casualties and financial losses. It is important to detect smoke early in the fire and to recognize the danger signal to the surrounding people in order to minimize the damage to the person, because the fire is more likely to cause suffocation due to toxic smoke occurring during combustion of certain substances rather than direct heat. . It is also important to detect flames and quickly spread fire alarms in order to prevent property damage due to heat generated after the smoke.

Generally, fire sensors include a smoke sensor that detects smoke generated during a fire, a heat sensor that senses heat, and a flame sensor that detects a fire by infrared and ultraviolet rays. However, in the case of the smoke sensor, the sensing performance of the sensor deteriorates rapidly because it diffuses rapidly from the well ventilated area to the outside air. In the case of a thermal sensor, it is detected only when the ambient temperature of the sensor rises. This is a disadvantage in that the detection time is not valid because the fire has advanced to a certain level or more. Since the flame detection sensor uses the infrared and ultraviolet ray detection method, the infrared ray and ultraviolet ray are absorbed by the smoke or other floating matter caused by the fire, and the sensitivity is suddenly attenuated. Also, there is a disadvantage that the malfunction probability such as responding to the welding light is high. In addition, smoke detection in the outdoors may be costly due to the fact that the sensing range is too wide to detect, or it is necessary to install a large number of sensors. In order to overcome and overcome the disadvantages due to the characteristics of each sensor described above, it is necessary to construct a fire detection system applying the technology of the image processing field.

Criteria for judging whether a fire occurs can be classified into smoke, temperature, flame, and combustion products. First, in the case of smoke, it refers to solid and liquid fine particles which are generated when incendiary combustible materials are incompletely burned when a fire occurs, and the fine particles are characterized by rising in the air. Depending on the combustion conditions and conditions, there may be a case where the flame appears immediately without generating smoke. However, most of the cases accompanied with smoke when a fire occurs, so it is important to detect the smoke in the early stage of the fire in order to recognize the fire early . Second, in the case of temperature, flame accompanied by a fire raises the ambient air temperature. If it exceeds the specific reference temperature, it can be regarded as a fire. Depending on the type and type of fire, the surrounding air temperature can rise rapidly within a short time, and in the case of slow combustion, it may take a long time to reach a specified reference temperature. However, since the surrounding air temperature is higher than the specific temperature, it may be a factor to determine whether or not a fire occurs. However, since the detection time is generally slower than smoke or flame, It can be regarded as a situation after the progress of the operation. Third, in the case of a flame, the substance emits ultraviolet infrared rays and visible rays when it is burned while emitting a flame. The flame detector is driven by a mechanism that detects the fire by detecting ultraviolet or infrared rays. When a fire is detected by using a camera image, a visible light region except for ultraviolet rays and infrared rays, that is, a color space due to light or a flame, The fire is perceived as the change of the fire. Fourth, in the case of combustion products, if the combustible material is combusted, combustion products are generated. Depending on the main constituents of the burning material, sulfur dioxide (SO ₂ ), phosphorus pentoxide (P ₂ O ₅ ), carbon monoxide CO) and carbon dioxide (CO ₂ ). The combustion products are also generated after the fire has passed for a certain period of time, so it is a factor for confirming the fire, but it does not have a great influence on the early detection side for preventing fire damage. Therefore, in order to minimize fire damage, it depends on how quickly smoke and flames generated in the early stage of fire can be detected and recognized.

The smoke is not constant in shape due to the circulation and diffusion of air, and appears in various colors depending on the type of combustion material. In order to detect this, methods such as fast cumulative motion direction model based on integral image using color difference, motion vector and smoke characteristics and smoke detection through static and dynamic analysis have been proposed. In order to detect the flame, it is necessary to detect the background and the flame separately. A background modeling method is used to detect the background, and a method of determining whether or not the flame is used by using various color space model values for the flame detection has been proposed. However, there is no optical fire detection algorithm that is suitable for all fire detection. When there is a delayed smoke or flame suddenly generated, the wind direction and wind speed of the surrounding environment, the difference of smoke generation position and distance of the sensing part, The fire detection may not be possible. In addition, it is not possible to detect with an image taken with a camera in a visible light absence space or a nighttime environment. Accordingly, the present invention provides a highly reliable fire monitoring method capable of early detection of signs of fire such as smoke and flames without being affected by environmental factors and variables by using a smoke and flame detection system using a depth camera .

A similar prior art to disclose a fire monitoring method using the depth camera described above is a study of 'indoor smoke detection based on RGB-Depth camera' in the Journal of the Institute of Electronics, Information and Communication Engineers. And other similar prior arts include KR 10-0891549 (B1) registered in the Korean Intellectual Property Office; KR 10-1224494 (B1); KR 10-1273869 (B1); KR 10-1336139 (B1); KR 10-1420684 (B1). However, the prior art does not provide a technique for a highly reliable fire monitoring method that minimizes the occurrence of false detection and malfunction due to environmental factors and variables, and can detect signs of fire such as smoke and flames at an early stage.

KR 10-0891549 (B1) KR 10-1224494 (B1) KR 10-1273869 (B1) KR 10-1336139 (B1) KR 10-1420684 (B1)

Park, Jang - Sik, 2014, "Indoor smoke detection based on RGB - Depth camera", Journal of Korean Institute of Electromagnetic Engineering and Electronics, Vol.9, No.2, pp. 155-160.

The present invention aims to satisfy the technical needs required from the background of the above-mentioned invention.

More specifically, it is an object of the present invention to minimize the occurrence of erroneous detection and malfunction due to environmental factors and variables frequently occurring in a fire detection sensor such as a smoke detection sensor, a flame detection sensor and a heat detection sensor, And to provide a highly reliable fire monitoring method capable of early detection of signs of a fire such as a flame.

The technical objects to be achieved by the present invention are not limited to the above-mentioned problems, and other technical subjects not mentioned can be clearly understood by those skilled in the art from the following description. There will be.

In order to achieve the above object, a fire monitoring method using a depth camera according to the present invention comprises: selecting a fire surveillance object, which is an area of change in depth image data analysis; installing a depth camera module; . Then, all the depth image data is stored in the DB module while performing depth image capturing in real time. At this time, depth image capture is performed in real time, depth image data change is analyzed in real time, and a depth image frame suspecting a fire is detected through contrast with background data, and transmitted to a console to output a fire alarm sound An algorithm for propagating to the speaker module is sequentially applied, and the individual modules are configured as one system.

As described above, the present invention is advantageous in that it is possible to provide a highly reliable fire monitoring system capable of minimizing the occurrence of erroneous detection and malfunction due to environmental factors and parameters, and early detection of signs of fire such as smoke and flames .

It is to be understood that the technical advantages of the present invention are not limited to the technical effects mentioned above and that other technical effects not mentioned can be clearly understood by those skilled in the art from the description of the claims There will be.

1 is an explanatory diagram for extracting depth information of a fire monitoring method using a depth camera according to an embodiment of the present invention;
2 is an exemplary view of a fire monitoring state of a fire monitoring method using a depth camera according to an embodiment of the present invention;
3 is a flowchart illustrating a fire monitoring algorithm proposed in a fire monitoring method using a depth camera according to an embodiment of the present invention;
4 is a layout diagram of major modules proposed in a fire monitoring method using a depth camera according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It is not. In the following description of the present embodiment, the same components are denoted by the same reference numerals and symbols, and further description thereof will be omitted.

An image-based fire detection system is generally proposed to distinguish flame and smoke from color images using edge, color distribution, frequency characteristics, and statistical features . Smoke detection using a camera is required to reduce false positives due to background motion or changes in the lighting environment. To this end, it is necessary to construct and utilize highly reliable fire monitoring system based on smoke detection algorithm using depth camera widely used for NUI (natural user interface) implementation. The depth camera can acquire depth information by analyzing the characteristics of the point pattern by projecting a specific infrared ray spot pattern onto an object as well as the RGB image. Infrared points projected onto smoke do not have a certain pattern and thus do not determine depth. Since the smoke detected by the depth camera has a white, gray or black color in the color image and has no depth value in the depth image, the region satisfying these two conditions can be determined as the smoke region. The depth information acquisition method of the depth camera uses a IR image of a complementary metal-oxide semiconductor (CMOS) camera having an IR pass filter simultaneously radiating a specific IR pattern, Position is used to calculate the depth displacement for each pixel location in the depth image.

FIG. 1 shows the relationship between the infrared speckle position k of the object on the object plane and the measurement variation d in the presence of an object on the reference plane. Depth Displays the 3D position of the object with the center point of the camera as the origin. The Z axis is perpendicular to the image plane, the X axis is perpendicular to the Z axis, and the direction from the depth camera to the laser. The Y axis is perpendicular to the Z axis and the X axis. It is assumed that the distance from the depth camera to the reference plane is Z ₀ , and the infrared spot on the object appears on the camera image plane. When the object moves closer or farther toward the depth camera direction, the variation d changes. The image space variation d is measured corresponding to the change of k in the object space. The following relation is defined by the similarity of triangles.

Equation (1)

Equation (2)

Z _k is the distance from the depth camera to the object plane, and b is the distance between the center of the depth camera and the center of the infrared laser. f is the focal length of the depth camera, and D is the variation in the object plane. Substituting D in Equation (1) from Equation (2) gives Equation (3).

Equation (3)

Equation (3) is a basic equation for extracting the depth, and constants Z ₀ , f and b are determined by correction.

According to the depth estimation principle of depth camera, depth is calculated for each point on the object plane, and points having similar depths are set as one object. Smoke varies periodically in concentration and does not determine the depth because the depth information of each point on the object plane is different. Therefore, since the depth information of a specific pixel can not be determined with respect to the smoke area, it is possible to regard a negative pixel as a smoke area in the depth image. FIG. 2 is a result of acquiring the color image 10 and the depth image 20 by artificially generating smoke and using a depth camera. It can be confirmed that the smoke region identified in the color image 10 can not be determined in the depth image 20 and is displayed in black. In the present invention, as shown in FIG. 2, a smoke detection method using characteristics of smoke of a depth camera is proposed. The smoke detection algorithm proposed in the present invention consists of two stages. The first step is to set the smoke candidate region for the region where the depth value is not determined in the depth image of the depth camera. However, since the depth camera can not determine the depth of an object having a black color, if the brightness of the color image is used for the smoke candidate region whose depth is not determined in the second step, You can decide. The above description will be described in more detail with reference to FIG. 3,

(S100) of selecting a depth of field image data acquisition and a depth observation of a fire object to be performed;

An object recognition process and a depth information acquisition step (S200) for the fire monitoring object selected in step S100;

A depth threshold value setting step S210 as a reference for extracting a candidate region from the current depth image acquired in the step S200;

Extracting background image data from the current depth image acquired in step 200 (S300);

Extracting depth information of a difference image using difference between current depth image data and background image data (S400);

Determining whether a depth threshold of the depth image set in step S210 is greater than a depth threshold of the difference image extracted in step S400;

Extracting the depth image data exceeding the depth threshold value in step S500 as a candidate area (S510);

(S600) analyzing the statistical characteristics by applying a probability distribution function algorithm to the depth image data constituting the candidate region extracted in the step S510;

Analyzing the smoke characteristic (S610) to determine whether the candidate region extracted in S510 is due to fire from the statistical characteristic calculated in operation S600;

If the smoke characteristic analyzed in step S610 is due to fire, step (S700) of extracting the actual smoke image and the depth-converted smoke image data in the fire monitoring area;

And transmitting the image data extracted in the step S700 to the control console and transmitting a fire alarm (S800).

FIG. 4 is a schematic diagram of a fire monitoring system 100 for implementing a fire monitoring method using a depth camera according to the present invention.

A fire monitoring object 110 to perform depth image data acquisition;

A depth camera module (120) for acquiring depth image data;

A DB module (130) for storing depth image data of the fire monitoring object (110) acquired in real time;

A data transmission module 140 for transmitting real-time depth image data and fire suspicious frame data to the control console 200 when fire suspect frame data is detected;

And a speaker module 150 for outputting a fire alarm sound upon activation of the data transmission module 140 is provided.

The speaker module 150 is installed not only near the fire monitor 110 but also on the control console 200 to synchronize with each other in real time.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, will be. Accordingly, the true scope of the present invention should be determined only by the appended claims.

100: Whole system
110: Fire watch
120: Depth camera module
130: DB module
140: Data transfer module
150: Speaker module

Claims

The implementation flow of the fire monitoring method using the depth camera,
(S100) of selecting a depth of field image data acquisition and a depth observation of a fire object to be performed;
An object recognition process and a depth information acquisition step (S200) for the fire monitoring object selected in step S100;
A depth threshold value setting step S210 as a reference for extracting a candidate region from the current depth image acquired in the step S200;
Extracting background image data from the current depth image acquired in step S200 (S300);
Extracting depth information of a difference image using difference between current depth image data and background image data (S400);
Determining whether a depth threshold of the depth image set in step S210 is greater than a depth threshold of the difference image extracted in step S400;
Extracting the depth image data exceeding the depth threshold value in step S500 as a candidate area (S510);
(S600) analyzing the statistical characteristics by applying a probability distribution function algorithm to the depth image data constituting the candidate region extracted in the step S510;
Analyzing the smoke characteristic (S610) to determine whether the candidate region extracted in S510 is due to fire from the statistical characteristic calculated in operation S600;
If the smoke characteristic analyzed in step S610 is due to fire, step (S700) of extracting the actual smoke image and the depth-converted smoke image data in the fire monitoring area;
And transmitting the image data extracted in the step S700 to the control console and propagating a fire alarm (S800).

The method according to claim 1,
The depth value of each point on the object plane is calculated by the depth camera presumption principle, the points having similar depths are set as one object, and the negative pixels are regarded as the smoke area in the depth image data. Fire detection method using camera.

The method according to claim 1,
The smoke candidate region is set to an area where the depth value is not determined in the depth camera image of the depth camera and the smoke region is determined to be the threshold value set by the user using the color image brightness of the smoke candidate region Fire monitoring method using depth camera.

delete