JP2018088105A - Monitoring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a fire monitoring system capable of coping with changes in environment and of improving the accuracy in determining abnormality by being combined with a multilayer type neural network.SOLUTION: A first fire detector 24-1 comprising a multilayer type neural network 34-1 learned by previously prepared learning information detects whether there is a fire or not by inputting a monitoring image of a monitoring camera 16. A second fire detector 24-2 comprising a multilayer type neural network 34-2 learned by previously prepared learning information and learning information collected in a monitoring area detects whether there is a fire or not by inputting the monitoring image of the monitoring camera 16. An alarm control unit 36 outputs a fire sign alarm from a fire receiver 12 when detecting fire with the first fire detector 24-1, and outputs a fire alarm from the fire receiver 12 when detecting the fire with the second fire detector 24-2.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a monitoring system in which an abnormality is determined by a neural network from a sensor such as a fire sensor or a human sensor, or data such as an image captured by a camera, and an alarm is issued.

  Conventionally, a system for detecting a fire using a sensor for monitoring a specific physical quantity such as a smoke sensor or a heat sensor has been put into practical use.

  On the other hand, various devices and systems have been proposed in which a fire is detected by performing image processing on an image of a monitoring area captured by a monitoring camera.

  Various devices and systems have also been proposed in the crime prevention field, such as monitoring intrusions and criminal acts using human sensors and surveillance cameras.

  In such disaster prevention and crime prevention systems, early detection of abnormalities is important from the viewpoint of initial response to abnormalities.

  For this reason, in the conventional device (Patent Document 1), as a phenomenon caused by smoke from a fire from an image, a decrease in transmittance or contrast, a convergence of a luminance value to a specific value, a luminance distribution range is narrowed and a luminance distribution is reduced. , A change in average value of brightness due to smoke, a decrease in the total amount of edges, and an increase in intensity in the low frequency band are derived, and these can be comprehensively judged to detect smoke.

JP 2008-046916 A JP-A-7-245757 JP 2010-238028 A JP-A-6-325270

  However, disaster prevention and crime prevention systems that use sensors that monitor specific physical quantities can be regarded as a fire even if the monitoring criteria are met by an event that is not a fire, or the intrusion cannot be detected by avoiding human sensors, etc. There was a problem that the abnormality could not be correctly identified.

  In addition, in a conventional fire monitoring system that detects a fire from a smoke image associated with a fire, smoke features such as transmittance, contrast, and edge in the smoke image are determined in advance, and an image captured by the monitoring camera is captured. The characteristics of smoke must be generated by processing, and there are a wide variety of smoke generation situations due to fire, and it is extremely difficult to find out what the characteristics of smoke are in it. Therefore, a fire monitoring system that accurately determines smoke from a fire from a monitoring image and outputs a fire alarm is on the way to practical use.

  In addition, a system for monitoring intrusions and criminal activities with a monitoring camera has been put into practical use, but there is room for improvement in that the monitoring accuracy depends on the conditions of light and darkness in the field.

  On the other hand, in recent years, for example, images of many cats and dogs are labeled and learned by a multilayer neural network equipped with a convolutional neural network, so-called deep learning is performed, and new images are learned. A technique for presenting an already-described multi-layered neural network and determining whether it is a cat or a dog is disclosed.

  In addition, deep learning is not only used for image analysis but also for use in natural language processing and behavior analysis.

  Using such a multilayer neural network as input information, physical quantities obtained from sensors such as fire detectors and human sensors, detection results, and images of the monitoring area captured by a monitoring camera are used as input information, and abnormalities are determined from the input information. In the determination device, a large number of abnormal and non-abnormal input information is prepared at the time of learning and learned in a multilayer neural network, and the input information is learned into a learned multilayer neural network at the time of monitoring If an input is made, it is possible to construct an abnormality monitoring system that estimates an abnormality from the output with high accuracy and outputs an alarm.

  In this case, a multi-layer neural network is trained by using a large number of abnormal and non-abnormal input information prepared in the manufacturing stage of the abnormality monitoring system as supervised learning information. A judgment device equipped with a network is installed in the facility to be monitored, and sensor data such as fire detectors and human sensors installed in the surveillance area and images taken by the camera are input to the judgment device and monitored for abnormalities. It will be.

  However, the learning of the multilayer neural network performed at the manufacturing stage is not the data acquired in the actual monitoring area but the learning using the input information prepared as standard, and is input by a local sensor or monitoring camera When an actual physical quantity or monitoring image is input, there is a possibility that an abnormality cannot be estimated with sufficiently high accuracy.

  In addition, the situation during normal times and abnormalities varies depending on the site environment, and learning with standard input information may reduce the detection accuracy of abnormalities due to changes in the site environment. There is.

  In addition, if anomaly is detected only with an anomaly detector that has been learned in the installation environment, it can detect anomalies that have occurred in the field in the past, but overlearning such as a decrease in detection accuracy for anomalies that have not occurred in the past There is a problem that the detection performance cannot be guaranteed.

  The present invention combines a multilayer neural network learned from input information prepared in advance with a multilayer neural network learned from input information input at the installation location, thereby allowing for changes in the environment. An object of the present invention is to provide a monitoring system that can cope with and can improve the accuracy of abnormality determination while guaranteeing the accuracy of abnormality detection.

(Monitoring system)
The present invention provides a monitoring system comprising:
A first abnormality detector configured by a multilayer neural network learned by learning information prepared in advance, and detecting abnormality or non-abnormality by input of input information from a monitoring area;
A second abnormality detector configured by a multilayer neural network learned from learning information prepared in advance and learning information collected in the monitoring region, and detecting whether the input information is abnormal or non-abnormal;
An alarm control unit for determining and alarming an abnormality in the monitoring area based on the detection results of the first abnormality detector and the second abnormality detector;
Is provided.

(Abnormality judgment by weighted addition of two detection results)
The alarm control unit determines an abnormality from a value obtained by setting and adding a predetermined weight to each detection result of the first abnormality detector and the second abnormality detector and alarms the abnormality.

(Weight that changes according to learning amount)
The predetermined weight varies depending on the amount of learning information.

(Input information)
The first abnormality detector and the second abnormality detector detect whether the physical quantity detected by the sensor and / or the image of the monitoring area imaged by the imaging unit are input information as abnormal or non-abnormal.

(Abnormal warning warning and abnormal warning)
The alarm control unit outputs an abnormality predictor alarm when abnormality detection is determined by either the first abnormality detector or the second abnormality detector, and an abnormality is detected by the first abnormality detector and the second abnormality detector. An abnormality alarm is output when it is determined that the detection is detected.

(Learning by input information of monitoring area of second abnormality detector)
The second anomaly detector is
A learning control unit for learning by deep learning;
A storage unit that stores, as input information, a physical quantity detected by a sensor and / or an image of a monitoring area imaged by an imaging unit;
Is provided,
The learning control unit reads the input information stored in the storage unit and inputs it as learning information to the multilayer neural network of the second abnormality detector for learning.

(Fire monitoring)
The first abnormality detector and the second abnormality detector detect the fire or non-fire by inputting the input information,
The alarm control unit determines and alarms a fire in the monitoring area based on the detection results of the first abnormality detector and the second abnormality detector,
The learning control unit reads the input information stored in the storage unit based on the monitoring result of the fire receiver monitoring the fire with the fire detector, and learns the learning information to the multilayer neural network of the second abnormality detector. To learn as input.

(Fire learning by input information corresponding to the fire alarm detector)
When the fire control signal is input by the fire receiver, the learning control unit detects the fire detector that has fired out of the input information from the predetermined time until the signal input by the fire alarm. Is read from the storage unit and input to the multilayer neural network of the second anomaly detector as learning information for fire.

(Fire learning at the time of fire determination based on input information from a predetermined time before the fire bulletin)
If the fire control signal based on the fire determination operation is input after the fire receiver inputs the fire signal based on the fire alarm of the fire detector by the fire receiver, the learning control unit will fire The input information up to the time of signal input is read from the storage unit, and is input to the multilayer neural network of the second anomaly detector as learning information for fire for learning.

(Fire learning based on input information from the fire warning level)
The fire detector detects the temperature or smoke concentration and sends the detected analog value to the fire receiver to determine the fire.
When a fire is detected by the fire detector, the learning control unit receives input information from when the detected analog value exceeds a predetermined fire predictive level lower than the fire judgment level to when the fire is detected by the fire detector. The data is read out from the storage unit and input to the multilayer neural network of the second anomaly detector as learning information for fire for learning.

(Non-fire learning at the time of restoration by input information from a predetermined time before the fire transfer)
If a fire alarm signal based on a fire alarm from a fire detector is input by the fire receiver and then a recovery alarm signal based on a recovery operation is input, the learning control unit The input information up to the time of input is read from the storage unit, and is input to the multilayer neural network of the second anomaly detector as non-fire learning information for learning.

(Non-fire learning based on input information from the level of fire warning)
The fire detector detects the temperature or smoke concentration and sends the detected analog value to the fire receiver to determine the fire.
The learning control unit detects that the detection analog value is lower than the fire judgment level when a fire alarm signal based on the fire alarm of the fire detector is input by the fire receiver and then a recovery alarm signal based on the recovery operation is input. The input information from when the predetermined fire predictive level is exceeded to when the fire signal is input is read from the storage unit, and is input to the multi-layer neural network of the second anomaly detector as non-fire learning information for learning. .

(Initialization learning based on normal input information in the monitoring area)
The learning control unit reads the input information stored in the storage unit in the normal monitoring state, and inputs it as non-fire learning information to the multilayer neural network of the second anomaly detector for initialization learning.

(Timing of initialization learning)
The initialization learning timing is any one of the following: when the apparatus is started up, when a predetermined operation is performed, when there is almost no change in the input information, and when the initial time is changed every predetermined time interval. Including one or more.

(Basic effect)
The present invention is a first abnormality detection which is constituted by a multilayer neural network learned by learning information prepared in advance in a monitoring system and detects abnormality or non-abnormality by inputting input information from a monitoring area. And a second anomaly detector configured to detect abnormality or non-abnormality according to input of input information, and a multilayer neural network learned from learning information prepared in advance and learning information collected in a monitoring area Since the alarm control unit for determining and alarming the abnormality of the monitoring area based on the detection results of the first abnormality detector and the second abnormality detector is provided, the first learning information prepared in the manufacturing stage is used for the first. By learning multi-layer neural network of anomaly detector and 2nd anomaly detector, it is high from input information that cannot be judged abnormal or non-abnormal by artificial analysis In addition to this, the multi-layer neural network of the second anomaly detector can learn from actual input information in the monitoring area to cope with changes in the environment that vary depending on the installation location. The function to estimate the abnormality with high accuracy from the input information is realized, and by judging the abnormality from the detection result of the first abnormality detector and the second abnormality detector and alarming, compared to the case of a single abnormality detector, It is possible to improve abnormality determination accuracy.

(Effect of abnormality determination by weighted addition of two detection results)
In addition, the alarm control unit determines an abnormality from a value obtained by setting and adding a predetermined weight to each detection result of the first abnormality detector and the second abnormality detector and warns the abnormality. For example, a weight of 0.3 is set to the detection result of the first abnormality detector having a low abnormality estimation accuracy, and the detection result of the second abnormality detector having a high abnormality estimation accuracy corresponding to the installation location is, for example, 0. A weight of .7 is set and added, and the presence or absence of an abnormality is determined by comparing with a predetermined determination threshold value, so that an abnormality can be determined with high accuracy in response to a change in the environment of the installation location.

(Weight that changes according to learning amount)
In addition, since the predetermined weight changes depending on the amount of learning information, for example, when the amount of learning information is small soon after installation, for example, the weight of the first abnormality detector is 0.7 and the weight of the second abnormality detector is By reducing the weight of the second abnormality detector such as 0.3, the dependence on the second abnormality detector with a small learning amount can be reduced. In addition, when the time has elapsed sufficiently after installation and there is a lot of learning information, for example, the weight of the first abnormality detector is 0.3, the weight of the second abnormality detector is 0.7, and so on. By increasing the weight, it is possible to increase the dependence on the fully-detected abnormality detector.

(Effects of input information)
In addition, the first abnormality detector and the second abnormality detector detect whether the abnormality is normal or non-abnormal by inputting the physical quantity detected by the sensor or the image of the monitoring area captured by the imaging unit as input information. Therefore, by inputting the physical quantity detected by the sensor or the image of the monitoring area imaged by the imaging unit to the learned first and second abnormality detectors, the abnormality can be estimated and alarmed with high accuracy. To do.

(Effect of abnormal warning and abnormal warning)
The alarm control unit outputs an abnormality predictor alarm when abnormality detection is determined by either the first abnormality detector or the second abnormality detector, and the first abnormality detector and the second abnormality detector are output. An abnormality alarm is output when an abnormality is detected in, so that an abnormality can be detected only by one of the abnormality detectors by detecting an abnormality with one of the abnormality detectors and issuing a predictive alarm. It is possible to report any abnormalities without omission, and when both abnormality detectors detect an abnormality, the possibility of abnormality is high, so an abnormality alarm (this alarm) is output by alarming the abnormality. It is possible to reliably notify.

(Effects of fire monitoring)
In addition, the first abnormality detector and the second abnormality detector input the input information to detect whether it is a fire or a non-fire, and the alarm control unit displays the detection results of the first abnormality detector and the second abnormality detector. Based on the monitoring result by the fire receiver monitoring the fire by the fire detector, the learning control unit reads the input information stored in the storage unit, Since learning information is input to the multilayer neural network of the abnormality detector and learned, the multilayer neural network of the first abnormality detector and the second abnormality detector is a standard fire at the manufacturing stage. If the fire alarm is output based on the fire monitoring result of the receiver of the fire alarm equipment, the multi-layer neural network of the second anomaly detector has been learned from the non-fire images. For example, an image recorded in the recording device at that time is read as a fire image to learn a multilayer neural network, and if a fire alarm is output but a non-fire is detected, then the recording device A fire corresponding to a monitoring area that is actually monitored by a monitoring camera as an image pickup unit by reading a multi-layer neural network of the second anomaly detector by reading out the image recorded in the image as a non-fire image And the learning by the non-fire image is efficiently performed, and the second anomaly detector makes it possible to alarm by estimating the fire with high accuracy from the monitoring image captured by the monitoring camera.

  The same applies to physical quantities such as temperature and smoke density detected by the sensor, and the second abnormality detector can make a warning by estimating fire with high accuracy from the detection signal of the sensor.

(Fire learning by input information corresponding to the sensor that fires)
In addition, if the fire control unit receives a fire alarm signal from the fire receiver, the learning control unit will detect the fire that has been fired out of the input information from the specified time before the fire alarm signal is input. Since the input information corresponding to the sensor is read from the storage unit and input to the multilayer neural network of the second anomaly detector as learning information for fire, the input information at the location of the fire is the second anomaly. It is obtained as fire learning information used for the detector, and learning information can be automatically obtained.

  When sensor data of a fire detector or the like is used as input information, for example, when sensor data from 5 minutes before the fire alarm is read from the storage unit and the second abnormality detector is learned as fire sensor data, a predetermined time If sensor data is detected at intervals, for example, every 5 seconds, 60 sets of sensor data are obtained. Further, when the image of the monitoring camera is used as input information, for example, when the image from 5 minutes before the fire alarm is read from the recording device and the second abnormality detector is learned as a fire image, the recorded image is 30 frames. Assuming that recording is performed at a rate of 9000 seconds, 9000 images are obtained from a recorded image of 5 minutes. As mentioned above, the second anomaly detector fires from the input information with higher accuracy by easily realizing learning with a plurality of input information corresponding to the fire detector by issuing a fire detector once. Can be estimated and alarmed.

(Fire learning at the time of fire determination based on input information from a predetermined time before fire alarm)
In addition, when the fire control signal based on the fire determination operation is input after the fire detection signal from the fire detector is input, the learning control unit will start from the predetermined time before the fire transfer signal is input. The input information is read from the storage unit and input to the multi-layered neural network of the second anomaly detector as learning information for fire so that it can be learned. Therefore, it is possible to learn the second anomaly detector using learning data without errors without having to label the learning data with fire.

(Fire learning based on input information from the fire warning level)
In addition, the fire detector detects the temperature or smoke concentration and sends a detection analog value to judge the fire, and the learning control unit detects the fire when the fire detector signal detects a fire. The input information from when the predetermined fire predictive level lower than the judgment level is exceeded to when the fire is detected is read from the storage unit, and is input to the multi-layer neural network of the second abnormality detector as learning information for fire. Therefore, input information from the storage unit when the temperature and smoke density of the monitoring area detected by the analog fire detector reaches a predetermined fire sign level that is a sign of fire lower than the fire judgment level is read from the storage unit. From reading and learning the second anomaly detector to reading out a lot of input information from the initial stage of the fire until it is judged as a fire, the second anomaly detection Vessel can learn, allowing sign detection of a fire.

(Non-fire learning at the time of restoration based on input information from a predetermined time before fire alarm)
The learning control unit inputs the fire alarm signal from a predetermined time before the recovery alarm signal based on the recovery operation is input after the fire alarm signal based on the fire alarm of the fire detector is input by the receiver. Since the input information up to the time is read from the storage unit and input to the multi-layer neural network of the second anomaly detector as learning information for fire, for example, input information from 5 minutes before the fire is triggered When the second anomaly detector is learned as non-fire input information by reading from the storage unit, the second anomaly detector is learned as input information at the time of non-fire when the fire does not occur reliably As a result, learning can be performed with learning data without errors without having to label learning data with non-fire.

(Non-fire learning at the time of restoration based on input information from the fire warning level exceeded)
In addition, the fire detector detects the temperature or smoke concentration and sends a detection analog value to judge the fire, and the learning control unit inputs a fire alarm signal based on the fire alarm of the fire detector by the receiver. Storage unit that stores input information from when the detected analog value exceeds a predetermined fire warning level that is lower than the fire judgment level to when the fire warning signal is input. Because the non-fire learning information is input to the multilayer neural network of the second anomaly detector and learned, the temperature and smoke concentration in the monitoring area detected by the analog fire detector are Reached the fire predictive level when a predetermined fire predictive level that indicates a fire predictor lower than the fire judgment level has been reached, but the non-fire is due to an increase in temperature or smoke concentration due to a cause other than fire From the learning of the second anomaly detector using non-fire input information as the non-fire input information, a large amount of input information from the initial stage until it is determined non-fire is read from the storage unit. For non-fire conditions that can be learned and can be regarded as a sign of fire, non-fire can be estimated with higher accuracy and false alarms can be reliably prevented.

(Effect of initialization learning using normal monitoring input information in the monitoring area)
Further, the learning control unit reads the input information stored in the storage unit in the normal monitoring state, and inputs it as non-abnormal learning information into the multilayer neural network of the second abnormality detector so as to perform initialization learning. Therefore, the non-abnormality estimation accuracy by the second abnormality detector for the input information in the monitoring area in the normal monitoring state is improved, and as a result, the estimation accuracy for the abnormality is further improved.

(Effects of initialization learning timing)
The initialization learning timing is any of the following: when the device is started up, when a predetermined operation is performed, when there is almost no change in the input information, and when the initial time changes every predetermined time interval As a result, the non-abnormal state in the installation environment can be learned by performing initialization learning at the time of starting up the device for the second abnormality detector, and a predetermined operation has been performed. Occasionally, by performing initialization learning, a non-abnormal state can be educated at an arbitrary timing. For example, a non-abnormal state can be educated immediately when the interior changes.

  In addition, for the second abnormality detector, by performing initialization learning when there is almost no change in the sensor output / camera image, the non-abnormal state is automatically educated in a state in which the monitoring area is stable. Furthermore, by performing initialization learning so that the initial time is changed every predetermined time interval, for example, the initialization learning timing is shifted every predetermined time. It is possible to learn non-abnormality by obtaining non-abnormal learning data at time.

Explanatory drawing showing the outline of a fire monitoring system that monitors fire with a surveillance camera and fire detector Explanatory diagram showing the functional configuration of a judgment device using a multilayer neural network that estimates fire from images captured by a surveillance camera Explanatory drawing showing the functional configuration of the multilayer neural network shown in FIG. The flowchart which showed the learning control of the multilayer type neural network of a 2nd fire detector in conjunction with the fire monitoring of the fire receiver by the learning control part of FIG. Explanatory drawing showing the time change of the detected analog value detected by the analog fire sensor Explanatory drawing which showed other embodiment of the determination apparatus using the multilayer type neural network which estimates a fire from the image imaged with the surveillance camera Explanatory diagram showing the outline of a fire monitoring system that monitors fire with an analog fire detector that functions as a sensor Explanatory drawing showing the functional configuration of a judgment device using a multilayer neural network that estimates fires from detection signals from analog fire detectors The time chart which showed the time change of the smoke density detected with the analog fire detector memorize | stored in the time series data generation part of FIG.

[Outline of fire monitoring system]
FIG. 1 is an explanatory diagram showing an outline of a fire monitoring system for monitoring a fire with a monitoring camera and a fire detector.

  As shown in FIG. 1, monitoring cameras 16-1 and 16-2 functioning as imaging means are installed in monitoring areas 14-1 and 14-2 of facilities such as buildings, respectively. 16-1, the monitoring area 14-2 is captured by the monitoring camera 16-2.

  The monitoring areas 14-1 and 14-2 are described as the monitoring area 14 when there is no need for distinction, and the monitoring cameras 16-1 and 16-2 are described as the monitoring camera 16 when there is no need for distinction.

  The surveillance camera 16 captures an RGB color image at, for example, 30 frames / second and outputs it as a moving image. One frame has a pixel arrangement of, for example, vertical and horizontal 4056 × 4056 pixels.

  In addition, on-off type fire detectors 18-1 and 18-2 are installed in the monitoring areas 14-1 and 14-2, respectively, and the temperature or smoke density due to the fire is detected and exceeds a predetermined threshold level. In this case, a fire alarm signal is output. The fire detectors 18-1 and 18-2 are described as the fire detectors 18 when there is no need to distinguish them.

  A judgment device 10 and a fire receiver 12 of a fire alarm facility are installed in a disaster prevention monitoring center or a manager room of the facility with respect to the monitoring area 14. The determination device 10 and the fire receiver 12 may be integrated. A monitoring camera 16 installed in the monitoring area 14 is connected to the determination apparatus 10 by a signal cable 20 and a moving image captured by the monitoring camera 16 is input.

  From the fire receiver 12, a sensor line 22 is drawn out to the monitoring area 14, and a fire sensor 18 is connected to each sensor line 22.

  The determination device 10 includes a first fire detector and a second fire detector each having a multilayer neural network. The multilayer neural network of the first fire detector and the second fire detector is learned by input information prepared in advance as standard in the manufacturing stage, and the multilayer neural network of the second fire detector is When a fire alarm signal from the fire detector 18 is input by the fire receiver 12, a moving image as input information corresponding to the fire alarm that has fired is read from the recording device, and the second fire detector multi-layered. It is input and learned as fire learning information in the neural network.

  The moving image sent from the monitoring camera 16 is input in parallel to the first fire detector and the second fire detector in the determination device 26 in units of frames, and detection by the first fire detector and the second fire detector is performed. Based on the combination of results, a fire is determined and a fire determination signal is output to the fire receiver 12 to output a fire warning alarm indicating a fire warning or a fire alarm indicating a fire. In addition, the moving image from the monitoring camera 16 is continuously recorded by a recording device provided in the determination device 10.

  When the fire receiver 12 receives a fire alarm signal generated by the fire detector 18, the fire receiver 12 outputs a fire alarm and also includes a fire transfer signal including a signal for identifying which fire detector has been notified to the determination device 10. Is output. When a fire alarm is output from the fire receiver 12, the manager or disaster prevention officer goes to the site where the fire detector 18 has been notified to check for fire, and if a fire is confirmed, A fire determination operation is performed by the fire receiver 12. When a fire determination operation is performed by the fire receiver 12, the temporarily stopped district acoustic alarm is released, and a fire determination transfer signal is output to the determination device 10.

  If the fire alarm is non-fired in the on-site confirmation, after removing the non-fire factor, the fire receiver 12 performs a recovery operation to cancel the fire alarm state and return to the normal monitoring state. After the fire alarm is output from the fire receiver 12 in this way, when a fire recovery operation is performed without performing a fire determination operation, a recovery transfer signal is output from the fire receiver 12 to the determination device 10. .

  The judgment device 10 is recorded on the recording device corresponding to the place where the fire detector is notified, based on the fire monitoring result by the fire signal, the fire determination signal and the recovery signal output from the fire receiver 12. A moving image in which the monitoring area 14 up to the output of the fire alarm is picked up by the monitoring camera 16 is read out from the recording device, and this is read as a fire image or a non-fire image. Control to learn the neural network. For example, when the fire detector 18-1 issues a report, the moving image captured by the monitoring camera 16-1 is read from the recording device.

[Judgment device]
(Functional configuration of judgment device)
FIG. 2 is an explanatory diagram showing a functional configuration of a determination apparatus using a multilayer neural network that estimates a fire from an image captured by a monitoring camera.

  As shown in FIG. 2, the determination apparatus 10 is provided with a first fire detector 24-1 that functions as a first abnormality detector and a second fire detector 24-2 that functions as a second fire detector. The first fire detector 24-1 includes an image input unit 32-1 and a multilayer neural network 34-1, and the second fire detector 24-2 includes an image input unit 32-2 and a multilayer neural network 34-2. The outputs of the multilayer neural networks 34-1 and 34-2 are input to the alarm control unit 36.

  The multilayer neural networks 34-1 and 34-2 are described as the multilayer neural network 34 when there is no need to distinguish between them, and the first fire detector 24-1 and the second fire detector 24-2 are particularly distinguished. When it is not necessary, it is described as a fire detector 24.

  In order to learn the multilayer neural network 34-2 for the second fire detector 24-2, a recording device 26, a learning image holding unit 28, and a learning control unit 30 functioning as a storage unit are provided. Here, the functions of the first fire detector 24-1, the second fire detector 24-2, the learning image holding unit 28, the learning control unit 30, and the alarm control unit 36 are the functions of the computer circuit corresponding to the processing of the neural network. This is realized by executing a program by the CPU.

  The multilayer neural networks 34-1 and 34-2 of the first fire detector 24-1 and the second fire detector 24-2 are based on input information that becomes a large number of fire images prepared in advance as standard in the manufacturing stage. Have been learning. Furthermore, the multilayer neural network 34-2 of the second fire detector 24-2 issues a fire when a signal from the fire alarm of the fire detector 18 is input to the learning control unit 30 by the fire receiver 12. A moving image as input information corresponding to the fire detector 18 is read from the recording device 26, and is input to the multilayer neural network 34-2 of the second fire detector 24-2 as learning information for fire to be learned. .

  In the normal monitoring state, the first fire detector 24-1 and the second fire detector 24-2 are multi-layered images of the monitoring area captured by the monitoring camera 16 via the image input units 32-1 and 32-2. It inputs to the neural networks 34-1 and 34-2 in parallel, detects whether it is fire or non-fire from the output value, and outputs it to the alarm control unit 36.

  The alarm control unit 36 determines a fire in the monitoring area 14 based on the detection results of the first fire detector 24-1 and the second fire detector 24-2, outputs a fire determination signal to the fire receiver 12, and issues an alarm. Is output.

  In the present embodiment, when the first fire detector 24-1 detects a fire, the alarm control unit 36 outputs a fire predictor signal to the fire receiver 12 to output a fire predictor alarm, thereby detecting the second fire detection. When a fire is detected by the device 24-2, a fire determination signal is output to the fire receiver 12 to output a fire alarm.

  The first fire detector 24-1 learns using input information prepared as standard, and has sufficiently high accuracy when an actual monitoring image input by the monitoring camera 16 in the monitoring area 14 is input. The second fire detector is learning from the actual fire image input from the surveillance camera 16 in addition to learning using the input information prepared as standard. Therefore, there is a possibility that it is impossible to estimate a fire that has not occurred in the past due to over-learning. Therefore, either the first fire detector 24-1 or the second fire detector 24-2 detects a fire. If this happens, a fire warning warning is output to call attention.

  On the other hand, when the fire is detected by both the first fire detector 24-1 and the second fire detector 24-2, the fire can be estimated with sufficiently high accuracy, and a fire alarm (this alarm) is output. The fire can be reported reliably.

  The recording device 26 records a moving image in the monitoring area imaged by the monitoring camera 16, and can partially read the recorded moving image in response to an external reproduction instruction.

  The learning control unit 30 records the recording device 26 when a fire alarm is output from the fire receiver 12 based on the fire report signal E1, the fire determination report signal E2, and the recovery report signal E3 from the fire receiver 12. The necessary part of the moving image is read out and temporarily stored in the learning image holding unit 28. Images are sequentially read out from the moving image held in the learning image holding unit 28 in units of frames, and the second fire detector 24-2. Is input as a supervised fire image to the multilayer neural network 34-2 through the image input unit 32-2, and the multilayer neural network 34-2 is learned by a learning method such as a back propagation method (error back propagation method). To learn the weights and bias.

  Here, the multi-layer neural network 34-1 of the first fire detector 24-1 learned by the standard input information prepared in advance and the standard input information and the recording device 26 are read out. When an image of the monitoring area captured by the monitoring camera 16 is input to the multilayer neural network 34-2 of the second fire detector 24-2 learned from the fire image at the time of fire, the estimated value corresponding to the fire image Is output.

  This estimated value has an expected value of 1 for a fire image used for learning, 0 for a non-fire image, and has a value in the range of 0 to 1 when an actual image is input. The expected value is compared with a predetermined threshold value, for example, 0.5, and if it is equal to or greater than the threshold value, a fire detection signal is output to the alarm control unit 36. Is input to the fire receiver 12 to output a fire predictor alarm, and when a fire detection signal is input from the second fire detector 24-2, a fire determination signal is output to the fire receiver 12. Output a fire alarm.

  The determination device 10 is provided with a monitor device, and when the alarm control unit 36 determines a fire sign or fire, an image of the monitoring area where the fire is detected by the monitoring camera 16 is displayed on the screen, and the fire receiver The fire manager may be able to confirm the fire from the person in charge of the fire warning warning from 12 or the person in charge of disaster prevention. In this case, if a fire determination switch is provided in the operation unit of the determination apparatus 10 and the fire determination switch is operated when a fire is confirmed from the monitor image, a fire notification signal is transmitted to the fire receiver 12 in the same manner as when the transmitter is operated. It is also possible to output a fire alarm from the fire receiver 12.

[Multilayer neural network]
FIG. 3 is an explanatory diagram showing a functional configuration of the multilayer neural network shown in FIG. 2, schematically showing in FIG. 3 (A) and schematically showing details in FIG. 3 (B).

  As shown in FIG. 3A, the multi-layer neural network 34 of this embodiment includes a feature extraction unit 38 and a recognition unit 40. The feature extraction unit 38 is a convolutional neural network, and the recognition unit 40 is a fully connected neural network.

  The multilayer neural network 34 is a neural network that performs deep learning (deep learning), has a deep hierarchy in which a plurality of intermediate layers are connected, and performs expression learning for feature extraction.

  An ordinary neural network requires an artificial trial-and-error operation to extract features for estimating a fire from an image. In the multilayer neural network 34, a convolutional neural network is used as a feature extraction unit 38. Then, the pixel value of the image is input, an optimum feature is extracted by learning, and is input to the fully connected neural network of the recognition unit 40 to identify whether it is fire or non-fire.

  As shown schematically in FIG. 3B, the fully connected neural network of the recognition unit 40 includes an input layer 46, a connection layer 48, a repetition of the intermediate layer 50 and the connection layer 48, and an output layer 52. .

(Convolutional neural network)
FIG. 3B schematically shows the structure of a convolutional neural network constituting the feature extraction unit 38.

  Convolutional neural networks are slightly different from normal neural networks and incorporate biological structures from the visual cortex. The visual cortex contains a receptive field that is a collection of small cells that are sensitive to a small area of the field of view, and the behavior of the receptive field can be imitated by learning weights in the form of a matrix. This matrix is called a weight filter (kernel) and becomes sensitive to similar subregions of an image, as well as the role biologically receptive fields play.

  The convolutional neural network can represent the similarity between the weighting filter and the small area by a convolution operation, and through this operation, an appropriate feature of the image can be extracted.

  As shown in FIG. 3B, the convolution neural network first performs a convolution process on the input image 42 by a weight filter 43. For example, the weighting filter 43 is a matrix filter with predetermined weights of 3 × 3 in the vertical and horizontal directions, and by performing a convolution operation while aligning the filter center with each pixel of the input image 42, nine pixels of the input image 42 are obtained. A plurality of feature maps 44a are generated by convolution with one pixel of the feature map 44a, which is a small area.

  Subsequently, a pooling operation is performed on the feature map 44a obtained by the convolution operation. The pooling calculation is a process of removing feature quantities unnecessary for identification and extracting feature quantities necessary for identification.

  Subsequently, the feature maps 44b and 44c are obtained by repeating the convolution operation and the pooling operation using the weight filters 45a and 45b in multiple stages, and the feature map 44c of the last layer is input to the recognition unit 40, and the normal full combination A fire or non-fire is estimated by the recognition unit 40 using a neural network.

  Note that the pooling calculation in the convolutional neural network does not perform the pooling calculation because the feature quantity unnecessary for distinguishing between fire and non-fire is not always clear and the necessary feature quantity may be deleted. You may do it.

[Learning multi-layer neural networks]
(Backpropagation)
A neural network composed of an input layer, a plurality of intermediate layers, and an output layer is provided with a plurality of units in each layer and combined with a plurality of units in other layers, and a weight and a bias value are set for each unit. The vector product of the input value and the weight is calculated and the bias value is added to find the sum, which is output to the next layer unit through a predetermined activation function, and reaches the final layer Forward propagation in which the value is propagated is performed.

  In order to change the weight and bias of such a neural network, a learning algorithm known as backpropagation is used. In backpropagation, supervised learning when a data set of an input value x and an expected output value (expected value) y is given to the network, and unsupervised learning when only the input value x is given to the network. In this embodiment, supervised learning is performed.

  When backpropagation is performed with supervised learning, for example, a function of the mean square error is used as an error for comparing the estimated value y * and the expected value y as a result of forward propagation through the network. To do.

  In backpropagation, the magnitude of error between the estimated value y * and the expected value y is used to propagate the value while correcting the weight and bias from the back to the front of the network. The amount corrected for each weight and bias is treated as a contribution to the error, calculated by the steepest descent method, and the error function value is minimized by changing the weight and bias values.

The learning procedure by backpropagation for the neural network is as follows.
(1) An input value x is input to a neural network and forward propagation is performed to obtain an estimated value y *.
(2) Calculate an error with an error function based on the estimated value y * and the expected value y.
(3) Backpropagation is performed on the network while updating the weight and bias.

  This procedure is repeated using different combinations of input value x and expected value y until the weight and bias error of the neural network is minimized as much as possible to minimize the value of the error function.

[Learning control linked to receiver fire monitoring]
(Learning with fire images)
The learning control unit 30 provided corresponding to the second fire detector 24-2 of the determination apparatus 10 illustrated in FIG. 2 is configured to receive a fire transfer signal E1 based on the fire alarm of the fire detector 18 by the fire receiver 12. Is input, a fire alarm is output, a fire is confirmed by an on-site confirmation by a supervisor or the like, and based on this, a fire decisive transfer signal E2 based on a fire determinating operation of the fire receiver 12 is input, for example, 5 The image of the monitoring area from the predetermined time before minutes before the fire signal E1 is input is read from the recording device 26, and the image input unit 32 is input to the multilayer neural network 34-2 of the second fire detector 24-2. -2 is input as a fire image via -2, and the estimated value y * is obtained, and backpropagation is repeated while changing the supervised image so as to minimize the error function value with respect to the expected value of fire y = 1 go It performs learning to modify the weights and biases by minimizing the difference function.

  In this case, if the recorded image is recorded as a moving image of 30 frames / second on the recording device 26, 9000 fire images are obtained from the recorded image of 5 minutes, and learning with a large number of fire images such as 9000 is easy. This makes it possible to estimate a fire with higher accuracy from a monitoring image captured by a camera and enable an alarm.

  Note that when a fire image is generated from a 5-minute video, the change between each frame image having a 1/30 second period is very small. For example, a frame image thinned out with a 1 second period is used as a supervised image. It is also good. In this case, 300 fire images are obtained from a 5-minute moving image, and a sufficient number of images are obtained for learning of the multilayer neural network 34-2 by back propagation.

(Learning with non-fire images)
In addition, the learning control unit 30 receives a fire alarm signal E1 based on the fire alarm of the fire detector 18 by the fire receiver 12 and then confirms that the fire is not fired by an on-site confirmation by a manager or the like. Since the recovery operation is performed by the fire receiver 12 and the recovery notification signal E3 based on the recovery operation is input, in this case, the fire notification signal E1 is input from a predetermined time, for example, five minutes before. Images from the recording device 26 are read out from the recording device 26, and input to the multilayer neural network 34-2 of the second fire detector 24-2 as a non-fire image via the image input unit 32-2, and the estimated value y * Learning to change the weighting and bias by repeatedly performing backpropagation while changing the supervised image so as to minimize the value of the error function with respect to the expected non-fire value y = 0.

  Also in this case, if the recorded image is recorded as a 30-frame / second moving image on the recording device 26, 9000 fire images are obtained from the 5-minute recorded image, and there are a large number of non-fire images such as 9000 images. Since learning can be easily realized, non-fire can be estimated with higher accuracy from a monitoring image captured by a camera, and false alarms can be prevented.

  In addition, when a non-fire image is generated from a 5-minute video, there is very little change between each frame image having a 1/30 second period. It is also good.

(Judgment device initialization learning)
The multilayer neural networks 34-1 and 34-2 provided in the first fire detector 24-1 and the second fire detector 24-2 of the determination apparatus 10 illustrated in FIG. The bias is initialized at random, and it is initialized by learning by back propagation using standard fire images and non-fire images prepared in advance, and in this state, it becomes a monitoring target as shown in FIG. Installed in the facility.

  In this case, the image of the monitoring area 14 captured by the monitoring camera 16 varies depending on the monitoring area 14, and is different from the standard supervised image used in the initialization learning. The multi-layer neural network 34-2 of the fire detector 24-2 is a moving image in a normal monitoring state that is captured by the monitoring camera 16 and recorded in the recording device 26 at the time of start-up after installation in the facility, that is, A non-fire movie is recorded on the recording device 26, and a recording for a predetermined time, for example, 5 minutes is read out from the non-fire video and is stored as a non-fire image in the learning image holding unit 28. The non-fire image is input as an image. Input to the multi-layer neural network 34-2 via the unit 32-2 to obtain the estimated value y *, and backpropagation so as to minimize the error from the expected non-fire value y = 0 , It is desirable to perform learning to change the weights and bias by minimizing the error function is performed repeatedly while changing the teacher image.

  As the supervised image used for this learning, it is desirable to read out from the recording device 26 different videos depending on the time zone such as morning, noon, and night, and learn them as non-fire images. .

  Initialization learning may be performed at the time of starting the apparatus. This makes it possible to first educate the non-fire state in the installation environment.

  The initialization learning may be performed when a predetermined operation is performed. Thereby, it is possible to educate the non-fire state at an arbitrary timing. For example, it is possible to educate the non-fire state immediately when the interior changes.

  The initialization learning may be performed when there is no or almost no change in sensor output / camera image. As a result, it is possible to automatically educate the non-fire state in a state where the monitoring area is stable.

  Further, the initialization learning timing may be shifted every predetermined time. For example, initialization learning is performed at 6:00, 12:00, 18:00, and 24:00 at the first time, and initialization learning is performed at 7:00, 13:00, 19:00, and 1 at the second time. Thereby, non-fire learning data can be obtained at various times, and non-fire can be learned including special states such as cooking, sunrise and sunset.

  The multi-layer neural network 34-2 of the second fire detector 24-2 that uses the monitoring image of the monitoring area 14 as a non-fire image improves the non-fire estimation accuracy for the monitoring area image in the normal monitoring state. To do. Thereafter, learning of the multilayer neural network 34-2 of the second fire detector 24-2 is performed by using a fire image or a non-fire image linked to the fire monitoring by the fire receiver 12, and the multilayer neural network 34-2 is learned. The accuracy of estimations for fires and non-fires is further improved.

(Control action by learning control unit)
FIG. 4 is a flowchart showing learning control of the multilayer neural network of the second fire detector in conjunction with the fire monitoring of the fire receiver by the learning control unit of FIG.

  As shown in FIG. 4, the learning control unit 30 causes the recording device 26 to record the moving image from the monitoring camera 16 installed in the monitoring area in step S1, and in step S2, the fire transfer signal from the fire receiver 12 is recorded. In step S3, a recorded image from a predetermined time before is read from the recording device 26 and held in the learning image holding unit 28.

  Then, it progresses to step S4, and if the learning control part 30 discriminate | determines the input of the fire decision transfer signal from the fire receiver 12, it will progress to step S5, and will discriminate | determine the input of the recovery report signal from the fire receiver 12 after that. In step S6, the recorded images for a predetermined time held in the learning image holding unit 28 are read out, and the recorded images are transferred to the multilayer neural network 34-2 via the image input unit 32-2 of the second fire detector 24-2. It is input as a fire image and learning to change weight and bias by back propagation.

  On the other hand, the learning control unit 30 does not determine the input of the fire determination signal in step S4, and determines whether or not the recovery signal is input in step S7. If it is determined in step S7 that the restoration transfer signal has been input without determination, the process proceeds to step S8, where a recorded image for a predetermined time held in the learning image holding unit 28 is read, and the second fire detector 24- The non-fire image is input to the multi-layer neural network 34 via the second image input unit 32-2, and learning to change the weight and bias by back propagation is performed.

[Fire learning by monitoring images from the fire warning level]
(Learning with fire images)
As another embodiment of the learning control by the learning control unit 30 of the determination apparatus shown in FIG. 2, an analog fire detector is installed in the alert area, and the detected analog value is detected by detecting the temperature or smoke density with the analog fire detector. In the case where a fire is judged by being sent to the fire receiver 12, images from the time when a fire sign is judged until the fire is judged are read out from the recording device 26, and the multi-layer neural network of the second fire detector 24-2 is read. The network 34-2 is learned by backpropagation.

  As shown in FIG. 5, when the temperature detected by the fire detector rises with time due to the fire that occurred at time t0, the fire predictor level is set to the fire predictor level TH1 lower than the fire judgment level TH2 at time t1. The backpropagation is performed with the image recorded during the time T from reaching the time t2 to reaching the time t2 at the fire level TH2.

  The images recorded during this time T are all fire images and do not include non-fire images. Therefore, the fire images are stored in the multilayer neural network 34-2 of the second fire detector 24-2. Learning to change the weight and bias by back-propagation can be performed to reliably improve the accuracy of identifying a fire from the input image.

  More specifically, the learning control unit 30 of the determination apparatus 10 shown in FIG. 2 detects that the detected analog value of the temperature or smoke density from the analog fire detector from the fire receiver 12 reaches the predetermined fire predictive level TH1. A warning warning is output, then the detected analog value reaches the fire level TH2, a fire alarm signal is input based on the fire alarm, a fire alarm is output, and a fire is confirmed by an on-site confirmation by the supervisor Based on this, when the fire determination signal E2 based on the fire determination operation of the fire receiver 12 is input, the video recording device records the image of the monitoring area from when the fire sign is detected until the fire notification signal is input 26, is input as a fire image via the image input unit 32-2 to the multilayer neural network 34-2 of the second fire detector 24-2, and weighted by backpropagation. It performs learning to change the bias.

(Learning with non-fire images)
In addition, the learning control unit 30 outputs a warning alarm when the detected analog value of the temperature or smoke density from the analog fire detector reaches the predetermined fire warning level TH1 from the fire receiver 12, and then the detected analog value is When fire level TH2 is reached and a fire warning signal is input based on the fire alarm, a fire alarm is output, and if the fire is not confirmed in the field confirmation by the manager, etc., the fire receiver 12 can perform a recovery operation. Since the recovery signal is input based on the recovery operation, the image of the monitoring area from when the fire warning sign is detected until the fire signal is input is read from the recording device 26, and the second fire detection is performed. The multi-layer neural network 34-2 of the device 24-2 is input as a non-fire image via the image input unit 32-2, and learning to change the weight and bias by back propagation is performed.

[Fire judgment by weighted addition of detection results of the first and second fire detectors]
FIG. 6 is an explanatory diagram showing another functional configuration of a determination apparatus using a multi-layer neural network that estimates a fire from an image captured by a monitoring camera, and weights and adds detection results of the first and second fire detectors. The feature is that a fire is judged.

  As shown in FIG. 6, the determination apparatus 10 is provided with a first fire detector 24-1 and a second fire detector 24-2, and the first fire detector 24-1 is connected to the image input unit 32-1. The second fire detector 24-2 is composed of an image input unit 32-2 and a multilayer neural network 34-2.

  The second fire detector 24-2 is provided with a recording device 26 that functions as a storage unit, a learning image holding unit 28, and a learning control unit 30 in order to learn the multilayer neural network 34-2. .

  Such a configuration is the same as that of the embodiment of FIG. 2, but the alarm control unit 36 to which detection signals of the first fire detector 24-1 and the second fire detector 24-2 are input is different from that of the embodiment of FIG. Is different.

  The alarm control unit 36 of the present embodiment includes a weight multiplier 70-1, a weight multiplier 70-2, an adder 72, a comparator 74, and a threshold setting device 76. The weight multiplier 70-1 multiplies the detection signal of the first fire detector 24-1 by the weight W1, and the weight multiplier 70-2 multiplies the detection signal of the second fire detector 24-2 by the weight W2.

  Here, the multilayer neural network 34-1 of the first fire detector 24-1 is learned from input information prepared in advance as a standard in the manufacturing stage, and the fire estimation accuracy corresponding to the installation location is low. Since it is assumed, for example, a low weight such as W1 = 0.3 is set.

  On the other hand, the multi-layer neural network 34-2 of the second fire detector 24-2 not only learns based on input information prepared in advance as standard in the manufacturing stage, but also receives a fire by monitoring the fire with a fire detector. Based on the monitoring result by the machine 12, the recorded fire and non-fire recorded images of the recording device 26 are read out and input as learning information to the multilayer neural network 34-2 of the second fire detector 24-2 for learning. Therefore, it is assumed that the fire estimation accuracy corresponding to the installation location is high, and a high weight such as 0.7 is set.

  Further, the weight of the fire detector 24 may be changed according to the amount of learning. When the learning amount is small, the weight of the first fire detector 24-1 is increased to, for example, 1/1, and when the learning amount is large, the weight of the first abnormality detector 24-1 is set to 0.1. The weight of the second fire detector 24-2 is set to 0.9, and the weight of the second fire detector 24-2 is increased. Thus, by increasing the weight of the second fire detector 24-2 according to the learning amount, the degree of dependence on the second fire detector 24-2 can be increased according to the detection accuracy.

  The learning amount may be based on the sum of simple input information, but in the case of abnormality type, that is, fire detection, the type and location of fire may be used as the learning amount reference. By learning fires of different types and locations, comprehensive fire detection is possible, and a more appropriate fire detector is preferable.

  The adder 72 adds the values of the first fire detector 24-1 and the second fire detector 24-2 multiplied by the weights W1 and W2 in the weight multipliers 70-1 and 70-2, and adds them to the comparator 74. Output. The output of the adder 72 is normalized to a range of, for example, 0 to 1, the threshold value setting unit 76 sets, for example, 0.5 as a threshold value, and the output of the adder 72 becomes equal to or higher than the threshold value 0.5. In this case, a fire determination signal is output to the fire receiver 12 to output a fire alarm.

  As a function for judging fire from the output of the adder 72, a predetermined predictive threshold value and a predetermined fire threshold value larger than that are set, and when the output of the adder 72 exceeds the predictive threshold value, a fire predictive signal is received by fire. The fire warning signal may be output by outputting to the machine 12 and a fire determination signal may be output to the fire receiver 12 when the output of the adder 72 exceeds the fire threshold.

[Fire monitoring system that monitors fire with sensors]
(Outline of fire monitoring system)
FIG. 7 is an explanatory diagram showing an outline of a fire monitoring system for monitoring a fire with an analog fire detector functioning as a sensor.

    As shown in FIG. 7, analog fire detectors 60-1 and 60-2 that function as sensors are installed in the monitoring areas 14-1 and 14-2 of facilities such as buildings, and are pulled out from the fire receiver 12. Connected to the transmission path 62, serial data transmission is possible. The analog fire detectors 60-1 and 60-2 are described as the analog fire detector 60 when it is not necessary to distinguish between them.

  The analog fire detector 60 detects the smoke density by the smoke detector and outputs a smoke density detection signal, periodically A / D-converts by sending a batch AD conversion command from the fire receiver 12 and smoke density in the memory Smoke density data is transmitted in response to polling from the fire receiver 12 that has been stored as data and designated a sensor address. When the smoke density exceeds a predetermined threshold level, it is determined that a fire has occurred, and a fire interrupt signal is sent. A fire alarm is output by transmitting to the fire receiver 12. Note that the analog fire detector 60 may detect temperature, CO concentration, etc. in addition to smoke concentration.

  The determination device 10 includes a first fire detector and a second fire detector each having a multilayer neural network. The multi-layer neural network of the first fire detector and the second fire detector is learned from sensor data as input information prepared in advance as a standard in the manufacturing stage. Further, the multi-layer neural network of the second fire detector is used. When a fire interrupt signal is received from the analog fire detector 60 by the fire receiver 12, the neural network reads out input information from a sensor corresponding to the analog fire detector 60 that issued the fire from the storage unit, It is input and learned as fire learning information in a multilayer neural network of fire detectors.

  Moreover, the determination apparatus 10 inputs the smoke density data detected by the analog fire sensor 60 via the receiver 12 and stores it as input information from the sensor in the storage unit.

  When the fire receiver 12 receives the fire interrupt signal from the analog fire detector 60, the fire receiver 12 outputs a fire alarm and outputs a fire report signal to the determination device 10. When a fire alarm is output from the fire receiver 12, the person in charge of management or the person in charge of disaster prevention visits the installation site of the analog fire detector 60 that has issued the alarm and confirms whether there is a fire. The fire determination operation is performed by the fire receiver 12. When a fire determination operation is performed by the fire receiver 12, the temporarily stopped district acoustic alarm is released, and a fire determination transfer signal is output to the determination device 10.

  If the fire alarm is non-fired in the on-site confirmation, after removing the non-fire factor, the fire receiver 12 performs a recovery operation to cancel the fire alarm state and return to the normal monitoring state. After the fire alarm is output from the fire receiver 12 in this way, when a fire recovery operation is performed without performing a fire determination operation, a recovery transfer signal is output from the fire receiver 12 to the determination device 10. .

  The determination device 10 corresponds to the location of the analog fire sensor 60 that has transmitted the fire interrupt signal based on the fire monitoring result by the fire signal, the fire determination signal, and the recovery signal output from the fire receiver 12. For example, time-series data is generated from smoke density data detected by the analog fire detector 60 in the monitoring area 14 until the output of the fire alarm stored in the storage unit, and this is used as learning information to the determination device 10. Control is performed by inputting into the multilayer neural network of the second fire detector provided and learning.

  For example, when the analog fire sensor 60-1 transmits a fire interrupt signal, the time series data of the analog fire sensor 60-1 is read from the storage unit. Further, when a sensor other than the analog fire detector is further provided, sensor data arranged in the monitoring area is read from the storage unit.

  For fire monitoring input information to monitor fires with sensors, in addition to time-series data, multiple data from multiple sensors may be used, or time-series data of multiple sensors that are a combination of them may be used. Anyway.

(Judgment device)
FIG. 8 is an explanatory diagram showing a functional configuration of a determination device using a multilayer neural network that estimates a fire based on a detection signal from an analog fire sensor.

  As shown in FIG. 8, the determination apparatus 10 includes a first fire detector 24-1 and a second fire detector 24-2, and the first fire detector 24-1 is a time-series data input unit 66-. 1 and a multilayer neural network 34-1 and the second fire detector 24-2 is composed of a time series data input unit 66-2 and a multilayer neural network 34-2. The output of 34-2 is input to the alarm control unit 36.

  The multi-layer neural networks 34-1 and 34-2 of the first fire detector 24-1 and the second fire detector 24-2 learn from time series data as input information prepared in advance as standard in the manufacturing stage. Has been.

  Furthermore, the multilayer neural network 34-2 of the second fire detector 24-2 issues a fire when a signal from the fire alarm of the fire detector 18 is input to the learning control unit 30 by the fire receiver 12. Time-series data is read from the storage unit of the time-series data generation unit 64 as input information corresponding to the fire detector 18 and input as fire learning information to the multilayer neural network 34-2 of the second fire detector 24-2. To learn.

  A time series data generation unit 64 is provided for the first fire detector 24-1 and the second fire detector 24-2, and a learning data holding unit 68 for learning the multilayer neural network 34-2. And a learning control unit 30 is provided.

  The multilayer neural network 34 according to the present embodiment includes only a fully connected neural network constituting the recognition unit 40 shown in FIG. 3A, and excludes a convolutional neural network serving as a feature extraction unit 38.

  The time-series data generation unit 64 stores smoke concentration data detected by the analog fire detector 60 via the fire receiver 12 in the storage unit. The smoke density data stored in the storage unit of the time-series data generation unit 64 is, for example, data indicating a change in smoke density over time as shown in FIG.

  The smoke density data in FIG. 9 is an example of a temporal change in smoke density due to a fire. The smoke density starts increasing at time t0, reaches a predetermined pre-alarm level TH1 at time t1, and then fire level TH2 at time t2. This shows the case where the fire alarm is output and the fire determination operation is performed.

  The learning control unit 30 generates time-series data when a fire alarm is output from the fire receiver 12 based on the fire alarm signal E1, the fire determination signal E2 and the recovery alarm signal E3 from the fire receiver 12. The time series data is generated based on the sensor data of the smoke density shown in FIG. 9 stored in the storage section and sent to the time section data input section 66 of the second fire detector 24-2. Then, it is input to the multilayer neural network 34-2 as fire time-series data, and the weight and bias of the multilayer neural network 34-2 are learned by the back propagation method.

  The time-series data generation by the time-series data generation unit 64 is, for example, given that smoke density data for each predetermined unit time Δt from time t0 to time t1 in FIG. 9 until reaching the pre-alarm level TH1 is S1 to S18. Generate time series data (S1 to S10), (S2 to S11),... (S9 to S18) for each period T1, T2,... T9 while shifting by unit time Δt, The data is stored in the learning data holding unit 68.

  The learning of the multilayer neural network 34-2 using the time series data (S1 to S10), (S2 to S11),... (S9 to S18) is, for example, learning using the time series data (S1 to S10). Then, the density values S1 to S10 are input in parallel to the input layer of the multilayer neural network 34-2. Similarly, the remaining time series data (S2 to S11),... (S9 to S18) are learned by sequentially inputting them in parallel to the input layer.

  In addition, the learning control unit 30 receives a fire alarm signal E1 based on the fire alarm of the fire detector 18 by the fire receiver 12 and then confirms that the fire is not fired by an on-site confirmation by a manager or the like. Since the recovery operation is performed by the fire receiver 12 and the recovery transfer signal E3 based on the recovery operation is input, the non-fire time series is similar to the case of the fire time series data shown in FIG. Data is generated and input to the multi-layer neural network 34-2 of the second fire detector 24-2 as non-fire time-series data via the time-series data input unit 66-2. The weight and bias of the neural network 34-2 are learned.

  Here, the multilayer neural network 34-1 of the first fire detector 24-1 learned from the input information prepared in advance as standard, and the input information and time-series data generation unit 64 prepared as standard in advance. For the multilayer neural network 34-2 of the second fire detector 24-2 learned from the fire time-series data read out from the time-series data generation unit 64 for each predetermined unit time Δt. Thus, time series data for a predetermined time is generated and inputted in parallel via the time series data input units 66-1 and 66-2, so that the fire is monitored.

  The determination device 10 uses as input information the time-series data corresponding to the analog fire detector 60 that has transmitted the fire interrupt signal, but detects each set of the first fire detector 24-1 and the second fire detector 24-2. It is preferable that each container is independent. In other words, the learning method is the same for any combination of the first fire detector 24-1 and the second fire detector 24-2, but the first fire detector 24-1 and the second fire detector 24 are respectively the same. Different input information is given to the group -2, and the fire determination is also performed by different determinations. Thereby, learning specialized for the installation environment is performed.

  As with the alarm control unit 36 in FIG. 6, the alarm control unit 36 in FIG. 8 determines fire by weighted addition of the detection results of the first fire detector 24-1 and the second fire detector 24-2. You may make it do.

[Modification of the present invention]
(Alarm control unit)
As a fire determination of the monitoring area based on the detection results of the first fire detector 24-1 and the second fire detector 24-2 by the alarm control unit 30, in addition to the above embodiment, the first fire detector 24-1 A fire alarm may be output by taking the OR of the fire detection signal of the second fire detector 24-2, or the first fire detector 24-1 and the second fire detector 24-2. A fire alarm may be output by taking an AND of the detection signals.

  Further, a fire warning alarm is output by an OR output of the fire detection signals of the first fire detector 24-1 and the second fire detector 24-2, and the first fire detector 24-1 and the second fire detection are output. A fire alarm (this alarm) may be output by an AND output of the detection signal of the detector 24-2.

(Theft monitoring)
The above embodiment takes fire monitoring as an example, but can also be applied to theft monitoring. In the case of theft monitoring, in the monitoring system using the monitoring camera of FIG. 1, the fire receiver 12 may be a theft receiver and the fire detector 18 may be a theft detector. In the monitoring system of FIG. 7, the fire receiver 12 may be a theft receiver and the analog fire sensor 60 may be a theft detector.

(Monitoring illegal activities)
It can also be applied to tort detection in advance. It learns the movement of a person who attempts illegal activities such as theft and arson, such as frequently looking around, and detects anomalies before performing illegal activities and outputs an alarm. As an alarm output method, for example, an image of a monitoring camera may be displayed on a monitor in a security room that monitors the monitoring camera, and the person may be displayed in a red frame or the like.

(Cooperation with entry / exit system)
It can also be applied to monitoring for abnormal intrusions, etc., in cooperation with the entrance / exit system. The entry / exit system determines, for example, whether a target person can enter or leave a specific area using a card or a fingerprint. However, in the entrance / exit system, there is a problem such as a companion where a person who should not enter the room enters along with a person who can enter the room. For illegal activities such as the above-mentioned companionship, the present invention can also be applied to learning and detecting abnormalities using camera images and information on the entrance / exit system as inputs.

(Clarification of fire judgment grounds)
In the above embodiment, the determination result of the presence / absence of a fire is reported, but in addition to this, a factor determined to be a fire may be displayed. For example, in monitoring a camera image, an image determined to be a fire is displayed, and an area where the contribution rate of the fire determination is high is highlighted. This makes it easy to visually check the area that the fire detector has determined to be a fire, makes it possible to easily determine whether or not a true fire has occurred, and can help determine the response according to the situation. .

(Arson monitoring)
In the above embodiment, the fire monitoring in the warning area is taken as an example, but in addition to this, a fire constituted by a multilayer neural network for the arson monitoring performed by installing a sensor such as a surveillance camera or a flame detector outdoors. A detector may be provided, the fire detector may be learned from deep learning, and arson may be monitored.

(Feature extraction)
In the above embodiment, an image is input to a convolutional neural network and features due to fire are extracted. However, preprocessing is performed to extract features such as contour and shading from the input image without using the convolutional neural network. Alternatively, a predetermined feature may be extracted, and an image from which the feature has been extracted may be input to a fully connected neural network that functions as a recognition unit to estimate whether a fire or non-fire occurs. As a result, the processing load for image feature extraction can be reduced.

(About the learning method)
In the above embodiment, learning is performed by backpropagation, but the learning method of the multilayer neural network is not limited to this.

(Composite image and sensor)
In the embodiment described above, fire monitoring using images and fire monitoring using sensors are different forms, but image data and sensor data may be handled in parallel as input information. For example, monochrome values per pixel are treated as input terms for image data, and detection values for each sensor are treated as input terms for sensor data, for example. In this case, the term of the intermediate layer from which the image features have been extracted in the intermediate layer and the term of the intermediate layer that is affected by the sensor data have an effect on the terms of the subsequent intermediate layer that determines fire detection. It is desirable as a result of education, but it is not limited to this if fire monitoring can be effective.

(Infrared illumination and imaging of infrared images)
In the above embodiment, the monitoring area is imaged in a state where the monitoring area is illuminated by the monitoring camera and / or in a natural light state. However, the infrared light from the infrared illumination device is irradiated to the monitoring area, and the sensitivity to the infrared area A multi-layered neural network of a judgment device is learned by back-propagation by capturing an infrared image with a surveillance camera with an image, and it is judged whether the learning is a fire or non-fire by inputting the infrared image of the monitoring area into the multi-layer neural network You may do it.

  By inputting the infrared image of the monitoring area to the determination device in this way, it is possible to perform fire monitoring using the monitoring image without being affected by the illumination state of the monitoring area or the change in brightness of day and night.

(Abnormal alarm from abnormal warning warning)
In the above-described embodiment, when an abnormality is detected by any one of the abnormality detectors, an abnormality sign alarm is output. However, the abnormality sign may be determined and the abnormality alarm may be performed by continuing the abnormality sign alarm. For example, when an abnormality is continuously detected only by the abnormality detector 2, an abnormality alarm is given after a predetermined time. Further, at the time of an abnormal sign warning, the video of the monitoring camera or the like may be displayed to the administrator from the determination device or the like. The administrator may determine whether there is an abnormality based on the display, and may make an abnormal determination or non-abnormal determination.

(Fire detector performance check)
In addition to the above embodiment, the abnormality detection accuracy of the abnormality detector may be tested. The abnormality detection accuracy is confirmed using an abnormal image created by a method such as computer graphics, image composition, or actual generation of an abnormality such as a fire.

(Selection of abnormality detector)
In the above-described embodiment, the abnormality detector is used together. However, the abnormality detector may be selectively executed. For example, when the learned second abnormality detector is in a state that can sufficiently detect a fire and a non-fire, the administrator is bothered if the first abnormality detector that does not learn detects a non-fire as a fire. It becomes. By making it possible to selectively execute the abnormality detector, it is possible to prevent annoyance due to a false report of the abnormality detector having a low detection accuracy.

(Share input information)
In the above embodiment, the abnormality detector performs learning based on each input information. However, the abnormality detector may perform learning based on input information from another input information acquisition terminal in the same system. For example, the captured image of the monitoring camera corresponding to the first abnormality detector may be learned as input information of the second abnormality detector.

(Other)
The present invention is not limited to the above-described embodiment, includes appropriate modifications without impairing the object and advantages thereof, and is not limited by the numerical values shown in the above-described embodiment.

10: determination device 12: fire receivers 14-1, 14-2: monitoring areas 16-1, 16-2: monitoring cameras 18-1, 18-2: fire detector 20: signal cable 22: sensor line 24 -1: 1st fire detector 24-2: 2nd fire detector 26: Recording device 28: Learning image holding part 30: Learning control part 32-1, 32-2: Image input part 34-1 and 34-2 : Multilayer neural network 36: alarm control unit 38: feature extraction unit 40: recognition unit 42: input images 43, 45 a and 45 b: weight filters 44 a, 44 b and 44 c: feature map 46: input layer 48: coupling layer 50: intermediate Layer 52: Output layers 60-1, 60-2: Analog fire detector 62: Transmission path 64: Time series data generation unit 66-1, 66-2: Time series data input unit 68: Learning data holding unit 70-1 , 70-2: Weight multiplier 7 2: Adder 74: Comparator 76: Threshold setting unit

Claims (14)

A first abnormality detector configured by a multilayer neural network learned by learning information prepared in advance, and detecting abnormality or non-abnormality by input of input information from a monitoring area;
A second abnormality detector configured by a multilayer neural network learned from the learning information prepared in advance and the learning information collected in the monitoring area, and detecting whether the input information is abnormal or non-abnormal;
An alarm control unit for determining and alarming an abnormality in the monitoring region based on detection results of the first abnormality detector and the second abnormality detector;
A monitoring system characterized by that.
2. The monitoring system according to claim 1, wherein the alarm control unit determines an abnormality from a value obtained by setting and adding a predetermined weight to each detection result of the first abnormality detector and the second abnormality detector. Monitoring system characterized by alarming abnormalities.
The monitoring system according to claim 2, wherein the predetermined weight varies depending on the amount of the learning information.
2. The monitoring system according to claim 1, wherein the first abnormality detector and the second abnormality detector are configured to input a physical quantity detected by a sensor and / or an image of the monitoring area captured by an imaging unit as the input information. A monitoring system characterized by detecting whether it is abnormal or non-abnormal.
2. The monitoring system according to claim 1, wherein the alarm control unit outputs an abnormality predictive alarm when abnormality detection is determined by either the first abnormality detector or the second abnormality detector. 3. An abnormality alarm is output when abnormality detection is determined by the first abnormality detector and the second abnormality detector.
The monitoring system according to claim 1,
The second abnormality detector is
A learning control unit for learning by deep learning;
A storage unit that stores, as input information, a physical quantity detected by a sensor and / or an image of a monitoring area imaged by an imaging unit;
Is provided,
The fire control system, wherein the learning control unit reads input information stored in the storage unit, and inputs the learning information into the multilayer neural network of the second abnormality detector as learning information.
The monitoring system according to claim 6, wherein
The first abnormality detector and the second abnormality detector detect the fire or non-fire by inputting the input information,
The alarm control unit determines and alarms a fire in the monitoring area based on detection results of the first abnormality detector and the second abnormality detector,
The learning control unit reads the input information stored in the storage unit based on a monitoring result by a fire receiver that monitors a fire with a fire detector, and the multi-layer type of the second abnormality detector. A monitoring system characterized by being input to a neural network as learning information for learning.
8. The monitoring system according to claim 7, wherein when the fire receiver receives a fire alarm signal from the fire detector, the learning control unit receives a signal from the fire alarm signal from a predetermined time before. Of the input information up to the time of input, the input information corresponding to the fire detector that has fired is read out from the storage unit and input to the multi-layer neural network of the second anomaly detector as fire learning information A monitoring system characterized by learning.
8. The monitoring system according to claim 7, wherein the learning control unit receives a fire determination signal based on a fire determination operation after a fire transmission signal based on a fire alarm of the fire detector is input by the fire receiver. When a signal is input, the input information from a predetermined time before the fire transfer signal is input is read from the storage unit, and fire learning information is stored in the multilayer neural network of the second abnormality detector. A monitoring system characterized by input and learning.
The monitoring system according to claim 7, wherein the fire detector detects a temperature or smoke concentration and sends a detected analog value to the fire receiver to determine a fire.
The learning control unit, when a fire is detected by the fire sensor, from when the detected analog value exceeds a predetermined fire predictive level lower than a fire judgment level to when a fire is detected by the fire sensor The monitoring system is characterized in that the input information is read from the storage unit, and is input to the multilayer neural network of the second anomaly detector as learning information for fire to learn.
8. The monitoring system according to claim 7, wherein the learning control unit receives a restoration message based on a restoration operation after a fire announcement signal based on a fire alarm of the fire detector is input by the fire receiver. When input, the input information from a predetermined time before the fire transfer signal is input is read from the storage unit and input as non-fire learning information to the multilayer neural network of the second anomaly detector A monitoring system characterized by learning.
The monitoring system according to claim 7, wherein
The fire detector detects temperature or smoke concentration and sends a detected analog value to the fire receiver to determine fire,
The learning control unit, when a recovery signal based on a recovery operation is input after a fire notification signal based on a fire alarm of the fire detector is input by the fire receiver, the detected analog value is The input information from when the predetermined fire predictive level lower than the fire judgment level is exceeded to when the fire signal is input is read from the storage unit, and is not stored in the multilayer neural network of the second anomaly detector. A monitoring system characterized by inputting and learning as fire learning information.
The monitoring system according to claim 6, wherein the learning control unit reads input information stored in the storage unit in a normal monitoring state, and the non-abnormality is detected in the multilayer neural network of the second abnormality detector. A monitoring system characterized in that it is input as learning information and is subjected to initialization learning.
  14. The monitoring system according to claim 13, wherein the initialization learning timing is performed at predetermined time intervals when a predetermined operation is performed when the apparatus is started up, and when there is almost no change in the input information. A monitoring system comprising any one or more of the first time changes.

Images