JP2021087031A - Information processing device, information processing method, monitoring system, and program - Google Patents

Abstract

To reduce the effort required to train a discriminator to perform a detection process on an image.SOLUTION: An information processing device acquires a plurality of images obtained by an imaging apparatus taking images of a predetermined area, acquires information indicating a change in a situation of the predetermined area from a detection device, classifies the plurality of images based on the information indicating the change in the situation of the predetermined area, and trains a discriminator for determining the situation of the predetermined area from the images using at least a part of the plurality of images according to the classification results.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device, an information processing method, a monitoring system, and a program.

In recent years, a surveillance system has been proposed that analyzes images captured by a network camera (hereinafter referred to as a camera) with a computer, automatically detects illegal intrusion into private land and falls of passersby, and notifies security guards. There is. Conventionally, as a monitoring system, a rule-based method has been adopted in which each person defines as a rule one by one what kind of movement on the screen means trespassing into private land or falling of a passerby. There is. Furthermore, recently, even if a person does not define a rule, a computer analyzes a large amount of captured images called learning data by a technology called machine learning or deep learning, automatically understands the situation, and defines the rule by itself. The technology to do so is also beginning to be put into practical use.

JP 2015-070401

In the rule-based image analysis system as described above, if the rules to be defined are clear, the rules can be defined relatively easily. For example, if you want to detect trespassing on private land, specify the location of the fence or fence that is the boundary of the private land with a line so that it overlaps the image captured by the camera, and an object that passes through that line in one direction You can define the rules to mean that someone has trespassed in some cases. However, in such a case, it is necessary to define the minimum necessary rules such as the size and moving speed of the object in order to prevent false detection by animals such as birds.

On the other hand, when it is desired to detect a passerby's fall, the rule becomes complicated, and the rule must be defined so as to detect various fall methods (abnormal states) in various situations. This problem is solved by preparing captured images of abnormal movements that should be detected and images of normal movements that should not be detected by machine learning or deep learning, and learning the captured images from a computer. It can be solved by letting them do it. However, there remains the problem that it is necessary for a person to perform the classification work of classifying a large number of captured images into abnormal movements and normal movements, which requires a great deal of labor (Patent Document 1).

An object of the present invention is to reduce the labor required for learning a classifier that performs detection processing on an image.

In order to achieve the object of the present invention, for example, the information processing apparatus according to one embodiment has the following configurations. That is, a first acquisition means for acquiring a plurality of images obtained by the imaging device imaging a predetermined area, and a second acquisition means for acquiring information indicating a change in the situation of the predetermined area from the detection device. A classification means for classifying the plurality of images based on information indicating a change in the situation of the predetermined area, and at least a part of the plurality of images according to the classification result by the classification means, the predetermined image is used. It is characterized by comprising a learning means for learning a classifier for determining the situation of an area.

It is possible to reduce the labor required for learning the classifier that performs the detection process for the image.

The figure which shows an example of the functional structure of the information processing apparatus which concerns on Embodiment 1. FIG. The figure which shows an example of the hardware configuration of the information processing apparatus which concerns on Embodiment 1. FIG. The figure which shows an example of the situation which the information processing apparatus which concerns on Embodiment 1 makes a judgment. The figure which shows an example of the learning phase and the operation phase which concerns on Embodiment 1. The flowchart of the processing example in the information processing method which concerns on Embodiment 1. The figure which shows an example of the learning data classification UI of the information processing apparatus which concerns on Embodiment 1. FIG. The figure which shows an example of the functional structure of the information processing apparatus which concerns on Embodiment 2. FIG. The figure which shows an example of the situation which the information processing apparatus which concerns on Embodiment 2 makes a judgment. The figure which shows an example of the situation which the information processing apparatus which concerns on Embodiment 2 makes a judgment. The flowchart of the processing example in the information processing method which concerns on Embodiment 2.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

[Embodiment 1]
[System configuration]
FIG. 1 is a block diagram showing an example of a system configuration of a monitoring system 101 having an information processing device according to an embodiment. The information processing device according to the present invention acquires information indicating a change in the situation of a predetermined area from the detection device, and classifies the captured image of the predetermined area based on the information. Then, using the captured image of the classified predetermined area, a classifier for determining the situation of the predetermined area is generated. That is, when inputting the learning data to the classifier, it is possible to exclude the abnormal operation image from the captured image and automatically perform the classification work of separating the learning data of only the normal operation image. The monitoring system 101 according to the present embodiment includes an acquisition unit 111, a state change detection unit 112, an imaging unit 121, a recording unit 122, a storage unit 123, a generation unit 131, an identification unit 141, and an alarm unit 142. The imaging unit 121 according to the present embodiment is realized by a camera. Further, each component of the monitoring system 101 excluding the imaging unit 121 and the acquisition unit 111 is realized by one or a plurality of servers, and each is connected by a LAN or a dedicated cable.

The imaging unit 121 acquires an image. In the present embodiment, the imaging unit 121 is composed of a plurality of cameras, and acquires an image composed of images sequentially captured by those cameras in a predetermined area, but is particularly limited to that. Do not mean. Further, the imaging unit 121 may be, for example, a network camera arranged at a remote location with respect to the information processing device. The recording unit 122 stores the image acquired by the imaging unit 121 in the storage unit 123. In the example of FIG. 1, the recording unit 122 stores the captured image in the storage unit 123 via, for example, LAN 151. The acquisition unit 111 detects information indicating a change in the situation in a predetermined area, and outputs the information to the state change detection unit 112. In the present embodiment, the acquisition unit 111 is composed of a plurality of sensors, and when, for example, the sensors detect an abnormal state in a predetermined area, the state change detection detects a signal having information indicating that a state change has occurred. It is possible to transmit to the unit 112. Further, the acquisition unit 111 may be, for example, a sensor installed at a remote location with respect to the information processing device. The state change detection unit 112 determines whether or not a change in the situation has occurred in a predetermined area in response to a signal from the acquisition unit 111 (sensor in this example). Next, the state change detection unit 112 stores the state change information indicating whether or not the state change has occurred in the storage unit 123 in the situation at the time when the signal is transmitted. In the following, a situation in which the state change detection unit 112 determines that a state change has occurred is defined as an abnormal state, and a situation in which the state change is not determined is defined as a normal state. In the present embodiment, the state change detection unit 112 stores the state change information in the storage unit 123 via the LAN 151.

The storage unit 123 stores and stores the captured image acquired from the image pickup unit 121 and the state change information acquired from the state change detection unit 112. Further, the storage unit 123 can associate the stored captured image with the state change information. That is, the storage unit 123 classifies the captured image acquired from the imaging unit 121 based on the state change information acquired from the state change detecting unit 112. The storage unit 123 can classify the captured image acquired from the imaging unit 121 into, for example, an image that captures a normal state and an image that captures an abnormal state. In the following, unless otherwise specified, it refers to classifying an image into an image that captures a normal state or an image that captures an abnormal state, and is referred to as classifying an image.

The generation unit 131 reads the image stored in the storage unit 123, and based on the read image, generates a classifier that operates inside the identification unit 141. The image read by the generation unit 131 when generating the classifier is not particularly limited, but in this example, for the sake of explanation, the classifier is selected by selectively using the images that capture the normal state classified by the storage unit 123. It shall be generated. A classifier is a computer that analyzes a large amount of images called training data to automatically understand the situation and define rules for the training data. In the present embodiment, the classifier means an algorithm that can classify whether or not an image captures an abnormal state by inputting an image. The classifier may perform the above classification based on, for example, a feature vector extracted from the image data input to the classifier. The above is the configuration used for installing the camera and preparing to identify the abnormal state.

When an abnormal state actually occurs, the monitoring system 101 detects the abnormal state by using the above-mentioned classifier and notifies the detection result. The identification unit 141 acquires an image from the recording unit 122, and classifies whether or not the image captures an abnormality by using the classifier generated by the generation unit 131. When the identification unit 141 is classified as having an abnormal state in the image input to the identification unit 141, the identification unit 141 transmits an instruction to execute the notification process to the alarm unit 142. The alarm unit 142 performs notification processing in response to an instruction from the identification unit 141. The notification process performed by the alarm unit 142 is not particularly limited, and for example, information indicating that the status is normal may be notified, but in the following, the alarm unit 142 shall issue an alert.

[Hardware configuration]
FIG. 2 is a block diagram showing an example of the hardware configuration of the computer operated by the service user according to the embodiment. Operations such as viewing captured image data and analyzed data, and performing various settings are executed by an application program running on an operating system (OS). Such application programs are stored in HDD 202 or ROM 203. The CPU 205 loads and executes the OS and the application program from the HDD 202 or the ROM 203 to the RAM, thereby realizing operations such as viewing captured image data and analyzed data, and performing various settings. The application program acquires the input of the service user from the input device 207 connected to the computer. Further, the information is output to the output device 206 and the processing result is displayed. Further, the application program communicates with another computer, server, device, or the like connected to the network via the communication device 208. These hardware are connected to each other by bus 201 and are configured to be operated by an application program. Similarly, each configuration of the monitoring system 101 such as the state change detection unit 112, the recording unit 122, the storage unit 123, the generation unit 131, the identification unit 141, and the alarm unit 142 is also a computer processor as shown in FIG. Can be achieved by executing a program in memory.

[Example of situation classified as abnormal state]
FIG. 3 is an example of a situation in which an abnormal state is identified by a classifier generated by the generation unit 131 according to the present embodiment. In the example of FIG. 3, the state change detection unit 112 determines whether or not a state change has occurred based on the signal from the acquisition unit 111 including the sensor and the notification button. FIG. 3A is an example of a situation in which it is determined that a state change has occurred when a passage sensor temporarily installed in the exclusion zone reacts. In the example of FIG. 3A, intrusion detection passage sensors 302 are installed on both sides of the entrance to the passage 301. For example, when the passage sensor 302 reacts, the state change detection unit 112 treats the captured images acquired by the cameras 303 and 304 installed in the vicinity of the responding sensor as captured images that capture the abnormal state. be able to. The area set as the vicinity of the sensor is not particularly limited, and is appropriately set according to desired conditions. In the following, the area near the sensor shall refer to the area within a predetermined range determined for each sensor based on the position of the sensor.

FIG. 3B is an example of a situation in which a PTZ (Pan Til Zoom) camera 311 capable of swinging and zooming is installed. In the example of FIG. 3B, a guard looking at a live video screen installed in the guard room may find a suspicious person 312 in the corner of the video acquired by the PTZ camera 311. In such a case, it is conceivable that the guard operates the operation panel to operate the swing and zoom of the PTZ camera 311 and to observe the face and hand of the suspicious person 312. In response to the security guard noticing some abnormality and operating the PTZ camera 311 in this way, it is considered that some abnormality has occurred at that time, and the acquisition unit 111 outputs a signal indicating that the state change has occurred. It can be transmitted to the state change detection unit 112. Next, the state change detection unit 112 can treat the image acquired by the PTZ camera 311 at that time as an captured image capturing the abnormal state based on the signal from the acquisition unit 111.

The situation in which the acquisition unit 111 determines that the abnormal state is in an abnormal state and transmits a signal to the state change detection unit 112 (that is, the image acquired at that time is regarded as an image in which the abnormal state is captured) is described. In particular, it is not limited to the example of FIG. The acquisition unit 111 determines that the status of the predetermined area is abnormal when the sensor that detects the status of the detection area related to the predetermined area reacts or when the notification switch related to the detection area is operated. You may. Specifically, when the security guard room receives an emergency call, the security guard who was watching the live video screen notices an abnormality and gives a wireless instruction to the security guard at the site, the escalator is emergency. It may be determined that the abnormal state is also when the fire alarm is stopped or the button of the fire alarm is pressed.

[Learning phase and operation phase]
FIG. 4 shows an example of a data flow diagram of a learning phase and an operation phase performed by the monitoring system 101 according to the present embodiment. The monitoring system 101 can separately perform the flow of the learning phase and the flow of the operation phase described below. In the learning phase of FIG. 4A, a flow in which the generation unit 131 generates a classifier is shown based on all the image data acquired by the imaging unit 121 and the sensor signal acquired from the acquisition unit 111. Has been done. All the image data obtained by the imaging unit 121 is stored in the storage unit 123 for recording the image. Further, the state change detection unit 112 determines whether or not the signal of the sensor acquired from the acquisition unit 111 indicates that a state change has occurred, and the state change which is the result of the determination. Information is stored in the storage unit 123. Next, the storage unit 123 classifies the image at the time when the state change occurs from each image as the image in which the state change has occurred, based on all the accumulated image data and the state change information. Further, the storage unit 123 adds a learning data exclusion flag to the image data classified as having a state change among the stored image data. The generation unit 131 learns the abnormal state using the image data to which the learning data exclusion flag is not added among the image data stored in the storage unit 123, and generates the above-mentioned classifier. Next, the generation unit 131 applies the generated discriminator after learning to the identification unit 141.

In the operation phase of FIG. 4B, when it is identified that the image data acquired from the imaging unit 121 captures the abnormal state, the alarm unit 142 issues an alert. The identification unit 141 acquires the designated image data from all the image data acquired from the imaging unit 121 or the image data stored in the storage unit 123, and the acquired image data captures the abnormal state. Classify whether or not it is. In the example of FIG. 4B, the identification unit 141 identifies whether or not the acquired image data captures the abnormal state by using the classifier generated in the learning phase. For example, when the acquired image data is identified as capturing an abnormal state, the identification unit 141 may transmit a signal including an instruction to issue an alert to the alarm unit 142. Further, the identification unit 141 according to the present embodiment may identify the captured images acquired by the imaging unit 121 in real time, or may identify the captured images sequentially acquired from the accumulated captured images.

In the present embodiment, the discriminator is learned using the image data in the normal state in which the state change has not occurred, according to the classification result as described above. However, in another embodiment, the discriminator may be trained using the image data of the abnormal state in which the state change occurs. Further, the discriminator may be learned by using both the image data in the normal state and the image data in the abnormal state, and the above classification result.

[Learning phase operation]
FIG. 5 is a flowchart showing an example of processing performed by the monitoring system 101 according to the present embodiment in the learning phase. The (A) main routine and (B) subroutine indicate the main routine and the subroutine of such processing, respectively.

(A) In step S501 of the main routine, the storage unit 123 determines whether to continue the classification of the image data and the collection of the learning data. The storage unit 123 does not continue the process when, for example, a sufficient amount of image data classified as a normal state can be collected or the interruption operation is performed by the user, and the process proceeds to step S503. If the process is to be continued, the process proceeds to step S502. In step S502, the storage unit 123 calls a subroutine to classify the image data and assign a learning data exclusion flag to the image classified as having a state change, and the process proceeds to step S511 described later. In step S503, the generation unit 131 performs learning using the image data to which the learning data exclusion flag is not added, and generates a classifier.

(B) In step S511 of the subroutine, the state change detection unit 112 acquires a signal from a sensor or the like by the method described in FIG. In step S512, the state change detection unit 112 determines whether or not a state change has occurred in the situation at the time when the signal was transmitted. If it is determined that the state change has occurred, the process proceeds to step S513. If not, the process returns to the subroutine caller and proceeds to step S501. In step S513, the storage unit 123 adds a learning data exclusion flag to the image of the camera near the responding sensor at the time when the signal is transmitted. Then, the process returns to the subroutine caller and proceeds to step S501.

[Confirmation of learning data]
FIG. 6 is a diagram showing an example of a confirmation UI for confirming the classification result of the learning data used by the monitoring system 101 according to the present embodiment in the learning phase. In the example of FIG. 6, the confirmation UI601 of the classification result of the learning data is composed of a tab 602 for selecting a camera, an image reproduction panel 603, an image control panel 604, and a seek bar 605 for confirming the image time. Will be done.

The tab 602 displays the classification for each camera for the image to be learned by the classifier. By selecting tab 602 on UI601, the user can switch the image displayed on the playback panel 603 to the image acquired by the camera of the selected tab. The playback panel 603 displays an image. The playback panel 603 displays, for example, an image of the camera displayed by the tab selected as described above. By operating the control panel 604, the user can play or stop, and fast forward or rewind the image on the playback panel 603. The seek bar 605 visually displays the time of the image currently being played back and the time of the image in the abnormal state in which the learning data exclusion flag is set. Here, the time period of the abnormal state image is represented as a black fill in the seek bar 605.

According to such a configuration, the images of the predetermined area can be classified by acquiring a plurality of images of the predetermined area and detecting the information indicating the change of the situation of the predetermined area. Then, using the classified images, it is possible to learn a classifier that determines the situation of a predetermined area from the images. Therefore, it is possible to automatically perform the work of classifying the images that capture the normal state from the plurality of images as learning data.

[Embodiment 2]
In the first embodiment, a method of detecting the reaction of the sensor in the field and the swinging operation of the camera as a state change, and directly determining and classifying that the image at that time captures an abnormal state will be described. did. Here, for example, a method of recognizing the behavior of a guard by a sensor or an image, detecting a state change from the behavior of the guard, and indirectly determining and classifying that an abnormal state has occurred will be described.

[System configuration]
FIG. 7 is a block diagram showing an example of a system configuration of a monitoring system having the information processing device 701 according to the present embodiment. The monitoring system according to the present invention has the same configuration as that of FIG. 1 except that it has a transmission unit 711 and a signal detection unit 712 instead of the acquisition unit 111, and redundant description will be omitted. The transmission unit 711 is a device for the signal detection unit 712 to recognize the position information of a person. The transmitter 711 may be, for example, a transmitter that is always carried by a security guard and emits radio waves or infrared rays, or is a wireless device that performs two-way communication with another device (for example, signal detector 712). There may be. The signal detection unit 712 can accurately recognize the position information of a person in a certain area based on the transmission unit 711.

[Example of situation classified as abnormal state]
FIG. 8 is a diagram for explaining a situation identified as an abnormal state by the information processing apparatus 701 according to the present embodiment. The signal detection units 803 and 804, which are sensors, or the signal detection units 823 and 824, detect whether or not a state change has occurred in a predetermined area by detecting a specific action of a person designated in advance. In the example of FIG. 8, each signal detection unit can determine whether or not a state change has occurred in a predetermined area by detecting the behavior of the guard.

FIG. 8A shows a situation in which the guard is walking, and the state change detection unit 112 determines that the state change has not occurred because each signal detection unit detects the behavior of the guard. This is an example of a situation that is not considered an abnormal condition. The camera 801 on the left captures the guards 812 and 813 at the angle of view, and the camera 802 on the right captures the person 811 at the angle of view. Further, the guard 812 possesses a transmitter (transmitter 711), and the position of the transmitter can be detected by triangulation using two signal detection units 803 and 804. In this example, guard 812 is walking to 813 because he recognizes that person 811 is in sight but not in an urgent situation. By detecting such an action of the guard, the state change detection unit 112 determines that the situation in the predetermined area is not an abnormal state.

FIG. 8B shows a situation in which the guard is running, and it is determined that the state change has occurred due to the state change detection unit 112 detecting the behavior of the guard by each signal detection unit. This is an example of a situation that is considered to be an abnormal condition. The camera 821 on the left captures the guards 832 and 833 at the angle of view, and the camera 822 on the right captures the person 831 at the angle of view. Further, each guard 832 possesses a transmitter, and the position of the transmitter can be detected by triangulation using two signal detectors 823 and 824. In this example, guard 832 finds the fallen person 831 and recognizes that it is an urgent situation, so he runs to 833 and moves. The state change detection unit 112 determines that the situation in the predetermined area is in an abnormal state when the signal detection units 823 and 824 detect such an action of the guard.

FIG. 9 is a diagram for explaining a situation identified as an abnormal state by the information processing apparatus 701 according to the present embodiment, similarly to FIG. In the example of FIG. 9, the state change detection unit 112 detects the behavior of the guard by simple image analysis using the images captured by the cameras 901 and 902, or the cameras 921 and 922, in a predetermined area. Determine if a state change has occurred.

FIG. 9A shows a situation in which the guard is walking, and when the state change detection unit 112 detects the action of the guard, it is determined that no state change has occurred and it is regarded as an abnormal state. This is an example of a situation where it is not done. Further, the camera 901 on the left side captures the guards 912 and 913 at the angle of view, and the camera 902 on the right side captures the person 911 at the angle of view. The state change detection unit 112 uses, for example, an image analysis function for determining whether or not a guard is based on face recognition or clothing, and a person tracking function, from the images of the cameras 901 and 902 to the position of the guard. Can be detected. Guard 912 is walking to 913 because he recognizes that person 911 is in sight but not in an urgent situation. By detecting such an action of the guard, the state change detection unit 112 determines that the situation in the predetermined area is not an abnormal state.

FIG. 9B shows a situation in which the guard is running, and the state change detection unit 112 detects the behavior of the guard, so that it is determined that the state change has occurred and the state is abnormal. This is an example of a situation that is considered. Further, the camera 921 on the left side captures the guards 932 and 933 at the angle of view, and the camera 922 on the right side captures the person 931 at the angle of view. Guard 912 is running to 913 because he recognizes that the person 911 is in sight but is in an urgent situation. The state change detection unit 112 determines that the situation in the predetermined area is an abnormal state by detecting such an action of the guard.

In addition, the method of tracking the person of the guard is not particularly limited. The guard's person tracking recognizes the shape of the person, considering that there is a high possibility of false detection when multiple people overlap or when a moving object other than the person is reflected in the tracking by simple background separation. It may be done by doing. For example, the state change detection unit 112 recognizes the shape of a person's face or whole body by combining known techniques such as the brightness gradient of the contour line or the optical flow, and further tracks the movement between a plurality of frames. You may. According to such processing, the possibility of the above-mentioned false detection can be reduced.

[Learning phase operation]
FIG. 10 is a flowchart showing an example of processing performed in the learning phase by the monitoring system according to the present embodiment. The (A) main routine, and the (B) and (C) subroutines indicate the main routine and the subroutine of such processing, respectively. In this example, the information processing apparatus 701 performs the same processing as in FIG. 5 except for (B) S1011 that recognizes whether the guard in the subroutine has rushed and (C) the subroutine that shows the detailed processing in S1011. Therefore, the duplicate description will be omitted.

(B) In step S1011 of the subroutine, the state change detection unit 112 calls step S1021 (described later) of (C) the subroutine in order to recognize whether the guard has rushed, and then proceeds to step S512. In step S512, the state change detection unit 112 determines whether or not a state change has occurred according to the output of the subroutine (C), and performs processing in the same manner as in the first embodiment.

In step S1021, the state change detection unit 112 determines whether the guard is in the vicinity of the camera or within the imaging range of the camera. The state change detection unit 112 can determine, for example, whether or not the transmitter described with reference to FIG. 8 has detected that the guard is in the vicinity of any of the cameras. The area set as the vicinity of the camera is not particularly limited, and is appropriately set according to desired conditions. In the following, the area near the camera shall refer to the area within a predetermined range determined for each camera based on the position of the camera. In addition, the state change detection unit 112 can determine whether or not it has detected that a guard is in the imaging range of any of the cameras by the image analysis described with reference to FIG. When the state change detection unit 112 determines that the guard is not near the camera or within the imaging range of the camera, the state change detection unit 112 determines that the situation in the predetermined area is normal, and captures the situation. Do not add the training data exclusion flag to the image. Then, the process returns to the subroutine caller and proceeds to step S1012. If this is not the case, the image may capture an abnormal state, and the process proceeds to step S1022 to confirm the next condition.

In step S1022, the state change detection unit 112 determines whether the moving speed of the guard exceeds a predetermined threshold value in order to determine that the situation in which the guard is running up is an abnormal state. When the movement speed of the guard exceeds the threshold value, the state change detection unit 112 determines that the situation in the predetermined area is an abnormal state, and adds a learning data exclusion flag to the image capturing the situation. The value of the moving speed set as the threshold value here is not particularly limited. For example, since it is said that the average speed of walking is about 4 km / h, the state change detection unit 112 is visually checked by the guard when the moving speed of the guard exceeds twice that speed of 8 km / h. You may find an anomaly and determine that you are in a rushing situation. That is, it may be determined that the condition of the predetermined area is abnormal. When the movement speed of the guard is equal to or less than the threshold value, the state change detection unit 112 determines that the situation in the predetermined area is a normal state, and does not add the learning data flag to the image. Then, the process returns to the subroutine caller and proceeds to step S512.

When the state change detection unit 112 detects the behavior of a guard, the method of recognizing the guard is not particularly limited. The state change detection unit 112 may recognize whether or not the person in the captured image is a security guard by recognizing clothes, a hat, or the like from the captured image, for example. Further, the state change detection unit 112 may identify whether or not the person is a security guard based on a beacon (transmitter) carried by the security guard.

Further, the method in which the state change detection unit 112 measures the moving speed of the guard is not particularly limited. The state change detection unit 112 may calculate the moving speed of the guard by, for example, a person tracking technique based on an image pickup device that captures an image to be classified and another image pickup device. Further, the state change detection unit 112 may measure the moving speed of the guard by a speed sensor carried by the guard.

According to such a configuration, it is determined whether or not there is a change in the scene in the captured image based on the position and action of the guard recognized by the sensor or the image, and the normal state is captured. A classifier can be generated using the determined image as training data.

(Other embodiments)
In this embodiment, a method of classifying an image of an abnormal state by determining whether it is an abnormal state based on a state change signal from a sensor, a camera, a reporting facility, and a communication facility that are directly connected to the system and permanently installed. Although explained, it is not particularly limited to this. For example, the above-mentioned judgment and classification may be performed based on temporary sensors, cameras, reporting equipment, and communication equipment. In addition, as a signal indicating that a state change has occurred, the monitoring system monitors reports from personal communication terminals of customers who are not directly connected to the system, or earthquake early warnings and weather warnings announced by government agencies. You may use the receiving method.

Further, in the present embodiment, as a method of determining whether or not there is a guard within the imaging range of the camera, a method of detecting the signal of the transmitter held by the guard by the signal detection unit installed in the facility and the face or He explained how to analyze images by clothes. However, the method of determining whether or not there is a guard within the imaging range of the camera is not particularly limited to these. For example, the state change detection unit 112 may determine whether or not there is a guard within the imaging range of the camera based on information other than the image such as footsteps or voiceprints. Further, transmission and reception may be reversed, that is, a determination may be made by a method in which a transmitter is installed in the facility and a guard holds a signal detection unit having a communication function.

Further, in the present embodiment, it has been described that the target to be detected by the video analysis is the guard when determining whether or not there is a guard within the imaging range of the camera, but the present invention is not particularly limited to this. .. For example, it may be a target for detecting a person wearing an easily identifiable uniform such as a police officer or a clerk. Further, in the present embodiment, it has been explained that the state in which the guards are running up is determined to be an abnormal state, but the person running up here is not particularly limited to the guards. For example, it may be determined whether or not the predetermined area is in an abnormal state by detecting a general passerby who has determined that the situation is urgent and has taken a rushing action. Further, what is recognized in these processes is not limited to the person, and the above-mentioned process may be performed on a desired object appropriately set instead of the person (guard).

Further, the state change detection unit 112, the recording unit 122, the storage unit 123, the generation unit 131, and the identification unit 141 have been described as performing processing by a server prepared separately from the image pickup unit 121 which means a camera. It is not limited to that. For example, some or all of them may be built into the camera.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

101: Monitoring system, 111: Acquisition unit, 112: State change detection unit, 121: Imaging unit, 122: Recording unit, 123: Storage unit, 131: Generation unit, 141: Identification unit, 142: Notification unit, 151: LAN

Claims (19)

A first acquisition means for acquiring a plurality of images obtained by the imaging device imaging a predetermined area, and
A second acquisition means for acquiring information indicating a change in the situation in the predetermined area from the detection device, and
A classification means for classifying the plurality of images based on information indicating changes in the situation of the predetermined area, and
A learning means for learning a classifier that determines the situation of the predetermined area from the images by using at least a part of the plurality of images according to the classification result by the classification means.
An information processing device characterized by being equipped with. From the plurality of images, the classification means obtains an image captured by the imaging device when the predetermined area is in a normal state, or an image captured by the imaging device when the predetermined area is in an abnormal state. The information processing apparatus according to claim 1, wherein the information processing apparatus is separated. The first or second aspect of the present invention, wherein the learning means selectively uses an image captured by the imaging device when the predetermined area is in a normal state to learn the classifier. Information processing equipment. The information indicating the change in the situation of the predetermined area includes the position information of the object in the detection area related to the predetermined area.
The information according to any one of claims 1 to 3, wherein the classification means determines whether the predetermined area is in a normal state or an abnormal state based on the position of the object. Processing equipment. The detection device is a transmission device that transmits a signal including position information of the object.
The information processing apparatus according to claim 4, wherein the second acquisition means acquires the position information of the object by receiving the signal. The information processing apparatus according to claim 4 or 5, wherein the classification means determines that the predetermined area is in a normal state when the position of the object is not detected in the detection area. The information according to any one of claims 4 to 6, wherein the classification means determines whether the predetermined area is in a normal state or an abnormal state based on the speed of the object. Processing equipment. 7. The classification means is characterized in that the speed of the object is acquired based on a speed sensor that detects the speed of the object or a plurality of images acquired by the first acquisition means. The information processing device described. The information indicating the change in the situation in the predetermined area includes information indicating the behavior of the person.
The classification means according to any one of claims 1 to 8, wherein the predetermined area is determined to be in an abnormal state when a person designated in advance is performing a specific action. The information processing device described. The information processing device according to claim 9, wherein the detection device detects the person specified in advance based on the clothes of the person. The detection device is characterized by being a sensor that detects the status of the detection area related to the predetermined area, a notification switch related to the detection area, or an image pickup device different from the image pickup device that images the detection area. The information processing device according to any one of claims 1 to 10. A storage means for accommodating and accumulating the image and information indicating a change in the situation of the predetermined area when the image is captured is further provided.
The information processing apparatus according to any one of claims 1 to 11, wherein the learning means learns the classifier using the image stored in the storage means. The information processing apparatus according to any one of claims 1 to 12, further comprising a determination means for determining a situation in the predetermined area by using the discriminator after learning and the image. The information processing apparatus according to claim 13, further comprising a notification means that executes a notification process in response to a determination of the determination means. The information processing apparatus according to claim 14, wherein the notification process is an alert. A monitoring system including the information processing device according to any one of claims 1 to 15.
The monitoring system is characterized in that the imaging device is a network camera arranged at a remote location of the information processing device. The monitoring system according to claim 16, wherein the detection device is installed at a remote location of the information processing device. The process of acquiring a plurality of images obtained by the image pickup device capturing a predetermined area, and
The process of acquiring information indicating a change in the situation in the predetermined area from the detection device, and
A step of classifying the plurality of images based on information indicating a change in the situation of the predetermined area, and
A step of learning a classifier that determines the situation of the predetermined area from the images by using at least a part of the plurality of images according to the classification result of the plurality of images.
An information processing method characterized by being provided with. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 15.

Images