JP5236607B2 - Anomaly detection device - Google Patents

Description

  The present invention relates to an anomaly detection device that detects an anomaly from an image obtained by imaging a monitoring space, and more particularly to an anomaly detection device that detects anomalies such as bringing in a suspicious object by a suspicious person and taking an article away by a suspicious person.

  For the purpose of crime prevention in manned surveillance spaces such as stations and airports, an apparatus has been proposed that detects the presence of suspicious objects such as neglected dangerous goods and forgotten things from surveillance images.

  In Patent Document 1, a difference between an input image and a background image is sequentially obtained and compared to detect a baggage image separated from a person image, and a left time of the detected baggage image is measured to determine a predetermined time. An image monitoring system is described that issues a warning when it has elapsed.

  That is, it is possible to detect an abnormality when the change area with respect to the background image is continuously detected at the same position in the vicinity of the person image for a certain period of time.

JP-A-1-245395

  However, in a manned monitoring space, a chair or the like that originally existed in the space may be moved in the space. These objects such as chairs are hereinafter referred to as background objects because their images are included in the background image as part of the background. When the background object is moved, similarly to the case of the suspicious object, the change area is continuously detected at the same position in the vicinity of the person's image for a predetermined time or more. Therefore, there has been a problem that the movement of the background object is erroneously detected as abnormal.

  If there is a false detection, extra confirmation work and countermeasure work occur for the supervisor, and the monitoring efficiency is greatly reduced. Moreover, when the background object is moved, the change area due to the background object continues to be detected at the two locations of the movement source and the movement destination, so that the work of the monitor accompanying the erroneous detection is doubled.

  Further, if the change area due to the background object is continuously detected, there is a problem that the area becomes a non-sensing area in which an abnormality such as bringing in a new suspicious object cannot be detected. In addition, the non-sensing areas are generated at two places, the movement source and the movement destination of the background object.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an abnormality detection device that can accurately detect an abnormality occurring in a monitoring space without erroneously detecting the movement of a background object as an abnormality. . Another object of the present invention is to provide an abnormality detection apparatus that can quickly eliminate an insensitive area caused by a moved background object and detect an abnormality occurring in a monitoring space with high accuracy.

  An abnormality detection apparatus according to the present invention includes an imaging unit that images a monitoring space every predetermined time, a storage unit that stores a background image in which the background of the monitoring space is captured, and an input image input from the imaging unit. The change area extraction means for extracting the change area in comparison with the background image and the input image in the change area extracted at the preceding and following times are compared to track the change area, and the track for each tracked change area is obtained. A tracking means for detecting, an area classification means for detecting a movement amount from the trajectory, and classifying the change area into a movement area where the movement amount is detected and a stationary area where the movement amount is not detected; and between the movement area and the stationary area Derivation determination means for calculating a degree of separation and setting a derivation relationship between the moving area and the stationary area for which the degree of separation less than the derivation determination value is calculated, and setting the stationary area as an abnormal candidate area, and an abnormal candidate at a predetermined time Different if there is an area An abnormality determination unit that outputs a signal, and the abnormality determination unit detects a pair of static regions in which a derivation relationship is set for the same moving region, and sets an abnormality candidate region by excluding the pair It is characterized by.

When a person in the monitoring space moves the background object, the change area by the person is classified as a movement area, and the change area of the movement source and the movement destination by the background object is classified as a stationary area. Then, the stationary area of the background object and the stationary area of the movement destination are both detected as a pair because the derivation relationship with the same moving area is set, and are excluded from the abnormality determination targets.
Therefore, according to the above configuration, even if a person in the monitoring space moves the background object, the change area extracted at the movement source and the movement destination of the background object is not erroneously detected as an abnormality due to bringing in a suspicious object, etc. Abnormalities can be detected with high accuracy.

  Further, in a preferred aspect of the present invention, the abnormality determination means collates a background image in one of the pair with an input image in the other of the pair, and when the matching coincides, an abnormality candidate region is excluded except for the pair. Set.

Since the image of the moved background object is captured in the background image in the movement-source still area and the input image in the movement-destination still area, a match is obtained by the above collation. For this reason, the moved background object pair is excluded from the abnormality determination target.
On the other hand, if a pair is detected because a single suspicious person has left multiple suspicious objects, if the multiple suspicious objects are not similar, the above matching will not yield a match, and the multiple suspicious objects are similar. Even if the images of these suspicious objects are both captured in the input image in the stationary region of the pair, no coincidence is obtained in the above collation. Therefore, a pair due to a suspicious object is not excluded from the abnormality determination target.
Therefore, according to the above-described configuration, the pair due to the background object can be excluded from the abnormality determination target without excluding the pair due to the suspicious object from the abnormality determination target, so that the abnormality can be detected with high accuracy.

In a preferred aspect of the present invention, the image processing apparatus further includes background image generation means for updating a portion of the background image corresponding to the pair with an input image.
According to such a configuration, since the changed area extracted by the moved background object is not extracted after the update, a new abnormality occurring in the area can be detected.

In a preferred aspect of the present invention, the abnormality determination means sets the abnormality candidate area by further excluding a stationary area where no derivation relationship is set.
According to such a configuration, since a stationary region that is generated regardless of the suspicious person, such as light inserted from the window, is excluded from the abnormality determination target, it is possible to detect the abnormality with high accuracy.

In a preferred aspect of the present invention, the predetermined time point is a disappearance time point when the locus of the moving region in which the derivative relationship is set with the abnormality candidate region is not detected.
According to such a configuration, since the abnormality can be detected when the suspicious person disappears from the field of view, the abnormality can be detected quickly.

In a preferred aspect of the present invention, the predetermined time point is any one of the disappearance time point or an elapsed time point after a preset neglect determination time has elapsed since the locus of the abnormal candidate region started to be detected.
According to this configuration, even if a suspicious person stays in the field of view for a long time, an abnormality can be detected without waiting for the suspicious person to disappear out of the field of view, so that the abnormality can be detected quickly.

Further, in a preferred aspect of the present invention, the derivation determining means calculates the degree of separation between the position of the stationary area and the position of the moving area extracted immediately before the stationary area is newly extracted. .
Since the still area is extracted as a part of the derivation source moving area at the time immediately before the new extraction, according to such a configuration, the correct derivation relationship is set even if a plurality of persons pass around the stationary area. The correct pair is detected. Therefore, it is possible to detect an abnormality with high accuracy.

  ADVANTAGE OF THE INVENTION According to this invention, even if there is a movement of a background object, it becomes possible to provide the abnormality detection apparatus which can detect the abnormality which generate | occur | produced by the suspicious person with high accuracy.

It is drawing which shows the functional block diagram of an abnormality detection apparatus. It is a figure which shows an example of the image etc. in process. It is a figure which shows an example of the image etc. in process. It is a figure which shows an example of area | region attribute information. It is a figure which shows the flowchart of an abnormality detection process. It is a figure which shows a part of flowchart of an area | region discrimination | determination process. It is a figure which shows another part of flowchart of an area | region discrimination | determination process.

  As an example of a preferred embodiment of the present invention, an abnormality detection device will be described in which a waiting room where an unspecified number of facility users come and go is used as a monitoring space, and the introduction of a suspicious object or the removal of an article is detected as an abnormality.

[Configuration of Abnormality Detection Device 1]
A functional block diagram of the abnormality detection apparatus 1 is shown in FIG. The abnormality detection device 1 is configured by connecting an imaging unit 2, a storage unit 3, a display unit 5, and a recording device 6 to a control unit 4.

  The imaging unit 2 is a so-called surveillance camera. The imaging unit 2 sequentially inputs images obtained by capturing the monitoring space at predetermined time intervals to the control unit 4. Hereinafter, an image input to the control unit 4 is referred to as an input image, and a time unit that is engraved at the above time interval is referred to as a time.

  The storage unit 3 is a memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 3 stores various programs and various data, and inputs / outputs such information to / from the control unit 4. Various data includes a background image 30, a locus 31, a reference feature amount 32, and region attribute information 33.

  The background image 30 is an image in which only the background of the normal monitoring space is captured. That is, the background image 30 does not include an image of a suspicious object or an image of a facility user. The background image 30 is generated and stored prior to the abnormality detection process.

  The background image 30 is compared with the input image at each time, and an area that differs between the two images is extracted as a change area. Then, a change area having similar characteristics is identified among the change areas extracted at the preceding and following times by the tracking means 42 described later, and the identified change areas are the same as the change areas caused by the same factor. An area ID is given. The area ID is a numerical value or a character string for identifying a factor. A series of change areas caused by suspicious objects, a series of change areas caused by a facility user, a series of change areas caused by a moved background object, and the like are each assigned a different area ID and tracked.

  The locus 31 is a history of positions for each tracked change area. Specifically, data in which the center of gravity positions of the change areas are arranged in time series and an area ID for identifying the change area are stored in association with each other. The trajectory 31 is also used to identify the change area.

The reference feature quantity 32 is an image feature quantity such as a color histogram that characterizes each change area, is extracted from the input image in the change area, and is stored in association with the area ID of the change area. The reference feature 32 is used for identifying the change area.
In addition to the color histogram, various feature amounts such as the number of edge points, representative colors, or a combination of two or more of these can be employed as the image feature amount.

The area attribute information 33 is information stored in association with the area ID of the change area and the attribute determined for the change area. The attribute includes area type and derivation relationship information. FIG. 4 shows an example of the region attribute information 33.
Area types include moving areas with movement, stationary areas without movement, derived areas derived from other change areas, abnormal candidate areas that may be abnormal, abnormal areas that have been determined to be abnormal (such as those due to suspicious objects) A suspicious object area determined), a background object area determined to be due to a background object, and a noise area determined to be due to noise. The derivation relationship information is specifically the area ID of the derivation source change area, and is stored for the change area whose area type is the derivation area. In this example, the derivation source column is provided in the derivation destination change area, but the derivation destination column may be provided in the derivation source change area.

  The control unit 4 is an arithmetic device such as a DSP (Digital Signal Processor) or MCU (Micro Control Unit). The control unit 4 reads out a program describing the operations of the background image generation unit 40, the change region extraction 41, the tracking unit 42, the region classification unit 43, the derivation determination unit 44, the abnormality determination unit 45, and the like from the storage unit 3 and executes it. It functions as each means. The control unit 4 processes an image input from the imaging unit 2 (hereinafter referred to as an input image) to detect an abnormality caused by the involvement of a suspicious person, and displays an abnormality signal related to the detected abnormality on the display unit 5 and the recording device. 6 is output. The control unit 4 includes clock means (not shown) and can acquire the current date and time from the clock means.

  The background image generation means 40 causes the storage unit 3 to store the entire input image captured when there are no suspicious objects and facility users in the monitoring space as the background image 30. The background image generation means 40 updates the background image 30 by partially combining the input image with the background image 30 as follows.

  First, the background image generation means 40 updates the background image 30 by synthesizing the input image with a portion of the background image 30 in which no change area is extracted from the input image at each time. The synthesis is performed by weighted average processing or replacement processing. By this update, the background image 30 can be adapted to the lighting environment that changes every moment. Therefore, it is difficult to erroneously extract image changes due to illumination fluctuations, and the accuracy of abnormality detection is improved.

  Secondly, the background image generation unit 40 updates the background image 30 by synthesizing the input image with a portion of the background image 30 corresponding to the changed region excluded from the abnormality candidate region by the abnormality determination unit 45. The parts to be updated are the background object area and the noise area. By this update, the same part does not become a non-sensing area, and a new abnormality occurring in the same part can be detected. The synthesis is performed by weighted average processing or replacement processing.

  The change area extraction unit 41 extracts the change area by comparing the input image at each time with the background image 30, and outputs the information of the extracted change area to the tracking unit 42.

The change area extraction unit 41 performs background difference processing to extract a change area. That is, the change area extraction unit 41 detects pixels in which the absolute value of the difference between the pixel values is greater than or equal to a preset difference threshold for each corresponding pixel of the input image and the background image 30, and detects the pixels detected in proximity. Summarize as change areas.
In another embodiment, the change area extraction unit 41 performs background correlation processing to extract a change area. In this case, the change area extraction unit 41 obtains a correlation value between each pixel of the input image and the pixel of the background image 30 corresponding to the pixel, and is close to each other among pixels whose correlation value is less than a preset correlation threshold value. Summarize the pixels as change areas.

  The tracking means 42 compares the input images in the change area extracted at the preceding and following times, identifies the change area, and detects the trajectory 31 for each identified change area. Such a process is generally called a tracking process.

  The tracking means 42 performs identification at each time and advances the tracking process sequentially. That is, the tracking unit 42 compares the image feature amount and the gravity center position of the change region extracted at the current time with the reference feature amount 32 and the trajectory 31 stored for each past change region, and the feature amounts are similar. The change areas having the consistency in position are identified as change areas caused by the same factor, the center of gravity position of the change area at the current time is added to the identified trajectory 31, and is identified by the image feature amount of the change area at the current time. The reference feature value 32 is updated.

  2 and 3 exemplify data processed by the abnormality detection device 1.

  Images 70 to 75 are input images captured at times t to t + 5. In the input image, a person a, a chair b, a person c, and a bag d are captured.

  Images 300 to 305 are background images 30 stored at times t to t + 5. Images 300 to 305 include an image of a chair b that is a background object.

  Images 80 to 85 are difference images between the input images 70 to 75 and the background images 300 to 305 at the times corresponding to the input images 70 to 75, respectively. The difference image includes information on the change area.

  Graphs 320 to 325 are trajectories 31 stored at times t to t + 5. The locus 31 is actually time-series data of coordinates, but is illustrated as a graph for assisting understanding.

  The person a appeared in the monitoring space at time t, moved in the monitoring space until time t + 4, and left outside the monitoring space. The person a was captured in the input images 70 to 74 while changing the position thereof, and the change areas 80A, 81A, 82A, 83A, and 84A for the person a were extracted at times t to t + 4. These change areas were tracked as area A (change area group identified by area ID “A”), and trajectories 320A, 321A, 322A, 323A, and 324A of area A were detected.

  The chair b was present in the monitoring space from time t to t + 5. However, the chair b was moved from the original position at time t + 2 by the person a and left at time t + 4. As a result, change areas 82B, 83B, and 84B were extracted at the movement source position of the chair b. These change areas were tracked as area B (change area group identified by area ID “B”), and loci 322B, 323B, and 324B of area B were detected. As another result, a change area 84E is extracted at the position to which the chair b is moved. This change area was tracked as an area E (change area identified by the area ID “E”) different from the area B, and a locus 324E of the area E was detected.

  The person c appeared in the monitoring space at time t + 2, moved in the monitoring space until time t + 4, and left outside the monitoring space. The person c was picked up by the input images 72 to 74 while changing the position thereof, and the change areas 82C, 83C, and 84C for the person c were extracted at times t + 2 to t + 4. These change areas were tracked as area C (change area group identified by area ID “C”), and loci 322C, 323C, and 324C of area C were detected.

  鞄 d was carried by the person c and appeared in the monitoring space at time t + 2, and was left in the monitoring space at time t + 4.鞄 d is a suspicious object brought into the surveillance space, and person c is a suspicious person who brought a suspicious object. At time t + 2 to t + 3, the image of 鞄 d was captured as a part of the image of the person c. Therefore, the area C by the person c was tracked, but tracking of 鞄 d was not performed. After the time t + 4 after being left, the single change areas 84D and 85D of 鞄 d are extracted, and these change areas are areas D different from the area C (change area group identified by the area ID “D”). And traces 324D and 325D of the region D were detected.

  The area classifying unit 43 calculates the movement amount of the change area from the trajectory 31 of each change area and compares it with the still determination amount. The classification is made into still areas less than the static determination amount, and the classification result is stored in the area attribute information 33. The movement amount is calculated as a total movement distance obtained by sequentially calculating distances between successive data in the time series data constituting the locus 31 of each change area. As the stillness determination amount, a value as close to 0 as possible is set in advance within a range where the extraction error of the change region is not determined as movement. The static area is a change area in which a substantial movement amount is not detected.

  The change area generated by the facility user is classified as a movement area. There may be suspicious individuals among facility users. On the other hand, a change area caused by a suspicious object or a background object is classified as a stationary area. However, the image of the suspicious object or background carried by the facility user is integrated with the facility user image, and the facility user image is dominant in the integrated image. The change area is identified as the change area of the facility user.

  In the example of FIGS. 2 and 3, the areas A and C are classified as moving areas, the areas B, D, and E are classified as stationary areas. As a result, the contents of the area attribute information 33 at each time are shown in FIG. As shown in FIG. In the example of FIG. 4, the moving area is temporarily set as a stationary area for convenience (area A at time t and area C at time t + 2) because the moving amount cannot be confirmed at the time of initial appearance.

  The derivation determination means 44 calculates the degree of separation between the change area classified as the movement area and the change area classified as the stationary area, and the movement area and the stationary area where the separation degree less than the derivation determination value is calculated. During this period, a derivation relationship is determined that the moving area is a derivation source of the still area, and the determined derivation relation is written in the region attribute information 33.

The derivation determining unit 44 calculates the reciprocal of the overlap rate between the still region and the moving region immediately before the still region is newly detected as a separation degree, or separates a value obtained by subtracting a similar overlap rate from 1. Calculate as degrees. As the derivation determination value, a value close to 1 is set in advance so that the extraction error of the change region is not regarded as an overlap.
The time constraint is a constraint for correctly determining the derivation relationship between the stationary region and the moving region that caused the stationary region. Since multiple facility users come and go in the surveillance space, it is often the case that facility users unrelated to suspicious objects and background objects pass through. It should have been detected as a part of the derivation source moving area immediately before the new detection. Therefore, a true derivation relationship can be determined without determining an erroneous derivation relationship in the moving region that has just passed through the stationary region due to the time constraint.

In another embodiment, the derivation determination unit 44 calculates the distance from the trajectory 31 of the moving area to the trajectory 31 of the stationary area as the degree of separation. In this case as well, only the passage through the stationary region is made by providing a time constraint so as to calculate the distance using the section corresponding to the time immediately before the stationary region locus 31 starts to be detected among the locus 31 of the moving region. A true derivation relationship can be determined without determining an erroneous derivation relationship in the movement area.
When the distance between the trajectories is calculated as the degree of separation, a value about one half of the average height of a human is set as the derivation determination value. At this time, the derivation determination value is stored as a function of the position in the image, and the derivation determination unit 44 reads out the derivation determination value corresponding to the position of the moving region, thereby adapting to the change in the apparent size depending on the position in the image. Accurate derivation determination can be performed. Further, the derivation determining means 44 may measure the long axis length of the moving area and set the length of one half of the long axis length as the derivation determining value for the moving area. By doing this, it is possible to perform accurate derivation determination adapted to changes in the apparent size depending on the position in the image and individual differences.

  Typically, the derivation relationship is determined with respect to a static region generated from the behavior of the facility user as follows. In other words, typical events for which the derivation relationship is determined include (1) the facility user left the brought-in item, (2) the facility user brought out the background item, and (3) the facility user received the background item. (4) The facility user moved the background. Among these, (1) to (3) are events that should be detected as abnormal, and the facility users of (1) to (3) are suspicious persons. On the other hand, (4) is an event that should be excluded from detection targets as not abnormal.

  In the example of FIGS. 2 and 3, since the degree of separation between the region 82B and the region 81A or the degree of separation between the locus 322B and the locus 321A is less than the derivation determination value, the derivation relationship with the region A as the derivation source and the region B as the derivation destination. Is determined, and the degree of separation between the area 84E and the area 83A or the degree of separation between the locus 324E and the locus 323A is less than the derivation determination value. Since the degree of separation between 84D and the area 83C or the degree of separation between the locus 324D and the locus 323C is less than the derivation determination value, the derivation relationship with the area C as the derivation source and the area D as the derivation destination is determined. As a result, the contents of the region attribute information 33 at each time are as shown in FIG.

  When the abnormality determination unit 45 comprehensively determines the analysis results of each change region by the tracking unit 42, the region classification unit 43, and the derivation determination unit 44, and determines that there is a change region caused by the involvement of a suspicious person, an abnormality signal is output. Output. Therefore, the abnormality determination unit 45 sets a change area that satisfies a predetermined condition as an abnormality candidate area, observes the progress, and sets a change area that is still set as an abnormality candidate area at a predetermined time point after the setting as a suspicious person. It is determined that the region is an abnormal change region caused by the involvement of the.

  Specifically, the abnormality determination unit 45 detects a pair of still areas in which a derivation relationship is set for the same moving area, and sets a still area other than the detected pair as an abnormality candidate area. A pair can be detected by searching the region attribute information 33.

When the facility user moves the background object, the movement area by the facility user, the stationary area from which the background object is moved, and the stationary area to which the background object is moved are extracted, A derivation relationship with the same moving area is set for each stationary area of the moving destination. Therefore, by setting a stationary area other than the above pair as an abnormal candidate area, it is not necessary to determine an abnormality with respect to a change area due to the movement of the background object, so that an abnormality detection with high accuracy can be performed. Thereby, the monitoring efficiency of the monitoring staff can be increased.
In addition, since the derivation relationship with the same moving object is a condition, even if the suspicious person leaves the suspicious object and the movement of the background object by a person different from the suspicious person occurs at the same time, or a plurality of different persons Even if background objects move at the same time, they are not confused to detect a pair.

  Here, there may be a rare case in which one suspicious person leaves two or more suspicious objects, and a pair is also detected in this case. Therefore, the abnormality determination unit 45 collates the background image 30 in one of the pairs with the input image in the other of the pair, and sets a still region other than the pair that has been collated as an abnormality candidate region.

If the detected pair is due to the movement of the background object, the image of the background object captured on the background image side in the stationary area of the movement source and the image captured on the input image side in the stationary area of the movement destination The image of the background object matches. On the other hand, since the suspicious object originally does not exist in the monitoring space, the image of the suspicious object does not exist in the background image 30. Therefore, if the detected pair is due to a plurality of suspicious objects, a matching match cannot be obtained.
Therefore, since an abnormal candidate area is set without excluding a pair generated when a single suspicious person leaves a plurality of suspicious objects, the detection accuracy of the suspicious object can be maintained high.

  In addition, the abnormality determination unit 45 sets an abnormality candidate region by further excluding a still region in which no derivation relationship is set. A static region where no derivation relationship is set is a noise region caused by noise not originating from the facility user. An example of the noise is light inserted from a window. By not setting a static area in which a derivation relationship is not set as an abnormality candidate area, it is possible to detect abnormality with high accuracy since it is not determined abnormality for noise. Thereby, the monitoring efficiency of the monitoring staff can be increased.

  In this way, the abnormality determination unit 45 that has set the abnormality candidate area confirms at each time whether the abnormality candidate area has been set, and determines that the abnormality candidate area that has been set at any one of the following times is abnormal. Judge that there is.

  First, the abnormality determination unit 45 determines that the abnormality candidate region is abnormal when the trajectory 31 of the moving region determined as the derivation source of the abnormality candidate region is not detected (that is, when the derivation source moving region disappears). Judge that there is. The disappearance of the derivation source moving area means that there is no possibility that a pair including the abnormal candidate area is detected. As a result, the abnormality can be detected quickly when the suspicious person disappears from the field of view.

  Secondly, the abnormality determination unit 45 may continue to set the abnormality candidate region at a time point (elapsed time) after a predetermined determination grace time has elapsed since the start of detection of the locus 31 of the abnormality candidate region. The abnormality candidate area is determined to be abnormal. The passage of time can be measured from the data length of the locus 31. As the determination grace time, a time (for example, about several tens of seconds to one minute) that a normal facility user spends on moving the background object in the monitoring space is set in advance. Thereby, even if a suspicious person stays in the visual field for a long time, an abnormality can be detected quickly without waiting for the suspicious person to disappear out of the visual field.

In the above description, the background object that has been largely moved is accurately excluded and the abnormality is determined. However, the background object may be shifted to the extent that it overlaps the image range in which the background object was originally captured. In this case, the change areas due to the background object are extracted together and are not detected as a pair.
Therefore, the abnormality determination unit 45 compares the luminance distribution of the background image 30 and the luminance distribution of the input image in the still region where the derivation relationship is set, and the two luminance distributions are similar beyond a preset distribution similarity criterion. In the case where there is a failure, the abnormal candidate region is set by further excluding the still region.
A color histogram can be adopted as the feature amount of the luminance distribution. In another embodiment, the luminance average value and the luminance dispersion value can be used as the feature amount of the luminance distribution. A value corresponding to about 50% of the area (number of pixels) of the change area to be determined is set as the displacement determination threshold Ta.
In the change area generated by shifting the background object, the background image after the movement is included on the input image side and the background object before the movement is included on the background image side, and thus the two luminance distributions are similar. On the other hand, since there is no suspicious object image in the background image 30 in the change area caused by the suspicious object that does not originally exist in the monitoring space, the two luminance distributions are different.
Therefore, it is no longer necessary to determine an abnormality with respect to a change region caused by a small movement of the background object, so that it is possible to detect an abnormality with high accuracy. Thereby, the monitoring efficiency of the monitoring staff can be increased.

  When an abnormality is detected in this way, the abnormality determination unit 45 generates and outputs an abnormality signal including the input image, the change area, the date and the like where the abnormality is determined.

  The display unit 5 is a display device such as a liquid crystal display, and displays information included in the abnormality signal so as to be visible.

  The recording device 6 is a hard disk recorder, a DVD recorder, or the like, and records information included in the abnormal signal so that it can be stored for a long time.

[Operation of Abnormality Detection Device 1]
Hereinafter, the operation of the abnormality detection device 1 will be described with reference to FIGS.

  When an administrator who confirms that the monitoring space is unmanned turns on the apparatus, each unit and each means are initialized and start operating (S1). The background image generation means 40 stores the input image being initialized in the storage unit 3 as the background image 30, the tracking means 42 clears the reference feature 32 and the trajectory 31, and the area classification means 43 and the derivation determination means 44 The attribute information 33 is cleared.

  After the initialization, the process of S2 to S12 is repeated each time a new image is input from the imaging unit 2 to the control unit 4.

  When a new image is input (S2), the change area extraction means 41 of the control unit 4 performs a background difference process between the input image and the background image 30 to extract the change area (S3), and the control unit 4 tracks it. The means 42 advances the tracking of the change area by comparing the trajectory 31 and the reference feature quantity 32 with the extracted information of the change area to identify the change area (S4). As a result of the tracking, the trajectory 31 and the reference feature 32 are updated with information on the change area at the current time.

  The process of step S4 will be described in detail.

  First, the tracking unit 42 calculates the gravity center position of the change area extracted at the current time, and extracts the image feature amount from the input image in the change area.

  Similarly, at the time prior to the current time, the position of the center of gravity and the extraction of the image feature amount are performed, and an area ID is assigned to these pieces of information and stored as the trajectory 31 and the reference feature amount 32. The tracking means 42 applies a constant velocity linear model to the stored trajectory 31 up to the previous time to obtain the predicted position of each object at the current time. For the calculation of the predicted position, a Kalman filter or the like can be used instead of the constant velocity linear model.

  Subsequently, the tracking unit 42 sets all combinations of each change area at the current time and each past change area. For each combination, the tracking unit 42 uses the similarity between the image feature amount of the change area at the current time and the reference feature quantity 32 (similarity between features), and the similarity between the centroid position and the predicted position of the change area at the current time (centroid). (Similarity between) and {α × similarity between features + (1−α) × similarity between centroids} is calculated as the total similarity. However, 0 <α <1.

The tracking unit 42 identifies a likely combination based on a comparison between the total similarity and a preset identification threshold. That is, the tracking means 42 selects the maximum overall similarity for each change region at the current time and each past change region, compares it with the identification threshold, and if the identification threshold is exceeded, the combination for which the overall similarity is calculated Is identified. When two or more objects overlap on the image, a change area in which these objects are integrated is extracted, and two or more combinations exceed the identification threshold for one change area. Such one-to-many identification Is also acceptable.
It is recognized that a past change area that has not been identified as a change area at the current time has disappeared. Also, a change area at the current time that has not been identified as a past change area is recognized as a new area that has newly appeared in the field of view.

Subsequently, the tracking unit 42 adds the center of gravity position of the change area at the current time to the identified locus 31 and overwrites the identified reference feature quantity 32 with the image feature quantity of the change area at the current time. Further, the tracking unit 42 deletes the trajectory 31 and the reference feature amount 32 of the change area in which the disappearance is recognized from the storage unit 3. Thereafter, the change area is not tracked. Further, the tracking unit 42 assigns a new area ID to the center of gravity position of the new area and stores it as the trajectory 31, and assigns the same area ID to the image feature quantity of the same area and stores it as the reference feature quantity 32. This area is newly registered as a tracking target.
Further, the tracking unit 42 stores the coordinate information of each change region and the region ID in the storage unit 3 in association with each other in order to calculate the degree of separation in step S61 described later. The storage unit 3 holds coordinate information for two hours including the current time so that it can be traced back to coordinate information one hour before.

When the tracking progresses in this way, the control unit 4 sets a loop process in which each change area is set as the attention area sequentially, and executes the area determination process in step S6 for each change area (S5 to S7).
Details of the area determination processing will be described with reference to FIG.

  First, the control unit 4 confirms whether the attention area is a new area (S60). If the region is a new region, the control unit 4 creates an item of attention region in the region attribute information 33 and writes “static region” in the region type column (YES in S60 → S61). If the data length of the locus 31 of the attention area is 1, it can be determined that it is not a new area if it is 2 or more. It should be noted that the “still area” written in step S61 is an initial value, and is rewritten as a “moving area” when the time advances and a motion is detected from the locus 31 of the attention area.

  When the attention area is a new area, derivation determination by the derivation determination unit 44 of the control unit 4 and pair detection by the abnormality determination unit 45 of the control unit 4 are performed.

  The derivation determining means 44 searches the moving area from the area attribute information 33, calculates the degree of separation of the attention area with respect to the searched moving area (S61), compares the degree of separation with the derivation determination value, and determines the degree of separation. It is confirmed whether or not there is a proximity movement region that is less than the value (S62). If there is a proximity movement area (YES in S62), the derivation determining means 44 writes “derived area” in the area type field of the attention area in the area attribute information 33, and the proximity movement area is displayed in the derivation source column of the attention area. The area ID is written (S63). The degree of separation is calculated from the stored coordinate information of the change area at the current time and the coordinate amount information of the change area one time before.

  The abnormality determination unit 45 searches the region attribute information 33 to confirm whether or not the same derivation source as the attention region is set in the change region other than the attention region, that is, whether there is a pair including the attention region. (S64). When a pair is detected (YES in S64), abnormality determination means 45 performs pair verification (S65). That is, the abnormality determination unit 45 calculates the color histogram Hcb of the part corresponding to the change area paired with the attention area in the background image 30, and calculates the color histogram Hci of the part corresponding to the attention area in the input image. And Hci are calculated as the inter-region similarity Sc, and Sc is compared with the pair determination threshold Tc. If Sc is equal to or greater than Tc, it is determined that the pair matches, otherwise it is determined that they do not match.

  If a match is determined (YES in S66), abnormality determination unit 45 assumes that the pair is generated by the movement of the background object, and displays “background object region” in each region type field of the pair in region attribute information 33. Is written (S67). By updating the background image in step S10 described later, the input image is immediately combined with the background image of the pair part, and the changed area is not extracted in the pair part at a later time. It can be definitely eliminated. Further, by performing pair detection on the new area, the pair can be detected when the movement of the background object is completed, so that it is possible to quickly eliminate the non-sensing area.

  If the attention area is a new area but a separation degree less than the derivation determination value has not been calculated, or if the attention area is a new area but there is no moving area, the separation degree cannot be calculated. Assuming that there is no area (NO in S62), the processes in steps S63 to S67 are skipped. If there is a proximity movement area but no pair (NO in S64), the processes in steps S65 to S67 are skipped. If there is a pair but a mismatch is determined by collation (NO in S66), the process in step S67 is skipped.

  With reference to FIG. 7, a process when the attention area is not a new area (NO in S60) will be described.

  The abnormality determination unit 45 refers to the region attribute information 33 and confirms whether “abnormal region”, “moving region”, or “abnormal candidate region” is written in the region type field of the region of interest (S70, S71). . If “abnormal area” is written, step S71 and subsequent steps are skipped to avoid multiple detection (YES in S70 → S7), and if “moving region” is written, it cannot be an abnormal candidate, and therefore, step S71 and subsequent steps are skipped. (YES in S70 → S7), if “abnormal candidate area” is written, steps S72 to S82 already processed are skipped (YES in S71 → S83).

  If neither “abnormal area”, “moving area”, or “abnormal candidate area” is written (NO in S70 → NO in S71), the area classification means 43 of the control unit 4 moves from the locus 31 of the attention area. The amount is calculated (S72), and the movement amount is compared with the stationary determination amount (S73). When the movement amount is larger than the still determination amount (YES in S73), the region classification unit 43 rewrites “still region” written in the region type field of the region of interest in the region attribute information 33 to “moving region” ( S74). At this time, if “derived region” is already written in the region type column of the region of interest, these are deleted (S75). Confirmation of whether the attention area is a moving area is continued until the stationary determination time elapses. The reason for waiting for the stationary determination time is to avoid erroneous determination when it is difficult to detect the movement amount because the facility user moves slowly or the movement direction is the optical axis direction of the camera. The region classification means 43 compares the tracking time of the attention region with the stillness determination time (S76), and skips step S77 and subsequent steps until the tracking time exceeds the stillness determination time (NO in S76 → S7). The tracking time may be converted from the data length of the locus 31. A value of about 10 seconds is set in advance as the stillness determination time.

  When the stillness determination time has passed with the movement amount remaining below the stillness determination amount (NO in S73 → YES in S76), the abnormality determination means 45 displays “derived region” in the region type field of the region of interest in the region attribute information 33. Is checked (S77). If “derived region” has not been written (NO in S77), abnormality determination means 45 writes “noise region” in the region type field of the region of interest in region attribute information 33 (S78). The input image is combined with the background image of the noise region by updating the background image in step S11 described later, and the changed region is not extracted in the noise region at a later time. Thus, the generation of a non-sensitive area due to noise is immediately eliminated.

  When “derived area” is written (YES in S77), abnormality determination unit 45 calculates color histogram Hai of the part corresponding to the attention area in the input image and also corresponds to the attention area in background image 30. The color histogram Hab is calculated, the ratio of the common part of Hai and Hab is calculated as the intra-region similarity Sa (S79), and Sa is compared with the displacement determination threshold Ta (S80). If Sa is larger than Ta (YES in S80), abnormality determination means 45 writes “background object region” in the region type field of the region of interest in region attribute information 33 (S81). In other words, the attention area is due to the background object shifted to the extent that it overlaps the image range in which it was originally imaged. The input image is immediately combined with the background image of the attention area by updating the background image in step S10 to be described later, and the non-sensing area caused by shifting the background object is excluded.

  On the other hand, when Sa is equal to or smaller than Ta (NO in S80), the abnormality determination unit 45 writes “abnormal candidate region” in the region type field of the region of interest in the region attribute information 33 (S82). In this way, a derived area that is neither a noise area nor a single background object area is set as an abnormal candidate area.

  The abnormality determination unit 45 refers to the region attribute information 33 to check whether there is an item of the moving region written in the derivation source column of the region of interest (S83). If YES in S83, it is determined that there is no possibility that a pair including the attention area is detected, and “abnormal area” is written in the area type field of the attention area in the area attribute information 33 (S85).

  Further, the abnormality determination means 45 compares the tracking time of the attention area and the determination grace time (S84), and if the tracking time exceeds the determination grace time (YES in S84), a pair including the attention area is determined. Since the time to be detected has passed, “abnormal area” is written in the area type field of the attention area in the area attribute information 33 (S85). The tracking time can be calculated from the data length of the trajectory 31.

  If there is a moving area from which the attention area is derived and the tracking time is less than the determination grace time, a pair including the attention area may still be detected, and step S85 is skipped (S83). NO → S84 and NO → S7).

  Returning to FIG. 5, the processing after the region determination processing for all the changed regions is completed (YES in S7) will be described.

  The abnormality determination unit 45 searches the region attribute information 33 to confirm whether or not there is an abnormal region (S8), and if there is an abnormal region, draws the outline of the abnormal region on the input image and sends the current date and time from the clock unit. The input image group captured in the time range in which the trajectory of the moving area set as the abnormal area derivation source was detected was selected, and the input image on which the abnormal area was drawn and the current date and time were selected. An abnormal signal including the input image group is generated, and the generated abnormal signal is output to the display unit 5 and the recording device 6 (YES in S8 → S9).

The display unit 5 to which the abnormal signal is input displays information included in the signal so as to be visible. The observer who looked at the display confirmed the event that occurred by visually checking the image of the abnormal area. If the suspicious object was left behind, it was checked whether it was a forgotten object or a dangerous object, and recovered. Take action. Further, the suspicious person is identified by visually recognizing the input image group, and the situation is confirmed.
The recording device 6 to which the abnormal signal is input records information included in the signal for a long time. The recorded information will be used later as evidence.

The background image generation means 40 searches the region attribute information 33 to check whether there is a background object region, and if there is a background object region, combines the input image of the region with the background image 30 to combine the background image 30. Is updated (S10). Further, the background image generation means 40 searches the region attribute information 33 to check whether or not there is a noise region. If there is a noise region, the background image 30 is synthesized with the background image 30 by combining the input image of the region. Update (S11). After the next time, a change area due to a background object or noise is not extracted. Further, the background image generating means 40 updates the background image 30 by synthesizing the input image of the part where the change area is not detected with the background image 30 (S12).
In addition, you may perform the process of step S10-S12 collectively. In this case, the background image generation means 40 calculates the sum area of the portion where the background object area, the noise area, and the change area are not detected, and combines the input image of the calculated sum area with the background image 30 to thereby generate the background image. Update 30

When the above process is completed, the process returns to step S2 again, and the process for the input image at the next time is performed.

DESCRIPTION OF SYMBOLS 1 ... Abnormality detection apparatus 2 ... Imaging part 3 ... Memory | storage part 4 ... Control part 5 ... Display part 6 ... Recording apparatus 30 ... Background image 31 ... Trajectory 32. .. Reference feature amount 33... Region attribute information 40... Background image generation means 41... Changed area extraction means 42... Tracking means 43. ..Means for judging abnormalities

Claims (7)

An imaging unit that images the monitoring space at predetermined intervals;
A storage unit storing a background image obtained by capturing the background of the monitoring space;
A change area extraction means for extracting a change area by comparing an input image input from the imaging unit with the background image;
Tracking means for comparing the input images in the change area extracted at the time before and after, tracking the change area, and detecting a track for each tracked change area;
Area classification means for detecting a movement amount from the trajectory, and classifying the change area into a movement area where the movement amount is detected and a stationary area where the movement amount is not detected;
Derivation determining means for calculating a degree of separation between the moving region and the stationary region, and setting a derivation relationship between the moving region and the stationary region where the degree of separation is less than a derivation determination value;
An abnormality determining means for setting the stationary region as an abnormal candidate region and outputting an abnormal signal when the abnormal candidate region is present at a predetermined time point;
With
The abnormality detection unit detects the pair of the still areas in which the derivation relationship is set for the same moving area, and sets the abnormality candidate area by excluding the pair. .   The said abnormality determination means collates the said background image in one of the said pair, and the said input image in the other of the said pair, and when the collation coincides, the said abnormality candidate area | region is set except for the said pair. Anomaly detection device.   The abnormality detection device according to claim 1, further comprising background image generation means for updating a portion corresponding to the pair in the background image with the input image.   The abnormality detection device according to any one of claims 1 to 3, wherein the abnormality determination unit sets the abnormality candidate region by further excluding the static region where the derivation relationship is not set.   The abnormality detection device according to claim 4, wherein the predetermined time point is a disappearance time point when the locus of the moving region in which the derivation relationship is set with the abnormality candidate region is not detected.   The abnormality detection device according to claim 5, wherein the predetermined time point is either the disappearance time point or an elapsed time point after a preset neglect determination time has elapsed since the locus of the abnormality candidate region started to be detected. The derivation determining means calculates the degree of separation between the stationary region and the moving region extracted at a time immediately before the stationary region is newly extracted. The abnormality detection device described in 1.