JP2014048947A - Image monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the detection accuracy of a constant temperature moving object, such as a person, by effectively reducing residual heat erroneous reports.SOLUTION: An image monitoring device for sequentially acquiring thermal images obtained by imaging monitoring space, and performing image processing of the thermal images to determine the existence/absence of a constant temperature moving object in the monitoring space compares a thermal image with a reference image to extract a change area, temporally tracks the change area, calculates a feature quantity which makes the change area more unlike the constant temperature moving object as the temperature of the tracked change area approaches the temperature of an image area of the reference image corresponding to the change area in accordance with the elapse of time, and determines whether the change area is the constant temperature moving object by using the calculated feature quantity.

Description

  The present invention relates to an image monitoring apparatus that sequentially acquires thermal images obtained by imaging a monitoring space, and performs image processing on the sequentially acquired thermal images to determine the presence or absence of a constant temperature moving object such as a person in the monitoring space.

  Conventionally, image monitoring that detects the presence or the number of persons in a monitoring space by extracting an image area indicating the characteristics of a person from a change mode between a thermal image input from an imaging device such as a thermal image camera and a past thermal image. The device is known.

  In this type of image monitoring device, for example, as disclosed in Patent Document 1, a reference image (background image) stored in advance such as a thermal image currently acquired by the imaging device and a thermal image obtained by imaging a state where no person is present. ) And extracting the difference between both images as a change area indicating a moving object, and determining whether the change area is due to a person from the size and motion speed of the extracted change area, The presence / absence of a person and the number of persons are detected.

Japanese Patent Application Laid-Open No. 08-066953

  However, in this type of image monitoring apparatus using a thermal image, a background object such as a chair or a floor warmed by the body temperature of a person or the like existing in the monitoring space is extracted even after the person or the like has left. There has been a problem of continuing to be performed (hereinafter, the change area is referred to as “residual heat extraction”). When such residual heat extraction is extracted, there is a problem that a person continues to be erroneously detected as being present for a while even though no person is present in the monitoring space (hereinafter referred to as “ "Remaining heat misinformation").

  By the way, the image area corresponding to such residual heat extraction changes with time so as to approach the temperature of the reference image (hereinafter referred to as “background temperature”), which is the temperature of the original background object. It has properties. That is, the temperature of the change area such as the chair or the floor warmed by the human body is initially detected at a temperature close to the body temperature of the person, but changes in temperature so as to approach the background temperature with the passage of time. On the other hand, the surface temperature of a constant temperature moving object such as a person is substantially constant regardless of the passage of time. Therefore, the present invention pays attention to such a difference in properties, and determines a change region in which the temperature approaches the background temperature as time passes as a residual heat extraction, thereby effectively reducing the residual heat false alarm and The object is to improve the detection accuracy of a constant temperature moving object such as the above.

In order to achieve the above-described object, an image monitoring apparatus according to claim 1 of the present application includes:
An image monitoring apparatus that sequentially acquires thermal images obtained by imaging a monitoring space, and performs image processing on the thermal images to determine the presence or absence of a constant temperature moving object in the monitoring space,
Extraction means for extracting a change area by comparing the thermal image with a reference image stored in advance in the monitoring space, tracking means for tracking the change area in time, and the change area tracked by the tracking means A feature amount calculation means for calculating a feature amount that the change region is not likely to be the constant-temperature moving object, as the temperature approaches the temperature of the image region of the reference image corresponding to the change region with time, Determination means for determining whether the change region is the constant temperature moving object using the feature amount.

  With this configuration, the extraction unit of the present invention extracts the changed region by comparing the acquired thermal image with a previously stored reference image. In the present invention, a thermal image acquired in the past, such as a background image of the monitoring space, can be appropriately selected and used as the reference image. Then, the feature amount calculation means of the present invention examines the temperature change with time for each change region extracted by the extraction means and temporally tracked by the tracking means. In the present invention, as the temperature of the change area and the temperature of the corresponding image area of the reference image (that is, the background temperature), for example, the average temperature value in each pixel of those areas, or the maximum temperature value of the pixels included in the change area Minimum temperature value, median value, etc. can be used. Then, the feature amount calculation means is a feature amount that is less likely to be a person as the temperature of the change region approaches the background temperature as time elapses, and that indicates that the residual heat generated by the person is likely to be extracted. The amount of “remaining heat extraction likelihood” is calculated. For example, as the temperature difference between the temperature of the change region and the background temperature tends to decrease with time, the degree of residual heat extraction likelihood of the change region is calculated to be larger. Then, the determination means of the present invention determines that the change region is not a person but a residual heat extraction when the calculated degree of likelihood of residual heat extraction is equal to or greater than a predetermined determination threshold, for example, so that the change region is a person. It is determined whether or not there is. As described above, the temperature of the change region of a constant temperature moving object such as a person is substantially constant regardless of the passage of time, while the residual heat extraction changes in temperature so as to approach the background temperature with the passage of time. . Therefore, by obtaining a feature amount that is not human for the change region that is the temperature change pattern of such residual heat extraction (feature amount that seems to be residual heat extraction) and using it for the presence / absence determination of a person, It is possible to effectively reduce the residual heat false alarm and improve the detection accuracy of a constant temperature moving object such as a person.

The image monitoring apparatus according to claim 2 of the present application is the image monitoring apparatus of claim 1,
The feature amount calculating unit obtains a difference temperature value that is an absolute value of a temperature difference between the temperature of the change region tracked by the tracking unit and the temperature of the image region of the reference image corresponding to the change region, and a predetermined period When the amount of decrease in the differential temperature value is equal to or greater than a predetermined threshold value, the predetermined period is defined as a decrease period in which the differential temperature value is decreasing, according to the ratio of the decrease period in the predetermined evaluation period The feature amount is calculated.

  With this configuration, the feature amount calculation unit of the present invention uses the thermal image and the reference image of a plurality of frames in the predetermined evaluation period T from the present to the temperature and background temperature for each change region of each frame. By calculating the absolute value of the temperature difference (difference temperature value) and examining the change pattern of the difference temperature value in the evaluation period T, it is determined whether the temperature of the change region is approaching the background temperature over time. Investigate. For example, when the difference temperature value is calculated every predetermined period Δt in the evaluation period T and the difference temperature value indicates a change that decreases with the passage of time, the temperature of the change region in the evaluation period becomes the background temperature. It can be seen that the change is approaching. At this time, when the decrease amount of the differential temperature value in the predetermined period Δt is equal to or greater than a predetermined threshold value, it is considered that the differential temperature value has decreased in the predetermined period Δt. In general, in a thermal image camera, the output value of a pixel is not stable due to the influence of a disturbance such as heat from a camera housing or the like, or heat generated by a sensor itself, and noise is likely to occur. Therefore, as in the above configuration, when the amount of decrease in the differential temperature value is equal to or greater than a predetermined threshold, the predetermined period Δt is regarded as a decrease period, thereby making it less susceptible to noise. Further, the feature amount calculating means calculates the degree of likelihood of residual heat extraction according to the ratio of the decrease period to the evaluation period T. For example, only when all the predetermined periods Δt of the evaluation period T are reduction periods, a value indicating that the change area is residual heat extraction (remaining heat extraction likelihood = 1) is calculated. More preferably, the degree of heat extraction likelihood may be calculated as a value that varies between 0 and 1 depending on the ratio of the decrease period to the evaluation period T. Thus, even if the change area is residual heat extraction, the degree of likelihood of residual heat extraction is obtained even when there is a predetermined period Δt in which the differential temperature value does not decrease by more than the threshold due to the influence of noise. Can be made less susceptible to noise.

The image monitoring apparatus according to claim 3 of the present application is the image monitoring apparatus according to claim 2,
The feature amount calculating means sets the threshold value to a larger value as the difference temperature value in the predetermined period is larger.

  With this configuration, the feature amount calculation unit of the present invention uses the threshold value used when determining the amount of decrease in the predetermined period Δt as a threshold value at any time of the predetermined period Δt (for example, the first time of the predetermined period Δt or The larger the difference temperature value at the end time, etc.), the larger the value is set. For example, when the temperature difference between the residual heat extraction and the background temperature (difference temperature value) is large, such as immediately after the occurrence of the residual heat extraction, in general, the temperature changes relatively rapidly so that the residual heat extraction temperature approaches the background temperature. However, as the temperature difference (difference temperature value) between the residual heat extraction temperature and the background temperature decreases with time, the temperature change becomes relatively gradual. Thus, as the difference temperature value, which is the temperature difference between the temperature of the change region and the background temperature, increases, the amount of decrease that is the amount that the temperature of the change region approaches the background temperature per unit time tends to increase. Therefore, when a fixed threshold value is used regardless of the magnitude of the difference temperature value, it is easy to determine that the difference temperature value is greater than or equal to the threshold when the difference temperature value is large. The accuracy of determining whether or not is a residual heat extraction is deteriorated. Therefore, as in the above configuration, by setting a threshold according to the size of the differential temperature value, the accuracy of determining whether the label area is a person or residual heat extraction is improved, and residual heat false alarms are reduced. can do.

The image monitoring apparatus according to claim 4 of the present application is the image monitoring apparatus according to any one of claims 1 to 3,
The feature quantity calculating means calculates the feature quantity when the position of the change area tracked by the tracking means on the thermal image is substantially constant.

  With this configuration, the feature amount calculation unit of the present invention calculates, for example, the amount of movement of the center of gravity position during the evaluation period for the change area tracked by the tracking unit of the present invention. Then, the feature amount calculation means performs the process for calculating the degree of likelihood of residual heat extraction only when the position on the thermal image is substantially constant, and when the position is not substantially constant, The degree of residual heat extraction likelihood is kept at 0, and the process for calculating the residual heat extraction likelihood is omitted. The residual heat extraction is a background object whose temperature has changed due to the body temperature of a person or the like, and these background objects are generally stationary objects unlike a person. Therefore, by setting the degree of likelihood of residual heat extraction as 0 without calculating the degree of likelihood of residual heat extraction, the accuracy of determining whether the change area is a person or residual heat extraction can be improved. Furthermore, the amount of calculation required for the determination can be reduced.

The image monitoring device according to claim 5 of the present application is the image monitoring device according to any one of claims 1 to 4,
The feature amount calculating unit is configured to maintain a predetermined stay at a position on the thermal image including the change region, where the constant temperature moving object change region determined by the determination unit as the constant temperature moving object immediately before the appearance of the change region. It is characterized in that the feature amount is calculated when it stays for the determination period or longer.

  With this configuration, the feature amount calculation means of the present invention enables the change area (constant temperature moving object) determined by the determination means to be a constant temperature moving object in the change area existing in the thermal image of several frames immediately before the appearance time of the change area. It is determined whether or not there is a change area. Then, when the constant-temperature moving object change area exists, any one of the constant-temperature moving object change areas exists at a position on the thermal image including the changed area that has appeared, and the constant temperature moving object change area is determined in advance. The process for calculating the degree of likelihood of residual heat extraction is performed only when the stagnation is longer than the stagnation determination period. On the other hand, when such a constant temperature moving object change area does not exist, the degree of residual heat extraction likelihood for the change area is set to 0 and the process for calculating the residual heat extraction likelihood is omitted. To do. Since the residual heat extraction is caused by a background object whose temperature has changed due to the body temperature of a person or the like, in general, there may be a constant temperature moving object such as a person or the like causing the residual heat extraction just before the appearance of the residual heat extraction. In other words, it can be assumed that such a constant temperature moving object has stayed in contact with the background object, so that residual heat extraction has appeared. Therefore, when there is no constant-temperature moving object that causes such a cause, the newly appearing change region is not the residual heat extraction, and by setting the degree of residual heat extraction likelihood as 0, The accuracy of determining whether the change region is a person or residual heat extraction can be improved, and the amount of calculation for the determination can be further reduced.

  According to the image monitoring apparatus according to the present invention, the temperature of the change area changes so as to approach the background temperature as time passes, and the change area is more likely to be residual heat extraction than the change area due to the constant temperature moving object. By detecting the amount and detecting the constant temperature moving object using the feature amount, the detection performance of the constant temperature moving object can be improved.

Illustration of the case where residual heat extraction occurs Block configuration diagram showing the overall configuration of the image monitoring device Flow chart showing processing procedure of overall operation by image monitoring apparatus The flowchart which shows the specific process sequence of a feature-value calculation process Schematic diagram in which the residual heat extraction label area and the person label area are superimposed The flowchart which shows the specific process sequence of residual heat extraction likelihood degree calculation processing The figure which illustrated the time change of the difference temperature value of the tracking label area

  Hereinafter, an embodiment in which the inside of a building is set as a monitoring space, and a supervisor such as a security guard monitors the number of persons existing in the monitoring space with a thermal image obtained by imaging the monitoring space with a thermal image camera, refer to the attached drawings. The details will be described. In the present embodiment, it is determined whether or not the change area is residual heat extraction by determining whether or not the temperature change pattern of the change area extracted from the thermal image changes so as to approach the background temperature. Accordingly, an image monitoring apparatus including a function of performing image processing so that residual heat extraction is not erroneously counted as a person is provided.

  FIG. 1 is a diagram illustrating a case where residual heat extraction occurs. In FIG. 1, the column (a) is a thermal image 100 acquired in a certain period (time t to time t + n + 3), which is arranged in time series from the top. The image area represented by the dotted line 101 in FIG. 1A is an image area corresponding to a chair installed in the monitoring space, for convenience of explanation in the present specification, and is displayed for convenience. Actually, since the temperature is substantially the same as that of the other surrounding background areas, no outline is clearly displayed in the thermal image 100. In addition, an image region denoted by reference numeral 102 in FIG. 1A is a region corresponding to a person existing in the monitoring space (hereinafter referred to as “person region 102”). As shown in FIG. 1A, a person corresponding to the person area 102 appears for the first time at time t + 1, sits on a chair from time t + 2 to time t + n, leaves at time t + n + 1, and heats at time t + n + 2. You can see that the camera is out of the frame. Here, at time t + n + 1, the newly generated image area 103 is an area corresponding to the position of the chair warmed by the body temperature of the person corresponding to the person area 102 (hereinafter referred to as “residual heat area 103”). .

  Further, in FIG. 1, the column (b) is an extracted image 110 obtained by extracting a change area from the thermal image 100 at the same time by an extraction unit to be described later, and is arranged in order from the top. A region denoted by reference numeral 112 in FIG. 1B is a change region 112 corresponding to the person region 102 of the thermal image 100. In addition, a region denoted by reference numeral 113 in FIG. 1B is a change region corresponding to the residual heat region 103 of the thermal image 100, that is, the residual heat extraction 113. As shown in FIG. 1B, between time t + 1 and time t + n, it can be properly determined that there is one person in the monitoring space, but after time t + n + 1, the residual heat extraction 113 is erroneously equivalent to a person. There is a risk of erroneously determining that the change area is to be output and outputting the wrong number of people (two people). Therefore, the image monitoring apparatus 1 according to the present embodiment has a function of performing image processing so that the residual heat extraction 113 is not erroneously counted as a person based on the temporal temperature change pattern of the extracted change region. It is.

  As shown in FIG. 2, the image monitoring apparatus 1 of the present embodiment is schematically configured to include an imaging unit 2, a storage unit 3, an image processing unit 4, and an output unit 5 in order to realize the above-described functions.

  The imaging unit 2 includes, for example, an infrared detection element having predetermined pixels (for example, 320 × 320 pixels), collects and detects infrared rays emitted from the monitoring space by an optical system such as a lens and a mirror, and detects the amount of infrared rays detected. The thermal image camera outputs the temperature distribution of the monitoring space corresponding to the image processing unit 4 as a thermal image. The imaging unit 2 is installed on the ceiling or wall and images a space obliquely below from above. At that time, a thermal image of one frame is acquired at a predetermined time interval and output to the image processing unit 4.

  The imaging unit 2 includes, as imaging parameters, external parameters indicating the installation position and imaging direction, and internal parameters indicating focal length, angle of view, lens distortion, other lens characteristics, and the number of pixels of the imaging element. This imaging parameter can be obtained by actually performing measurement or the like, and is stored in the storage unit 3 in advance. Then, using this imaging parameter, the pixel position in the thermal image can be coordinate-converted between coordinates on the imaging surface of the imaging unit 2 (imaging surface coordinates) and coordinates in the real space (real coordinates). . In this embodiment, both (1) conversion from imaging surface coordinates to real coordinates and (2) conversion from real coordinates to imaging surface coordinates using the imaging parameters are collectively referred to as perspective transformation.

  The storage unit 3 stores camera setting information 3a related to the image monitoring apparatus 1 (for example, installation height of the imaging unit 2 and depression angle) and information used for various processes of the image processing unit 4. Examples of information used for various processes of the image processing unit 4 include a thermal image for each frame acquired by the imaging unit 2, a reference image 3b used for extracting a change region from the thermal image by the extraction unit 4a, and an image A setting value 3c (for example, an extraction threshold used by the extraction unit 4a, various thresholds used by the feature amount calculation unit 4c, a determination threshold used by the determination unit 4d, etc.) used for each process of the processing unit 4 is included.

  The image processing unit 4 is configured by a computer including a CPU and the like, receives an input of a digitized image from the imaging unit 2, and performs an extraction process, as a series of processes shown in FIGS. In order to execute a labeling process, a tracking process, a feature amount calculation process, and a determination process, an extraction unit 4a, a tracking unit 4b, a feature amount calculation unit 4c, and a determination unit 4d are included.

  The extraction unit 4a performs an extraction process for extracting an area having a temperature change from the thermal image acquired by the imaging unit 2 as a change area. In the present embodiment, for example, a past thermal image is stored in advance in the storage unit 3 as the reference image 3b, and the difference between the thermal image acquired this time and the reference image 3b is equal to or greater than a predetermined extraction threshold stored in the storage unit 3. It is determined that there is a temperature change, and a region with this temperature change is extracted as a change region. At this time, a thermal image acquired in the past, such as a thermal image in the background of the monitoring space, can be appropriately selected and used as the reference image 3b.

  Further, the extracting unit 4a performs a labeling process on each extracted change area. In the labeling process, when attention is paid to an extraction pixel having a change area, the extraction pixels adjacent to the vicinity of 8 of the attention extraction pixel are regarded as a single extraction pixel area. This is a process of assigning a unique label. Further, in the labeling process in the extraction unit 4a, a plurality of change areas within a predetermined range are regarded as change areas made of an integrated object, and label integration processing for integrating them into a single label is performed. Hereinafter, a change area (including one or a plurality of change areas integrated by the label integration process) to which a label is given by the labeling process is referred to as a “label area”. In addition, the change area | region in this invention respond | corresponds to the label area | region (including the tracking label area | region mentioned later) in this embodiment.

  In the present embodiment, the change region is extracted from the difference from the pre-stored reference image 3b in the currently acquired thermal image. However, the present invention is not limited to this, and the time change of the pixel value is subjected to frequency analysis such as a digital filter. It is also possible to use a method of extracting the change region by the above. Further, the label integration process can be omitted.

  The tracking unit 4b temporally tracks the change region extracted by the extraction unit 4a, and the label region (change region) obtained by the labeling process by the extraction unit 4a is compared with the tracking label region of the previously acquired image. A tracking process for associating is performed. Here, the tracking label region is a unique label (hereinafter, referred to as a label that is different from the label in the labeling process by the extraction unit 4a) so that the label region by the same tracking object is always the same label in the images acquired so far. This refers to the label area to which the tracking label is reassigned. Specifically, the tracking process is the same as the previously acquired thermal image tracking label area based on the positional relationship and size in the thermal image label area that is newly acquired and being processed. It is determined whether the object is due to an object. If it is determined that the object is the same, the same tracking label area as the tracking label area of the previously acquired image is assigned again. For a label area that is present in the currently processed image and is not associated with any of the previous tracking label areas, tracking is started as a new appearance tracking label area, and a new tracking label is assigned. For tracking label areas that existed last time and are not associated with any of the label areas that appear in the currently processed image, tracking is terminated and the tracking label is discarded.

  The feature amount calculation unit 4c calculates a feature amount for determining whether each tracking label region is a person or residual heat extraction based on the image feature of each tracking label region tracked by the tracking unit 4b. Execute the process. Details of the feature amount calculation processing will be described later. The feature amount calculated here is “residual heat” obtained from a time change of a temperature difference (differential temperature value) with respect to the reference image 3b for each label area. “Extractability” is included. Further, for each label area, the estimated actual size and the estimated actual area in the real space calculated using the aspect ratio of the circumscribed rectangle of the label area, the installation height of the apparatus stored in the storage unit 3 and the depression angle, The “personality degree”, which is a value representing the personness calculated from these, is also included in the feature amount.

  The determination unit 4d determines whether each tracking label region is in accordance with a predetermined determination condition based on the characteristic amount of the tracking label region calculated by the feature amount calculation unit 4c (personality degree, residual heat extraction degree). A determination process for determining whether or not a person is present is performed. Details of the determination process will be described later. In addition, the determination unit 4 d counts the number of label areas determined to be a person, and outputs the number of persons existing in the monitoring space to the output unit 5.

  The output unit 5 outputs a determination signal indicating the number of persons counted by the determination unit 4d of the image processing unit 4 to the outside. The output unit 5 includes, for example, a display or a sound generator. When the determination signal indicating the number of persons existing in the monitoring space is input from the determination unit 4d of the image processing unit 4, the output unit 5 drives the display and the sound generation device to inform the monitor of the number of persons. Performs notification output. The output unit 5 is connected to a security device (not shown) or a remote monitoring center via a communication line, and outputs a determination signal input from the determination unit 4d of the image processing unit 4 to the communication line. It can also be configured as F.

(About the processing operation of the image processing unit of the image monitoring apparatus 1)
Next, the processing operation of the image processing unit 4 when the presence or absence of a person in the monitoring space is determined using the image monitoring apparatus 1 having the above configuration and the number of persons is output will be described with reference to FIG. In the flowchart of FIG. 3, loop 1 means that each process of ST2 to ST8 is executed every time one frame of thermal image is acquired, and loop 2 executes each process of ST6 and ST7 of the thermal image of the current frame. This means that the number of tracking label areas extracted from is executed.

  When the image monitoring apparatus 1 is activated, the image processing unit 4 executes setting information acquisition processing (ST1). In the setting information acquisition process, camera setting information 3a related to the image monitoring apparatus 1 set in advance and various information used for various processes of the image processing unit 4 are acquired, and the acquired information is stored in the storage unit 3. The information referred to here includes the installation height and shooting direction of the image pickup unit 2, the number of pixels in the vertical and horizontal directions of the image obtained from the image pickup unit 2, the vertical angle of view and horizontal angle of view, the reference image 3b, various threshold values, and the like. is there.

  Next, the image processing unit 4 executes a thermal image acquisition process (ST2). In the thermal image acquisition process, the imaging unit 2 acquires a thermal image obtained by imaging the monitoring range of the monitoring space. Note that immediately after the image monitoring apparatus 1 is started, each pixel value of the reference image 3b used in the extraction unit 4a is initialized with the temperature value of the first acquired thermal image and stored in the storage unit 3. .

  Next, extraction processing is executed by the extraction means 4a of the image processing unit 4 (ST3). In the extraction process, the change area is extracted from the thermal image acquired in the thermal image acquisition process. In the present embodiment, the change region is extracted by determining whether each pixel has a difference between the thermal image acquired this time and the reference image 3b equal to or greater than a predetermined extraction threshold stored in the storage unit 3.

  Next, the extraction unit 4a performs a labeling process on the change area extracted in ST3 (ST4). In the present embodiment, a labeling process is performed in which extracted pixels adjacent to the vicinity of 8 of the target extracted pixel are regarded as a group of extracted pixel areas and labeled. In the labeling process, a label integration process for integrating a plurality of label regions in a predetermined range into one label is performed.

  Next, tracking processing is executed by the tracking unit 4b of the image processing unit 4 (ST5). In the tracking process, the label area labeled by the labeling process of ST4 is associated with the tracking label area of the previously acquired thermal image. In this process, it is determined whether or not there is a tracking label in the thermal image. If there is no tracking label in the thermal image, a reference image 3b stored in the storage unit 3 is newly acquired. Updates with the thermal image.

  Next, the feature quantity calculation means 4c of the image processing unit 4 uses a feature quantity (personality degree) for determining whether each tracking label area tracked temporally in the tracking process is a person or whether residual heat is extracted. , A characteristic amount calculation process for calculating the degree of likelihood of residual heat extraction) is performed (ST6). In the present embodiment, the degree of likelihood of residual heat extraction takes a value between 0 and 1, as will be described later, and the tracking label area has a value as large as that of residual heat extraction. Further, the degree of personality is 1 when the tracking label area is like a person, and 0 when it is not like a person. Details of this processing will be described later.

  Next, the determination unit 4d of the image processing unit 4 uses the feature amount obtained by the feature amount calculation unit 4c to determine whether or not the tracking label area to be processed is a person (ST7). In the determination process, first, it is determined whether or not the degree of likelihood of residual heat extraction in the tracking label area is equal to or greater than a predetermined residual heat determination threshold Th1 (for example, Th1 = 0.6). In the present embodiment, it is assumed that the remaining heat determination threshold Th1 is set in the storage unit 3 in advance. When the likelihood of residual heat extraction in the tracking label area is equal to or greater than the residual heat determination threshold Th1, the tracking label area is regarded as residual heat extraction and is not counted in the number of people existing in the monitoring space. On the other hand, when the degree of residual heat extraction likelihood of the tracking label area is smaller than the residual heat determination threshold Th1, it is determined whether or not the person likelihood degree of the tracking label area is 1. When the person-likeness degree of the tracking label area is not 1, the tracking label area is regarded as a moving object (for example, a small animal) other than a person, and is not counted as the number of persons existing in the monitoring space. On the other hand, when the person-likeness degree of the tracking label area is 1, the tracking label area is regarded as a person, and the number of persons existing in the monitoring space is counted. As a result of the determination process, the tracking label area regarded as the residual heat extraction is updated using the current thermal image in the image area of the reference image 3b corresponding to the tracking label area, and the next thermal image is acquired. The label area is not extracted in the process to be executed.

  Next, the image processing unit 4 refers to the storage unit 3 and performs output processing for outputting the number of persons counted as a person by the determination unit 4d to the output unit 5 (ST8). Upon receiving the result output (determination output) from the image processing unit 4, the output unit 5 can perform notification output for notifying the monitor of the number of persons existing in the monitoring space by display output, audio output, or the like. it can.

(Details of feature value calculation processing)
Next, details of the feature amount calculation processing executed by the feature amount calculation means 4c of the image processing unit 4 in ST6 of the flowchart of FIG. 3 will be described with reference to FIG. This process is performed for the tracking label area that is the target of processing in loop 2.

  The feature amount calculating means 4c of the image processing unit 4 first calculates the circumscribed rectangle coordinates of the tracking label area to calculate the coordinate values of the circumscribed rectangle vertices and the width and height of the circumscribed rectangle of the tracking label area to be processed. Processing is performed (ST10).

  Next, the feature amount calculation unit 4c is configured to calculate the coordinate values, widths and heights of circumscribed rectangles obtained by the circumscribed rectangle coordinate calculation processing of the tracking label area in ST10, and camera setting information 3a stored in the storage unit 3. The assumed actual size calculation process for obtaining the estimated actual width, estimated actual height, and estimated actual area of the tracking label area in the real space is performed by performing perspective transformation using the installation height and depression angle of the image monitoring apparatus (see FIG. ST11).

Next, the feature amount calculation unit 4c executes a person-likeness degree calculation process (ST12). In the person-likeness degree calculation process, the size and shape of the tracking label area are determined based on the estimated actual width and estimated height in the real space of the circumscribed rectangle of the tracking label area calculated in ST12. Calculate the degree of character that represents whether or not a person is likely. Specifically, when the assumed actual area of the tracking label region is 0.3 to 2.5 m ² and the ratio of the width to the height of the tracking label circumscribing rectangle is 0.2 to 1, the personness degree is set to 1. In other cases, the personness degree is set to zero. In addition to the above, for the tracking label area, conditions are set such that (1) the estimated actual width is 60 cm or more and 100 cm or less, (2) the estimated actual height is 70 cm or more and 200 cm or less, and both (1) and (2) It may be calculated so that the person-likeness degree is 1 only when the above condition is satisfied, and the person-likeness degree is 0 when any one of the conditions (1) and (2) is not satisfied. Then, a process of storing the degree of personality calculated here in the storage unit 3 in association with the label of the tracking label area is performed.

  Next, the feature amount calculation means 4c is a difference temperature value that is an absolute value of a temperature difference between the temperature of the tracking label area to be processed and the temperature (background temperature) of the area of the reference image 3b corresponding to the tracking label area. Differential temperature value calculation processing for calculating D is performed (ST13). In the present embodiment, in obtaining the differential temperature value D, first, the absolute value of the temperature difference between the temperature (pixel value) of the thermal image and the temperature (pixel value) of the corresponding reference image 3b in each pixel of the tracking label region. Ask for. And the difference temperature value is calculated by calculating | requiring the average value of the absolute value of the temperature difference calculated | required in each pixel in a tracking label area | region. In the difference temperature value calculation process, the calculated difference temperature value is stored in the storage unit 3 in association with the acquisition time of the thermal image and the tracking label so that it can be used in the residual heat extraction likelihood calculation process described later.

  Next, the feature amount calculation means 4c performs a movement amount calculation process for calculating the movement amount of the tracking label area to be processed (ST14). In the movement amount calculation process, for the tracking label area to be processed, the movement distance of the center of gravity between frames is obtained from the center of gravity position (coordinates) of the previous frame and the center of gravity position (coordinates) of the current frame, and the movement of the center of gravity is obtained. The distance value is calculated as the movement amount of the tracking label area.

  Next, the feature amount calculation means 4c determines whether or not the tracking label region to be processed is substantially constant at the position on the thermal image based on the movement amount calculated in ST13 (ST15). . “Substantially constant” refers to a range in which the tracking label area is not recognized as moving even if noise or the like is taken into consideration. Specifically, when the movement amount calculated in ST14 is equal to or less than a predetermined movement determination threshold Th2 (for example, 4 pixels), it is determined that the tracking label area is substantially constant at the position on the thermal image ( (ST15-Yes), the process proceeds to ST16. On the other hand, when the calculated movement amount is larger than the predetermined movement determination threshold Th2, it is determined that the tracking label area has moved (ST15-No), and a process of setting the residual heat extraction likelihood to 0 in ST20. Do. Residual heat extraction is a background object whose temperature has changed due to the body temperature of a person, etc., and these background objects are generally stationary objects unlike humans, so the degree of likelihood of residual heat extraction can be calculated for tracking label areas with movement. By setting it as 0, it is possible to improve the accuracy of determining whether the change area is a person or residual heat extraction, reduce residual heat false alarms, and further reduce the amount of calculation required for the determination Can do.

  In ST16, the feature quantity calculating means 4c determines that the label area determined to be a person staying at a position including the tracking label area for a predetermined period or more in a period immediately before the appearance of the tracking label area to be processed. A staying person label area detection process for detecting a person label area (hereinafter referred to as “staying person label area”) is performed. Here, the person label area refers to the tracking label area determined as the person label by the determination means 4d, and corresponds to the constant temperature moving object change area in the present invention.

  Hereinafter, the staying person label area detection process will be described in detail by taking the case of FIG. 1 as an example. Here, in the case of FIG. 1, a case where the current time is t + n + 2 and the change area 113 at the current time t + n + 2 is the tracking label area to be processed will be described. In the staying person label area detection process, first, the appearance time of the tracking label area to be processed is checked, and whether or not the person label area exists in a period immediately before the appearance time, for example, a period of 10 frames from the appearance time. Check out. As shown in FIG. 1, the appearance time of the tracking label area 113 to be processed is time t + n + 1, and it can be seen that the person label area 112 exists at time t + n, which is a frame immediately before the appearance time. Subsequently, in the staying person label area detection process, the relationship between the position at the appearance time of the tracking label area to be processed and the position of the person label area is checked, and the person label area exists at a position including the tracking label area to be processed. It is determined whether or not to do. FIG. 5 shows the tracking label area 113 to be processed at time t + n + 1 in FIG. 1 and the person label area 112 at time t + n, which is a frame immediately before the appearance time of the tracking label area 113, at positions on the thermal image. FIG. 5 is a schematic diagram of superposition, and in FIG. As shown in FIG. 5, it can be seen that the person label area 112a includes the tracking label area 113a to be processed at time t + n + 1. As described above, in the staying person label area detection process, it is determined whether or not a person label area exists at a position including the tracking label area to be processed by examining the positional relationship between both label areas. Subsequently, in the staying person label area detection process, if a person label area exists at a position including the tracking label area to be processed, is the person label area staying more than a predetermined stay determination threshold Th3? By determining whether or not there is a staying person label area, it is determined. For example, if the stay determination threshold Th3 is set in the storage unit 3 as 60 frames, and if n = 200 frames in the case of FIG. 1, the person label area 112 stays more than the stay determination threshold Th3. It is determined. Therefore, it is determined that the person label area 112 in the figure is a staying person label area. It is assumed that the retention determination threshold Th3 is set in the storage unit 3 after determining in advance by experiment how long the background object changes depending on the body temperature of the person.

  Next, the feature amount calculating means 4c branches the process depending on whether or not the staying person label area is detected in ST16 (ST17). When the staying person label area is detected in ST16 (ST17-Yes), the process proceeds to ST18. On the other hand, when the staying person label area is not detected in ST16 (ST17-No), a process of setting the degree of residual heat extraction likelihood to 0 is performed in ST20. Since the residual heat extraction is a background object whose temperature has changed due to the body temperature of a person or the like, it can be generally assumed that the person causing the residual heat stays in contact with the background object immediately before the appearance of the residual heat extraction. Therefore, when there is no person who causes such a cause, it is determined that the newly appearing label area is not the residual heat extraction, and the label area is set by setting the residual heat extraction likelihood degree to 0 without calculating it. It is possible to improve the accuracy of determining whether a person is a person or residual heat extraction, reduce residual heat false alarms, and further reduce the amount of calculation required for the determination.

  In ST18, the feature amount calculating unit 4c determines whether or not the elapsed time from the appearance time of the tracking label area to be processed is equal to or longer than a predetermined evaluation period T (for example, 60 seconds). When the elapsed time from the appearance time of the tracking label area is equal to or longer than the evaluation period T (ST18-Yes), the process proceeds to ST19. When the elapsed time from the appearance time of the tracking label area is not equal to or longer than the evaluation period T (ST18-No), processing for setting the degree of residual heat extraction to 0 is performed in ST29.

  In ST19, the feature amount calculation unit 4c executes a residual heat extraction likelihood degree calculation process. Hereinafter, details of the residual heat extraction likelihood degree calculation process will be described with reference to the flowchart of FIG. 6.

  In the residual heat extraction likelihood degree calculation process, first, in the past evaluation period T from the current time, a process of reading the differential temperature value D stored in the storage unit 3 in ST13 for each predetermined segment period Δt is performed (ST30). . The “predetermined period Δt” in the present invention corresponds to the segment period Δt in the present embodiment. In the present embodiment, the evaluation period T is set to 60 seconds, the divided period Δt is set to 10 seconds, and divided into six periods, and the differential temperature value D in each section is read out. FIG. 7 is a graph showing the change over time of the difference temperature value D in the evaluation period T of the tracking label region 113 when the change region 113 in the case of FIG. As shown in FIG. 7, in ST30, a process of reading from the storage unit 3 the differential temperature values D0 to D6 every segment period Δt (10 seconds) from the present to the past evaluation period T (60 seconds before).

  In the flowchart of FIG. 6, loop 3 means that each process of ST31 to ST34 is executed for each division period Δt. In the description of the residual heat extraction likelihood degree calculation process, the selection Δt refers to the segment period Δt that is the processing target in the loop 3.

  Next, based on the differential temperature value D read out in ST30, a process of calculating a decrease amount in the selection Δt is performed (ST31). For example, in the example of FIG. 7, when the selection Δt is a period between −60 seconds and −50 seconds, the decrease amount in the selection Δt can be calculated by D6-D5. Similarly, when the selection Δt is a period between −50 seconds and −40 seconds, the amount of decrease in the selection Δt can be calculated by D5-D4.

  Next, processing for setting the decrease amount determination threshold Th4 to be a larger value as the difference temperature value in the selection Δt is larger is performed (ST32). In the present embodiment, a value obtained by multiplying the difference temperature value D at the start time of the selection Δt by a coefficient α (for example, α = 0.2) smaller than 1 obtained in advance in the experiment is stored in the storage unit 3 as the reduction amount determination threshold Th4. Processing to set. For example, when the selection Δt is a period between −60 seconds and −50 seconds, the decrease determination threshold Th4 in the selection Δt can be calculated by D6 × α. In general, as the difference temperature value, which is the temperature difference between the label region temperature and the background temperature, increases, the amount of decrease, which is the amount that the temperature of the change region approaches the background temperature per unit time, tends to increase. Therefore, by setting the threshold value according to the magnitude of the difference temperature value, it is possible to improve the accuracy of determining whether the label region is a person or the residual heat extraction, and reduce residual heat false alarms. The “predetermined threshold value” in the present invention corresponds to the decrease amount determination threshold Th4 in the present embodiment.

  Next, processing for comparing the amount of decrease in selection Δt calculated in ST31 with the amount of decrease determination threshold Th4 set in ST32 is performed (ST33). When the decrease amount in the selection Δt is equal to or greater than the decrease amount determination threshold Th4 (ST33-Yes), the selection Δt is set as the decrease period (ST34). Here, the decrease period refers to a division period Δt in which it is determined that the temperature of the tracking label region is approaching the background temperature (the difference temperature value D is decreasing). On the other hand, when the decrease amount is smaller than the decrease amount determination threshold Th4 (ST33-No), the process proceeds to loop 3, the selection Δt is changed, and the process from ST31 is repeated.

  When it is determined in loop 3 whether or not all the divided periods Δt in the evaluation period T are decreasing periods, a process for calculating the degree of residual heat extraction is performed using these results (ST35). In the present embodiment, the degree of residual heat extraction likelihood is calculated from the ratio of the decrease period in the evaluation period T. For example, in the evaluation period T = 60 seconds, when the segment period Δt determined to be the decrease period is 50 seconds (the number of the segment periods Δt determined to be the decrease period is 5), the likelihood of residual heat extraction is set to 50. /60≈0.83 is calculated. And the process which memorize | stores the degree of residual heat extraction calculated here in the memory | storage part 3 in association with the tracking label of a tracking label area | region and time is performed. Thus, by calculating the degree of residual heat extraction likelihood according to the ratio of the decrease period to the evaluation period T, it is possible to obtain a more accurate residual heat extraction degree that is less affected by noise.

  When the degree of likelihood of extracting residual heat is calculated in ST35, the degree of likelihood of extracting residual heat is terminated, the feature amount calculating process is also terminated, and the process proceeds to the determination process of ST7 in FIG.

  By the way, the present invention is not limited to the above-described embodiment, and may be implemented in various different embodiments within the scope of the technical idea described in the claims. Further, the effects described in the embodiments are not limited to this.

  In the above embodiment, a value obtained by averaging the absolute value of the temperature difference between the current thermal image and the reference image 3b in the tracking label region is calculated as the difference temperature value D in ST13. However, the present invention is not limited to this, and the difference temperature value D may be calculated using another value that can be a representative value of the difference value between the thermal image and the reference image 3b in the tracking label area. For example, in the absolute value of the temperature difference between the thermal image and the reference image 3b, the differential temperature value D may be calculated using the maximum value, minimum value, median value, etc. in the tracking label area. Further, the absolute value of the difference value between the average temperature of the thermal image in the tracking label area and the average temperature of the corresponding image area of the reference image 3b may be calculated as the difference temperature value D.

  In the above embodiment, the difference temperature value D is read at each segment period Δt in ST30, and the process of calculating the decrease in Δt using these values as they are in ST31. However, the present invention is not limited to this, and all the differential temperature values D in the evaluation period T are read in ST30, a low-pass filter such as a moving average is applied in time series, and a value for each division period Δt is extracted for the result, Δt The amount of decrease in may be calculated. As a result, even if the output value of the thermal image is not stable and there is a lot of noise, the change pattern of the differential temperature value in the evaluation period can be evaluated without being affected by these noises.

  In the above embodiment, using the degree of likelihood of residual heat extraction obtained at ST35 at ST35, a threshold value is determined at ST7 as to whether the tracking label area is residual heat extraction. However, the present invention is not limited to this, and the threshold determination may be performed using the degree of residual heat extraction calculated in the past several frames from the current time. For example, it may be determined in ST7 whether or not the tracking label region is residual heat extraction using a value obtained by averaging the likelihood of residual heat extraction for the past five frames. As a result, even if the output value of the thermal image is not stable (there is a lot of noise) and the value of the degree of residual heat extraction is not stable, it is difficult to be affected by these effects.

  In the above embodiment, the amount of movement of the tracking label is calculated from the center-of-gravity movement distance of the frames before and after the tracking label region in ST14. However, the present invention is not limited to this, and the amount of movement may be calculated according to the overlapping degree of the tracking label areas of the preceding and following frames. Further, the accumulated center-of-gravity movement distance in the evaluation period T may be calculated as the movement amount.

  In the above embodiment, the decrease amount determination threshold Th4 is calculated according to the difference temperature value D at the start time of the selection Δt. However, the present invention is not limited to this, and for example, the difference temperature value D at the end time of the selection Δt or the central time. The decrease amount determination threshold Th4 may be calculated according to the above.

  In the above-described embodiment, the image monitoring apparatus uses a person as a constant-temperature moving object to be detected. However, the present invention is not limited to this, and is applicable to an image monitoring apparatus that uses a constant-temperature moving object other than a person such as a small animal as a detection target. .

DESCRIPTION OF SYMBOLS 1 ... Image monitoring apparatus 2 ... Imaging part 3 ... Memory | storage part 4 ... Image processing part 5 ... Output part 3a ... Camera setting information 3b ... Reference | standard image 3c ... Setting Value 4a ... Extraction means 4b ... Tracking means 4c ... Feature quantity calculation means 4d ... Determination means 100 ... Thermal image 110 ... Extraction image 101 ... Region corresponding to chair 102 ..People region 103 ... Residual heat region 110 ... Extracted image 112 ... Change region of person 113 ... Residual heat extraction

Claims (5)

An image monitoring apparatus that sequentially acquires thermal images obtained by imaging a monitoring space, and performs image processing on the thermal images to determine the presence or absence of a constant temperature moving object in the monitoring space,
An extraction means for comparing the thermal image with a pre-stored reference image of the monitoring space to extract a change area;
Tracking means for temporally tracking the change area;
As the temperature of the change area tracked by the tracking unit approaches the temperature of the image area of the reference image corresponding to the change area with time, the change area is less likely to be the constant temperature moving object. A feature amount calculating means for calculating the amount;
Determination means for determining whether the change region is the constant temperature moving object using the feature amount;
An image monitoring apparatus comprising:   The feature amount calculating unit obtains a difference temperature value that is an absolute value of a temperature difference between the temperature of the change region tracked by the tracking unit and the temperature of the image region of the reference image corresponding to the change region, and a predetermined period When the amount of decrease in the differential temperature value is equal to or greater than a predetermined threshold value, the predetermined period is defined as a decrease period in which the differential temperature value is decreasing, according to the ratio of the decrease period in the predetermined evaluation period The image monitoring apparatus according to claim 1, wherein the feature amount is calculated. The image monitoring apparatus according to claim 2, wherein the feature amount calculation unit sets the threshold value to a larger value as the difference temperature value in the predetermined period is larger.   The said feature-value calculation means calculates the said feature-value when the position on the said thermal image of the said change area tracked by the said tracking means is substantially constant, The said feature-value is described in any one of Claims 1-3. Image monitoring device. The feature amount calculating unit is configured to maintain a predetermined stay at a position on the thermal image including the change region, where the constant temperature moving object change region determined by the determination unit as the constant temperature moving object immediately before the appearance of the change region. The image monitoring apparatus according to any one of claims 1 to 4, wherein the feature amount is calculated when it remains for a determination period or longer.

Images