JP4238042B2 - Monitoring device and monitoring method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a monitoring device and a monitoring method, and more particularly to a monitoring device and a monitoring method for monitoring a plurality of monitoring objects moving in a monitoring space using an imaging device, a CMOS sensor, or the like.
[0002]
[Prior art]
In general, the center of the fiber grating element and the center of the imaging lens are set apart by a predetermined distance parallel to the plane, and based on the distance from the imaging lens to the imaging plane of the imaging device and the distance from the imaging lens to the plane, The height from the point of the imaging object to the plane is calculated, and when the imaging object is placed on the plane, it is detected that the point on the imaging plane has moved to a distant point. The mutual relationship between such distances or heights is calculated, and the posture of a person in the toilet is monitored three-dimensionally (see, for example, Patent Document 1).
[0003]
[Patent Document 1]
JP 2002-122417 A (paragraph 0016, FIG. 1)
[0004]
[Problems to be solved by the invention]
As described above, the conventional monitoring device can three-dimensionally monitor the posture of the imaging object. However, since the movement of the point on the imaging plane is detected, individual monitoring objects can be individually detected from a plurality of monitoring objects. It was difficult to identify and monitor an object to be monitored.
[0005]
Accordingly, an object of the present invention is to provide a monitoring device and a monitoring method for discriminating a plurality of monitoring objects moving in a monitoring space and monitoring the movement of each monitoring object.
[0006]
[Means for Solving the Problems]
  In order to achieve the above object, a monitoring device 1 according to the invention according to claim 1 is, for example, shown in FIG., FIG.As shown in
  A monitoring device that monitors a plurality of monitoring objects moving in a three-dimensional monitoring space, and is a first device that images the monitoring objects 12, 13, and 14 that are installed toward the monitoring space 10 and change over time. The second imaging device 16 and the first imaging device 16 are separated from the first imaging device 16 by a predetermined interval W and are directed toward the monitoring space 10 to image the monitoring objects 12, 13, and 14 that change over time. A difference image forming unit 24 that sequentially forms a difference image between an image picked up at the same time by the first image pickup device 18 and an image picked up after a predetermined time by the first image pickup device 1 and the second image pickup device 18; 25 and a plurality of monitoring objects 12 by correlation calculation based on the difference image of the image by the first imaging device 16 and the difference image of the image by the second imaging device 18 formed by the difference image forming units 24 and 25. Calculation unit 30 for obtaining the distances 13 and 14 respectively. ; It includes a calculation unit 30, a difference image of the image by the first image pickup device 16A plurality of edges on a certain scanning line 40,Image by second imaging device 18On the corresponding scanning line 40 of the difference imageA number is assigned to each of a plurality of edges in order, and a difference image between the difference image of the first image pickup device 16 and the difference image of the image of the second image pickup device 18 is defined as a calculation range between edges between two adjacent numbers. The correlation calculation is performed while shifting the position between the images, and the distances of the monitoring objects 12, 13, and 14 are calculated from the position at which the maximum correlation value is obtained in the correlation calculation, and changes in the distance over time are detected. Thus, it is determined that the region between the numbers is the object region.
[0007]
Here, the first image pickup device 16 and the second image pickup device 18 described in claim 1 typically include two image pickup devices such as a CCD area sensor and a MOS sensor, and passively measures the distance by trigonometry. A mold distance detector can be used.
[0008]
  If comprised in this way, while calculating the correlation of the distance of the some monitoring target object 12,13,14, at least 2 point on the edge which forms the outline of each monitoring target object 12,13,14 is the same monitoring target object Since it is provided with the calculating part 30 which determines whether it is an edge of each, the state of each monitoring object can be monitored.
  Even when a plurality of monitoring objects move at the same time, an area between two adjacent numbers can be determined as an object area.
[0009]
  In order to achieve the above object, a monitoring device 1 according to a second aspect of the present invention is a monitoring device that monitors a plurality of monitoring objects moving in a three-dimensional monitoring space, for example, as shown in FIGS. The first imaging device 16 that images the monitoring objects 12, 13, and 14 that are installed toward the monitoring space 10 and changes over time, and a predetermined interval W from the first imaging device 16. The second imaging device 18 that is installed toward the monitoring space 10 and images the monitoring objects 12, 13, 14 that change with time, and the first imaging device 1 and the second imaging device 18. The first imaging formed by the difference image forming units 24 and 25 that sequentially form the difference image between the image captured at the same time and the image captured with a predetermined time delay, and the difference image forming units 24 and 25. The difference image of the image by the device 16 and the second imaging device 18 An arithmetic unit 30, each determining the position and distance of a plurality of monitored objects 12, 13, 14 by the correlation calculation based on the difference image of the image; includes a calculation unit 30, a first imaging device 16A plurality of edges on a certain scanning line 40,Image by second imaging device 18On the corresponding scanning line 40 of the difference imageA number is assigned to each of the plurality of edges 12a, 12b, 13a, 13b, 14a, and 14b in order, and a difference image signal in an edge between two adjacent numbers is determined as a difference image by the first imaging device 1. By calculating the correlation between the difference images of the images by the second imaging device 18 while shifting the position between the images, and calculating the moving direction vector of each edge 12a, 12b, 13a, 13b, 14a, 14b with time. A region surrounded by an edge between two numbers (for example, between 12a and 12b) is determined as an object region, and monitoring is performed from a position where the maximum correlation value in the correlation calculation is obtained in the region determined as the object region. A distance between the objects 12, 13, and 14 is calculated, and a change with time of the distance is detected.
[0012]
If comprised in this way, in order to calculate the vector direction in which the several monitoring target object 12,13,14 moves, it will determine whether each monitoring target object approaches or moves away from the 1st and 2nd imaging devices 16 and 18. be able to.
[0013]
  In order to achieve the above object, a monitoring method according to the invention of claim 3 is, for example, shown in FIG., FIG.As shown in FIG. 4, a monitoring method for monitoring a plurality of monitoring objects moving in a three-dimensional monitoring space, which is installed toward the monitoring space 10 and changes over time, is monitored objects 12, 13, and 14. A first imaging step for imaging the image and the monitoring objects 12, 13, and 14 that are installed toward the monitoring space 10 and change over time are separated from the first imaging step by a predetermined interval W. A second imaging step, and a difference image forming step of sequentially forming a difference image between an image taken at the same time by the first imaging step and the second imaging step and an image taken after a predetermined time delay And a plurality of monitoring objects 12, 13, 14 by correlation calculation based on the difference image of the image by the first imaging step and the difference image of the image by the second imaging step formed in the difference image forming step. Calculating each of the distances, and Calculation unit processes the difference image of the image by the first imaging stepA plurality of edges on a certain scanning line 40,Difference image of image by second imaging stepOn the corresponding scan line 40 ofNumbers are assigned in order to a plurality of edges, and the difference between the difference image of the first imaging step and the difference image of the image of the second imaging step is calculated between the edges between two adjacent numbers as a calculation range. By calculating the correlation while shifting the position between the images, calculating the distance of the monitoring objects 12, 13, and 14 from the position where the maximum correlation value in the correlation calculation is obtained, and detecting the change over time of the distance Then, it is determined that the area between the numbers is the object area.
  If comprised in this way, while calculating the correlation of the distance of the some monitoring target object 12,13,14, at least 2 point on the edge which forms the outline of each monitoring target object 12,13,14 is the same monitoring target object And a determination step for determining whether or not the edge of each of the objects to be monitored can be monitored.
  Even when a plurality of monitoring objects move at the same time, an area between two adjacent numbers can be determined as an object area.
  In order to achieve the above object, a monitoring method according to a fourth aspect of the present invention is a monitoring method for monitoring a plurality of monitoring objects moving in a three-dimensional monitoring space, for example, as shown in FIGS. The first imaging process for imaging the monitoring objects 12, 13, and 14 that are installed toward the monitoring space 10 and change over time, and a predetermined interval W away from the first imaging process, The second imaging step for imaging the monitoring objects 12, 13, and 14 that are installed toward the monitoring space 10 and change over time, and the first imaging step and the second imaging step at the same time. A difference image forming step of sequentially forming a difference image between the picked-up image and an image picked up with a delay of a predetermined time; and the difference image of the image formed by the first image pickup step and the first image formed in the difference image forming step Correlation calculation based on the difference image of the two images Thus a calculation step of obtaining each of the positions and distances of the plurality of monitored objects 12, 13, 14; wherein the calculating step, a difference image of the image by the first imaging stepA plurality of edges on a certain scanning line 40;The difference image of the image by the second imaging stepOn the corresponding scan line 40 ofA number is assigned to each of the plurality of edges 12a, 12b, 13a, 13b, 14a, and 14b in order, and a difference image signal in an edge between two adjacent numbers is determined as a difference image obtained by the first imaging step. By calculating the correlation between the difference images of the images in the second imaging step while shifting the position between the images, and calculating a moving direction vector of each edge 12a, 12b, 13a, 13b, 14a, 14b with time. A region surrounded by an edge between two numbers (for example, between 12a and 12b) is determined as an object region, and monitoring is performed from a position where the maximum correlation value in the correlation calculation is obtained in the region determined as the object region. A distance between the objects 12, 13, and 14 is calculated, and a change with time of the distance is detected.
[0014]
With this configuration, at least two edge portions are made to correspond to each other based on the vector direction in which the plurality of monitoring objects 12, 13, 14 move, and the distance between the monitoring objects 12, 13, 14 is measured. can do.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol or a similar code | symbol is attached | subjected to the mutually same or equivalent member, and the overlapping description is abbreviate | omitted.
[0016]
FIG. 1 is a schematic partially broken perspective view of a monitoring device 1 according to an embodiment of the present invention. The monitoring device 1 monitors a person 12, a person 13, and a person 14 as a plurality of monitoring objects moving in the three-dimensional monitoring space 10. The three-dimensional monitoring space refers to an area that can be defined by, for example, orthogonal coordinates in the horizontal direction (X axis), the vertical direction (Y axis), and the depth (distance, Z axis).
[0017]
The monitoring device 1 is installed toward a monitoring space 10 composed of a passage 36 and a wall surface 37, and a first imaging device 16 that captures images of a plurality of persons 12, 13, and 14 whose shapes change over time. The second imaging device 18 that is placed toward the monitoring space 10 and is separated from the first imaging device 16 by a predetermined interval W and that captures the person 12, the person 13, and the person 14 whose shapes change over time. The first and second imaging devices are installed on, for example, a ceiling or a wall surface and image the person 12, the person 13, and the person 14.
[0018]
The first and second imaging devices are connected to the control unit 2 with wires, and transmit NTSC video signals to the interface I / F unit 19 of the control unit 2. The control unit 2 includes an I / F unit 19, a calculation unit 30 connected to the I / F unit 19, a storage unit 31 connected to the calculation unit 30, and a counter unit 29 connected to the calculation unit 30. .
[0019]
In addition, the control unit 2 detects that any person taking an image exceeds the base line 38 set in the passage 36, and transmits a detection signal to the counter unit 29. The counter unit 29 is connected to a display 32 such as an external display or an LED display device, and transmits the number of people data passing through the base line 38 to the display 32. The display 32 receives the number of people data from the counter unit 29 and digitally displays the numbers.
[0020]
The storage unit 31 includes an imaging signal storage unit 20 at a first time T0, an imaging signal storage unit 21 at a second time T1 that sequentially stores imaging signals captured by the first imaging device 16, and a second imaging. The imaging signal storage unit 22 at the first time T0 and the imaging signal storage unit 23 at the second time T1 that sequentially store the imaging signals captured by the device 18, and the monitoring targets corresponding to the person 12, the person 13, and the person 14, respectively. A distance information storage unit 34 that sequentially stores the distance between the object and the imaging device, and an edge number storage unit 35 that stores an edge number assigned to an edge appearing in the scanning line signal of the difference image captured by the first and second imaging devices. And are provided. The distance information storage unit 34 may store a mask area address set by orthogonal coordinates of the XYZ axes.
[0021]
The arithmetic unit 30 includes a first difference image forming unit 24 that sequentially forms difference images from the first imaging device 16 and a second difference image forming unit 25 that sequentially forms difference images from the second imaging device 18. The distance calculation unit 26 for calculating the position information and distance information of the monitoring objects corresponding to the person 12, the person 13, and the person 14, and the movement direction or vector direction of the person 12, the person 13, the person 14 and the person 12, respectively. Whether or not two adjacent edges appearing in the scanning line signals of the difference images formed by the movement calculation unit 27 and the difference image forming unit 24 and the difference image forming unit 25 are edges that form the outline of a person. And a pairing determination unit 28 for determining the above.
[0022]
The pairing determination unit 28 determines whether there are two edges corresponding to the contours of the person 12, the person 13, and the person 14 approaching or moving away from the first and second imaging devices, for example. Have moved in the same vector direction, pairing of two edges is determined as valid information, and when moving to different vectors, pairing of two edges is determined as invalid information.
[0023]
The operation of the monitoring device 1 will be described with reference to a partially broken perspective view of FIG. In the monitoring space 10, a person 12, a person 13, and a person 14 are moving toward a stereo camera configured by the first imaging device 16 and the second imaging device 18. For example, a base line 38 region in which the passage 36 is provided along the wall surface 37 in the stereo camera direction is set.
[0024]
Here, the first imaging device 16 has a predetermined number of scanning lines, the second imaging device 18 has a predetermined number of scanning lines, and both the imaging devices 16 and 18 share the scanning line direction in parallel. is set up. In the present embodiment, the first imaging device 16 and the second imaging device 18 are described as separate bodies, but may be configured as a single unit.
[0025]
The monitoring device 1 can count the person passing through the base line 38 by the counter unit 29, and by performing the pairing process of two edges appearing on each scanning line, the area sandwiched between the edges is processed as an object area, and the person Is detected to have passed through the base line 38.
[0026]
In addition, the number of scanning lines included in the first and second imaging devices is typically the same, and the number of scanning lines of each imaging device is, for example, about 500, and each scanning line includes It has approximately 500 pixels. Further, in the present embodiment, the first imaging device 16 and the second imaging device 18 are installed toward the three-dimensional monitoring space 10 with an angle on the wall or ceiling, but are not shown. It is also possible to monitor the persons with the monitoring space 10 oriented in the horizontal direction.
[0027]
The control unit 2 receives imaging signals from the first imaging device 16 and the second imaging device 18 and extracts images formed on the scanning lines (horizontal direction) corresponding to each other in synchronization with each other. Further, 500 scanning lines included in each imaging device may be extracted as the scanning lines to be synchronized. The number of imaging regions may be divided into predetermined intervals.
[0028]
For example, when detecting an entire image of a person, a scanning line for capturing an image with an interval of about 100 mm is extracted, and when detecting a part of a person, a scanning line for capturing an image with an interval of about 50 mm is extracted and the other approaches. When all the objects are detected, it can be set individually so as to extract scanning lines for capturing images at intervals of about 10 mm. For example, in order to detect that any part of a person has entered the monitoring space 10, when at least about 500 scanning lines of each imaging device are selected, one is selected at regular intervals and about 20 are selected. These scanning lines may be extracted.
[0029]
Furthermore, the scanning lines to be synchronized may be extracted in time series. In addition, when extracting scanning lines from the first imaging device 16 and the second imaging device 18, if extraction processing is performed with an external synchronization signal, a circuit that extracts and processes corresponding scanning lines. Can be simplified, and software processing becomes easy.
[0030]
A person far from the monitoring space 10 is imaged by the first and second imaging devices, but is determined by the arithmetic unit 30 as being present in the mask area and not in the monitoring space 10. On the other hand, when people enter the monitoring space 10 from the mask area, the computing unit 30 determines the approach of the people.
[0031]
The monitoring device 1 receives the image signals of the person 12, the person 13, and the person 14 captured by the first imaging device 16 at the time T 0 through the I / F unit 19, and receives the first signal of the storage unit 31 via the calculation unit 30. Is stored in the imaging signal storage unit 20 at time T0. In parallel, the image signals of the person 12, the person 13, and the person 14 imaged at the time T 0 by the second imaging device 18 are received through the I / F unit 19, and the first signal of the storage unit 31 is received via the calculation unit 30. It memorize | stores in the imaging signal memory | storage part 22 of the time T0.
[0032]
Subsequently, the image signals of the person 12, the person 13, and the person 14 at time T 1 captured by the first imaging device 16 after a predetermined time from the time T 0 are received through the I / F unit 19 and stored through the calculation unit 30. The image is stored in the imaging signal storage unit 21 at the second time T1 of the unit 31. In parallel, the image signals of the person 12, the person 13, and the person 14 at the time T 1 captured by the second imaging device 18 with a delay of a predetermined time from the time T 0 are received through the I / F unit 19, and are transmitted through the calculation unit 30. And stored in the imaging signal storage unit 23 at the second time T1 of the storage unit 31.
[0033]
The calculation unit 30 has a function as a difference image forming unit that forms a difference image from image information of each image formed on a scanning line included in each imaging signal. The image information is typically a pixel value of a pixel constituting the image.
[0034]
The calculation unit 30 sequentially calculates a difference image between the images of the person 12, the person 13, and the person 14 at time T0 and the images of the person 12, the person 13, and the person 14 at time T1 that are captured by the first imaging device 16. Then, a difference image is formed in the difference image forming unit 24 of the first imaging device 16. In parallel, the difference image between the image of the person 12, the person 13, and the person 14 at time T0 and the image of the person 12 at time T1 captured by the second imaging device 18 is sequentially calculated, and the second imaging device 18 is processed. The difference image forming unit 25 forms a difference image. The formed difference image is image information indicating the contours (edges) of the person 12, the person 13, and the person 14 obtained by extracting the moving parts of the person 12, the person 13, and the person 14.
[0035]
The two images for forming the difference image are respectively acquired at the imaging time T1 delayed by a predetermined time from the imaging time T0. However, the shift time of the person 12, the person 13, and the person 14 is large. However, it may be set to a time that can be regarded as substantially the same position, for example, about 0.1 second. Or it is set as 1-10 periods (1/30 second-1/3 second) of a television period. When such a difference image is calculated and acquired, the backgrounds of the person 12, person 13, and person 14 are removed, and only the images of the person 12, person 13, and person 14 in motion can be extracted.
[0036]
In addition, the calculation unit 30 specifies an area divided by pixels larger than a predetermined threshold corresponding to the person 12, the person 13, and the person 14 among the absolute values of the pixel values constituting the difference image, It has a region extraction function that extracts the maximum region. For example, a plurality of pixel values (light reception signals) arranged on the horizontal scanning line at the imaging time T0 corresponding to the contour of the person 12 are combined with a plurality of pixel values (light reception signals) arranged on the horizontal scanning line at the imaging time T1, While one horizontal scanning line is shifted in units of one pixel in the horizontal direction, the point at which the combined waveform at time T0 and time T1 becomes maximum can be regarded as a coincidence point between the two images. The computing unit 30 can extract the region where the person 12 has moved by repeating this maximum region extraction process for each predetermined scanning line. Subsequently, for the person 13 and the person 14, similarly, a plurality of pixels corresponding to the contours of each person are combined, and a plurality of pixels corresponding to the outlines of the person 13 and the person 14 are combined to extract the moved region.
[0037]
Since the image formed on the scanning line of each imaging device does not move in the background, the pixel values of the boundary portions of the moving person 12, person 13, and person 14 change abruptly. Therefore, the calculation unit 30 reads out each difference image from the first difference image forming unit 24 and the second difference image forming unit 25, and the absolute value of each pixel value constituting the difference image is larger than a predetermined threshold value. Among the areas divided by pixels, the largest area can be regarded as the boundary between the person 12, the person 13, and the person 14. Here, the boundary is defined as the maximum area among the areas separated by the movement of patterns such as clothes, gloves and the like worn by the person 12, person 13, and person 14, This is because a pixel value having a small threshold value may change.
[0038]
The distance calculation unit 26 correlates the difference image formed by the difference image forming unit 24 and the difference image forming unit 25 based on the difference image of the image by the first imaging device 16 and the difference image of the image by the second imaging device 18. By performing the processing, each distance between the first and second imaging devices and the person 12, the person 13, and the person 14 is calculated, and the distance values of the person 12, the person 13, and the person 14 are temporarily stored.
[0039]
Typically, the distance calculation unit 26 calculates a correlation output value between the maximum areas (pixels that have moved) extracted from the difference image of the person 12, so that the imaging apparatus and the person 12 are based on the trigonometry. The distance to is calculated.
[0040]
Similarly, the distance calculation unit 26 calculates the correlation output value between the maximum regions (pixels that have moved) extracted from the difference image of the person 13, and thus based on trigonometry, the distance between the imaging apparatus and the person 13 is calculated. Calculate the distance.
[0041]
Similarly, the distance calculation unit 26 calculates a correlation output value between the maximum areas (pixels that have moved) extracted from the difference image of the person 14, thereby obtaining the distance between the imaging device and the person 14 based on trigonometry. Calculate the distance.
[0042]
Here, the maximum region extracted from the difference image corresponds to the point that the change in contrast with the background was severe due to the movement of each person formed on the scanning line, and the scanning line of the first imaging device 16 When the image information of the maximum area of the difference image formed on the scanning line of the second imaging device 18 is extracted, and the image information (light reception signal) of the maximum area of each extracted difference image is compared, both imaging devices The position of the maximum area of each difference image is slightly different due to the parallax relationship. That is, the displacement for matching the maximum areas having different positions on the scanning line corresponds to the distance between the two imaging devices and the person.
[0043]
Correlation processing by the distance calculation unit 26 is performed by shifting the difference image by the second imaging device in units of one pixel with respect to the difference image by the first imaging device, and measuring the position where the correlation peak is maximized. The average correlation output value between the two difference images or the correlation output value at the approximate center of the extracted region is calculated to obtain distance information. This correlation processing is configured to perform arithmetic processing on each of the person 12, the person 13, and the person 14 as described above.
[0044]
Here, the correlation output value is a relative imaging position difference generated by parallax between the first imaging device 16 and the second imaging device 18, and typically, the number of pixels (pixels) is obtained by correlation processing. ) Is the output value. The computing unit 30 calculates the distance between the imaging device and the person 12 using trigonometry based on the parallax between the scanning line of the first imaging device and the scanning line of the second imaging device based on the correlation output value. Can do.
[0045]
Correlation processing means that either one of the images obtained from the scanning line of the first imaging device and the scanning line of the second imaging device is shifted in units of pixels until the two images substantially match, This is a process of calculating the shifted amount by indicating the number of pixels, for example. The determination of whether or not the two difference images match is performed based on the strength of the entire overlapping signal when one difference image is fixed and the other difference image is compared while shifting. The coordinates at which the signal peaked are the coincidence points and the correlation peak positions.
[0046]
Further, the correlation processing is performed by binarizing the difference images corresponding to the respective images captured by the first imaging device 16 and the second imaging device 18 with appropriate values and extracting the edge portions thereof, It is also possible to extract a certain area portion and perform correlation processing only in this extracted area. If comprised in this way, since the correlation process uses the binarized difference image, the calculation processing speed can be improved.
[0047]
The distance calculation unit 26 corresponds to each pixel larger than the threshold value in the maximum area extracted from the difference image corresponding to each image captured by the first imaging device 16 and the second imaging device 18. The distance between the imaging devices and the person 12 may be calculated based on the obtained positional deviation of the corresponding pixels. In this case, the distance calculation unit 26 does not calculate the correlation output value between the maximum areas extracted from the difference image as described above, and obtains the correspondence of pixels larger than the threshold value in the maximum area. The displacement of the pixel to be calculated is calculated.
[0048]
The distance calculation unit 26 calculates a distance from the imaging device to the person 12 using a trigonometric method based on the positional deviation of the corresponding pixels, that is, the parallax of both imaging devices. Thereby, for example, the distance calculation unit 26 assigns numbers from the same direction in order to pixels (edges) that exceed a plurality of threshold values existing in the difference image, and sets an in-edge pattern between two adjacent numbers. When correlation processing is performed, parallax is obtained from the positional deviation between the corresponding numbered pixels, and an in-edge pattern that has changed over time is detected, the area between the two numbers can be determined as the object area .
[0049]
With this configuration, the distance calculation unit 26 pairs the pixels (edges) exceeding the thresholds indicating the left and right ends of the person 12, the left and right ends of the person 13, and the left and right ends of the person 14. (In one set), the person 12, the person 13, and the person 14 imaged by the first image pickup device 16 and the second image pickup device 18 are respectively set to the first object region, the second object region, and the third object region. The distance information for distinguishing each object region can be calculated.
[0050]
In addition, by setting the optical axes of the first imaging device 16 and the second imaging device 18 in a substantially horizontal direction, the background and the person are more likely than the case where the difference in contrast between the passage 36 and the person is small. And contrast can be obtained. In addition, since images corresponding to the scanning lines of both imaging devices corresponding to the head and toes of the person can be obtained, the image information of the plurality of scanning lines can be averaged to improve the monitoring accuracy.
[0051]
For example, when the number of scanning lines of the imaging device is α, the accuracy of the standard deviation of the image information can be obtained by subtracting the root α by averaging the α value, thereby improving the reliability of the monitoring device. Can do. In this case, the background movements of the person 12, the person 13, and the person 14 and other persons who are walking in the distance of the passage 36 are simultaneously imaged. The first imaging device 16 and the second imaging device 18 Based on the captured image information, the distance between the moving background, another person existing in the distance, the person 12, the person 13, and the person 14 and the imaging device is calculated, and the person 12 that is the monitoring object equal to or smaller than the predetermined distance. The distance between only the person 13 and the person 14 is extracted, and other moving image information is removed.
[0052]
The calculation unit 30 transmits the distance information of each of the person 12, the person 13, and the person 14 calculated and output by the distance calculation unit 26 to the distance information storage unit 34, and sequentially calculates the distance information captured at time T0 and time T1. Remember me.
[0053]
The calculation unit 30 continues to receive the image pickup signals of the person 12, the person 13, and the person 14 from the first and second image pickup devices, and generates a difference image that is calculated from the next images at the time T0 and the time T1. Based on this, the distance between the person 12, the person 13, and the person 14 is calculated.
[0054]
In addition, the calculation unit 30 reads the distance information of each of the person 12, the person 13, and the person 14 stored in the distance information storage unit 34, and the current person 12, the person 13, and the person 14 acquired by the distance calculation unit 26. Compare with each distance information. By this comparison processing, it is possible to detect changes over time of the person 12, the person 13, and the person 14, and to calculate the movement vector of each person.
[0055]
On the other hand, when correlation processing is performed on the difference image between the first imaging device 16 and the second imaging device 18, the paired edges may not move with the same movement vector. For example, when correlation processing is executed by using a pair of edges corresponding to the right end portion of the person 12 and the left end portion of the person 13, the calculated distance information is the distance of the background located between the person 12 and the person 13.
[0056]
In this case, the pairing determination unit 28 provided in the calculation unit 30 determines that the portion sandwiched between the two edges subjected to the correlation processing indicates the background region instead of the object region. Based on the determination result of the pairing determination unit 28, the arithmetic unit 30 invalidates the pairing of these two edges, and reconfigures the assignment of the edge numbers stored in the edge number storage unit 35 provided in the storage unit 31.
[0057]
For example, an edge number reconstructed from the same direction is assigned to a plurality of edges present in the difference image formed by the first imaging device 16 formed in the difference image forming unit 24. In addition, an edge number reconstructed from the same direction is assigned to a plurality of edges present in the difference image formed by the second imaging device 18 formed in the difference image forming unit 25.
[0058]
The distance calculation unit 26 assigns a number to the edge in each difference image by the first imaging device 16 and the second imaging device 18 based on the edge number reconstructed in the edge number storage unit 35, and performs pairing. Correlation processing is performed on the two edges thus obtained, and the distance between the two edge numbers is calculated based on trigonometry.
[0059]
The calculation unit 30 reads the distance information of each of the person 12, the person 13, and the person 14 stored in the distance information storage unit 34, and re-calculates the current person 12, the person 13, and the person 14, which are recalculated by the distance calculation unit 26. Compare with distance information. By this comparison processing, it is possible to detect changes over time of the person 12, the person 13, and the person 14, and to calculate the movement vector of each person.
[0060]
With reference to the flowchart of FIG. 2, the processing flow of the monitoring apparatus is illustrated. Here, the left image is an image captured by the first imaging device 16, and the right image is described as an image captured by the second imaging device 18. The process starts from start step S10. In step S11, the control unit 2 performs stereo processing of the left image and the right image captured by the first imaging device 16 and the second imaging device 18 via the I / F unit 19. Obtain as an image. Further, the acquired stereo image is subjected to mask processing to remove image information outside the monitoring space 10.
[0061]
The control unit 2 acquires the left image of the t frame that is the first image obtained by capturing the person 12, person 13, and person 14 in the monitoring space at the time t (step S13). Next, the left image of the t + Δt frame, which is the second image captured at the time t + Δt when the predetermined time Δt has elapsed, is acquired (step S14).
[0062]
Subsequently, a difference image (left) between the left image of t frame and the left image of t + Δt frame is extracted (step S15). The difference image corresponds to either the t frame or the t + Δt frame, and after performing the dynamic region extraction process (step S16) from which the background is removed from the extracted difference image, the edge number of the subsequent step S21 is applied. Transition to processing (pairing).
[0063]
In addition, the control unit 2 acquires a right image of t frame, which is a third image obtained by capturing the person 12, person 13, and person 14 in the monitoring space at the time t (step S17). Next, a right image of a t + Δt frame that is a fourth image captured at a time point t + Δt after a predetermined time Δt has elapsed is acquired (step S18).
[0064]
Subsequently, a difference image (right) between the right image of t frame and the right image of t + Δt frame is extracted (step S19). The difference image corresponds to either the t frame or the t + Δt frame, and after performing the dynamic region extraction process (step S20) with the background removed from the extracted difference image, the edge number of the subsequent step S21 Transition to processing (pairing).
[0065]
In the present embodiment, the first imaging device and the second imaging device perform imaging at each time point (t, t + Δt) in parallel. In this case, Δt may be appropriately determined according to the setting conditions of the counting device, but it is desirable to set the time from when a certain frame N is imaged until the next frame N + 1 is imaged.
[0066]
In step S21, the contours of the person 12, person 13, and person 14 with strong contrast in the left image of the t frame are paired. For example, when three people are lined up side by side, there are at least six edges on the same scanning line, so an integer edge number is assigned in order from the left of the scanning line, and adjacent numbers are paired together. Can do.
[0067]
In this edge number processing, the left end portion “1” and the right end portion “2” as viewed from the stereo camera for the left end person 12, the left end portion “3” and the right end portion “4” for the center person 13, The person 14 may be numbered like a left end “5” and a right end “6”. Numbering from “1” to “6” may be performed from the rightmost person 14 to the leftmost person 12.
[0068]
Similarly, in step S21, the contours of the person 12, person 13, and person 14 with strong contrast in the right image of the t frame are paired. For example, edge numbers can be assigned in order from the left of the scanning line, and adjacent numbers can be paired, and the left end person “1” and the right end part “2” as viewed from the stereo camera with respect to the left end person 12, the center The left end portion “3” and the right end portion “4” of the person 13 may be numbered, and the left end portion “5” and the right end portion “6” of the right end person 14 may be numbered. Numbering from “1” to “6” may be performed from the rightmost person 14 to the leftmost person 12.
[0069]
Subsequently, the process proceeds to step S22, where a corresponding point between the difference image of the left image and the difference image of the right image is searched and correlation processing is executed. Corresponding point search is configured to correlate the left and right dynamic regions using the integer numbers assigned by the edge number processing described above. That is, a plurality of pixel signals received between the edge number “1” and the edge number “2” assigned on a certain scanning line of the left difference image and the edge number “assigned on the corresponding scanning line of the right difference image”. Correlation processing is performed in association with each of a plurality of pixel signals received between 1 ”and edge number“ 2 ”, and the distance between each of the regions sandwiched between the edges at the substantially maximum correlation peak position is calculated. When a change over time is detected in the distance between each of the sandwiched regions, the region sandwiched between these two edge numbers can be determined as the object region. This calculated distance can be calculated based on the number of pixels that are shifted due to the parallax of the stereo camera.
[0070]
Similarly, a plurality of pixel signals received between an edge number “2” and an edge number “3” assigned on a certain scanning line of the left difference image, and an edge number assigned on the corresponding scanning line of the right difference image Correlation processing is performed in association with each of a plurality of pixel signals received between “2” and edge number “3”, and the distance between each of the regions sandwiched between the edges at the substantially maximum correlation peak position is calculated. When a change with time is detected in the distance between the regions sandwiched between the two, the region sandwiched between these two edge numbers is determined as the object region. On the other hand, if no change over time is detected in the distance between the regions sandwiched between the edges, it is considered that the background is detected, and it is determined that pairing is invalid.
[0071]
Then, the process proceeds to step S23, and it is determined whether the correlation process of the final pair “5” and “6” has been completed. If the determination result is Yes (Yes), the process branches to step S24. If the determination result is non- (NO), the process branches to step S22, and the edge number correlation processing of the final pair “5” and “6” is performed. Repeat the process until is completed.
[0072]
When branching from step S23 to step S22, the pair edge number is incremented by two, the correlation processing between the next pair of edge numbers “3” and “4” is executed, and the process proceeds to step S23. Also when the process branches from step S23 to step S22 again, the edge number of the pair is incremented by 2 and the correlation process between the edge numbers “5” and “6” as the final pair is executed, and then step S23 is performed. Branch to
[0073]
Accordingly, step S22 correlates a predetermined number of scanning lines with the paired edge numbers “1” and “2”, and then performs the next paired edge numbers “3” and “4”. The predetermined number of scanning lines are correlated with each other, and finally, the predetermined number of scanning lines are correlated with the paired edge numbers “5” and “6”.
[0074]
When the correlation process of the final pair “5” and “6” is completed and the process branches to step S24, the first point is obtained from the corresponding points obtained by the correlation process of the region between the paired edge numbers. And distance information between the second imaging apparatus and the person 12, person 13, and person 14 is calculated. For example, the distances of the person 12, person 13, and person 14 are calculated and output based on the number of pixels that match the left image and the right image.
[0075]
In step S25, the distance information obtained in step S24 is associated with the time axis, and a spatio-temporal image including the distance information of the person 12, the person 13, and the person 14 is created in the distance information storage unit 34 and captured. This is a movement information calculation step for calculating movement information of edges (outlines) of the person 12, the person 13, and the person 14.
[0076]
Subsequently, in step S26, the spatiotemporal image created in the distance information storage unit 34 at the previous time and the current spatiotemporal image are compared, and the person 12, person 13, and person 14 existing in the current spatiotemporal image. This is a temporal change presence / absence determination step for determining whether or not there is a temporal change in a region sandwiched between two edge numbers associated with. For example, it is determined from the distance information based on the current correlation processing result whether the region between the edge number “1” and the edge number “2” indicating the outline of the person 12 is approaching or moving away from the stereo camera. can do. Similarly, the region between the edge number “3” and the edge number “4” corresponding to the person 13 and the region between the edge number “5” and the edge number “6” corresponding to the person 14 are also 2 It is determined whether or not there is a temporal change in the region between the two edge numbers.
[0077]
If there is no temporal change in any one of the regions sandwiched between the edge numbers constituting the pair according to the determination result in step S26 (determination result is NO), the processing is branched to step S30 as invalid information. After reconstructing the edge number, the process proceeds to step S22. For example, correlation processing is executed with the reconstructed edge number that invalidates the leading edge number and assigns the edge number from the next edge. On the other hand, if there is a temporal change in the region between the edge numbers constituting the pair (determination result is Yes), the process branches to step S27 as valid information.
[0078]
Step S27 is a process of storing the distance of the monitoring object and the edge number of the monitoring object in the distance information storage unit 34. In order to monitor the state of the person who changes sequentially, the distance between the correct person 12, person 13, and person 14 and the edge number of the pair corresponding to each person are updated and stored based on the paired edge numbers. The process proceeds to step S28.
[0079]
Here, a region sandwiched between edge numbers constituting a valid pair processed as valid information is determined as an object region. It should be noted that the area sandwiched between the paired edge numbers that are regarded as invalid information can be processed as a background area.
[0080]
Step S28 is a process of determining a person who has exceeded the baseline 38 among the persons captured by the stereo camera. Since the distance and moving direction (vector direction) between the stereo camera and the person 12, person 13, and person 14 are sequentially stored in the distance information storage unit 34 as described above, the traveling direction and distance of each person, etc. It is possible to determine whether or not the base line 38 is moved out of the field of view of the stereo camera.
[0081]
If the determination result of step S28 is non- (NO), the process is branched to the node N12 to acquire the next stereo image. On the other hand, if the determination result is Yes (Yes), since a certain person has exceeded the base line 38, the process branches to step S29 and increments the counter unit 29, and then the process branches to node N12. Get the next stereo image.
[0082]
The present invention is not limited to the sequence of the above-described embodiment. For example, step S11 can be omitted, mask processing can be omitted, and the person 12, person 13, and person 14 captured with a stereo camera can be monitored. it can. Further, correlation processing for only the person 12 (step S22), distance information calculation (step S24), and temporal change coincidence (step S26) are determined from the difference image of the left and right image frames, and edge numbers are reconfigured in step S30. It can also be configured as follows. In short, as long as the sequence can capture a plurality of persons with a stereo camera and monitor the distance of each person, the monitoring apparatus can be configured by omitting each step or changing the flow.
[0083]
As described above, the present embodiment is a monitoring method for monitoring a plurality of monitoring objects (persons) moving in the three-dimensional monitoring space 10, and includes a plurality of monitoring objects (persons) moving in the monitoring space 10. 12, person 13, person 14) and a difference image generation step (steps S 15, S 19) for sequentially forming difference images, and edges of a plurality of persons (person 12, person 13, person 14) based on the difference images A vector calculation step (step S25) for calculating a vector direction in which the person moves based on a change with time of a portion (or an area sandwiched between edges), and at least based on a vector direction in which a plurality of people move A correlation processing step (step S22) for executing correlation processing by matching two edge portions, and a distance measurement step (step S24) for detecting the distance of a person based on the result of the correlation processing. Monitoring method for monitoring the object can be provided.
[0084]
Here, with reference to the schematic block diagram of FIG. 3, the principle of the method of calculating the distance between the imaging device and the person 12 using the trigonometric method will be described. A method for measuring the distance when the person 12 is observed from the side and the person 12 is imaged from the vertical direction by the first imaging device 16 and the second imaging device 18 will be exemplified.
[0085]
Here, w is a distance between optical axes (base line length) between the first imaging device 16 and the second imaging device 18, f is a lens focal length when the light receiving lens of each imaging device is a single lens, and d. Is the parallax on the imaging plane of the first imaging device 16. The focal length f here is the focal length f of the combination lens when a commonly used combination lens is used. Accordingly, the distance a between the top of the target person 12 and the imaging device can be calculated by the following equation.
a = w × f / d (1)
[0086]
In this way, the sensor control unit 60 that has acquired the stereo image from the first and second imaging devices 16 and 18 causes the internal difference image forming unit 62 to perform the difference image and the second imaging of the first imaging device 16. A difference image of the device 18 is formed, the two difference images are subjected to correlation processing by the correlation output calculation unit 64, and a correlation processing signal is output to the control unit 2, whereby the distance (height) from the imaging device to the person 12. Is calculated.
[0087]
Further, the height z of the person 12 is calculated by subtracting the distance to the person 12 from the reference distance h between the passage and the imaging device measured in advance, and the height z is temporarily stored in the distance calculation unit 26. Further, it is also possible to configure so that video signals are directly transmitted from the two imaging devices to the control unit 2 without using the sensor control unit 60.
[0088]
In this way, the monitoring device can accurately monitor the presence and state of the person 12, person 13, and person 14 located in the monitoring space 10 by measuring the image information over time.
[0089]
Since the imaging device can easily take a wide angle of view, the use of a wide angle of view makes it possible to take a sufficiently large monitoring target area in the horizontal direction. By applying such a monitoring device to the monitoring of various facilities, increasing or decreasing the angle of view of the first imaging device 16 and the second imaging device 18 and setting the proximity detection area to an arbitrary width, for example, Persons walking in the passage can be monitored and the persons passing through the base line 38 can be counted.
[0090]
With reference to the conceptual diagram of FIG. 4, the monitoring method of the person 12, the person 13, and the person 14 imaged in the monitoring space 10 by this Embodiment is demonstrated. The first imaging device 16 and the second imaging device 18 are arranged in parallel, and the optical axis is set substantially horizontal in the monitoring space.
[0091]
FIG. 4A shows an image frame captured by a stereo camera. It is assumed that persons existing in the image frame are walking along the wall surface 37 in the direction of the base line 38 along the passage 36. Further, the persons are surrounded by the outlines of the person 12, the person 13, and the person 14 indicated by solid lines in the drawing. Here, a horizontal line indicated by a one-dot chain line in the figure is one scanning line 40 in an arbitrarily extracted image frame for imaging the vicinity of the heads of the people.
[0092]
On the scanning line 40, as viewed from the stereo camera, the left end edge 12a and the right end edge 12b of the person 12, the left end edge 13a and the right end edge 13b of the person 13, and the left end edge 14a and the right end part of the person 14 are displayed. Each of the edges 14b appears and a total of six edges are shown. The six edges are shown on the scanning line 40 based on the correlation values subjected to the above-described correlation processing.
[0093]
In the present embodiment, for example, the edge numbers “1” and “2” are assigned to a pair of the edge 12a and the edge 12b of the person 12, and the edge number is assigned to the edge 13a and the edge 13b of the person 13 as a pair. “3” and “4” are allocated, and the edge numbers “5” and “6” are allocated and configured with the edge 14 a and the edge 14 b of the person 14 as one pair. That is, integer edge numbers from “1” to “6” are assigned sequentially from the left person 12 shown in the figure. Then, adjacent edge numbers are processed as a pair.
[0094]
FIG. 4B is a schematic diagram of a scanning line at a certain point in time. 4A shows a scanning line 40a that represents the position of each edge by extending a broken line in the vertical direction from each edge on the scanning line 40 in FIG. That is, the left pair of edges 12a and 12b, the middle pair of edges 13a and 13b, and the right pair of edges 14a and 14b are arranged on the same scanning line 40a in this order from the left.
[0095]
FIG. 4C is a schematic diagram of the scanning line 40b after a predetermined time has elapsed. The edge 12a and edge 12b of the left pair in the figure are changed to positions separated from each other as compared with FIG. This state can be processed by the control unit 2 (not shown) as an approach vector “← →”.
[0096]
Further, the edge 13a and the edge 13b of the middle pair are changed to positions closer to each other as compared with FIG. This state can be processed as a separation vector “→ ←” in the control unit 2 (not shown).
[0097]
Furthermore, the edge 14a and the edge 14b of the right pair are changed to positions separated from each other as compared with FIG. This state can be processed by the control unit 2 (not shown) as an approach vector “← →”.
[0098]
At this time, the control unit 2 generates a separation vector and an approach vector from the distance information indicated by each edge, and detects that the two edges processed as a pair have moved in the same direction. Further, the change in the edge of the left pair measures the state where the person 12 approaches the stereo camera, and the change in the edge of the middle pair measures the state where the person 13 moves away from the stereo camera. Is measured when the person 14 approaches the stereo camera.
[0099]
FIG. 4D is a schematic diagram of the scanning line 40c after a predetermined time has passed. The edge 12a and edge 12b of the left pair in the figure are changed to positions separated from each other as compared with FIG. The control unit 2 processes this state as the approach vector “← →”. Further, the edge 13a and the edge 13b of the middle pair are changed to positions separated from each other as compared with FIG. The control unit 2 processes this state as the approach vector “← →”. Furthermore, the edge 14a and the edge 14b of the right pair are changed to positions separated from each other as compared with FIG. The control unit 2 processes this state as the approach vector “← →”.
[0100]
At this time, the control unit 2 generates a separation vector and an approach vector from the distance information indicated by each edge, and detects that the two edges processed as a pair have moved in the same direction. However, the change of the edge 12a and the edge 12b of the left pair is measured when the person 12 passes the base line 38 and is closest to the stereo camera direction.
[0101]
It is assumed that the first image pickup device 16 and the second image pickup device 18 are installed at an angle of view that does not capture an outline indicating the left end portion of the person 12 in an image frame picked up at the next time point. If the edge number reconstruction processing is not performed on a total of five edges appearing on the scanning line, one edge 12b of the left pair and one edge 13a of the middle pair are automatically paired, and the edge number “1”. And “2” are determined to be invalid information. That is, the distance representing the background is measured.
[0102]
Therefore, the edge numbers are reconfigured in accordance with the determination result that the temporal changes do not match in accordance with the sequence of the embodiment described above. For example, the leftmost edge that appears in the image frame at the next time point is skipped or the second edge number from the left is decremented to reconstruct the first pair of edge numbers.
[0103]
FIG. 4E is a schematic diagram of the scanning line 40d at the next time point. The edge of the left pair in the figure shows a state in which the edge number is reconfigured by the closest approach of the person 12 and excluded from the pairing process, and the person 12 passes the base line 38 and is counted up 12c. Although the edge of the right end (12b) corresponding to the person 12 is not displayed on the scanning line 40d in the figure, even if the edge position is displayed if it is excluded from the pairing target, it is not displayed. Or any.
[0104]
Further, the edge 13a and the edge 13b of the middle pair are separated from each other as compared with FIG. 4 (d) due to the temporal change of the person 13 given the reconstructed edge numbers “1” and “2”, respectively. The position has changed. The control unit 2 processes this state as the approach vector “← →”. Further, the edge 14a and the edge 14b of the right pair are similarly given reconstructed edge numbers “3” and “4”, respectively. It has changed to a distant position. The control unit 2 processes this state as the approach vector “← →”.
[0105]
In this way, the control unit 2 generates the separation vector and the approach vector from the distance information indicated by each edge, detects that the two edges processed as a pair have moved in the same direction, and the people have passed the base line 38. Alternatively, the state of approaching or moving away from the stereo camera direction can be accurately measured, and a plurality of persons captured on the same image frame can be monitored.
[0106]
In the present embodiment, the movements of the persons are detected using the separation vector “→ ←” and the approach vector “← →”, but the present invention is not limited to this. For example, a movement vector of a person who is moving in parallel can also be detected. That is, since the left and right difference images can be correlated and the position information of the person can be obtained from the distance information and the pixel address, even when a person moving horizontally with respect to the stereo camera is imaged. The monitoring device can generate an accurate horizontal movement vector. For example, when a person horizontally moving to the right is monitored, the vector direction “→” of the right edge of the person and the vector direction “→” of the left edge can be calculated as the same direction.
[0107]
Furthermore, as the people who are the monitoring objects approach in the direction of the stereo camera, the endpoint of the person entering the other imaging device is the other because of the parallax relationship between the right imaging device and the left imaging device. In some cases, the light does not enter the imaging device. For example, the end point 12a, the end point 12b, the end point 13a, the end point 13b, the end point 14a, and the end point 14b in FIG. 4D are incident on the scanning line 40c of the left imaging device, but the scanning line 40c of the right imaging device. In this state, the image corresponding to the end point 12a is not incident. In the case where the number of endpoints captured between the left and right imaging devices is different, the region sandwiched between the endpoints 12a and 12b located on the scanning line 40c of the left imaging device having a large number of endpoints is set to the other (right side). ) When the correlation processing is performed over the entire scanning line 40c of the imaging apparatus or the scanning lines adjacent vertically, and the correlation value of the entire area does not exceed the threshold value, the area sandwiched between the end point 12a and the end point 12b is invalidated. Processing is performed so that the area is determined and the correlation value is not adopted. On the other hand, when the correlation value of the entire region exceeds the threshold value, the region between the end point 12a and the end point 12b is determined as an effective region (object region), the correlation value is adopted, and the person based on the calculated correlation peak position The distance between the camera and the stereo camera can be calculated respectively.
[0108]
As described above, the case where the scanning lines of the two imaging devices 16 and 18 are installed so as to correspond to each other or the corresponding scanning line is selected has been described. Adjustments may be made so that each scanning line corresponds and a fixed one may be used. In this way, since the two imaging devices 16 and 18 can be handled in the same manner as a distance sensor, the installation of the two imaging devices 16 and 18 becomes very simple.
[0109]
The monitoring device 1 distinguishes between the person 12, the person 13, and the person 14 located in the monitoring space 10, and the positions of the persons are standing, sitting, falling down, and the persons are A simple device can follow a series of movements, such as whether it is moving, stopping, or passing through the baseline. In this case, if the difference from the reference distance of the background is acquired, it can be used to determine the state even if the distance is relatively inaccurate.
[0110]
In addition, since the monitoring device 1 uses the difference image, for example, when the people no longer move, the region cannot be specified, but a series of methods such as extending the interval between the image frames in time is used. The state of the people can be determined comprehensively from the movement.
[0111]
Therefore, it is possible to automatically monitor the presence of persons who are stationary or move slowly. For example, it is effective when people walking in the surveillance space have little movement.
[0112]
Furthermore, the distance to the background when there are no persons in the monitoring space is set as the reference distance, and the changes in the persons as the monitoring object from that state are followed. Can be determined.
[0113]
According to the present embodiment as described above, the state of the people in the monitoring space is determined, and monitoring such as that the people are moving in a predetermined direction or passing a predetermined line is accurately performed. And it can be done very easily. In addition, since the information of the entire image frame is not acquired and processed from the imaging device, complicated and large-capacity image processing is not required, and a relatively simple digital circuit or analog circuit, or a combination thereof, is very An inexpensive device can be constructed.
[0114]
【The invention's effect】
As described above, according to the present invention, a monitoring device that monitors a plurality of monitoring objects moving in a three-dimensional monitoring space, is installed toward the monitoring space 10 and changes over time. The first imaging device 16 that images 12, 13, and 14 and the monitoring object 12 that is installed toward the monitoring space 10 at a predetermined interval W from the first imaging device 16 and changes over time. , 13, 14, and a difference image between an image captured at the same time by the first imaging device 16 and the second imaging device 18 and an image captured at a predetermined time later Based on the difference image forming units 24 and 25 sequentially formed, and the difference image of the image by the first imaging device 16 and the difference image of the image by the second imaging device 18 formed by the difference image forming units 24 and 25. , Correlate the distances of multiple monitoring objects 12, 13, 14 And calculating unit 30 for determining whether at least two points on the edge forming the outline of each monitoring object 12, 13, 14 are the edges of the same monitoring object, and moves in the monitoring space This provides an excellent effect of providing a monitoring device that discriminates a plurality of monitoring objects to be monitored and monitors the movement of each monitoring object.
[Brief description of the drawings]
FIG. 1 is a schematic partially broken perspective view of a monitoring device according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a flow of the monitoring apparatus according to the embodiment of the present invention.
FIG. 3 is a principle diagram of a monitoring method used in the embodiment of the present invention.
FIG. 4 is a conceptual diagram of an image frame of the monitoring apparatus according to the embodiment of this invention.
[Explanation of symbols]
1 Monitoring device
2 Control unit
10 Monitoring space
12 people
13 people
14 people
16 First imaging device
18 Second imaging device
19 I / F section
20 Imaging signal storage unit
21. Imaging signal storage unit
22 Imaging signal storage unit
23 Imaging signal storage unit
24 Difference image forming unit
25 Difference image forming unit
26 Distance calculator
27 Movement calculation part
28 Pairing judgment part
29 Counter section
30 Calculation unit
31 Memory unit
32 Display
34 Distance information storage
35 Edge number storage
60 Sensor control unit
62 Difference image forming unit
64 Correlation output calculator

Claims (4)

A monitoring device for monitoring a plurality of monitoring objects moving in a three-dimensional monitoring space;
A first imaging device that is installed toward the monitoring space and images the monitoring object that changes over time;
A second imaging device that is installed at a predetermined distance from the first imaging device and is directed toward the monitoring space and that images the monitoring object that changes over time;
A difference image forming unit that sequentially forms a difference image between an image captured at the same time by the first imaging device and the second imaging device and an image captured with a predetermined time delay;
Calculations for obtaining the distances of the plurality of monitoring objects by correlation calculation based on the difference image of the image by the first imaging device and the difference image of the image by the second imaging device formed by the difference image forming unit. And comprising;
The computing unit is
And edge the plurality of the one scanning line of the first difference image of the image by the imaging device, for the previous SL corresponding plurality of edges on a scanning line of the difference image of the image by the second imaging device, each Numbers are assigned in order, and the position between the adjacent images is shifted between the difference image of the first image pickup device and the difference image of the second image pickup device with the interval between two adjacent numbers as the calculation range. By calculating the correlation, calculating the distance of the monitoring object from the position where the maximum correlation value in the correlation calculation is obtained, and detecting the change over time of the distance, the region between the numbers is the object A monitoring device that determines an area. A monitoring device for monitoring a plurality of monitoring objects moving in a three-dimensional monitoring space;
A first imaging device that is installed toward the monitoring space and images the monitoring object that changes over time;
A second imaging device that is installed at a predetermined distance from the first imaging device and is directed toward the monitoring space and that images the monitoring object that changes over time;
A difference image forming unit that sequentially forms a difference image between an image captured at the same time by the first imaging device and the second imaging device and an image captured with a predetermined time delay;
The positions and distances of the plurality of monitoring objects are respectively calculated by correlation calculation based on the difference image of the image by the first imaging device and the difference image of the image by the second imaging device formed by the difference image forming unit. A computing unit to be obtained;
The computing unit is
And edge the plurality of the one scanning line of the first difference image of the image by the imaging device, for the previous SL corresponding plurality of edges on a scanning line of the difference image of the image by the second imaging device, each Numbers are assigned in order, and the difference image signal in the edge between two adjacent numbers is correlated between the image of the first imaging device and the image of the second imaging device while shifting the position between the images. By calculating and calculating a moving direction vector of each edge over time, an area surrounded by the edge between the two numbers is determined as an object area,
Calculating the distance of the monitoring object from the position where the maximum correlation value in the correlation calculation is obtained in the area determined to be the object area, and detecting a change with time of the distance;
Monitoring device. A monitoring method for monitoring a plurality of monitoring objects moving in a three-dimensional monitoring space;
A first imaging step of imaging the monitoring object in the monitoring space that changes over time;
A second imaging step of imaging the monitoring object at a position separated from the first imaging device by a predetermined interval;
A difference image forming step of sequentially forming a difference image between the image picked up by the first image pickup step and the second image pickup step performed at the same time and an image picked up after a predetermined time;
Calculations for obtaining the distances of the plurality of monitoring objects by correlation calculation based on the difference image of the image obtained by the first imaging step and the difference image of the image obtained by the second imaging step formed in the difference image forming step. And including a process
The calculation step includes
And edge the plurality of the one scanning line of the difference image of the image by the first imaging step, with respect to prior SL corresponding plurality of edges on a scanning line of the difference image of the image by the second imaging step, each Numbers are assigned in order, and the position between the images is shifted between the difference image of the first imaging step and the difference image of the image of the second imaging step, with the edge between two adjacent numbers as the calculation range. By calculating the correlation, calculating the distance of the monitoring object from the position where the maximum correlation value in the correlation calculation is obtained, and detecting the change over time of the distance, the region between the numbers is the object It is determined to be an area ,
Monitoring method. A monitoring method for monitoring a plurality of monitoring objects moving in a three-dimensional monitoring space;
A first imaging step of imaging the monitoring object in the monitoring space that changes over time;
A second imaging step of imaging the monitoring object at a position separated from the first imaging device by a predetermined interval;
A difference image forming step of sequentially forming a difference image between the image picked up by the first image pickup step and the second image pickup step performed at the same time and an image picked up after a predetermined time;
The positions and distances of the plurality of monitoring objects are respectively calculated by correlation calculation based on the difference image of the image obtained by the first imaging step and the difference image of the image obtained by the second imaging step formed in the difference image forming step. A calculation process to be obtained;
The calculation step includes
And edge the plurality of the one scanning line of the difference image of the image by the first imaging step, with respect to prior SL corresponding plurality of edges on a scanning line of the difference image of the image by the second imaging step, each Numbers are assigned in order, and the difference image signal in the edge between two adjacent numbers is correlated between the image of the first imaging step and the image of the second imaging step while shifting the position between the images. By calculating and calculating a moving direction vector of each edge over time, an area surrounded by the edge between the two numbers is determined as an object area,
Calculating the distance of the monitoring object from the position where the maximum correlation value in the correlation calculation is obtained in the area determined to be the object area, and detecting a change with time of the distance;
Monitoring method.

Images