JP4040651B2 - Camera shake correction method and image system - Google Patents

Description

The present invention relates to an image system that identifies a specific moving object from the moving system, about the particular camera shake correction.

  When recognizing a moving body such as a person from a moving image in the monitoring target area, it is necessary to correct the vibration of the installation camera and the vibration due to the wind. As a similar technique, there is a camera shake correction method at the time of shooting. That is, with respect to a predetermined small block in the camera field of view, a search is performed by shifting one pixel at a time, and the position of the most similar block is detected to correct the blur amount.

  In addition, as a method of recognizing a moving object from a moving image of a shaking camera, a plurality of attention points are set and matching between successive image frames is performed, and affine transformation is applied in two stages based on the result. In this method, the background is aligned and the moving object region is extracted from the difference image in the time axis direction after the alignment (Non-patent Document 1).

IEICE Technical Report (PRU95-181, 1995-December)

  A conventional camera shake correction method has a relatively small amount of camera shake shift, so that a small reference block at a predetermined position in the visual field range can be used as a template pattern, and a matching search range may be narrow. However, if the reference block is small or the position is fixed, matching becomes impossible if disturbance or a moving body enters the block.

  On the other hand, the moving object extraction method based on the cited document performs matching between image frames with a plurality of set attention patterns, but if there is an attention pattern with no features (regular pattern), the alignment is incorrect. Will occur. In addition, if there are matching regions in some of the target patterns, applying the two-step affine transformation based on these matching results to perform background alignment will take too much time, and real-time online processing will take place. It becomes difficult.

An object of the present invention is to overcome the problems of the prior art, and to provide a camera shake correction method that corrects camera shake accurately and at high speed even in an environment where the amount of camera shake is large and disturbances exist. The purpose is to provide a used image system .

The above objects are achieved by a method of the camera to correct the camera shake of the captured image, while the reference image of two frames of the image, while the processed image, the processed image using a template pattern created from the reference image By performing pattern matching, this is achieved by calculating the amount of camera shake and correcting image shift.

  Image shift due to camera shake has regularity that is the same amount of movement in the same direction over the entire processed image. Therefore, the camera shake amount can be calculated by specifying a movement vector having the regularity, at least the movement direction, among the movement vectors between the template area and the matching area. The matching area is an area having the maximum similarity extracted by pattern matching.

  As one method, pattern matching is performed on the processed image using a plurality of template patterns created from the reference image, a movement vector with a matching area is calculated for each template pattern, and the movement direction is the same or the movement direction and The movement vector (group) having the maximum frequency with the same movement amount is obtained, the camera shake amount is calculated from the movement direction and the movement amount of the movement vector (group), the image is shifted by the camera shake amount, and the reference image and The camera shake can be corrected by aligning the background of the processed image.

  The template pattern is set as a rectangular area having a predetermined area which is designated one by one in the area of the reference image or is automatically positioned according to a predetermined rule.

  Further, using a plurality of template patterns set as described above as candidates, the density dispersion value of each is obtained, and only the template pattern having a density dispersion value equal to or greater than a certain value is used for pattern matching. In this way, by using a characteristic template pattern having a large variance value, the accuracy of pattern matching is improved. Further, when a large number of patterns are set as a whole of the reference image, the pattern having no features is discarded, so that the processing efficiency is improved.

The above objects are achieved by a imaging system camera to correct the camera shake of the captured image, while the reference image of two frames of the image, image storage means for storing the other as the processed image, the reference image or Latin plate patterns generating template pattern creating means calculates a camera shake amount performs pattern matching for the processed image to the template pattern, is achieved by providing the camera shake correction means for performing a position alignment of the processed image and the reference image The

  Further, based on the difference image between the reference image after the alignment and the processed image, a feature amount including a moving distance of the moving image is calculated, and a specific moving object is identified according to a predetermined determination criterion using the feature amount. A moving object identification means is provided.

  Further, a binarization threshold value is determined from the density frequency distribution of the difference image, binarization processing means for binarizing the difference image is provided, and the feature amount is calculated from the binarized image. Features.

  Further, the present invention is characterized in that there is provided display processing means for accumulating the position coordinates of the specific moving object in time series and displaying a movement locus from the current image of the moving object and the past position coordinates.

According to the present invention, when the camera is shaken by vibration or wind, a moving image in which the shake is accurately corrected can be obtained. Furthermore, it is possible to view the image of the identified from various mobile outside disturbance or target moving objects on the monitor online.

According to the camera shake correction method of the present invention, one frame of an input image obtained by photographing a target area with a camera is taken as a reference image and the next frame is taken as a processed image, and pattern matching between the template pattern created from the reference image and the processed image is performed. performed to calculate the camera shake amount, since the positioning of the background of the reference image and the processed image, Ru can accurately detect the camera shake even if the mobile and noise present in the input image.

  Further, since the template pattern used for pattern matching uses a characteristic pattern having a density dispersion value of a certain value or more, the pattern matching is highly reliable and can be processed efficiently.

According to the image system of the present invention, it is possible to provide an image system capable of correcting image shift due to camera shake when identifying a target object such as a person from a feature amount including a moving amount of a moving object in a moving image. .

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Equivalent elements are denoted by the same reference numerals throughout the drawings.

  FIG. 1 is a functional block diagram of an image monitoring system according to an embodiment. In this system, the image input unit 200 sequentially captures images obtained by capturing the target area at the sampling timing by the ITV camera 100 as image signals and performs A / D conversion. The template pattern creation unit 400 creates a plurality of template patterns using the captured image of one frame as a reference image, obtains a variance value of density for each template pattern, and adopts a template pattern having a large variance value.

  The camera shake amount calculation unit 600 calculates the amount of shake by performing pattern matching using the adopted template pattern using the image of one frame captured from the image input unit 200 as a processed image at the next sampling timing. The camera shake correction unit 800 moves the processed image by the camera shake amount obtained by the camera shake amount calculation unit 600, and aligns the background of the moved image and the reference image.

  The moving object extraction unit 1000 creates a difference image by performing a difference between the moving image and the processed image, calculates a density frequency distribution of the difference image, and calculates a threshold value calculated from density information obtained by removing noise in the density frequency distribution The moving object area is extracted from the binary image or the difference image binarized in step 1, and the feature amount of the moving object area is calculated.

  The moving object identification unit 1200 discriminates the moving object by using the feature amount of the moving object region, discriminates the moving object, and stores the image of the moving object in the image storage unit 1500 when recognizing the presence of the moving object to be monitored. To do. The display control unit 1400 performs control to display on the display device 1600 using the image of the moving object stored in the image storage unit 1500 and the movement locus thereof. These image input unit 200, template pattern creation unit 400, camera shake amount calculation unit 600, camera shake correction unit 800, moving object extraction unit 1000, moving object identification unit 1200, image storage unit 1500, and display control unit 1400 are sequentially repeated. The image is displayed on the display device 1600 in real time, and the monitor monitors the video of the identified target object online using the display device 1600 such as a monitor.

  FIG. 2 is a conceptual diagram for detecting a moving object from a moving image in which the camera is shaking. As shown in FIG. 4A, when the camera shakes, the building 50 of the processed image 40 is shifted from the building 30 of the reference image 10. Therefore, after the processed image 40 is aligned with the reference image 10 by the amount displaced by the camera shake, the difference of the area of the common scene 20 is performed. In this case, nothing is detected in the area of the common scene 20 because there is no moving object or disturbance.

  On the other hand, when there is a moving person in the scene with camera shake as shown in FIG. 5B, the processed image 40 is aligned with the reference image 10 and corrected for camera shake. However, since the person 70 in the reference image 10 has moved to the person 80 in the processed image 40, when the difference between the reference image 10 and the processed image 40 is performed, the image 90 before the movement of the person 70 and the image 95 after the movement are obtained. It can be detected with the correct distance. In this way, when the influence of camera shake is eliminated, a specific moving object, for example, a person can be identified using a feature amount such as a moving distance.

  FIG. 3 is a functional block diagram showing the configuration of the image input unit. When the ITV camera 100 captures a target scene, the image signal captured from one image capturing unit (Fi) 220 is used as a reference image, and the image signal captured from the other image capturing unit (Fi + 1) 240 is processed. As an image, A / D conversion is performed by the A / D conversion unit 260, and the reference image is transmitted to the template pattern creation unit 400 and the processed image is transmitted to the camera shake calculation unit 600.

  FIG. 4 is a functional block diagram showing the configuration of the template pattern creation unit. The template pattern creation unit 400 takes in the reference image from the image input unit 200, the template pattern generation unit 420 generates a plurality of template pattern candidates, and the template pattern determination unit 440 calculates the density dispersion value for each candidate pattern. Then, a candidate pattern having a variance value equal to or greater than the threshold value is determined as a template and stored in the template pattern storage unit 460. As the template pattern to be adopted, for example, in the case of an image of an office room with a window, the threshold value of the dispersion value is about 35-40 or more. In this way, the position of the template pattern with a small variance value is not searched for a shift position as will be described later.

  FIG. 5 is an explanatory diagram showing a template pattern generation method, and is an example of generating a light and dark template pattern. The template pattern generation unit 420 gives the attention point 423 to the reference image from the image input unit 200 and generates a rectangle centered on the attention point 423 as the template pattern 425. The attention point 423 is instructed by using an input device such as a mouse or the position of one reference point. From this point, a plurality of candidate patterns are set with a predetermined regular position as the attention point. Generate automatically.

  FIG. 6 is an explanatory diagram showing another method for generating a template pattern. The template pattern generation unit 420 in this example generates a region of a rectangle 427 directly designated by a mouse or the like on the reference image as a candidate pattern.

  The template may be a binary pattern. In the case of a binary pattern, the template is generated from a binary image obtained by binarizing the reference image with a predetermined threshold value. Or you may produce | generate from the binary image which calculated | required the edge of the reference | standard image and binarized with the predetermined threshold value.

  FIG. 7 is a functional block diagram illustrating a configuration of the camera shake calculation unit. The camera shake calculation unit 600 inputs the created template and the processed image, and the pattern matching region setting unit 620 sets a pattern matching processing region for performing pattern matching on the processed image (window setting). The pattern matching processing area is not necessarily the entire processed image. For example, the template pattern area itself may be enlarged by m pixels on the left and right, n pixels on the top and bottom, or arbitrarily specified, and may be set in accordance with the width of camera shake.

  The pattern matching unit 640 performs pattern matching for each template pattern with respect to the pattern matching processing region, and detects the matching region that is the best match. When the template is a light / dark pattern, light / dark pattern matching is performed, and when the template is a binary pattern, binary pattern matching is performed. The camera shake vector calculation unit 660 calculates a movement vector for each pattern.

  FIG. 8 is a flowchart showing the processing procedure of the camera shake calculation unit. First, a pattern matching area including a template pattern area is set (S650). Next, in the set pattern matching area, the matching area that matches most is searched while shifting the template pattern pixel by pixel (S660). Further, the position of the template pattern is compared with the matching region that most matches in step S660, and a movement vector composed of the movement direction and the movement amount is calculated (S670). Furthermore, it is checked whether or not all pattern matching using the set template pattern has been completed (S680). If not completed, the process returns to step S650, and a series of processing steps are repeated.

  FIG. 9 is an explanatory diagram illustrating an example of the movement vector calculated by the camera shake calculation unit. For each of a plurality of template patterns generated on the reference image, pattern matching is performed on the processed image to search for a matching region having a maximum similarity not less than a predetermined threshold value.

  When the matching region is detected, the representative position (for example, the upper left position of the rectangle) 918 of the processed image that matches the representative position (for example, the upper left position of the rectangle) 915 of the reference image is set as the shift position. The representative position 915 of the reference image is the starting point of the vector, the representative position 918 of the found processed image is the ending point of the vector, and the vector 910 connecting the starting point and the ending point is the moving vector. Similarly, for the next template pattern, pattern matching is performed to find a matching region, the representative position 925 of the reference image is the start point of the vector, the representative position 928 of the found processed image is the end point of the vector, A vector 920 connecting the end points is set as a movement vector.

  When the illustrated moving person 1430 exists, a moving vector 692 corresponding to the moving person 1430 indicates the moving direction and moving amount of the person. However, the movement vectors 694, 696, 698, etc. of the part whose background has been changed by the moving person 1430 have no regularity in the moving direction. Further, since a template pattern having a small variance value is not used (the shift position is not searched), the movement vector is missing as shown.

  FIG. 10 is a functional block diagram illustrating the configuration of the camera shake correction unit. With respect to the movement vector for each template calculated by the camera shake calculation unit 600, the distance between the corresponding vectors (reference vector) and the comparison vector (target vector) between the start points of the corresponding vector distance calculation unit 820 And the distance between the end points. In addition, the inclination calculation unit 830 between corresponding vectors calculates the inclination of the reference vector and the inclination of the target vector.

  Furthermore, when the distance between the start points and the distance between the end points are equal and the inclination of the reference vector and the inclination of the target vector are equal, the frequency distribution calculation unit 840 of the same vector calculates the frequency distribution using both as the same vector. Some errors are allowed in the distance and inclination here. For example, the difference between the distance and the inclination may be the same for ± 1 to 2 pixels or less.

  The X-direction swing amount calculation unit 850 and the Y-direction swing amount calculation unit 860 calculate the X-axis swing amount (movement amount) and the Y-axis swing amount (movement amount) from the start point and the end point of the maximum frequency vector of the frequency distribution. calculate.

  The image shift unit 870 shifts the processed image (or the reference image) in the X direction and the Y direction with the calculated amount of shaking in the X direction and the Y direction. The shift target may be the pattern matching processing area.

  FIG. 11 is a flowchart showing a correction procedure by the camera shake correction unit. First, with respect to the i-th movement vector, the distances between the start points and the end points are calculated for the i + 1-th to m-th movement vectors (S825), and i + 1-th to the i-th movement vector. The inclination is calculated for the m-th movement vector (S835). Next, the movement vectors having the same distance and the same inclination between the start points and the end points are set as the same vector (S845), the number of the same vectors is counted, and the vector frequency distribution is calculated (S855). It is checked whether all the movement vectors have been processed (S865). If not, the process returns to step S825. When all the vectors have been processed, a vector frequency distribution is created.

  Next, the maximum frequency vector is searched from the created frequency distribution (S875), and the X-direction fluctuation amount and the Y-direction fluctuation amount are calculated from the start point and end point of the vector (S880). Finally, the processed image is shifted in the X and Y directions by the calculated amount of shaking (S890). In step S875, if the maximum frequencies are all equal to or less than 1, the vector directions are disjoint and there is no regularity, so it is considered that camera shake has not occurred.

FIG. 12 is an explanatory diagram showing a method for calculating the distance and inclination between corresponding vectors. The movement vector 910 and the movement vector 920 shown in FIG. 9 will be described as an example. Distance calculating section 820 between the corresponding vectors, the distance L1 of the start point of the reference vector V _k P1 (x1, y1) and the object vector V _{k + 1} of the starting point P2 (x2, y2), the reference vector V _k endpoint P1 * The distance L2 between (x1 *, y1 *) and the end point P2 * (x2 *, y2 *) of the target vector V _{k + 1} is calculated by Equation 1.
(Equation 1)
L1 = √ ((x2-x1) + (y2-y1) ² )
L2 = √ ((x2 * −x1 *) ² + (y2 * −y1 *) ² )
Here, in order for the reference vector V _k and the target vector V _{k + 1 to} be the same, it is only necessary to satisfy L1 = L2. Actually, | L1−L2 | ≦ 1 to 2 pixels is regarded as an error range and regarded as the same vector.

The slope calculation unit 830 between corresponding vectors sets the slope M1 of the reference vector V _k from the start point P1 (x1, y1) and the end point P1 * (x1 *, y1 *), and the slope M2 of the target vector V _{k + 1} as the start point P2. From (x2, y2) and the end point P2 * (x2 *, y2 *), the calculation is performed by Equation 2.
(Equation 2)
M1 = (y1 * −y1) / (x1 * −x1)
M2 = (y2 * −y2) / (x2 * −x2)
Here, when M1 = M2 is satisfied, the gradients of the reference vector V _k and the target vector V _{k + 1} are the same. Actually, it is regarded as the same up to about M1-M2 = ± 1.

FIG. 13 is a table in which the frequency distribution of the movement vector in FIG. 9 is calculated. That is, for the movement vector 910, it is determined whether it is the same vector as the other movement vectors 920, 692, 694..., The vector number (for example, 920) compared in the same case is deleted, and the frequency of the movement vector 910 is updated. To do. Next, the same method as the movement vectors 692 and 694 is repeated to create a frequency distribution table in which different movement vectors and their frequencies in the processed image are tabulated. In the table shown in the figure, the frequency of the movement vector 910 is the maximum, and the amount of camera shake is determined by Equation 3 from the start point coordinates and the end point coordinates.
(Equation 3)
X-direction shaking (Δx) = x1 * −x1
Y-direction swing (Δy) = y1 * −y1
The camera shake correction unit 800 uses the X-direction shake amount Δx and the Y-direction shake amount Δy as described above, and the image shift unit 870 shifts the position of the processed image by Δx in the X direction and Δy in the Y direction. Correct camera shake by aligning with the background.

  In the above-described embodiment, it is assumed that both the movement direction and the movement amount of the movement vector match in calculating the frequency of the same vector. However, it is possible to match only the movement direction due to the regularity of image shift caused by camera shake. Further, although a plurality of template patterns are used, when the shift direction due to camera shake can be predicted in advance, the camera shake amount can be calculated even from the result of pattern matching using one template pattern.

FIG. 14 is a functional block diagram illustrating a configuration of the moving object extraction unit. The moving object extraction unit 1000 captures the reference image from the image input unit 200 and the processed image after correction from the camera shake correction unit 800, and the difference image calculation unit 1020 calculates the difference for each pixel between the images. Create an image. Next, after the density frequency distribution calculation unit 1040 calculates the density frequency distribution of the difference image, the noise removal unit 1060 smoothes the density frequency distribution Hj (j = 0, 1,..., N) according to Equation 4. To remove noise. n is the maximum concentration.
(Equation 4)
H _j = (H _j-1 + 2H _j + H _{j + 1} ) / 4-1
H ₀ = (2H ₀ + H ₁ ) / 3-1
_{H n = (2H n + H} n-1) / 3-1
However, J = 1,. . . , N-1.

  A binarization threshold value calculation unit 1070 calculates a binarization threshold value of the difference image from the density frequency distribution smoothed by the noise removal unit 1060. The binarization processing unit 1080 performs binarization processing with the calculated binarization threshold value to create a binary image of the difference image, excludes pixel blocks having an area inappropriate for the detection target as noise, and expands -Create a binary image shaped by shrinking or the like.

  The feature amount calculation unit 1090 calculates the feature amount of the detection target from the binary image by the binarization processing unit 1080, and uses it to identify the monitoring target object. In addition, when the monitoring place is dark and the difference image cannot be extracted even if the target object is present, illumination is performed so that the brightness can be extracted.

  FIG. 15 is an explanatory diagram showing a binarization threshold value calculation method. The binarization threshold value calculation unit 1070 automatically calculates the binarization threshold value th from the maximum density value (max) obtained from the density frequency distribution smoothed by the noise removal unit 1060.

  When there are many change areas of the difference image, the smoothed density frequency distribution is as shown in (a). When the density is searched from the maximum density value (max) 1110 to a smaller value, at least one local maximum value 1120 exists. When the density is searched for on the basis of the maximum value 1120, a minimum value 1130 closest to the maximum density exists. The density value from the minimum value 1130 to the maximum density 1110 is the density distribution of the changed portion. Therefore, the density value of the minimum value 1130 is set as the binarization threshold th. This is a method for determining the binarization threshold value by the mode method.

On the other hand, when the difference area of the difference image is small, the density frequency distribution is as shown in (b) without the local maximum value 1120 as shown in (a) because the frequency of density of the changed portion is small. In this case, the binarization threshold th is set to a value obtained by subtracting the constant C from the maximum density (max) 1110 according to Equation 5.
(Equation 5)
th = max-C
However, th ≧ thmin (lower threshold).

  When the calculated threshold value th is smaller than a preset thmin, the lower limit threshold value thmin is set. The constant C is a value of a difference between the upper limit threshold thmax and the lower limit threshold thmin of white noise in a normal state calculated in advance from the normal image of the target image. Alternatively, any value that indicates the maximum fluctuation range of noise in the normal state of the target image may be used. For example, in a scene where the outdoor environment is monitored in the daytime, the upper limit threshold thmax is about 12 to 14 gradations, and the lower limit threshold thmin is about 4 to 5 gradations. Therefore, the constant C is about 7 to 10 gradations. Even when white noise is added to the maximum value of the changed portion, the noise is reduced by the smoothing process of Equation 4, so that it can be accurately set even for a slight change in density difference.

  FIG. 16 is a functional block diagram showing the configuration of the binarization processing unit. The binarization processing unit 1080 takes in the difference image and the binarization threshold value, the binarization unit 1082 creates a binary image with the binarization threshold value, and the minute area removal unit 1084 is excluded from extraction. A region with a small area is removed as noise, and a binary image of a change region of the difference image is created.

  For example, when the extraction target is a person, the binarization processing unit 1080 binarizes the area having an appropriate area as a person corresponding to the visual field range of the camera. The area below that is noise.

  In many cases, the binary image of the change area is separated from the object to be extracted, and the image correction unit 1086 expands and connects these separation areas and then contracts to correct the binary image. The number of times of expansion and contraction of the image correction unit 1086 is the same, for example, about 2 to 3 in the example of person extraction.

  FIG. 17 is a functional block diagram illustrating a configuration of the feature amount calculation unit. The feature amount calculation unit 1090 of this example calculates the following feature amounts from the binary image. That is, the area calculated by the area calculating unit 1094, the aspect ratio calculated by the aspect ratio calculating unit 1095, and the moving distance calculated by the moving distance calculating unit 1096, and these are time-sequentially at every image input timing i. calculate.

  The area calculation unit 1094 calculates the area by calculating the number of pixels of the binary image. The aspect ratio calculation unit 1095 projects a binary image in the X direction and accumulates the number of pixels for each position of the X axis, and projects in the Y direction and accumulates the number of pixels for each position of the Y axis. The ratio of the Y projection distribution is calculated as the aspect ratio. The movement distance calculation unit 1096 calculates, as the i-th distance, the length between the (i−1) th and i-th centroids using the centroid of the region having the largest area in the binary image. However, i = 1 is the first moving distance.

  FIG. 18 is a table for explaining the relationship between the detection target and the feature amount in the present embodiment. The detection target here is a moving object, which is a person, a small animal, or a vehicle. The feature amount uses an area, an aspect ratio, and a moving distance. When the detection target is a person, the characteristics of the person are that the moving distance is small, the area is medium, and the shape is vertically long. For example, when detecting a person who can be visually confirmed in an outdoor scene in the daytime, the threshold value of the moving distance may be about 20 to 30 pixels and the aspect ratio may be about 3: 1.

  When the detection target is a small animal, the movement distance is medium, the area is small, and the shape is horizontally long or the aspect ratio is approximately 1 as a feature of the small animal. When the detection target is a vehicle, as a feature of the vehicle, both the moving distance and the area are large, the shape is horizontally long, and the aspect ratio is about 1.

  The moving object identification unit 1200 uses the movement distance obtained by the movement distance calculation unit 1096, the area obtained by the area calculation unit 1094, and the aspect ratio obtained by the aspect ratio calculation unit 1095 to detect a person, a small animal, or a vehicle to be detected. Identify and detect. When the target detection target is identified, the image is transferred to the display control unit 1400, and the coordinates of a predetermined position such as the center of gravity are accumulated in the image storage unit 1500 in time series.

  FIG. 19 is a flowchart showing the processing procedure of the moving object identification unit. First, the area is checked (S1300). If the area is medium, it is checked whether the shape is vertically long using the X projection distribution and the Y projection distribution (S1310). If the shape is vertically long, the moving distance is checked (S1315), and if the moving distance is small, it is determined as a person (S1320). If the shape is not vertically long in step S1310 or if the moving distance is not small in step S1315, both are determined to be disturbances (S1330).

  Next, if it is determined in step S1300 that the area is small, the shape is checked (S1340). If the shape is horizontally long or the aspect ratio is the same, the moving distance is checked (S1350). If the moving distance is medium Determined as a small animal (S1360). If the shape is vertically long in step S1340 or not moderate in step S1350, it is determined as a disturbance (S1370).

  If it is determined in step S1300 that the area is large, the moving distance is checked (S1380). If the moving distance is large, the shape is determined (S1390). If the shape is horizontally long or the aspect ratio is the same, the vehicle is determined. (S1395). If the moving distance is not large in step S1380 or the shape is vertically long in step S1390, it is determined as a disturbance (S1370).

  FIG. 20 is a display diagram of the movement trajectory of the identified person. When the display control unit 1400 receives a moving object to be detected, for example, a person identification notification and its image data from the moving object identification unit 1200, the display control unit 1400 displays the image and the movement locus on the display device 1600 online.

  That is, the moving object identification unit 1200 displays the i + 1th image 1430 of the person identified in time series every i−1th, ith, i + 1th, etc., and the i−1th and ith movement trajectory. 1450, i-th and i + 1-th movement trajectory 1460 is displayed. Thereby, the monitoring person can monitor a person and its movement state on a screen.

The functional block diagram which shows the structure of the image monitoring system by one Embodiment of this invention. Explanatory drawing which detects a moving object in the state where the camera is shaking. The functional block diagram which shows the structure of an image input part. The functional block diagram which shows the structure of a template pattern creation part. Explanatory drawing which shows the production | generation method of a template pattern. Explanatory drawing which shows another production | generation method of a template pattern. The functional block diagram which shows the structure of a camera shake calculation part. The flowchart which shows the process sequence of movement vector calculation. Explanatory drawing which shows the example of calculation of the movement vector of the image of 1 frame. The functional block diagram which shows the structure of a camera shake correction part. The flowchart which shows the process sequence of camera shake correction. Explanatory drawing which shows the calculation method of the distance between corresponding | compatible vectors and inclination. The table which shows an example of the frequency distribution of a movement vector. The functional block diagram which shows the structure of a moving object extraction part. Explanatory drawing which shows the calculation method of a binarization threshold value. The functional block diagram which shows the structure of a binarization process part. The functional block diagram which shows the structure of a feature-value calculation part. A table explaining the detection target and its characteristics. The flowchart which shows the identification procedure of a moving object identification part. The display figure which shows the identification result of a moving person.

Explanation of symbols

  100 ... ITV camera, 200 ... Image input unit, 400 ... Template pattern creation unit, 420 ... Template pattern generation unit, 440 ... Template pattern determination unit, 600 ... Camera shake calculation unit, 640 ... Pattern matching unit, 660 ... Camera shake vector Calculation unit, 800 ... camera shake correction unit, 820 ... distance calculation unit between corresponding vectors, 830 ... inclination calculation unit between corresponding vectors, 840 ... frequency distribution calculation unit of the same vector, 850 ... X-direction shake amount calculation unit, 860 ... Y-direction fluctuation amount calculation unit, 870 ... image shift unit, 1000 ... moving object extraction unit, 1020 ... difference image calculation unit, 1040 ... density frequency distribution calculation unit, 1060 ... noise removal unit, 1070 ... binary threshold calculation , 1080 ... binarization processing unit, 1090 ... feature amount calculation unit, 1200 ... moving object identification unit, 1400 ... display control unit, 1500 ... image storage unit, 1600 ... display device.

Claims (4)

In the method of correcting camera shake of images taken by the camera,
Using one of the two frames of the image as a reference image and the other as a processed image, pattern matching of the processed image is performed using a plurality of template patterns created from the reference image, and a movement vector with a matching region for each template pattern Calculating a movement vector (group) having the same frequency of movement or having the same movement direction and movement amount, calculating a camera shake amount from the movement direction and movement amount of the movement vector (group), A camera shake correction method, wherein the image is shifted by the camera shake amount to align the reference image and the processed image. In a camera shake correction method of an image system for identifying a moving object from an image taken by a camera using a feature amount such as a movement amount,
Using one of the two frames of the image as a reference image and the other as a processed image, pattern matching of the processed image is performed using a plurality of template patterns created from the reference image, and a movement vector with a matching region for each template pattern Is calculated, a movement vector (group) having the maximum frequency in which the movement direction and the movement amount are the same is obtained, a camera shake amount is calculated from the movement direction and movement amount of the movement vector (group), and the feature amount is calculated. A camera shake correction method comprising aligning the reference image and the processed image in accordance with the camera shake amount. In an image system that corrects camera shake in images taken by the camera,
While the reference image of two frames of the image, the pattern image storing means for storing as the other processed images, and the template pattern generating means for generating a plurality of template pattern from the reference image, for a more said processed image to said template pattern Matching is performed, a movement vector with the matching area is calculated for each template pattern, a movement vector (group) having the same movement direction or a movement frequency and the same amount of movement is obtained, and the movement vector (group) is obtained. ) to calculate the camera shake amount from the moving direction and the moving amount, characterized in that in the image shifted by said camera shake amount provided, and the camera shake correction means for aligning the processed image and the reference image Imaging system. In claim 3,
Based on the difference image between the reference image after alignment and the processed image, a feature amount including a moving distance of the moving image is calculated, and a specific moving object is identified according to a predetermined determination criterion using the feature amount. An image system comprising an object identification unit.

Images