JP2019027882A - Object distance detector - Google Patents

Abstract

To provide an object distance detector with which, when obtaining a distance image from the image of a stereo camera having a fisheye lens, it is possible to change a resolution to be lowered so that the resolution is differentiated depending on image position before obtaining the distance image.SOLUTION: The object distance detector comprises an image analysis unit 14 for calculating a distance image from the image outputted from a pair of imaging sensors 23 and a suspicious person detection unit 16 for performing image recognition from the distance image. The image analysis unit 14 includes an image selection unit 32 for dividing the image imaged by each of a pair of fisheye cameras 11 into a preset plurality of sections and converting the image into images of resolutions set for each section, and a corresponding point selection unit 33 for finding corresponding points that correspond to the same point on the imaged subject in each of two images. The image analysis unit 14 includes a distance calculation unit for finding the distance from the stereo camera to the corresponding point on the basis of two corresponding points that correspond to the same point on the subject.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an object distance detection apparatus that includes a stereo camera including a camera having a pair of fisheye lenses and obtains a distance from an image of the stereo camera to an object.

With recent improvements in image recognition technology, it is possible to avoid danger and drive automatically when driving a car using image recognition of in-vehicle camera images, and to detect known criminals using image recognition of surveillance camera images. It has become. In addition, using a stereo camera having a pair of cameras as an in-vehicle camera or a surveillance camera, for example, based on the difference in position (parallax) between corresponding points on each image corresponding to the same target point on the subject, The distance from each point to the point on the subject corresponding to each point on the image is detected, and a distance image that represents each pixel of the image by the distance to the subject instead of the luminance and color difference is obtained, and image recognition is performed using this distance image. It is possible. In this case, a corresponding point corresponding to each point on the photographed subject in each of the two photographed images is searched, and the difference in position between corresponding points on the two images corresponding to the same point on the subject is determined. The above distance is calculated from the parallax. In addition, when the distance image is obtained, for example, image recognition such as human detection or moving body detection by hail recognition, etc., and further, avoidance of danger when driving a car, automatic driving, nomination arrangement offense by hail recognition, etc. Detecting a specific person, detecting a suspicious person such as an illegal intruder, and the like. In the image recognition in the distance image, since a part of the three-dimensional shape of the object on the object to be photographed is known from the distance image, the subject on the image can easily and accurately identify people, dogs, cats, horses, vehicles, etc. Can be done.
In consideration of the accuracy of image recognition, it is preferable to use a high-resolution image sensor. However, when an arithmetic process such as distance detection described above is performed using an image having a large number of pixels, the amount of calculation is enormous. Therefore, when processing a moving image such as a video from an in-vehicle camera or a surveillance camera, it takes a long time to process one frame. Depending on the calculation speed of the integrated circuit that performs calculation processing, the calculation processing time of one frame becomes too long, and there is a possibility that subsequent processing may not be in time when a real-time response is required. In this case, it is conceivable to reduce the processing time by first thinning out the pixels of the image to lower the resolution (number of pixels) of the image.

  A fish-eye lens is known as a lens unit (lens) of a camera. The fisheye lens is a lens unit that adopts a projection method that is not the central projection method used in a normal wide-angle lens or a telephoto lens. For example, most fisheye lenses adopt an equidistant projection method. In addition to this, a fisheye lens employs an equisolid angle projection method, an orthographic projection method, a stereo projection method, and the like. On the telephoto side, there is little difference in the captured image between the central projection method and other projection methods, there is no telescopic fisheye lens, and the fisheye lens is an ultra-wide-angle lens. In general, the angle of view of a fisheye lens is often 180 degrees, but there are lenses with an angle of view of less than 180 degrees and lenses with an angle of view of greater than 180 degrees.

  There has been proposed a distance calculation device for obtaining a distance to a subject with a stereo camera using such a fisheye lens (see Patent Document 1). In a stereo camera using this fisheye lens, an image captured by the stereo camera is converted into a spherical image obtained by projecting the image onto a spherical surface, and the distance to each subject is obtained.

JP2013-109779A

By the way, calculating a distance image from each frame of a moving image shot with a stereo camera using a fisheye lens requires an enormous amount of computation as in the case of a normal lens or more, and processing per frame The time will be longer. Further, if an image in which an image is formed from a fish-eye lens unit on a planar image sensor surface is detected and output as it is, distortion at the center of the image is small and distortion at the peripheral edge of the image is large. In addition, when monitoring indoors with a surveillance camera, depending on the size, the stereo camera should be installed vertically so that the floor directly below the center ceiling of the indoor space is centered. It is efficient to do.
In addition, when shooting in front with an in-vehicle camera or when shooting with a monitoring camera from the corner of the monitored space, for example, a stereo camera so that the horizontal direction or a diagonally downward direction is the center. It is efficient to set up and shoot. In these cases as well, the periphery is greatly distorted from the center of the image taken with the fisheye lens.

In addition, with a surveillance camera that faces the vertical direction of the ceiling, even if there is a person underneath, no wrinkle appears and the head appears, making it difficult to identify a person by recognizing a wrinkle or the like. However, in order for a person to come directly under the surveillance camera, it is necessary to move from the peripheral part to the center part of the photographing range, and since the face of the person who moves at this time is captured, it is possible to recognize wrinkles.
Also, in a horizontal outdoor surveillance camera or in-vehicle camera, the sky appears above the image, and there is often no need to specify the subject by image recognition. When obtaining a distance image from a stereo camera image having such a fisheye lens, it is inefficient to process the entire image uniformly, and some processing device is required to reduce the processing time. Seem.

  The present invention has been made in view of the above circumstances, and when a distance image is obtained from a stereo camera image having a fisheye lens, the distance is changed after the resolution is changed so as to be different depending on the position of the image. An object of the present invention is to provide an object distance detection device capable of obtaining an image.

In order to solve the above problems, an object distance detection device of the present invention includes a stereo camera including a pair of fisheye cameras having a fisheye lens unit and an image sensor,
A distance image calculation unit that calculates a distance image from images output from the pair of imaging sensors;
A distance image recognition unit for performing image recognition including identification of a subject from the distance image;
With
The distance image calculation unit is a partitioning unit that partitions each of the images captured by the pair of fisheye cameras into a plurality of preset partitions.
A resolution conversion unit that converts the image into a resolution image set for each section;
A corresponding point search unit for obtaining corresponding points corresponding to the same point on the photographed subject in each of the two images captured substantially simultaneously with a pair of fisheye cameras;
A distance calculation unit that is searched by the corresponding point search unit and obtains a distance from the stereo camera to the corresponding point based on a difference in position between two corresponding points corresponding to the same point on the subject. To do.

According to such a configuration, the image captured by the fisheye camera can be divided into a plurality of sections, and the resolution can be changed according to the sections, so when considering the image captured by the fisheye camera, the central portion of the image is The distortion caused by the fisheye lens is small, and it is possible to detect, for example, a shark or a person even without modification. On the other hand, since the image has a large distortion at the peripheral edge of the image and the image is distorted and compressed, it is difficult to recognize the image unless the distortion is removed. Here, if the resolution of the image is first lowered uniformly in order to facilitate processing, it becomes difficult to perform highly accurate image recognition at the image peripheral portion having a large distortion.
Therefore, in the central section of the image, the resolution of the image is lowered to reduce the amount of processing, and the processing is performed at a high resolution without reducing the resolution in the peripheral portion of the image, thereby reducing the resolution of the image. Even if the processing speed is improved, the accuracy of image recognition can be maintained. In this case, if the resolution is lowered at a large area in the center of the image, the overall processing amount can be reduced, the processing speed can be increased, and image recognition by lowering the resolution can be performed. It can suppress that accuracy falls. The resolution here is, for example, the number of pixels per unit area of the image. When the resolution is lowered, the pixels of the image are thinned out by a well-known method. In this case, a process similar to a known image reduction process may be performed.

The sections of the images of the two fisheye cameras are basically sectioned in the same direction range. That is, when the section A, the section B, and the section C correspond to the respective images, corresponding points of the other image corresponding to the corresponding point A in the section A of one image at least other than the boundary portion of the section. A exists in the section A of the other image. Therefore, the search for corresponding points is basically performed in two corresponding sections of two images. The search for corresponding points is performed by, for example, extracting feature points (singular points) in image recognition in each section of two images of a pair of fisheye lenses, and the other image corresponding to the feature point in one image section. The feature points of the sections are determined, and corresponding points are determined by basic image recognition.
In the search for corresponding points, the epipolar geometry described in Patent Document 1 described above may be used for determination.

In the configuration of the present invention, the distance image calculation unit includes pixels arranged vertically and horizontally and is divided into pixel areas each including one or a plurality of pixels, and the correspondence from the stereo camera for each pixel area. Output a distance image with a color that changes according to the distance to the point,
In the section where the resolution of the distance image is different, it is preferable that the number of the pixels constituting the pixel region is different depending on the resolution.

  According to such a configuration, by increasing / decreasing the number of pixels constituting the pixel area of each section in accordance with the resolution of each section, images of each section having different resolutions can be displayed on the same distance image at substantially the same display magnification. Can be expressed as Each pixel region is colored with one color, and in the distance image, the color of each pixel region changes according to the distance. For example, a distance image is an image showing a numerical value of a distance as compared to an image showing a numerical value of temperature as in a thermography, and a color change that becomes an image like a thermography is, for example, a change in brightness (luminance) or a hue. Or a combination of both luminance and hue changes.

In the configuration of the present invention, it is preferable that the distance image calculation unit includes a distortion removal unit that removes distortion caused by a fisheye lens for each of the sections.
According to such a configuration, since it can be handled as a two-dimensional image from which distortion is removed for each section, it is possible to easily search for corresponding points. It should be noted that a method for removing distortion photographed with a fisheye lens is well known, and an IP core that enables distortion removal by setting parameters necessary for distortion removal of each fisheye lens is sold. It is possible to remove distortion.

In the above configuration of the present invention, the pair of fisheye cameras are arranged in a substantially vertical direction,
It is preferable that the resolution conversion unit changes the resolution so that the resolution is higher in the section at the periphery of the image than in the section at the center of the image.

  Considering the image distortion caused by the fisheye lens as described above, it is preferable to lower the resolution of the central portion of the image with little distortion and not to lower the resolution of the peripheral portion of the image with large distortion. Furthermore, with a fisheye camera attached to the ceiling or some columnar structure facing down, the center of the image is directly below, but in this part, the subject is photographed from directly above. Since it is difficult to recognize the image, it is difficult to recognize the image. On the other hand, the image is recognizable at a position slightly deviated from above, and the image can be recognized. Further, the angle of view of the fisheye camera is 180 degrees. There is a possibility that the front of the kite will be photographed nearby. Therefore, by reducing the resolution of the central portion of the image and not reducing the resolution of the peripheral portion of the image, it is possible to achieve both shortening of the processing time and maintaining the accuracy of image recognition. In this case, it is preferable that the area of one section of the central portion with low resolution is larger than the area of one section of the peripheral portion with high resolution.

In the configuration of the present invention, the pair of fisheye cameras are arranged in a substantially horizontal direction,
Preferably, the resolution conversion unit changes the resolution so that the resolution in the lower section of the image is higher than that in the upper section of the image.

  According to such a configuration, when the horizontal direction is taken with a fisheye camera, the sky is displayed at the top of the image when it is outdoors and the ceiling is displayed when it is indoors. When monitoring driving, criminal or suspicious person identification, etc., it is a less important part, so reducing the resolution to the bottom of the image will reduce the accuracy of image recognition and reduce processing time. Can do. In particular, when the height of the fisheye camera is high, for example, when the fisheye camera is a surveillance camera, it is higher than a human, or when the fisheye camera is an in-vehicle camera, it is at a higher position in the car. It is preferable to maintain the resolution because the importance of the lower part of the image becomes high. In this case, the resolution at the center of the image is preferably lower than that at the bottom of the image. In addition, when the installation height of the fisheye camera is lower than the height of the person (the position of the shark), the lower section of the image is more likely than the upper section of the image where the shark is likely to be captured. The resolution may be changed so that the resolution is lowered.

  According to the present invention, a distance image can be calculated by a stereo camera using a fisheye lens at high speed and with high accuracy without imposing a heavy load on the arithmetic processing device.

It is a block diagram which shows the object distance detection apparatus of embodiment of this invention. It is a block diagram which shows the image analysis part of an object distance detection apparatus similarly. 4 is a flowchart showing processing of an image analysis unit of the object distance detection device. It is a figure which shows the division of the image of an object distance detection apparatus similarly. It is a figure which shows the division of the image of an object distance detection apparatus similarly. (A), (b) is a figure for demonstrating the difference in the resolution for every division of a distance image.

Embodiments of the present invention will be described below.
The object distance detection device of the present embodiment uses a fisheye lens stereo camera as a camera mainly related to monitoring, such as a surveillance camera and an in-vehicle camera, but is not for outputting a stereoscopic image, A distance image in which each pixel is represented by the distance from the camera to the subject is generated, and a monitoring operation such as detection of a suspicious person is automatically enabled by image recognition from the distance image.

As shown in FIG. 1, the object distance detection device includes a pair of fisheye cameras 11 having a fisheye lens unit 21, a color filter 22, an image sensor 23, and the like, and images from the image sensors 23 of the pair of fisheye cameras 11. A pair of image input units 12 for inputting signals, a pair of image signal correction processing units 13 for performing image signal correction processing for removing distortion based on the fisheye lens of the input image, and image signal correction processing from image signals. A correction parameter calculation unit 15 for obtaining correction parameters, an image analysis unit (distance image calculation unit) 14 for obtaining a distance image from a corrected image signal from which distortion by the fisheye lens has been removed, and a distance image generated by the image analysis unit 14 And a suspicious person detecting unit 16 as a distance image recognizing unit that automatically performs monitoring work such as suspicious person detection.
The pair of fisheye cameras 11 constitutes a stereo camera, and outputs an image connected to the image sensor 23 via the color filter 22 by the fisheye lens unit 21 as an image signal. At this time, the image is output as a moving image.

  The fish-eye lens unit 21 is a fish-eye lens by adopting a projection method that is not a central projection method, and is a lens unit that employs an equidistant projection method in the present embodiment. The projection method of the fisheye lens unit 21 is not limited to the equidistant projection method, and any projection method other than the central projection method may be employed. For example, a lens unit of a projection method other than the above-described central projection method may be used. May be used as a fisheye lens. In the present embodiment, the angle of view of the fisheye lens unit 21 is 180 degrees, but may be, for example, an angle of view of about 160 degrees to 200 degrees.

  For example, the pair of fisheye cameras 11 are arranged adjacent to each other so that the optical axes of the fisheye lens units 21 are parallel to each other, and each of them has an angle of view of 180 degrees so that the other fisheye lens unit 21 can be photographed. It has become. This makes it possible to use epipolar geometry for searching for corresponding points described later.

  The output from the image sensor 23 of the fisheye camera 11 is input from the image input unit 12 in the object distance detection device, and the image signal correction processing unit 13 performs color synchronization processing, white balance processing, gamma processing, color matrix processing, luminance. Performs matrix processing, color difference / luminance processing, and the like. In the present embodiment, since a color image is not necessarily output on a monitor or the like, for example, a configuration without using the color filter 22 may be used, and processing relating to color may not be performed. Note that, as described later, feature points are extracted from two images by image recognition, and corresponding points are searched based on the feature points. Therefore, in image recognition, a color image is selected from a luminance image (grayscale image). If the recognition rate is higher, a color image may be generated from the image signal as described above. Also, parameters based on the image signal required by the image signal correction processing unit 13 are calculated from the image signal by the correction parameter calculation unit 15.

In the image analysis unit 14, the image corrected as described above is input, and an image conversion unit 31 as a distortion removal unit performs image conversion to remove distortion by the fisheye lens as necessary. For example, an equidistant projection image is converted into a central projection image. Note that distortion removal can be performed by a known method. For example, a known integrated circuit for image conversion for distortion removal is used. Next, partitioning is performed to divide the converted image from which distortion has been removed by the partitioning unit and the image selection unit 32 serving as a resolution conversion unit into set partitions.
This partitioning is not limited to a straight line partitioning, but may be a curve partitioning. For example, when an equidistance projection image is used without image conversion, segmentation by a curve is preferable.

  Here, FIG. 4 shows the sections K11 to K33 in the case where the pair of fisheye cameras 11 are arranged with the optical axis directed in the vertical direction (downward) on the ceiling or the like when the converted image is rectangular. As shown, the area of each partition is large in the central sections K11, K12, and K21, and the peripheral sections K13, K23, K31, K32, and K33 are narrow in area. Further, the resolutions of these sections K11 to K33 are low in the central sections K11, K12, and K21, while the resolutions in the peripheral sections K13, K23, K31, K32, and K33 are high. For example, by setting each section to the same number of pixels, 400 × 200, the resolution of the central sections K11, K12, and K21 is lowered, whereas the peripheral sections K13, K23, K31, and K32 The resolution will be higher.

  In addition, when distortion is corrected by image conversion before partitioning, the distorted portion is stretched in a contracted state at the peripheral edge of the image. However, the resolution is higher than that at the peripheral portion, and even if the pixel is thinned out so that the resolution is reduced at the center portion of the image, the influence is less than when the resolution is reduced at the peripheral portion. Therefore, the resolution is lowered by performing the process of thinning out pixels in the central sections K11, K12, and K21 of the image, and the pixels are not thinned out in the peripheral sections K13, K23, K31, K32, and K33 of the image, The resolution is not lowered. It should be noted that the pixels are thinned out in all the sections K11 to K33, and the degree may be increased in the sections K11, K12, and K21 in the central portion than the sections K13, K23, K31, K32, and K33 in the peripheral portion of the image. In order to shorten the processing time, the area of the central sections K11, K12, and K21 where the resolution is lowered is made wider than the peripheral sections K13, K23, K31, K32, and K33 where the resolution is not lowered. . In addition, when performing correction for removing distortion after dividing an image into a plurality of sections, the image is distorted so that the image shrinks at the peripheral portion. Therefore, in order to correct such distortion, Since the area is enlarged, the area of the peripheral section is made smaller than that of the central part when partitioning. In this case, by removing the distortion after partitioning, if the resolution is lowered in the peripheral view having a large distortion, the image is greatly deteriorated, and there is a possibility that the calculation of the distance and the image recognition are adversely affected. Therefore, when removing distortion after partitioning, it is preferable to thin out more pixels at the central portion than at the peripheral portion.

  Further, in FIG. 5, when the converted image is rectangular, for example, a pair of fisheye cameras 11 are arranged at a position higher than a human back with the optical axis directed horizontally or obliquely downward with respect to the horizontal direction. The sections K11 to K42 are shown in the figure, and the sections K11 to K33 below the central portion are the same sections as the fisheye camera 11 facing in the vertical direction, and each section K11 to K33 is shown. The resolution and area are set in the same manner.

  On the other hand, in the upper sections K41 and K42 above the center portion of the image, for example, the sky is outdoors and the ceiling is reflected indoors. Therefore, the importance is low, and the resolution is lower and the area is wider than the section K11 in the center portion. Yes. In the upper sections K41 and K42, the upper center section K41 has the lowest resolution and the largest area. On the other hand, the upper left and right sections K42 have a higher resolution than the section K41 and a smaller area. It has become.

  4 and 5 show examples of division, and based on the distortion of the image by the fisheye lens unit 21 of the fisheye camera 11, the resolution of the central division is lowered and the area is widened. In comparison, the resolution and the area can be adjusted according to the importance based on the arrangement of the fisheye camera 11 on the basis of increasing the resolution of the peripheral portion and reducing the area. That is, the fish-eye camera 11 having an angle of view of about 180 degrees or more has a wide shooting range. For example, a position where a person cannot exist may enter the shooting range. It is preferable to improve the processing speed by lowering the resolution of such a portion.

The image selection unit 32 selects the same sections K11 to K33 for each of the pair of images. When a pair of sections is selected, the corresponding point selection unit 33 as the next corresponding point search unit extracts feature points of each image by image recognition and associates feature points of each image as described above. Do. In this case, epipolar geometry can be used, and corresponding points to be paired in each image are sequentially determined and selected. When selection of corresponding points for a pair of sections is completed, corresponding points are extracted in the next pair of sections. When the extraction of the corresponding points is completed in all pairs of sections, the distance calculation unit 34 next determines the difference between the positions of the corresponding points on the image and the distance between the pair of fisheye cameras 11. The distance from the fisheye camera 11 to the target point corresponding to the pair of corresponding points is obtained. Calculation of distance based on the so-called triangulation method, but when using parallax, the baseline distance (distance between a pair of cameras) is much smaller than the distance to the target for general triangulation, both points When the difference in distance from each fisheye camera 11 to the target point does not matter, the distance to the target point is obtained by dividing the baseline distance by the parallax (unit radians).
Also, the three-dimensional coordinates of the target point on the three-dimensional coordinate in the real space are calculated from the coordinate position (projection position) on the two-dimensional coordinate on each image of the corresponding point, the distance between the cameras, and the focal length of the camera. The distance from the camera to the target point can be calculated from the target point and the three-dimensional coordinate position in the real space of the camera.

  The flowchart of FIG. 3 shows the processing in the image analysis unit 14 described above. After the image is captured by the pair of fisheye cameras 11 and the image is corrected, the image is input to the image analysis unit 14 for each pair of frames. The process which calculates the distance image which shows the distance from the fisheye camera 11 for every corresponding point used as each pixel to the object point is shown. Note that the flowchart of FIG. 3 shows a case where a process for removing distortion is performed after partitioning. As shown in the flowchart, when a corrected image for each frame is input from the pair of fisheye cameras 11, partitioning is performed (step S1). In partitioning, an image is divided into a plurality of partitions and a process of reducing the resolution of each partition (for example, reduction processing) is performed, but the resolution is changed depending on the partition. Moreover, the area of one division is adjusted with a division. Basically, the resolution is low and the area is increased in the central section of the image, while the resolution of the section is higher than that in the central area and the area of the section is smaller than that in the central section.

After partitioning, distortion is removed in each partition (step S2). Note that an already established method is used as a method for removing distortion of an image using a fisheye lens.
Next, feature points (singular points) as corresponding points in each section are extracted. As feature points, for example, singular points such as edges are detected by well-known edge detection. A corresponding point is a point on the image corresponding to a target point in the real space, a point corresponding to the same target point in two images is a corresponding point, and each point in the real space is shown in both images. If so, there will be corresponding points, and it is preferable to search for many corresponding points using the above-described epipolar geometry method.
Next, the distance from the fisheye camera 11 to the target point in the real space corresponding to the corresponding point is calculated based on the difference in the position of the corresponding point on the image and the distance between the fisheye cameras 11 ( Step S4). Next, a distance image is generated and output with the distance of each corresponding point determined as the value of each pixel (step S5).

  This distance image is used for monitoring. A person is detected by recognizing a distance image, or a living thing or an article (vehicle or the like) other than a person is detected. In addition, with regard to people, the person is identified from the registered person's three-dimensional shape or photograph, or the person's three-dimensional shape detected in the distance image, for example, height, physique, clothing shape, etc. A child and an adult may be distinguished, an age may be distinguished, and a man and a woman may be distinguished. Moreover, it is good also as what detects the shape for every vehicle model of vehicles, such as a motor vehicle and a motorcycle, and detects a vehicle type.

  According to the object distance detection device of the present embodiment, distance detection is performed by a stereo camera including a plurality of fisheye cameras 11 having an angle of view of, for example, about 180 degrees. For example, up to a target point as each object in real space. The distance can be obtained and the distance image can be output. However, since the angle of view is wide, the amount of information in one frame image is large, and the amount of calculation processing including image recognition necessary for distance detection is large. Depending on the capability of the processing circuit, a long time is required for processing, and the processing time for one frame is long, making it difficult to process a moving image. In addition, there are many target points that can be calculated for distance in the image of the fisheye camera 11. And in the peripheral part of the image where the amount of information (target point) is larger than the central part of the image, the image is distorted from the central part, and the resolution is not necessarily high. On the other hand, the image of the fish-eye camera 11 includes many less important parts in monitoring work such as sky, ceiling, floor and ground. Therefore, by dividing the part with low distortion and high resolution and the part with low importance and calculating the distance by reducing the resolution of this part, the processing amount can be reduced without greatly reducing the accuracy of distance calculation. This can reduce the processing time. Thereby, the image recognition in the monitoring apparatus using the distance image calculated | required with the stereo camera which has the several fisheye camera 11 can be performed smoothly.

  Here, each pixel region D of the distance image output from the object distance detection device will be described. In the distance image, each point on the image is represented by a distance from the stereo camera. In the present embodiment, the distance image is represented by a change in color shade according to the distance value. The color change may be, for example, a monochrome gradation or a gradation of another color. When mechanically recognizing an image using a distance image, each point on the image may be represented by a numerical value indicating the distance. FIG. 6A shows a part of the section K11 as a part of the distance image corresponding to the sectioned image shown in FIG. 4, and FIG. 6B shows a part of the section K33. The sections K11 to K33 are arranged in the same manner as in FIG. 4 on one distance image. And the resolution changes with each division K11-K33. However, as shown in FIG. 6, the size of the pixel P (the portion divided by both the double line and the dotted line) which is the minimum unit on the distance image is the same. The pixel P is, for example, a monitor pixel that displays a distance image.

  In the distance image, the image is divided into pixel areas D (parts separated by double lines), and each pixel area D is composed of one pixel P or a plurality of pixels P. For example, black and white shading corresponding to the distance from the stereo camera to the corresponding point (photographing target) is attached to each pixel area D, and the distance image is a color (color shading) corresponding to the distance of each pixel area D. It is expressed. Here, the resolution of the section K11 is lower than that of the section K33, while the number of pixels in each pixel area D of the section K11 is 4, whereas the number of pixels of the pixel area D of the section K33 is 2. In addition, the pixel area D of the section K11 having a lower resolution has a larger number of pixels P and a larger area than the pixel area D of the section K33 having a higher resolution. In one distance image, it is not necessary to change the size of the minimum unit pixel depending on the section, and it is possible to cope with the difference in resolution by changing the number of pixels P in the pixel region D. Can be displayed on a single monitor at substantially the same display magnification.

11 Fisheye Camera 14 Image Analysis Unit (Distance Image Calculation Unit)
16 Suspicious person detection part (distance image recognition part)
21 Fisheye lens unit 23 Image sensor 31 Image conversion unit (distortion removal unit)
32 Image selection unit (partitioning unit: resolution conversion unit)
33 Corresponding point selection part (corresponding point search part)
34 Distance calculator

Claims (5)

A stereo camera comprising a pair of fisheye cameras having a fisheye lens unit and an imaging sensor;
A distance image calculation unit that calculates a distance image from images output from the pair of imaging sensors;
A distance image recognition unit for performing image recognition including identification of a subject from the distance image;
With
The distance image calculation unit is a partitioning unit that partitions each of the images captured by the pair of fisheye cameras into a plurality of preset partitions.
A resolution conversion unit that converts the image into a resolution image set for each section;
A corresponding point search unit for obtaining corresponding points corresponding to the same point on the photographed subject in each of the two images captured substantially simultaneously with a pair of fisheye cameras;
A distance calculation unit that is searched by the corresponding point search unit and obtains a distance from the stereo camera to the corresponding point based on a difference in position between two corresponding points corresponding to the same point on the subject. An object distance detection device. The distance image calculation unit is composed of pixels arranged vertically and horizontally, and is divided into pixel regions each composed of one or a plurality of pixels, and each pixel region corresponds to a distance from the stereo camera to the corresponding point. Output a distance image with a changing color,
In the section where the resolution of the distance image is different, the number of the pixels constituting the pixel area is different according to the resolution in the section where the resolution of the distance image is different. The object distance detection apparatus according to claim 1.   The object distance detection apparatus according to claim 1, wherein the distance image calculation unit includes a distortion removal unit that removes distortion caused by a fisheye lens for each of the sections. The pair of fisheye cameras are arranged in a substantially vertical direction,
4. The resolution conversion unit according to claim 1, wherein the resolution conversion unit changes the resolution so that the resolution in the peripheral section of the image is higher than that in the central section of the image. The object distance detection device according to item. The pair of fisheye cameras are arranged in a substantially horizontal direction,
4. The resolution conversion unit according to claim 1, wherein the resolution conversion unit changes the resolution so that the resolution in the lower section of the image is higher than that in the upper section of the image. 5. The object distance detection apparatus described.

Images