JP2006031171A - Pseudo three-dimensional data generation method, apparatus, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and apparatus for generating pseudo three-dimensional data from an image to be photographed. <P>SOLUTION: The method for generating pseudo three-dimensional data from an image to be photographed comprises: an image acquiring process for acquiring a first image photographed in a first illumination status and a second image photographed in a second illumination status which is brighter than the first illumination status; and a depth value calculation process for calculating a depth ratio by dividing the pixel value of the first image by the pixel value of the corresponding pixel of the second image to calculate a depth ratio. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method and apparatus for generating pseudo three-dimensional data from an image to be photographed, a program for causing a computer to execute the method, and a computer-readable recording medium storing the program.

  Conventionally, various methods are used to input three-dimensional data of an object. Three-dimensional data input methods can be broadly classified into a passive method and an active method. For example, a stereo method is a representative example of the passive method, and a light cutting method is a typical example of the active method.

  The stereo method is a method in which two photographing devices are arranged at a certain interval, and the position of the photographing object is obtained by the principle of triangulation from the difference in projection position in both images with respect to the same photographing object, that is, the difference between corresponding points. . The light cutting method is a method in which the slit light projection device and the photographing device are arranged at a certain interval, and the position of the photographing target is obtained by the principle of triangulation from the position where the slit light projected on the photographing target is projected in the image of the photographing device. is there.

  On the other hand, as a technique for obtaining a depth value by irradiating a subject with illumination light, illumination whose brightness changes at least depending on the position on the subject (for example, illumination whose brightness increases from left to right) is used. There is a method of irradiating, detecting reflected light for each light intensity, and obtaining a depth value using the principle of triangulation. However, in this technique, since the principle of triangulation is used for calculating the depth value, it is necessary to provide a certain distance between the camera and the illumination device, and it is difficult to reduce the size of the device (see Patent Document 1). .

  In addition, as a technique similar to the above, so-called Time is calculated by irradiating a subject with intensity-modulated illumination light whose light intensity changes with time, and calculating the distance to the subject from the intensity of the reflected light based on the light propagation time. There is a method for obtaining a depth value by using the principle of of flight. However, this technology requires a special illumination device for projecting intensity-modulated illumination light and a camera equipped with a high-speed shutter for capturing changes in intensity-modulated illumination light, which is suitable for reducing the cost of the device. (See Patent Document 2).

Furthermore, as a technique for obtaining a pseudo depth value of a subject, a gray value of a transmission image used for creating a divided image with respect to a divided image for each discrete depth value of the subject obtained using an X-ray CT apparatus or the like. There is a method of pseudo-calculating a depth value so as to smoothly connect to adjacent divided images. However, this technique is based on the premise that a transmission image divided in advance for each discrete depth value of an object is obtained in advance by an X-ray CT apparatus or the like, and is difficult to apply to applications other than special purposes such as medical treatment. (See Patent Document 3).
JP 2000-329524 A JP 2000-121339 A JP-A-2-0779179

  In recent years, with the widespread use of digital cameras, it has become possible to capture and process two-dimensional images to be photographed. An image that can be acquired by a digital camera or the like is a two-dimensional image having no depth, but if a three-dimensional image can be input, the range of utilization of the image is widened and the field of use is increased. However, since both the passive method and the active method use the principle of triangulation, it is necessary to provide a distance between apparatuses such as an imaging apparatus and a slit light projection apparatus, and it is difficult to reduce the size. Also, the methods described in Patent Documents 1, 2, and 3 are difficult to apply to applications that require downsizing and cost reduction.

  In order to solve these problems, the inventor of the present invention compares the difference between pixel values of a plurality of images photographed in different illumination states or the result of division with a predetermined threshold value, thereby obtaining the depth of each part to be photographed. Invented a method and apparatus for separating values from perspective. However, when performing perspective separation based on the difference, there is a problem that perspective separation cannot be performed correctly if there is a difference in the reflectance of each part to be imaged. Further, when performing perspective separation by division, there is a problem that perspective separation cannot be performed correctly if, for example, the amount of light hitting the object to be imaged is not constant due to shadows.

  The present invention has been invented in view of the above problems, and an object of the present invention is to provide a method and apparatus for generating pseudo three-dimensional data from an image to be photographed. Furthermore, it aims at providing the program for making a computer perform these methods, and the storage medium which memorize | stored this program.

  According to one aspect of the present invention, a method for generating pseudo three-dimensional data from an image to be photographed includes a first image photographed in a first illumination state and a second illumination brighter than the first illumination state. An image acquisition process for acquiring a second image taken in a state, and a depth value for calculating a depth value by dividing a pixel value of the first image by a pixel value of a corresponding pixel of the second image And a value calculation process.

  By dividing the pixel value of the first image by the second image captured in a brighter illumination state, a depth ratio that continuously changes in the range of 0 to 1 can be obtained.

  The pseudo three-dimensional data generation method according to another aspect of the present invention may further include a resolution reduction process for reducing the resolution of the image before calculating the depth value. Further, it may include a contrast improvement process for adjusting the distribution range of the depth value after the depth value is calculated.

  According to one embodiment of the present invention, a relief-like depth having a smooth stereoscopic effect can be created from an image to be captured.

  According to another aspect of the present invention, the amount of calculation can be reduced because the resolution of the image is lowered before the depth value is calculated. In addition, the S / N ratio in division can be improved.

  According to still another aspect of the present invention, since the distribution range of the depth value is adjusted, pseudo three-dimensional data that gives a stronger stereoscopic effect can be created.

  According to still another aspect of the present invention, since the calculated depth value is smoothed, it is possible to create pseudo three-dimensional data that smoothes the depth and gives a natural stereoscopic effect.

  According to still another aspect of the present invention, since the distance between the photographing unit and the illuminating unit may be adjacent to each other, an apparatus that can be miniaturized can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram for explaining a pseudo three-dimensional data generation method according to an embodiment of the present invention. In FIG. 1, three images are shown. (A) An unilluminated image is an image photographed without illuminating a subject such as a flash. (B) An image with illumination is an image captured by applying illumination such as a flash to a subject to be imaged. (C) Depth map is the ratio of the depth value of the object to be imaged calculated by applying the pseudo three-dimensional data generation method according to one aspect of the present invention to (A) an image without illumination and (B) an image with illumination (Hereinafter also referred to as “depth ratio”) is shown as an image by associating with luminance. The pixel value of the corresponding pixel in each image is
1− (non-illuminated image ÷ illuminated image) = depth ratio

  In the following description, it is assumed that the pixel value of the image is proportional to the luminance of the shooting target. If the luminance of the shooting target is high, the pixel value of the image is large, and if the luminance of the shooting target is low, the pixel value of the image Shall be small. This assumption is for the sake of brevity and may be different from this assumption and should not be taken as limiting the technical scope of the present invention.

In an image to be photographed, in a portion where illumination such as a flash does not reach at all (that is, a portion farthest from photographing means / illuminating means (see FIG. 14)), (A) a pixel value of an image without illumination and (B) illumination Since it is equal to the pixel value of the image, the relationship of (non-illuminated image ÷ illuminated image) = 1 holds. Therefore,
Depth ratio = 1- (non-illuminated image / illuminated image) = 1-1
= 0
It becomes. On the other hand, for a portion where illumination such as a flash is well reached (that is, a portion closest to the photographing means / illuminating means), the pixel value of the image with illumination (B) is larger than the pixel value of the image without illumination (A). Therefore, the relationship of (non-illuminated image ÷ illuminated image) ≈0 holds. Therefore,
Depth ratio = 1- (non-illuminated image ÷ illuminated image) ≈1-0
≒ 1
It becomes. Therefore, the maximum value of the depth ratio is 1 and the minimum value is 0, which respectively correspond to the closest part and the farthest part from the imaging means to be imaged.

Depth ratio is
It may be defined as depth ratio = non-illuminated image ÷ illuminated image. In this case as well, the maximum value of the ratio of depth values is 1 and the minimum value is 0, but these correspond to the portion farthest from the photographing means to be photographed and the portion closest thereto.

  FIG. 2 is a graph showing the relationship between the pixel value of the illuminated image and the ratio of the depth value in this case. It can be seen that the depth value ratio approaches 1 when the pixel value of the illuminated image is small, and the depth value ratio approaches 0 when the pixel value of the illuminated image is large.

In the following description, the depth ratio is
Depth ratio = 1-(Unilluminated image / Illuminated image)
However, it should be noted that the effect of the present invention can be obtained regardless of which definition is adopted. In this case, the portion of the subject to be photographed whose depth value ratio is close to 1 is close to the photographing means / illuminating means, and the portion of the subject to be photographed whose depth value ratio is close to 0 is far from the photographing means / illuminating means.

  The photographing means / illuminating means will be described later (see FIG. 14).

  FIG. 3 is a flowchart for explaining a pseudo three-dimensional data generation method according to an embodiment of the present invention.

  In step S110, an image without illumination (A) and an image with illumination (B) are input from the photographing means. Both images shall have (x × y) pixels. The unilluminated image (A) does not necessarily have to be an image captured in the absence of illumination by the illumination means, and may be an image captured in an illumination state that is not brighter than the illuminated image (B).

  In step S120, a parameter i specifying each pixel and a parameter S ′ indicating the number of pixels of the input image are initialized.

In step S130, the depth ratio Ci of the pixel designated by the parameter i is set.
Depth ratio Ci = 1- (Ai / Bi)
Calculated by Here, Ai and Bi indicate the pixel values of the corresponding pixels specified by the parameter i of the unilluminated image (A) and the illuminated image (B), respectively. In other embodiments, the depth value ratio Ci is set to
Depth ratio Ci = (Ai / Bi)
As described above, it may be calculated by. Here, when expressed in the binary system, Ai / Bi and 1- (Ai / Bi) have a complement relationship, and the other can be calculated from one by the inversion operation for each bit.

  In step S140, the parameter i is incremented, and in step S150, it is determined whether or not the depth value ratio Ci has been calculated for all pixels (i <S '). If there is a pixel whose depth ratio Ci has not yet been calculated (YES in step S150), the process returns to step S130. When the depth ratio Ci has been calculated for all pixels (NO in step S150), the process ends.

  As a result, the parameter Ci contains the ratio of the depth value of the corresponding pixel i. The result of displaying the parameter Ci as an “image” is the (C) depth value shown in FIG.

  In order to reduce the calculation load, the resolution of the image may be reduced before calculating the depth ratio. By reducing the resolution, the amount of calculation can be reduced. In addition, even when a shift occurs between pixels in an image shot under different lighting conditions due to movement of the shooting target, etc., the depth ratio resolution can be made coarser than in the image. This has the effect of blurring and making the shift appear less.

  Further, if the resolution is lowered by taking the average value of the pixels, the effect of improving the S / N ratio can be obtained as in the case where the low-pass filter is applied.

  Note that the process of reducing the resolution and the process of obtaining the average value of the pixels may be executed as necessary.

  FIG. 4 is a diagram for explaining the improvement of the contrast of the depth map according to an embodiment of the present invention. The (C) depth map shown in FIG. 4 is the (C) depth map shown in FIG. From the depth map of FIG. 4C, it can be seen that the image is dark as a whole and the contrast as an “image” is poor. This (C) depth map originally represents the ratio of the depth of the object to be imaged, so that “contrast” as an “image” means that the depth ratio of the object to be imaged is wide within the maximum range (0 to 1). Means not distributed. By improving the contrast of the “image” having a low contrast, the depth ratio of the photographing target can be widely distributed in the maximum range (0 to 1). As a result, FIG. 4D shows a depth map with improved contrast. This will be further described using a depth ratio histogram.

FIG. 5A shows a histogram of (C) depth map and (D) depth map with improved contrast in FIG. From FIG. 5A, it can be seen that the depth ratio of the depth map of FIG. 4C is concentrated in a narrow range. The ratio is represented by x, and the histogram is represented by f (x). This histogram f (x) is distributed between t _min and t _max . What is necessary is just to expand f (x) in the horizontal direction so that it is widely distributed between 0 and 1. From FIG. 5A, it can be seen that in FIG. 4D, in the histogram of the depth map with improved contrast, the distribution range is expanded by converting f (x) to g (x). Here, t ( _min ) of f (x) is converted to correspond to 0 of g (x), and _{tmax of} f (x) is converted to correspond to 1 of g (x).

The above conversion converts f (x)
g (x) = max × ( f (x) -t min) ÷ (t max -t min) (1)
Can be achieved by conversion. Here, max is the maximum value of the depth ratio, and is 1.0 when 100%.

  FIG. 6 is a flowchart for explaining a pseudo three-dimensional data generation method with contrast improvement according to an embodiment of the present invention described with reference to FIG.

  In step S210, an image without illumination (A) and an image with illumination (B) are input from the photographing means. Both images have (x × y) pixels. As described with reference to FIG. 3, the non-illuminated image (A) does not necessarily have to be an image photographed without illumination by the illumination means, and is not brighter than the illuminated image (B). Any image captured in a state may be used.

In step S220, a parameter t _max indicating the ratio assigned to the maximum value after conversion and a parameter t _min indicating the ratio assigned to the minimum value are initialized. Also, a parameter i for designating each pixel and a parameter S ′ indicating the number of pixels of the input image are initialized.

In step S230, the ratio Ci of the depth value of the pixel designated by the parameter i is set.
Depth ratio Ci = 1- (Ai / Bi)
Calculated by Here, Ai and Bi indicate the corresponding pixels specified by the parameter i of the unilluminated image (A) and the illuminated image (B), respectively. In other embodiments, the depth ratio Ci is set to
Depth ratio Ci = (Ai / Bi)
As described above, it may be calculated by. Here, when expressed in the binary system, Ai / Bi and 1- (Ai / Bi) have a complement relationship, and the other can be calculated from one by the inversion operation for each bit.

  In step S240, the parameter i is incremented.

In step S250, it is determined whether the depth ratio Ci is greater than the current _tmax . If Ci is larger than the current t _max (YES in step S250), t _max is updated with Ci in step S260, and then the process proceeds to next step S270. If Ci is not larger than the current t _max (NO in step S250), the process proceeds to the next step S270 without updating t _max .

In step S270, it is determined whether the depth value ratio Ci is smaller than the current t _min . If Ci is smaller than the current t _min (YES in step S270), t _min is updated with Ci in step S280, and then the process proceeds to next step S290. If Ci is not smaller than the current t _min (NO in step S270), the process proceeds to the next step S290 without updating t _min .

  In step S290, it is determined whether or not the depth value ratio Ci has been calculated for all pixels. If there is a pixel whose depth value ratio Ci has not yet been calculated (YES in step S290), the process returns to step S230. When the depth value ratio Ci is calculated for all the pixels (NO in step S290), the process proceeds to the next step S300.

Through the above steps, the depth value ratio Ci is calculated for all the pixels, and the parameters t _max and t _min contain the maximum value and the minimum value of the ratio Ci. A depth map with improved contrast can be obtained by applying the conversion formula (1) to Ci.

  In step S300, the parameter i specifying the pixel is initialized again.

In step S310, the conversion formula (1) is applied to Ci, and the result is Ci. By this operation, when max = 1, the ratio Ci of each pixel i is distributed between 0 and 1. For example, assume that t _max = 0.6 and t _min = 0.3. Pixel i with ratio Ci = 0.3, 0.5, 0.6 is
Ci = 1 × (Ci-0.3) ÷ (0.6-0.3)
Thus, new Ci values are converted to 0, 0.67, and 1, respectively.

  In step S320, the parameter i is incremented, and in step S330, it is determined whether or not the conversion of the depth value ratio Ci has been completed for all the pixels. If there is a pixel whose depth value ratio Ci has not yet been converted (YES in step S330), the process returns to step S310. When the depth value ratio Ci is converted for all the pixels (NO in step S330), the process ends.

FIG. 5A shows an example in which the histogram f (x) is enlarged so that the maximum value t _max and the minimum value t _min of f (x) become 1 and 0, respectively. There are various other ways of expansion. For example, as shown in FIG. 5B, it is possible to enlarge until t _max and t _min are 1 or more and 0 or less, respectively, and round 1 or more to 1 and 0 or less to 0. . Thereby, the center part of the depth ratio can be emphasized.

  This depth map contrast improving method is effective when the ratio of depth values is a group within a narrow range, as shown by f (x) in FIG. However, when the ratio of the depth map is separated into a plurality of ranges, there is a problem that a sufficient contrast improvement effect cannot be obtained.

  FIG. 7 is a diagram for explaining a case where the histogram of the depth map is separated into a plurality of ranges. In the depth map shown in FIG. 7A, it can be seen that only a portion 71 surrounded by a broken-line ellipse is white, and only this portion has a depth ratio close to 1 (representing that it is close to the photographing means / illuminating means). The depth ratio histogram of this image has two peaks 73 and 75 as shown in FIG.

  FIG. 8 is a diagram showing a histogram when the contrast of the depth map of FIG. 7 is improved by the contrast improving method described with reference to FIG. FIG. 8A is the same histogram as shown in FIG. When the contrast improving method described with reference to FIG. 6 is applied to the depth map, the resulting depth map histogram is as shown in FIG. In FIG. 8B, the two peaks 73 and 75 shown in FIG. 8A are converted into the two peaks 83 and 85 shown in FIG. 8B, respectively. As can be seen, there is a depth ratio between the two peaks 83 and 85 that is not yet assigned to any pixel, and the contrast is not sufficiently improved. In order to solve this problem, a characteristic curve is used.

  FIG. 9 is a diagram for explaining a contrast improvement method using a characteristic curve. FIG. 9A shows a histogram f (x) of a depth map to be subjected to the contrast improvement method. In FIG. 9B below, a characteristic curve h (x) of the depth ratio, which is a curve obtained by integrating the histogram f (x), is drawn. In FIG. 9C on the right side of FIG. 9B, a 45 ° line is drawn. FIG. 9D above shows a histogram g (y) of the depth map in which the contrast is improved by the characteristic curve.

The relationship between each figure will be described. In the depth map before conversion, there are f (x ₀ ) pixels having a depth ratio x ₀ (corresponding to the point P ₀ in FIG. 9A). The converted ratio y ₀ is determined by the point P ₁ on the characteristic curve corresponding to the point P ₀ . In FIG. 9B, the coordinates of the point P ₁ are (x ₀ , y ₀ ) (y ₀ = h (x ₀ )). In FIG. 9C, a point P ₂ on the 45 ° line corresponding to the point P ₁ is folded upward while maintaining the converted ratio y ₀ . The corresponding point P ₃ in FIG. 9D is determined by the number of pixels f (x ₀ ) before conversion and the ratio y ₀ after conversion. Similarly, other points before conversion correspond to the points on FIG. 9D, and the histogram g (y) of the depth map after conversion is obtained.

  Here, in FIG. 9C, the ratio x before conversion may be converted to the ratio y after conversion using a continuous function of monotonous increase instead of the 45 ° straight line passing through the origin.

  FIG. 10 is a flowchart for explaining a pseudo three-dimensional data generation method including a contrast improvement method according to an embodiment of the present invention described with reference to FIG.

  In step S410, an unilluminated image (A) and an illuminated image (B) are input from the photographing means. Both images have (x × y) pixels. As described with reference to FIG. 3, the non-illuminated image (A) does not necessarily have to be an image photographed without illumination by the illumination means, and is not brighter than the illuminated image (B). Any image captured in a state may be used.

In step S420, a parameter t _max indicating the ratio assigned to the maximum value after conversion and a parameter t _min indicating the ratio assigned to the minimum value are initialized. Also, a parameter i for designating each pixel and a parameter S ′ indicating the number of pixels of the input image are initialized. Also, the number p _j of pixels having the ratio j is initialized.

In step S430, the ratio Ci of the depth value of the pixel designated by the parameter i is set.
Depth value ratio Ci = 1- (Ai / Bi)
Calculated by Here, Ai and Bi indicate the corresponding pixels specified by the parameter i of the unilluminated image (A) and the illuminated image (B), respectively. In other embodiments, the depth value ratio Ci is set to
Depth value ratio Ci = (Ai / Bi)
As described above, it may be calculated by. Here, when expressed in binary, Ai / Bi and 1- (Ai / Bi) have a complement relationship, and the other can be calculated from one by the inversion operation for each bit.

In step S440, the number p _Ci of pixels having the ratio Ci is incremented, and in step S450, the parameter i specifying the pixel is incremented.

  In step S460, it is determined whether or not the depth ratio Ci has been calculated for all the pixels. If there is a pixel whose depth value ratio Ci has not yet been calculated (YES in step S460), the process returns to step S430. When the depth ratio Ci is calculated for all pixels (NO in step S460), the process proceeds to the next step S470.

Through the above steps, the depth ratio is calculated for all the pixels, and the number of pixels having the ratio Ci (histogram) is calculated by p _Ci .

  Next, in step S470, the depth ratio j (0 ≦ j ≦ 1) is initialized.

In step S480, p _j (= p _Ci ) is added from 0 to the current value of _j , and the result of division by the total number of pixels S ′ is defined as p _j . This operation corresponds to the operation of integrating f (x) in FIGS. 11A to 11B to obtain the characteristic curve h (x). The reason for dividing by the total number of pixels S ′ is to normalize the maximum value to be 1. p _j is the depth ratio after conversion.

  In step S490, it is determined whether step S480 has been completed for all ranges of parameter j. If not (YES in step S490), j is increased by Δj in step S500, and then the process returns to step S480, and steps S480 and S490 are repeated for the new j. If step S480 has been completed for all ranges of parameter j (NO in step S490), the process proceeds to next step S510.

  Through the above steps S470 to S500, all p's are added for depth values 0 to j, and the number of pixels up to the depth value j is obtained. The depth ratio after conversion is obtained by dividing the result by the total number of pixels S ′.

  In step S510, a parameter i for specifying a pixel is initialized.

In step S520, the ratio Ci is replaced with the normalized pixel count p _Ci after conversion. By this operation, the converted histogram shown in FIG. 9D is obtained.

  In step S530, the parameter i is incremented. In step S540, it is determined whether or not the replacement of step S520 has been completed for all pixels (i <S ′). If not completed (YES in step S540), the process returns to step S520. If completed (NO in step S540), the process ends.

  FIG. 11 is a diagram for explaining a result of improving the contrast of the depth map shown in FIG. 7 using the contrast improvement method using the characteristic curve described with reference to FIG. FIG. 11A is a histogram of the depth map before improvement. FIG. 11B is a histogram of the depth map after improvement. By introducing the contrast improvement method using the characteristic curve, it can be seen that the improved depth map improves the contrast so that almost the entire range of the ratio is used.

  FIG. 12 is a diagram for explaining smoothing of a depth map according to an embodiment of the present invention. FIG. 12A is a depth map before smoothing. In this depth map, it can be seen that the stuffed eyeball portion to be photographed becomes black, and the depth ratio changes abruptly as compared with the peripheral portion. Therefore, this problem can be solved by applying a low-pass filter to the depth map for smoothing. FIG. 12B is a depth map smoothed by applying low-pass fill. It can be seen that the abrupt change in the depth ratio that occurred in the eyeball portion of the stuffed animal that is the subject of photography has been eliminated.

  FIG. 13 is a diagram illustrating an example of the coefficients of the low-pass filter used in the smoothing process. Here, as the low-pass filter, the pixel values of all the pixels within the range of the filter size (7 × 7) centering on the pixel to be processed are multiplied by the reciprocal (1/49) of the filter size as a coefficient, and added. A simple moving average filter was applied. The filter size can be arbitrarily changed, and another filter having a smoothing effect may be applied. Examples of other filters having a smoothing effect include a weighted moving average filter that weights each matrix in the filter as needed, and a Gaussian filter that performs weighting according to a normal distribution.

  FIG. 14 is a block diagram illustrating a pseudo three-dimensional data generation apparatus according to an embodiment of the present invention. The pseudo three-dimensional data generation apparatus 600 illustrated in FIG. 14 includes a processing unit 610, an imaging unit 650, and an illumination unit 660. The illumination unit 660 is connected to the imaging unit 650, and the imaging unit 650 is connected to the processing unit 610.

  The processing unit 610 includes an image storage unit 620, a calculation unit 630, and a control unit 640. The image storage unit 620 includes two frame memories 621 and 622, and captures and holds image data to be photographed by the photographing unit 650. The calculation unit 630 includes an image calculation unit 631 and a depth value calculation unit 632, and calculates a pseudo depth ratio of the shooting target by comparing images of the shooting target shot in different illumination states. The control means 640 controls each of the above means. For example, the image calculation unit 631 performs processing for reducing the resolution of the image to be captured stored in the image storage unit 620 and sends the image with the reduced resolution to the depth value calculation unit 632. The depth value calculation unit 632 generates pseudo three-dimensional data to be imaged by applying the pseudo three-dimensional data generation method described above based on the image input from the image calculation unit 631.

  The imaging unit 650 captures light reflected by the imaging target and converts it into an electrical signal, and an A / D conversion unit converts an analog electrical signal captured by the solid-state imaging device 651 into a digital electrical signal. 652.

  The processing means 610 of the present invention can also be realized by a computer and a program that runs on the computer, and the program can be recorded on a recording medium that can be read by the computer. It is also possible to download this program from a server or the like on the network by connecting the computer to the network.

  As mentioned above, the invention made by the present inventor has been specifically described based on the above embodiments. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Of course.

It is a figure for demonstrating the pseudo | simulated three-dimensional data generation method by one Embodiment of this invention. It is a graph which shows the relationship between the pixel value of an image with illumination, and a depth ratio. 5 is a flowchart for explaining a pseudo three-dimensional data generation method according to an embodiment of the present invention. It is a figure for demonstrating the contrast improvement of the depth map by one Embodiment of this invention. 5 is a histogram of the depth map shown in FIG. 4 and a depth map with improved contrast. 6 is a flowchart for explaining a pseudo three-dimensional data generation method with contrast improvement according to an embodiment of the present invention. It is a figure for demonstrating the case where the histogram of a depth map is isolate | separated into the several range. FIG. 8 is a diagram showing a histogram when the contrast of the depth map of FIG. 7 is improved by the contrast improving method described with reference to FIG. 6. It is a figure for demonstrating the contrast improvement method by a characteristic curve. 5 is a flowchart for explaining a pseudo three-dimensional data generation method including a contrast improvement method according to an embodiment of the present invention. It is a figure for demonstrating the result of improving the contrast of the depth map shown in FIG. 7 using the contrast improvement method using the characteristic curve demonstrated with reference to FIG. It is a figure for demonstrating the smoothing of the depth map by one Embodiment of this invention. It is a figure which shows an example of the coefficient of the low-pass filter used by the smoothing process. 1 is a block diagram illustrating a pseudo three-dimensional data generation device according to an embodiment of the present invention.

Explanation of symbols

600 ... Pseudo three-dimensional data generation device 610 ... Processing means 620 ... Image storage means 621, 622 ... Frame memory 630 ... Calculating means 631 ... Image calculating means 632 ... Depth ratio calculating means 640 ... Control means 650 ... Shooting means 651 ... Solid Image sensor 652 ... A / D conversion means 660 ... illumination means

Claims (20)

A method of generating pseudo three-dimensional data from an image to be captured,
An image acquisition process for acquiring a first image captured in a first illumination state and a second image captured in a second illumination state brighter than the first illumination state;
A depth value calculating step of calculating a depth value by dividing a pixel value of the first image by a pixel value of a corresponding pixel of the second image. The method of claim 1, comprising:
In the depth value calculating step, a result of dividing a pixel value of the first image by a pixel value of a corresponding pixel of the second image is used as a depth value ratio. The method of claim 1, comprising:
In the depth value calculating step, a complement of a result obtained by dividing a pixel value of the first image by a pixel value of a corresponding pixel of the second image is used as a depth value ratio.   4. The method according to claim 1, further comprising a resolution reduction process for reducing the resolution of the first and second images acquired in the image acquisition process.   5. The method according to claim 1, further comprising a contrast improvement step of adjusting a distribution range of depth values calculated in the depth value calculation step.   6. The method according to claim 5, wherein, in the contrast improving process, a distribution range of depth values is expanded using a characteristic curve.   7. The method according to claim 6, wherein the characteristic curve is obtained by integrating a histogram of depth values before contrast improvement.   8. The method according to claim 1, further comprising a smoothing step of smoothing the depth value calculated in the depth value calculating step. An apparatus for generating pseudo three-dimensional data from an image to be captured,
A first image is obtained by photographing the photographing subject in the first illumination state, and a second image is obtained by photographing the photographing subject in a second illumination state brighter than the first illumination state. Photographing means to
Image storage means for storing the first and second images acquired by the photographing means;
Depth value calculating means for calculating a depth value by dividing the pixel value of the first image stored by the image storage means by the pixel value of the corresponding pixel of the second image. Device to do. The apparatus of claim 9, comprising:
The depth value calculation means uses the result of dividing the pixel value of the first image by the pixel value of the corresponding pixel of the second image as a depth value ratio. The apparatus of claim 9, comprising:
The depth value calculating means uses the complement of the result of dividing the pixel value of the first image by the pixel value of the corresponding pixel of the second image as the ratio of depth values.   12. The apparatus according to claim 9, further comprising resolution reducing means for reducing the resolution of the first and second images stored in the image storage means.   13. The apparatus according to any one of claims 9 to 12, wherein the depth value calculating means adjusts the distribution range of the calculated depth value.   14. The apparatus according to claim 13, wherein the depth value calculation means expands a depth value distribution range using a characteristic curve.   15. The apparatus according to claim 14, wherein the characteristic curve is obtained by integrating a histogram of depth values before adjusting the distribution range.   16. The apparatus according to claim 9, further comprising a smoothing unit that smoothes the depth value calculated by the depth value calculating unit.   17. The apparatus according to claim 9, further comprising an illumination unit that illuminates the object to be photographed, wherein the first illumination state corresponds to a state in which the illumination unit is not turned on, The lighting state of 2 corresponds to a state in which the lighting means is turned on.   A program for causing a computer to execute the method according to any one of claims 1 to 8.   The computer-readable recording medium which recorded the program of Claim 18.   Pseudo three-dimensional data generated by the method according to any one of claims 1 to 8.