WO2011132364A1 - 立体画像撮影装置および立体画像撮影方法 - Google Patents
立体画像撮影装置および立体画像撮影方法 Download PDFInfo
- Publication number
- WO2011132364A1 WO2011132364A1 PCT/JP2011/001726 JP2011001726W WO2011132364A1 WO 2011132364 A1 WO2011132364 A1 WO 2011132364A1 JP 2011001726 W JP2011001726 W JP 2011001726W WO 2011132364 A1 WO2011132364 A1 WO 2011132364A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- eye
- image
- vertical
- timing
- images
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B35/00—Stereoscopic photography
- G03B35/08—Stereoscopic photography by simultaneous recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/246—Calibration of cameras
Definitions
- the present invention relates to a stereoscopic image capturing apparatus and a stereoscopic image capturing method provided with a pair of left and right imaging units, and more specifically, corrects a shift between left-eye and right-eye images caused by an optical axis shift between a pair of left and right imaging units.
- the present invention relates to a technique for ensuring the simultaneity of photographing by the left eye and right eye imaging units.
- a twin-lens stereoscopic camera that captures a subject with a pair of left and right imaging units and obtains a stereoscopic image from the obtained left and right images is known.
- the pair of left and right imaging units generally cannot avoid an optical axis shift in the vertical and horizontal directions between them.
- the optical axis shift in the vertical direction is a problem.
- the deviation is detected, and the detected deviation amount is determined by each mechanical drive unit in the multiaxial pan head.
- a technique is known in which the depression angle and elevation angle of the left-eye and right-eye imaging units are adjusted to control the shift so as to be eliminated. This is mechanical deviation correction (see, for example, Patent Document 1).
- Patent Documents 1 and 2 both perform mechanical adjustment, and the adjustment is naturally limited and cannot be completely corrected. This is because there is always an individual difference even if a pair of left and right imaging units having the same specifications is prepared. In addition, it is difficult to avoid an optical axis shift due to a mounting error. Furthermore, an optical axis shift occurs between the imaging units due to a change over time or vibration accompanying movement, and if a three-dimensional image is synthesized as it is, an appropriate three-dimensional image cannot be obtained.
- the present invention has been created in view of such circumstances, and even if a vertical optical axis shift occurs between a pair of left and right imaging units, this is reliably corrected and left and right eye imaging units. It is an object to ensure a high-quality three-dimensional image without misalignment between left and right images.
- ⁇ 1 The present invention solves the above problems based on the following considerations.
- the basis is the technical idea of adjusting the spatial displacement, which is a shift in the image, by replacing it with the temporal displacement of adjusting the drive timing. This will be described with reference to FIG.
- the left-eye and right-eye imaging units 1L and 1R and the frame-synchronized drive control unit that controls the image sensors 2L and 2R in both the imaging units 1L and 1R in a frame-synchronized manner.
- 3 and a camera processing unit 4 that performs image processing on the left and right captured images acquired by the left-eye and right-eye imaging units 1L and 1R to generate a parallax image that is a source of stereoscopic image synthesis.
- the frame synchronous drive control unit 3 compensates that the left-eye and right-eye imaging units 1L and 1R are matched at the operating frequency.
- the left-eye and right-eye captured images PL and PR obtained by the left-eye and right-eye imaging units 1L and 1R
- the left-eye and right-eye captured images PL and PR due to the optical axis shift in the image sensors 2L and 2R.
- a vertical shift which is a spatial shift
- the vertical timing adjuster 5 synchronizes the drive timings of the image sensors 2L and 2R with the frame synchronization so that the vertical shift amounts of the framing positions of the subject images AL and AR in the left-eye and right-eye captured images PL and PR approach zero.
- the drive controller 3 is configured to be adjusted.
- the drive timing of one of the left-eye and right-eye image sensors 2L and 2R is adjusted.
- the adjustment of the drive timing of the image sensor is configured to be possible with any image sensor. This is because the right-eye photographic image PR may be shifted downward in the vertical direction with respect to the left-eye photographic image PL, or conversely, the left-eye photographic image PL may be shifted downward in the vertical direction with respect to the right-eye photographic image PR. In order to cope with both cases. That is, the drive timing adjustment of the left-eye and right-eye image sensors 2L and 2R can be performed independently of each other.
- both the captured images PL and PR are developed on the drawing so that the spatial vertical direction axis is the time axis direction.
- the vertical direction of the drawing corresponds to the spatial horizontal direction of both captured images PL and PR.
- the notation of such a figure including ( ⁇ 1) (details will be described later with reference to FIG. 5) is specific to the MOS image sensor of the rolling shutter system, and from the image sensor immediately after completion of exposure for a predetermined time.
- the reading is sequentially performed in the screen vertical direction (right direction in the figure) in units of lines.
- the line unit line is one column of pixel groups along the vertical direction of the drawing.
- AL is a subject image to be read in the left-eye photographed image PL
- AR is a subject image to be read in the right-eye photographed image PR.
- the subject images AL and AR advance in time as they go downward (right side).
- the left eye and right eye image sensors 2L and 2R are displaced within one frame of the same subject at the same time with respect to the imaging surface of the image sensor due to optical axis misalignment.
- the position (AL, AR) of the right-eye shot image PR is displaced vertically downward (rightward in the figure) relative to the left-eye shot image PL. That is, it is assumed that the subject image AR in the right-eye photographic image PR is shifted downward in the vertical direction with respect to the subject image AL in the left-eye photographic image PL.
- ⁇ t is a shift time amount obtained by converting a vertical shift amount of the framing position of the subject image AR in the right-eye shot image PR with respect to the framing position of the subject image AL in the left-eye shot image PL into time.
- the stereoscopic image has a defect with a vertical shift. End up.
- the left and right trimming areas TL are used to detect the amount of vertical displacement with respect to the framing positions of the subject images AL and AR in the left-eye and right-eye photographed images PL and PR.
- TR are set.
- the left and right trimming areas TL are set so that the relative positional relationship of the subject image AL in the left-eye trimming area TL and the relative positional relationship of the subject image AR in the right-eye trimming area TR are equivalent.
- TR are set.
- These left-eye and right-eye trimmed images PTL and PTR are spatially equivalent in phase relationship based on the trimming areas TL and TR, as shown in FIG.
- a spatially equivalent phase relationship is not an immediate solution. This is because the problem of time gap remains. Solving this point is the greatest point of the present invention. This will be described in detail below.
- both the subject images AL and AR are of a scene that is shifted in time. Furthermore, if the subject is a moving subject, both the subject images AL and AR are further different in position (coordinates), contour shape, size, etc., and further differ from the subject image to be originally extracted, and synthesized into a three-dimensional image. In this case, the shift is not avoided, and the depth position changes depending on the moving speed and the moving direction, and the image is recognized as an image with a sense of incongruity that cannot be found in nature. This is the details of the problem to be solved.
- the reading of the left-eye trimming image PTL takes a longer time than the reading start pulse SP, which is different from the reading start pulse SP of the right-eye trimming image PTR.
- a read start pulse SP * delayed by ⁇ t is used. It is the role of the vertical timing adjuster 5 to generate the read start pulse SP * with a delay of time ⁇ t from the standard read start pulse SP.
- the left-eye trimmed image PTL whose optical axis is relatively below is of the same scene that is not substantially shifted in time from the right-eye trimmed image PTR. It becomes. Therefore, when the left-eye trimmed image PTL and the right-eye trimmed image PTR are combined with a stereoscopic image as a parallax image, a stereoscopic image in which the shift is eliminated is obtained.
- the upper right eye subject image AR corresponds to the timing (ts + ⁇ t) to (te + ⁇ t)
- the lower left eye subject image AL also corresponds to the timing (ts + ⁇ t) to (te + ⁇ t).
- the left and right trimmed images PTL and PTR are matched in the vertical positional relationship. The reason for this is that although it is repeated, as shown in FIG. 3A, a read start pulse SP * delayed in time by ⁇ t from the upper read start pulse SP is used.
- the image sensor on the side where the photographed image is relatively lower is assumed to be the left-eye image sensor 2L, but naturally the image sensor on the side where the photographed image is relatively below is the image sensor for the right eye. It may be 2R. In that case, the above may be considered symmetrically by replacing the left and right.
- both the left-eye and right-eye photographic images for the parallax images of the same subject at the same time through the adjustment of the time factor of the drive timing of the two image sensors 2L and 2R by the action of the vertical timing adjuster 5 (The spatial positional relationship in the trimmed images PTL, PTR) is converted into an equivalent one.
- the spatial displacement of vertical deviation is adjusted so as to approach zero, and is also performed with the spatial positioning of the same dimension. That is mechanical adjustment. Because of the mechanical adjustment, there was a limit, and the ultimate fine adjustment was difficult.
- the mechanical adjustment in the case of the prior art is adjustment of the spatial relative positional relationship between the subject and the imaging equipment in a state before capturing the optical image of the subject. In other words, this is an “external adjustment”.
- adjustment is performed in a state where an optical image of a subject is captured and digitized. This is an “internal adjustment”.
- In the external adjustment there is inevitably a discrepancy between the cause and the result, but in the case of the internal adjustment, the relationship between the cause and the result is close and a fundamental solution.
- the spatial displacement which is a displacement in the image
- a temporal displacement called a drive timing adjustment
- the stereoscopic image capturing apparatus of the present invention is A left-eye imaging unit 1L having a left-eye image sensor 2L; A right-eye imaging unit 1R having a right-eye image sensor 2R; A frame-synchronized drive control unit 3 that controls the left-eye and right-eye image sensors 2L and 2R in frame-synchronization, and
- the left-eye captured image PL and the right-eye image are processed by performing image processing on the left-eye captured image PL acquired by the left-eye image capturing unit 1L and the right-eye captured image PR acquired by the right-eye image capturing unit 1R.
- a camera processing unit 4 that generates a parallax image with the captured image PR;
- the left eye is adjusted so that the amount of vertical deviation between the framing position of the subject image in the left-eye photographic image PL and the framing position of the subject image in the right-eye photographic image PR approaches zero.
- a vertical timing adjuster 5 that adjusts the drive timing of the right-eye image sensors 2L and 2R; Is provided.
- the stereoscopic image capturing method of the present invention includes, as an essential processing step, The framing position of the subject image in the left-eye captured image PL obtained by the left-eye image sensor 1L having the left-eye image sensor 2L, and the subject in the right-eye captured image PR obtained by the right-eye image sensor 1R having the right-eye image sensor 2R.
- a first step for determining a vertical shift amount from the framing position of the image A second step of adjusting the drive timing of the left-eye and right-eye image sensors 2L and 2R based on the vertical direction shift amount ⁇ h; A third step of sequentially reading out image data after completion of exposure acquired by the left-eye and right-eye image sensors 2L and 2R driven according to the adjusted drive timing; including.
- ⁇ 1 >> and ⁇ 18 >> described above even if a vertical optical axis shift occurs between the pair of left and right imaging units 1L, 1R, the shift is made through electronic timing adjustment by the vertical timing adjuster 5.
- a parallax image for display / recording is obtained that has a high degree of synchronism in shooting with no spatial vertical deviation between the left-eye and right-eye images.
- a stereoscopic image can be obtained.
- the vertical timing adjuster electronically adjusts the left eye and right eye captured images in the vertical direction.
- a display / recording parallax image in which synchronization of shooting without any deviation is ensured is obtained, and a high-quality stereoscopic image can be obtained.
- FIG. 1 is a block diagram showing a basic configuration of a stereoscopic image photographing apparatus of the present invention.
- FIG. 2 is a conceptual diagram (part 1) of read timing adjustment with a rolling shutter in the present invention.
- FIG. 3 is a conceptual diagram (part 2) of timing adjustment for reading with the rolling shutter in the present invention.
- FIG. 4 is a block diagram showing a specific configuration of the vertical timing adjuster in the embodiment of the present invention.
- FIG. 5 is a conceptual diagram of exposure / reading processing with a general rolling shutter.
- FIG. 6 is a block diagram showing the configuration of the stereoscopic image capturing apparatus in the embodiment of the present invention.
- FIG. 7 is an explanatory diagram showing a state of vertical shift of the left-eye and right-eye photographed images in the embodiment of the present invention.
- FIG. 8 is an explanatory diagram of the control of the read timing with respect to the vertical shift in the embodiment of the present invention.
- FIG. 9 is a timing chart for explaining the operation corresponding to FIG. 7 in the embodiment of the present invention.
- FIG. 10 is a timing chart for explaining the operation corresponding to FIG. 8 in the embodiment of the present invention.
- FIG. 11 is a view (No. 1) showing a display state of the left-eye and right-eye photographed images on the monitor display in the embodiment of the present invention.
- FIG. 12 is a diagram (No. 2) illustrating a display state of the left-eye and right-eye captured images on the monitor display according to the embodiment of the present invention.
- FIG. 13 is a diagram showing how the face area is detected in the embodiment of the present invention.
- FIG. 14 is a flowchart showing the operation of the stereoscopic image capturing apparatus according to the embodiment of the present invention.
- FIG. 15 is a diagram showing an intermediate stage of trimming area adjustment in the embodiment of the present invention.
- FIG. 16 is a perspective view (No. 1) showing the state of the camera system in the embodiment of the present invention.
- FIG. 17 is a perspective view (No. 2) showing the camera system in the embodiment of the present invention.
- ⁇ 2 >> In the above ⁇ 1 >>, the trimming areas TL and TR are mentioned. In the trimming areas TL and TR, the delay amount necessary for the drive timing is finally obtained.
- the delay amount of the drive timing can be obtained by other methods, but if the trimming areas TL and TR are set, the delay amount can be easily obtained.
- the vertical timing adjuster 5 is set so that the relative positional relationship of the same subject at the same time is the same in the left-eye photographed image PL and the right-eye photographed image PR.
- the amount of deviation between the left and right trimming areas TL and TR is obtained, and the delay amount of the drive timing corresponding to the obtained amount of deviation is set in the frame synchronous drive control unit 3.
- image data corresponding to the number of lines corresponding to the vertical shift amount from the upper side in the image displaced downward in the vertical direction may be invalid data.
- the image sensors 2L and 2R in the pair of left and right imaging units 1L and 1R are provided with electronic shutters for exposure control.
- a rolling shutter system is standard.
- a global shutter system is a standard electronic shutter in the case of a CCD (Charge-Coupled Device) image sensor.
- the global shutter method of the CCD image sensor is a method in which all pixels simultaneously perform a shutter operation. Therefore, if the drive is synchronized by the left and right image sensors, the left-eye image and the right-eye image are displaced in the vertical direction. However, there is no timing shift.
- the left-eye and right-eye image sensors 2L and 2R are MOS image sensors having a rolling shutter as an electronic shutter for exposure control. is there.
- the invention of the above ⁇ 1 >> was originally based on a rolling shutter type MOS image sensor.
- the invention ⁇ 1> does not necessarily require a rolling shutter as an electronic shutter for exposure control.
- the inventions ⁇ 1 >> and ⁇ 2 >> are established under such conditions.
- ⁇ 3 >> the specification with higher applicability is limited.
- the rolling shutter MOS image sensor that performs line-by-line reading is suitable for the timing adjustment of the read start pulse SP from the image sensors 2L and 2R by the vertical timing adjuster 5 of the above ⁇ 1 >> and ⁇ 2 >>. It is a thing.
- the present invention is preferably applied to such a rolling shutter type image sensor.
- ⁇ 4 In the above-described structures ⁇ 1 >> to ⁇ 3 >>, the drive timing adjusted by the vertical timing adjuster 5 is set as the timing of the vertical synchronization signal, and the vertical timing adjuster 5 then takes a picture.
- the vertical sync signal timing corresponds to the vertical deviation from the sync reference signal timing. It is preferable to delay by a vertical scanning period comparable to the number of lines. In this case, in the image sensor on the side where the subject image framing position in the captured image is relatively below, the timing of the vertical synchronizing signal coincides with the timing of the synchronizing reference signal.
- the timing of the vertical synchronization signal in the image sensor with the subject image framing position on the upper side is the timing of the vertical synchronization signal on the image sensor with the subject image framing position on the lower side (upper side of the optical axis). This means that the timing is delayed by the vertical scanning period corresponding to the vertical shift amount.
- Both the image sensor 2L in the left-eye imaging unit 1L and the image sensor 2R in the right-eye imaging unit 1R are driven based on a vertical synchronization signal that is a timing reference.
- a vertical synchronization signal that is a timing reference.
- the timings of both vertical synchronizing signals are adjusted independently of each other.
- a common signal that is the source of the vertical synchronization signal of both the image sensors 2L and 2R is a synchronization reference signal.
- the timing of the vertical synchronization signal of the image sensor with the subject image framing position on the lower side is the same timing as the synchronization reference signal, and the subject image framing position is on the upper side (
- the timing of the vertical synchronization signal of the image sensor on the lower side of the optical axis is delayed from the synchronization reference signal.
- the amount of delay ⁇ t corresponds to a vertical scanning time equivalent to the number of lines corresponding to the amount of vertical deviation ⁇ h of the framing positions of the subject images AL and AR in the left-eye and right-eye captured images PL and PR.
- the dimension of the delay amount ⁇ t is time, and the dimension of the vertical deviation amount ⁇ h is length.
- the delay amount ⁇ t may be converted into the number of lines in the setting display mode (resolution standard such as VGA).
- the vertical shift amount is adjusted to approach zero.
- the left-eye and right-eye captured images (trimmed images PTL and PTR) for parallax images can be provided with consistency without vertical deviation.
- the framing position of the subject image AL of the left-eye photographed image PL is relatively displaced upward.
- the read start timing is delayed in the image sensor 2L for the left eye on the side where the framing position of the subject image of the photographed image is relatively upward (the optical axis lower side).
- the photographed image PR for the right eye is displaced relatively upward.
- the read start timing is delayed in the right-eye image sensor 2R on the side where the framing position of the subject image of the captured image is relatively upward.
- the vertical shift is eliminated in the left-eye and right-eye images constituting the parallax image, and the spatial / temporal positional relationship of the right-eye subject image AR is spatial / temporal of the left-eye subject image AL. It is adjusted to the same as the relative positional relationship.
- the spatial and temporal positional relationships of the subject images AL and AR in the left-eye and right-eye captured images (trimmed images PTL and PTR) for the parallax images are the same. Adjusted.
- the spatial displacement which is a shift in the image
- a temporal displacement called a delay of the read start timing. That is, the vertical timing adjuster 5 is configured to adjust the timing of the vertical synchronization signals of the left-eye and right-eye image sensors 2L and 2R independently of each other.
- ⁇ 5 the driving timing of the left-eye and right-eye MOS image sensors 2L and 2R is determined by the electronic shutter start pulse ESSP (shutter gate pulse) and the read start pulse SP. There is an aspect in which the timing is delayed by the same time in parallel with the delay of the vertical synchronization signal of the MOS image sensor.
- ESSP shutter gate pulse
- the electronic shutter start pulse ESSP sequentially resets the charge accumulation accompanying the incidence of the optical image of the subject in units of lines in response to the output of this pulse, and starts the exposure period (accumulation time) from the pulse release timing. It is.
- the read start pulse SP is sequentially read in line units starting from this pulse, and one frame period (one field period) is determined based on the cycle.
- the period from the release timing of the electronic shutter start pulse ESSP to the timing of the next readout start pulse SP corresponds to the exposure period.
- the electronic shutter start pulse ESSP and the read start pulse SP having such a function are generated based on the vertical synchronization signal. Therefore, when the vertical synchronization signal is delayed, it is preferable that the electronic shutter start pulse ESSP and the read start pulse SP are delayed by the same time.
- the vertical timing adjuster 5 converts the vertical shift amount ⁇ h into the shift amount of the number of lines and then converts the left-eye and right-eye captured images PL and PR.
- the shift amount of the number of lines is set as the delay amount of the drive timing in the left-eye image sensor 2L or the right-eye image sensor 2R that acquires the other one in which the subject image framing position is relatively higher than the other.
- the amount of vertical deviation is captured by the number of pixels in the vertical direction. Adjustment of the drive timing of the image sensor is performed based on the number of lines.
- the line number shift amount (delay amount information) obtained by converting the vertical shift amount in this way is fed back to the corresponding image sensor (image sensor of optical axis shift, image shift), and matching control is performed. To serve.
- ⁇ 7 In the above ⁇ 1 >> to ⁇ 6 >>, in order to give a drive timing adjustment instruction from the vertical timing adjuster 5 to the corresponding image sensor via the frame synchronous drive control unit 3, it corresponds to the deviation amount. It is necessary to send the drive timing delay amount ⁇ t. Therefore, the vertical timing adjuster 5 is provided with a function for sending the drive timing delay amount ⁇ t. In this section, the function of sending the drive timing delay amount ⁇ t is assumed to be by an artificial manual operation (manual operation by the operator of the stereoscopic image capturing apparatus).
- the left-eye shot image PL and the right-eye shot image PR are displayed on the monitor.
- the captured image displayed on the monitor may be raw image data or image data that has been resized or trimmed for display.
- the operator grasps whether or not the left-eye subject image AL and the right-eye subject image AR are displaced in the vertical direction, and if so, what the vertical displacement amount ⁇ h is. After that, when the operator manually operates the operation device 6 (see FIG. 1), the position of the trimming area of one of the image sensor images is adjusted so that the operation device approaches zero.
- a deviation adjustment instruction signal for adjustment is generated, and the deviation adjustment instruction signal is sent to a position adjuster (mounted on a camera processing unit or the like) of an image trimming area.
- a state in which the trimmed image shift is reduced is displayed on a monitor, for example. While checking the display, the operator performs an adjustment instruction operation until the vertical shift amount becomes zero.
- the shift adjustment instruction signal at the time when the vertical alignment of the image is completed by changing the trimming position is sent to the vertical timing adjuster 5 to adjust the read start timing, and the above-described adjustment operation is executed here. .
- This adjustment reduces the amount of vertical deviation between the left-eye trimming area TL including the left-eye subject image AL and the right-eye trimming area TR including the right-eye subject image AR.
- the decreasing state is displayed on a monitor, for example.
- the vertical deviation ⁇ h between the left-eye trimming area TL and the right-eye trimming area TR becomes zero, and the synchronism of the vertical vertical deviation between the left-eye and right-eye photographic images is ensured.
- a display / recording parallax image can be obtained, and a high-quality stereoscopic image can be obtained.
- this aspect is the configuration of the above ⁇ 1 >> to ⁇ 7 >> An operating device that generates a shift adjustment instruction signal for instructing the vertical timing adjuster 5 to adjust the drive timing based on a manual operation by an operator of the stereoscopic image capturing apparatus;
- the vertical timing adjuster 5 is configured to adjust the drive timing in the frame synchronous drive control unit 3 based on the shift adjustment instruction signal.
- the function of the camera processing unit 4 is used.
- the camera processing unit 4 has a function of reducing and resizing the image data corresponding to the left and right trimming areas TL and TR to a monitor size, and displaying the reduced and resized images side by side on the left and right sides on the monitor. (See FIG. 11). This can be said to be a shift detection function through vision.
- the reason why the image data corresponding to the left and right trimming areas TL and TR is reduced and resized is to be halved in the horizontal direction so that the two left and right images fit within one monitor size.
- a reduced resized image based on the right eye trimmed image PTR is displayed in the right half display area
- a reduced resized image based on the left eye trimmed image PTL is displayed in the left half display area. If there is an optical axis shift in the pair of left and right imaging units 1L and 1R, a vertical shift occurs in the displayed two left and right images.
- the operator (photographer) cancels the vertical shift of the left-eye display image and the right-eye display image by manual operation with the operation unit 6 while checking the shift of the two images shifted in the vertical direction on the monitor.
- the start line position of the trimming area may be adjusted up and down so that the continuity of a common horizontal subject in the image can be obtained. Or, outdoors, the landscape of the distant view may be adjusted based on the horizon.
- the drive timing delay amount ⁇ t corresponding to the shift amount obtained in this way is captured internally and reflected as the drive timing delay amount ⁇ t in the vertical timing adjuster 5.
- the vertical alignment of the left-eye and right-eye display images by manual operation eventually becomes the vertical alignment of the start line position of the trimming area.
- ⁇ 11 This section is an alternative to the above ⁇ 10 >>, and displays the left-eye and right-eye trimmed images PTL and PTR by half each.
- the camera processing unit 4 reduces and resizes the left half of the image data corresponding to the left eye trimming area TL to one half of the monitor size and displays it on the left side on the monitor, as well as the right eye trimming area TR.
- Of the right half of the image data corresponding to is reduced to half the monitor size and displayed on the right side on the monitor (see FIG. 12). This is also a function of detecting a shift through vision.
- the line position may be adjusted up and down. For example, when the body of one subject image is divided between the left and right body, and the left and right body images are shifted to a state where they are cut off in the vertical direction, the left and right body images are manually manipulated. The adjustment to match the image exactly is to balance the left and right sides of the body of one person including the face.
- the drive timing delay amount ⁇ t corresponding to the shift amount thus obtained is captured internally and reflected as the drive timing delay amount ⁇ t.
- the vertical alignment of the left-eye and right-eye display images by manual operation eventually becomes the vertical alignment of the start line position of the trimming area.
- the vertical timing adjuster 5 uses the delay information ⁇ t of the drive timing as the start line position information of the left eye trimming area TL and the right eye trimming area. A delay amount obtained by converting the difference from the position information of the TR start line into time is used. Vertical deviation can be corrected based on the start line positions of the left and right trimming areas.
- ⁇ 14 >> In this section, the automatic method is used to grasp the vertical direction deviation and the vertical direction deviation amount ⁇ h and to instruct the vertical matching. This will be described with reference to FIG.
- the components of the vertical timing adjuster 5 left eye and right eye attention point coordinate calculators 7L and 7R, a vertical deviation amount calculator 8, a delay amount calculator 9, and a drive timing setter 10 are provided.
- the left eye attention point coordinate calculator 7L calculates the vertical coordinate YL of the attention point QL (eye, mouth, etc. of the human face) of the subject image AL in the left eye photographed image PL.
- the right eye attention point coordinate calculator 7R calculates the vertical coordinate YR of the attention point QR of the subject image AR in the right eye photographed image PR.
- the vertical shift amount calculator 8 calculates the vertical shift amount ⁇ h from the difference between the vertical coordinates YL and YR of the points of interest QL and QR.
- ⁇ h>0 the subject image AL in the left-eye photographed image PL is displaced downward in the vertical direction from the subject image AR in the right-eye photographed image PR, and when ⁇ h ⁇ 0, the right-eye photography is performed.
- the subject image AR in the image PR is displaced vertically downward from the subject image AL in the left-eye photographed image PL.
- the vertical deviation amount ⁇ h is expressed by the number of pixels, it is also the number of lines. This difference in the number of lines (number of pixels) in the vertical direction corresponds to the delay time of the drive timing. That is, the drive timing delay amount ⁇ t can be obtained from the vertical shift amount ⁇ h by calculation.
- the delay amount ⁇ t is positive, the drive timing for the right-eye photographed image PR is delayed. Conversely, when the delay amount ⁇ t is negative, the drive timing for the left-eye photographed image PL is delayed.
- the drive timing setting unit 10 sets the drive timing in consideration of the delay amount ⁇ t of the drive timing in the frame synchronous drive control unit 3.
- the vertical timing adjuster 5 The left and right attention points are calculated by extracting the left attention point QL in the left eye photographed image PL and the right attention point QR in the right eye photographed image PR, and then determining the vertical coordinates of the left and right attention points QL and QR.
- Point coordinate calculators 7L and 7R Point coordinate calculators 7L and 7R;
- a vertical shift amount calculator 8 for calculating the vertical shift amount ⁇ h from the difference between the vertical coordinate YL of the left target point QL and the vertical coordinate YR of the right target point QR;
- a delay amount calculator 9 for converting the vertical direction shift amount ⁇ h by the vertical direction shift amount calculator 8 into a delay amount ⁇ t in units of time;
- a drive timing setting unit 10 for setting the drive timing in consideration of the delay amount ⁇ t of the drive timing in the frame synchronous drive control unit 3; It is the aspect of having.
- the stereoscopic image capturing method of ⁇ 18 >> is a stereoscopic image capturing method related to the stereoscopic image capturing apparatus of ⁇ 13>, and
- the first step includes A 1-1 step of displaying the left-eye captured image PL and the right-side captured image PR side by side on the same screen; In each of the left-eye and right-eye photographic images PL and PR displayed on the same screen, the relative positions of the same subject at the same time are the same between the left-eye photographic image PL and the right-eye photographic image PR.
- the delay amount ⁇ t corresponding to the vertical shift amount ⁇ h is fed back to the drive timing, so that the vertical optical axis shift is between the pair of left and right imaging units 1L, 1R. Even if the image is generated, the deviation is surely corrected through electronic and automatic timing adjustment by the vertical timing adjuster 5 and, at the same time, there is no spatial vertical deviation between the left-eye and right-eye photographed images. Therefore, a display / recording parallax image in which the quality is ensured can be obtained, so that a high-quality stereoscopic image can be obtained.
- the first to fourth steps include the trimming area TL of the left-eye captured image QL and the trimming of the right-eye captured image QR.
- the amount of shift ⁇ h in the vertical direction is detected by detecting the shift amount when the one is shifted in the vertical direction. is there.
- ⁇ 21 This item is the above-described stereoscopic image capturing method according to ⁇ 19 >>
- the number of the line to which the target point QL in the trimming area TL of the left-eye photographed image PL belongs and the target point QR in the trimming area TR of the right-eye photographed image PR are determined.
- the vertical deviation amount ⁇ h is detected by calculating the difference between the line number of the line to which it belongs.
- ⁇ 15 In ⁇ 14 >> above, A face area detector for detecting a face area of a subject in the left-eye and right-eye captured images QL and PR;
- the attention point coordinate calculators 7L and 7R are a preferred mode in which specific portions in the face area detected by the face area detector are set as the left and right attention points QL and PR.
- ⁇ 22 In the three-dimensional image capturing method of the above ⁇ 19> to ⁇ 21 >>, there is an aspect in which a specific part of the face area of the subject in the left-eye and right-eye captured images is used as the attention point.
- ⁇ 16 For the left-eye and right-eye imaging units 1L and 1R, a zoom lens having a zooming function in its optical system is often used. In the specification in which the left-eye and right-eye imaging units 1L and 1R have zooming lenses in the zoom lens, the zooming lenses of the left-eye and right-eye imaging units 1L and 1R are configured to be linked to each other. Regardless of the shift to the telephoto side or the shift to the wide-angle side, if a magnification error occurs on the left and right sides, the left-eye and right-eye images will be inconsistent, and when combined as a stereoscopic image, an appropriate image cannot be obtained. . When the zoom lens is telephoto zoom, the degree of mismatch between the left-eye and right-eye images increases.
- the left-eye and right-eye imaging units 1L and 1R each include an optical system having a variable-power lens.
- the variable-power lens of the left-eye image-capturing unit 1L and the variable-power lens of the right-eye image pickup unit 1R Are configured to be linked to each other.
- zoom lenses that are reciprocated along the optical axis.
- this section is not limited to this type, and exhibits a variable magnification function in a form other than reciprocal movement. Also good.
- ⁇ 17> In the configuration of ⁇ 16> above, when the zoom ratio of the variable power lens of the two imaging units changes, the amount of vertical deviation ⁇ h between the two images changes. This section adjusts this shift. Specifically, the magnification on the most telephoto side with respect to the image on the widest angle side is set to n times, the number of displacement lines on the widest angle side is Lw, and the number of displacement lines on the most telephoto side is Lt.
- the deviation amount line number Lb at an arbitrary magnification b is ⁇ n: the amount of change per side per n times ⁇ b: the amount of change per side at magnification b ( ⁇ n means the square root of n)
- Lb [(Lt ⁇ Lw) / ⁇ n] ⁇ ⁇ b + Lw It becomes. This is the number of shift amount lines corresponding to the magnification b, and the drive timing delay amount ⁇ t is calculated from this and fed back.
- the number of lines which is the amount of deviation in the vertical direction in the imaging area on the widest angle side and the farthest telephoto side of the zoom lens, is registered, and converted into the number of lines at an arbitrary zoom position.
- the stereo image shooting method in this section is A fourth step of starting to shoot the left-eye and right-eye photographic images PL and PR; A fifth step of reading the left-eye and right-eye photographed images PL and PR; A sixth step of initially setting the trimming areas TL and TR in the left-eye and right-eye photographed images PL and PR, A seventh step of extracting the attention points QL and QR in the trimming areas TL and TR of the left-eye and right-eye photographed images PL and PR; An eighth step of confirming whether or not there is a vertical shift between the attention point QL of the left-eye photographed image PL and the attention point QR of the right-eye photographed image PR; A ninth step of adjusting one position of the trimming area TL of the left-eye photographed image PL and the trimming area TR of the right-eye photographed image PR in the vertical direction when there is a deviation in the vertical direction; A tenth step in which the seventh to ninth steps are
- FIG. 5A shows the passage of time
- FIG. 5B shows a photographed image by the image sensor
- FIG. 5C shows exposure and readout control.
- (A), (b), and (c) correspond temporally in the vertical direction.
- the right direction is the time advance direction
- ESSP is the electronic shutter start pulse
- SP is the read start pulse.
- P is a photographed image
- A is a subject image to be read
- the vertical direction of the figure is the horizontal direction (direction along the line) of the photographed image
- the horizontal direction of the figure is This is the vertical direction of the captured image.
- a strip-shaped object represents imaging data for each line.
- the horizontal direction corresponds to the time axis
- the vertical direction corresponds to the vertical direction of the shooting screen.
- the strip-shaped one shows the imaging situation (temporal transition) for each line (Hiroo Takemura, “Introduction to CCD / CMOS Camera Technology”, p. 64 of Corona Corp. b)
- CMOS reading method in the case of a rolling shutter.
- FIG. 5B corresponds to an overhead view of FIG. 5C.
- the electronic shutter start pulse ESSP sequentially resets the charge accumulation accompanying the incidence of the optical image of the subject by the output of this pulse in each line, and the exposure period (accumulation time) is started from the timing of releasing the pulse.
- the read start pulse SP is used to sequentially read out image data in units of lines starting from this pulse, and one frame period (one field period) is determined based on the cycle.
- the electronic shutter start pulse ESSP and the readout start pulse SP correspond to the exposure of the first line and the readout of data in line units. Although not shown, the same electronic shutter start pulse ESSP and set of read start pulse SP are used for each line. They are output sequentially shifted by a time corresponding to one line.
- the exposure period Tex is from the timing tsh of releasing the electronic shutter start pulse ESSP to the timing tsp of the next read start pulse SP.
- An elongated parallelogram in the range corresponding to Tex (“exposure period in units of lines” in (c)) represents a common exposure period for all lines.
- the exposure period Tex can be made equal for all lines.
- the readout start pulse SP immediately after the end of the exposure period Tex is sequentially performed from the first line to the last line in line units.
- Read image data This is an exposure / readout system using a rolling shutter system of a MOS image sensor.
- Each part of the vertical direction of the subject image A forming one image has a property of being shifted from each other in time.
- the leg portion of the subject image A is an image at a timing slightly delayed compared to the head portion. This is because the set of exposure start timing, exposure end timing, and readout start timing is sequentially delayed from the head of the subject image A to the legs.
- the electronic shutter start pulse ESSP and the read start pulse SP having such a function are generated on the basis of the vertical synchronization signal. Therefore, if the vertical synchronization signal is delayed, the electronic shutter start pulse ESSP and the read start pulse SP are naturally delayed by the same time in conjunction with the delay.
- FIG. 6 is a block diagram showing the configuration of the stereoscopic image capturing apparatus in the embodiment of the present invention.
- reference numeral 100 denotes a stereoscopic image photographing unit
- reference numeral 200 denotes a camera processing unit.
- the stereoscopic image capturing unit 100 and the camera processing unit 200 are electrically connected via a data communication unit 150 formed of a cable.
- the stereoscopic image imaging unit 100 includes a right-eye imaging unit 1R, a left-eye imaging unit 1L, and a dual sensor frame synchronization drive control unit 3.
- the right-eye imaging unit 1R includes a zoom lens 11, an optical lens 12, an optical low-pass filter (LPF) 13, a color filter 14, a MOS type image sensor 2R, and an analog front end unit 15 including an AD converter.
- the left-eye imaging unit 1L includes a zoom lens 21, an optical lens 22, an optical low-pass filter (LPF) 23, a color filter 24, a MOS type image sensor 2L, and an analog front end unit 25 including an AD converter.
- the frame synchronous drive control unit 3 is configured to drive and control the left and right image sensors 2L and 2R in frame synchronization.
- the left and right image sensors 2L and 2R are MOS image sensors having a rolling shutter mechanism.
- the left-eye imaging unit 1L and the right-eye imaging unit 1R receive subject light flux independently of each other, and subject image image data obtained by photoelectric conversion in the image sensors 2L and 2R is input to the camera processing unit 200 as parallax image data. It is designed to output.
- the configuration and operation of the left-eye and right-eye imaging units 1L and 1R are the same.
- the right-eye imaging unit 1R will be described.
- Light that has passed through the variable power lens 11 and the optical lens 12 driven along the optical axis passes through the optical LPF 13 and the color filter 14 and enters the image sensor 2R.
- the optical LPF 12 removes high-frequency components and prevents aliasing (high-frequency component folding phenomenon).
- the subject image formed on the light receiving surface of the image sensor 2R is converted into a signal charge corresponding to the amount of incident light by each photodiode, and sequentially read out as a voltage signal (image signal) corresponding to the signal charge.
- the image signal read from the image sensor 2R is sent to the analog front end unit 15 and subjected to processing such as analog gain and CDS (correlated double sampling), and then the image signal is converted into a digital signal by A / D conversion processing. And is output to the preprocessing unit 44 of the camera processing unit 200.
- processing such as analog gain and CDS (correlated double sampling)
- CDS correlated double sampling
- the left-eye and right-eye image sensors 2L and 2R are driven based on a pulse supplied from the driver circuit of the frame synchronous drive control unit 3. That is, the driver circuit generates the vertical synchronization signals VL and VR for each image sensor based on the synchronization reference signal common to the image sensors 2L and 2R. Then, the driver circuit generates an electronic shutter start pulse ESSP (shutter gate pulse) and a read start pulse SP based on the vertical synchronization signals VL and VR, respectively, and individually controls the image sensors 2L and 2R. Based on the electronic shutter start pulse ESSP, the charges accumulated in the photodiodes of the individual pixels are sequentially reset in units of lines.
- ESSP shutter gate pulse
- the period from the release timing of the electronic shutter start pulse ESSP to the timing of the subsequent read start pulse SP is an exposure period (accumulation time).
- the read start pulse SP in each of the image sensors 2L and 2R is output in line units, and image data is read out in line units. As a result, the image data acquired by the image sensors 2L and 2R is output as the left-eye and right-eye photographed images PL and PR.
- a delay amount ⁇ t which is a timing shift time between the two vertical synchronization signals VL and VR, is supplied from the CPU 41 in the camera processing unit 200 described later to the frame synchronization drive control unit 3 via the preprocessing unit 44. It is configured. With the unit time corresponding to the line unit as a reference, the delay amount ⁇ t is increased or decreased by an integral multiple of the unit time.
- the electronic shutter start pulse ESSP and the read start pulse SP are also delayed by the same delay amount ⁇ t.
- the electronic shutter start pulse ESSP and the read start pulse SP are repeatedly and periodically output from the frame synchronous drive control unit 3 in both the left-eye image sensor 2L and the right-eye image sensor 2R.
- the camera processing unit 200 41 is a CPU (Central Processing Unit), 42 is a ROM (Read Only Memory), 43 is a RAM (Random Access Memory), 44 is a preprocessing unit, 45 is a memory control unit, and 46 is an image memory.
- 47 is an image signal processing unit
- 48 is a compression / decompression unit with motion vector detection
- 49 is a resizing processing unit
- 50 is a face area detection unit
- 51 is a recording media interface
- 52 is a display processing unit
- 53 is a monitor interface
- 55 is a recording medium
- 56 is a camera shake detection unit
- 57 is a synchronization signal generation unit (SSG)
- 58 is an external device control interface.
- the CPU 41 is a control unit that performs overall control according to a predetermined program, and controls the operation of each circuit based on an operation signal from the operation panel 54. That is, the CPU 41 controls the left-eye and right-eye imaging units 1L and 1R through control by the frame synchronous drive control unit 3 according to various shooting conditions (exposure conditions, strobe light emission, shooting mode, etc.) according to the operation signal. Control. Further, the CPU 41 performs automatic exposure (AE) control, automatic focus adjustment (AF) control, auto white balance control, lens drive control, image processing control, read / write control of the recording medium 55, and the like.
- AE automatic exposure
- AF automatic focus adjustment
- auto white balance control lens drive control
- image processing control read / write control of the recording medium 55, and the like.
- the CPU 41 drives and controls the focus lenses in the optical lenses 12 and 22 and controls the zoom lenses 11 and 21 according to a zoom instruction from the operation panel 54 so that the zoom magnifications of both optical systems become the same. Further, the CPU 41 performs AF control when detecting half-press of the release switch, and starts matching exposure and reading control to capture a recording image when detecting full-press of the release switch. Further, the CPU 41 sends a command to a strobe control circuit (not shown) as necessary to control light emission of a flash light emitting tube (light emitting unit) such as a xenon tube.
- a flash light emitting tube light emitting unit
- the ROM 42 stores programs executed by the CPU 41 and various data necessary for control, and the RAM 43 is used as a work area for the CPU 41.
- the pre-processing unit 44 includes an auto calculation unit that performs calculations necessary for AE and AF control, and performs a focus evaluation value calculation, an AE calculation, and the like based on an imaging signal that is captured in response to a half-press of the release switch.
- the calculation result is transmitted to the CPU 41.
- the CPU 41 controls a lens driving motor (not shown) based on the result of the focus evaluation value calculation, and moves the optical lenses 12 and 22 to the in-focus positions where matching is achieved. Also, matching exposure control is performed in the setting of the electronic shutter.
- the pre-processing unit 44 and the image signal processing unit 47 are synchronized (processing for interpolating a spatial shift of the color signal associated with the color filter array), white balance (WB) adjustment, gamma correction, luminance / color difference signal generation, contour Various processes such as enhancement, scaling (enlargement / reduction) processing by the electronic zoom function, and conversion (resizing) processing of the number of pixels are performed. These processes are performed in accordance with commands from the CPU 41. Further, the preprocessing unit 44 and the image signal processing unit 47 process the image signal using the image memory 46 via the memory control unit 45 provided between the preprocessing unit 44 and the image signal processing unit 47. The image data of the processing result is temporarily stored in the image memory 46.
- the resizing processing unit 49 changes the image size of the image data that has undergone predetermined signal processing in the preprocessing unit 44 and the image signal processing unit 47 to a standard size. This functions when recording image data in a standardized size or when displaying it on a monitor display connected to the monitor interface 53.
- the face area detection unit 50 detects information such as the position / size / tilt of a person's face as necessary for the resized image data.
- the compression / decompression unit 48 compresses the resized image data according to various compression formats. At this time, a compression encoding algorithm corresponding to the compression format used is used. MPEG format, H.264 When compressing data at a standardized size in a moving image compression data format such as H.264 format, the compression / decompression unit 48 periodically reads and loads resized image data from the image memory 46 via the memory control unit 45 in parallel processing. After the captured frame data is compressed, the compressed data is stored in the memory space by being written back to the image memory 46.
- the recording medium interface 51 relays the memory control unit 45 and the recording medium 55, and transfers the compressed image data to the recording medium 55 for recording.
- the two types of image data of the left-eye and right-eye imaging units 1L and 1R thus captured are recorded on the recording medium 55 according to the recording mode.
- the recording medium 55 is not limited to a semiconductor memory represented by a memory card, and various media such as a magnetic disk, an optical disk, and a magneto-optical disk can be used.
- the recording medium 55 is not limited to a removable medium, and may be a built-in recording medium (internal memory).
- the operation panel 54 is configured for an operator to input various instructions to the camera system.
- a mode selection switch for selecting an operation mode of the camera system, a menu item selection operation (cursor movement operation), and the like.
- a desired target such as a cross key for inputting an instruction for frame advance / rewind of a playback image, an execution key for instructing selection (registration) or executing an operation, and a selection item, or canceling the instruction
- Various operation devices such as a cancel key, a power switch, a zoom switch, a release switch, and a shift correction operation for correcting a vertical shift.
- FIG. 7 is an explanatory diagram of a state of vertical shift of the left-eye and right-eye shot images.
- the left-eye and right-eye imaging units 1L and 1R that are synchronously controlled by the frame synchronization drive control unit 3 capture the human subject 70, thereby obtaining the left-eye and right-eye captured images PL and PR.
- the left-eye imaging unit 1L has the optical axis displaced relatively downward. Since the optical axis of the left-eye imaging unit 1L is shifted downward relative to the right-eye imaging unit 1R, the left-eye subject image AL is shifted upward relative to the right-eye subject image AR during framing. It has a shape. The amount of deviation of the optical axis is assumed to be ⁇ h in the direction perpendicular to the reference position of the light receiving surface of the image sensor.
- the left-eye trimming area TL and the right-eye trimming area TR are used as common recording effective areas for the subject images AL and AR in the left-eye and right-eye captured images PL and PR, respectively.
- the trimming areas TL and TR are set so that the relative positional relationship of the subject image AL in the left-eye trimming area TL and the relative positional relationship of the subject image AR in the right-eye trimming area TR are equivalent.
- FIG. 8 is an explanatory diagram of the control of the read timing with respect to the vertical shift. Adjustment from the state of FIG. 7 to FIG. 8 is the point of this embodiment.
- the read start timing is delayed from the standard timing in the case where the optical axis is relatively shifted downward and as a result the subject image AL is relatively shifted upward (in this case, the left-eye image sensor 2L).
- the delay amount ⁇ t is an amount obtained by converting the number of lines into time, assuming that the vertical deviation amount ⁇ h is counted by the number of lines.
- the read start timing may be the standard timing.
- the output timing of the read start pulse SP may be delayed.
- the output timing of the vertical synchronization signal generated from the synchronization reference signal in the frame synchronization drive control unit 3 may be delayed. If the output timing of the vertical synchronizing signal is delayed, the output timing of the electronic shutter start pulse ESSP, the read start pulse SP, and other timing pulses is also delayed in parallel translation.
- the top of the hat of the father image which is the left-eye subject image AL
- the top of the hat of the father image is to be read at a time T1 when a very short time has elapsed from the start line (reading start time) of the left-eye photographed image PL.
- the top of the hat of the father image which is the right-eye subject image AR
- T2 when a longer time has elapsed, which is a time further delayed from the start line (reading start time) of the right-eye photographic image PR. It becomes a target.
- Reading of the right-eye photographic image PR including the right-eye subject image AR is based on the output timing of the read start pulse SP based on the standard vertical synchronization signal. That is, the reading of the right-eye captured image PR is performed as standard with no time delay.
- the reading of the left-eye photographic image PL in which the vertical shift of the subject image is downward starts from the point of time delayed by ⁇ t from the standard reading timing.
- the time from the timing corresponding to the time of the upper edge to the timing corresponding to the time of the top of the father's hat is T1.
- ⁇ t + T1 T2
- the subject is not a stationary state as in the illustrated example and is a moving subject that is moving at a certain speed or higher.
- T2 in the stationary state T2 + ⁇ t ⁇
- the adjustment is made in a state including the movement, in other words, the adjustment is made in response to the movement, so that it is not affected by the vertical deviation of the optical axis. It is possible to obtain left-eye and right-eye images that constitute a parallax image that is the basis of a high-quality stereoscopic image.
- the moving direction of the moving subject is the horizontal direction, there is no relation to the vertical direction deviation. At this time, the sense of depth is shifted.
- the moving direction is diagonal, the horizontal component is irrelevant, and only the vertical component is the object of adjustment in this embodiment, and is corrected well.
- FIG. 9 is a timing chart for explaining the operation corresponding to FIG. 7, and FIG. 10 is a timing chart for explaining the operation corresponding to FIG.
- the left-eye and right-eye photographed images PL and PR for three frames are shown.
- 9 corresponds to FIG. 2
- FIG. 10 corresponds to FIG. 9 and 10
- the falling timings of the vertical synchronization signals VL and VR correspond to the timing of the read start pulse SP (not shown) of the first line.
- the timing of the vertical synchronization signal VL for the left-eye image sensor 2L matches the timing of the vertical synchronization signal VR for the right-eye image sensor 2R.
- the spatial vertical deviation is reflected as it is, and there is a deviation also in the time axis direction.
- the vertical synchronization signal VL * of the image sensor 2L of the photographic image for left eye PL in which the subject image AL is shifted upward because the optical axis is shifted downward is shown.
- the timing is delayed by a delay amount ⁇ t from the timing of the vertical synchronization signal VR of the right-eye image sensor 2R.
- FIG. 11 shows the display of the left-eye and right-eye photographed images PL and PR on the monitor display.
- FIG. 11A shows the state before adjustment
- FIG. 11B shows the state after adjustment.
- a monitor display (not shown) connected to the monitor interface 53 by converting the left-eye captured image PL and the right-eye captured image PR to the monitor size by image processing in the camera processing unit 200.
- the display on the monitor display is as follows: the left-eye shot image PL obtained by reducing and resizing the original image in half in the horizontal direction is displayed on the left half of the screen, and the original image is displayed in the horizontal direction on the right half.
- the right-eye photographed image PR that has been resized to a fraction is displayed.
- the left-eye and right-eye photographed images PL and PR have a vertical shift due to the optical axis shift.
- the operator looks at the screen and confirms the vertical shift of the left-eye and right-eye shot images PL and PR, and performs manual adjustment to eliminate the vertical shift by operating the operation panel 54. To do. That is, the operator performs a manual operation so that the left-eye photographed image PL on which the subject image AL is shifted upward is shifted downward. At this time, if there is a subject extending in the horizontal direction in the image, vertical adjustment is performed so that continuity is obtained in the left-eye and right-eye photographed images PL and PR. In the outdoors, it is desirable to make adjustments with reference to the horizon or with reference to distant scenery.
- the CPU 41 obtains the number of lines corresponding to the operation amount, further converts the number of lines into time to calculate the delay amount ⁇ t, and the delay amount ⁇ t is transmitted to the preprocessing unit 44.
- the frame synchronous drive control unit 3 sets the given delay amount ⁇ t to the corresponding image sensor (image sensor with optical axis misalignment or image misalignment) of the two image sensors 2L and 2R.
- the number of adjustment lines is obtained based on the amount of operation in the vertical direction adjusted by manual adjustment performed by the operator from the initial screen display state of the monitor display.
- the trimming areas TL and TR are set in the left-eye and right-eye photographed images PL and PR on the initial screen, and the number of lines of deviation is obtained from the information on the difference in the Y coordinate of the start position of the trimming area.
- the number of pixels in the vertical direction may be obtained instead of the number of lines.
- FIG. 12 shows another manner of manual adjustment, and shows how the left-eye and right-eye captured images PL and PR are displayed on the monitor display.
- 12A shows the state before adjustment
- FIG. 12B shows the state after adjustment.
- the left half image data of the left-eye photographed image PL is reduced to half the monitor size by image processing in the camera processing unit 200, and the monitor display screen is displayed.
- the image data is displayed on the left side
- the right half of the right-eye photographed image PR is reduced to one half of the monitor size and displayed on the right side on the screen. Due to the optical axis shift, the left-eye and right-eye shot images PL and PR have a vertical shift.
- the deviation is conspicuous on both sides of the boundary line (vertical center line) between the left display area and the right display area.
- the operator eliminates the misalignment by performing the operation on the operation panel 54 while checking the vertical direction of the left-eye and right-eye photographed images PL and PR while viewing the screen.
- Perform manual adjustment of That is, the operator manually operates so that the subject image AL shifted upward (in this case, the left half image PL) is shifted downward.
- the vertical adjustment is performed so that the left half image and the right half image are continuous, and the subject image located on the center line is vertical. Check the direction balance.
- the body of one subject person image (children's image) is divided into a left half body and a right half body, and the left half body image and the right half body image are shifted to a state where they are cut off in the vertical direction. Align the left and right body images exactly by manual operation. Such adjustment is relatively easy to understand because it balances the left and right sides of one person including the face. It should be noted that adjustment methods such as adjustment with reference to the horizontal line or with reference to a distant landscape are also considered outdoors.
- the CPU 41 obtains the number of lines corresponding to the operation amount, further converts the number of lines into time to calculate the delay amount ⁇ t, and the delay amount ⁇ t is transmitted to the preprocessing unit 44.
- the frame synchronous drive control unit 3 sets the given delay amount ⁇ t to the corresponding image sensor (image sensor with optical axis misalignment or image misalignment) of the two image sensors 2L and 2R.
- the face regions KL and KR are detected in the left-eye and right-eye photographed images PL and PR, respectively.
- the points of interest QL and QR are extracted in the detected face areas KL and KR.
- the attention points QL and QR the eyes, mouth, ears, nose and the like can be considered.
- the Y coordinates of the extracted attention points QL and QR are obtained, and the difference between the two Y coordinates is defined as the vertical deviation amount ⁇ h.
- the vertical shift amount ⁇ h is converted into a delay amount ⁇ t, and the delay amount ⁇ t is sent to the frame synchronous drive control unit 3.
- the output timing of the read start pulse SP of the image sensor whose object image framing position is on the upper side is delayed by the delay amount ⁇ t.
- the delay amount ⁇ t As a result, as shown in FIG. 13B, left-eye and right-eye photographed images PL and PR that are matched as parallax images are obtained.
- FIG. 14 is a flowchart showing the operation of the stereoscopic image capturing apparatus having the above configuration.
- the subject of this control is the CPU 41 and its peripheral part in the camera processing unit 200.
- the state of processing does not need to be displayed on the monitor display. However, it may be projected.
- step S1 shooting with adjustment of vertical shift is started.
- step S2 the left-eye photographic image PL and the right-eye photographic image PR acquired by the left-eye and right-eye image sensors 2L and 2R are read out.
- step S3 initial trimming areas TL and TR are set at the same positions in the effective pixel areas of the left-eye and right-eye image sensors 2L and 2R in the left-eye and right-eye captured images PL and PR.
- step S4 face area detection is performed on the left-eye and right-eye subject images AL and AR in the left-eye and right-eye trimming areas TL and TR.
- step S5 the amount of vertical deviation ⁇ h between the points of interest QL and QR in the detected face regions KL and KR is detected.
- step S6 it is confirmed whether or not there is a vertical shift. If there is a deviation, the process proceeds to step S7, and if there is no deviation, the process proceeds to step S8. When there is a vertical shift and the process proceeds to step S7, the trimming areas TL and TR are adjusted, and the process returns to step S4.
- FIG. 15 shows an intermediate stage of adjustment of the trimming areas TL and TR.
- step S4 of the face area detection described above face area detection may be performed on the original image read from the image sensor, or the face is reduced on the image reduced to an arbitrary size by the resizing process. Region detection may be performed. As long as the conditions for making the resizing ratios in the left-eye and right-eye photographed images PL and PR remain the same, the resizing ratio is arbitrary. When calculating the number of lines of vertical deviation, the number of detected lines may be multiplied by the inverse of the resizing ratio.
- step S8 it is determined whether the electronic shutter of the image sensor is a rolling shutter type or a global shutter type. If it is a rolling shutter type, the process proceeds to step S10, and if not, the process proceeds to step S11.
- step S10 which has been advanced as the rolling shutter type, the timing of the vertical synchronization signal generated from the synchronization reference signal common to the left-eye and right-eye image sensors 2L and 2R is determined based on the number of deviation lines previously determined. The intervening delay amount ⁇ t is sent to the frame synchronous drive control unit 3.
- the electronic shutter start pulse ESSP and the read start pulse SP are output at a timing delayed by the delay amount ⁇ t from the standard in the corresponding image sensor (image sensor having an optical axis shift or image shift). .
- a display / recording parallax image can be obtained in which synchronism of shooting without spatially vertical deviation between the left-eye and right-eye images is ensured, and a high-quality stereoscopic image can be obtained. It becomes.
- the horizontal shift is a parallax when a person stereoscopically views, and is related to a characteristic called a baseline length in stereoscopic image production.
- the shift is recognized as a shift in the sense of depth of the stereoscopic image.
- the baseline length is also a parameter that is actively fine-tuned in conjunction with the setting of the sense of distance from the subject and the adjustment of the convergence angle.
- the present invention does not mention the details of the horizontal automatic adjustment.
- ⁇ Variable lens / zoom ratio When the zoom ratio by the variable power lenses 11 and 21 changes, the delay amount ⁇ t of the read start pulse SP in the frame synchronous drive control unit 3 varies in proportion to the zoom ratio. In particular, when the enlargement zoom is performed, the deviation of the exposure timing increases. Therefore, it is necessary to adjust the delay amount ⁇ t of the read start pulse SP in conjunction with the zoom ratio.
- the amount of change per side per n times is ⁇ n ( ⁇ n means the square root of n), and the amount of change per side at the magnification b is ⁇ b.
- Lb [(Lt ⁇ Lw) / ⁇ n] ⁇ ⁇ b + Lw It becomes. This is the number of shift amount lines corresponding to the magnification b, and the drive timing delay amount ⁇ t is calculated from this and fed back. This delay amount ⁇ t is set in the image sensor on the side of the object image upward displacement by the optical axis downward displacement. As a result, regardless of the zoom ratio change of the zoom lens, display / recording is ensured in which there is no spatial vertical deviation between the left-eye and right-eye captured images PL and PR. Thus, it becomes possible to obtain a high-quality stereoscopic image.
- the subsequent camera processing unit 200 sets the trimming areas TL and TR in all the imaging areas of the left-eye and right-eye image sensors 2L and 2R, respectively, and performs stereoscopic image processing. It is assumed that image processing is performed as data.
- a camera system as shown in FIG. 16 can be considered.
- This is a hybrid photographing system including a stereoscopic image photographing adapter.
- This hybrid imaging system is composed of a combination of a stereo camera SC for stereoscopic image shooting and a non-stereo camera NS that is a general camera for normal shooting that is not for stereoscopic image shooting.
- the stereo camera SC includes a stereoscopic image capturing unit 100, a data communication unit 150, and a camera processing unit 200.
- the stereoscopic image capturing unit 100 and the camera processing unit 200 are electrically connected via a data communication unit 150 formed of a cable.
- Imaging signals output from the left-eye and right-eye imaging units 1L and 1R in the stereoscopic image imaging unit 100 are configured to be input to the camera processing unit 200 via the data communication unit 150.
- the camera processing unit 200 is configured to receive image signals acquired by the left-eye and right-eye imaging units 1L and 1R and perform image processing. This image processing includes recording of image data.
- the camera processing unit 200 is configured to be capable of driving the left-eye and right-eye imaging units 1L and 1R.
- the stereoscopic image capturing unit 100 includes a left-eye image capturing unit 1L and a right-eye image capturing unit 1R that receive a subject light beam independently on the left and right sides, and a lens hood 30 on which the image capturing units 1L and 1R are mounted.
- the non-stereo camera NS includes a camera main body 300 and a lens barrel 400.
- the lens barrel 400 is detachably attached to the mount portion of the camera body 300.
- the lens hood 30 is configured to be attachable to the tip of the lens barrel 400 of the non-stereo camera NS, and the camera processing unit 200 is configured to be attachable to the hot shoe 310 of the camera main body 300.
- the non-stereo camera NS and the stereo camera SC can exchange signals via the hot shoe 310.
- the other end of the lens barrel 400 is formed with a mount for combining with the camera body 300.
- the stereoscopic image capturing unit 100 that is detachably attached to the lens barrel 400 of the non-stereo camera NS in the lens hood 30 is configured such that the attachment opens the front of the lens barrel 400.
- the stereoscopic image capturing unit 100, the data communication unit 150, and the camera processing unit 200 configure a stereoscopic image capturing adapter, and the stereoscopic image capturing adapter configures a stereo camera SC.
- an optical lens In the left-eye and right-eye imaging units 1L and 1R, an optical lens, an iris for adjusting the amount of incident light, a mechanical shutter, an optical low-pass filter, and an image sensor are arranged in this order.
- the motor that moves the lens holding frame in the optical axis direction includes an autofocus motor (AF motor) that moves the focusing lens and a zoom motor that moves the zoom lens.
- AF motor autofocus motor
- zoom lens motor zoom lens motor
- iris motor The drive is controlled by a motor controller in the camera processing unit 200.
- the image sensor is arranged perpendicular to the optical axis of the photographing lens.
- the sub-circuit board of the stereoscopic image capturing unit 100 having the left-eye and right-eye image capturing units 1L and 1R is connected to the main circuit substrate of the camera processing unit 200 via the data communication unit 150. These contacts and terminals on the circuit board enable wired communication together with a serial interface described later.
- a plurality of contact points are provided on the front mounting portion of the stereoscopic image capturing unit 100 and the lens barrel 400, and the mounting portions on the lens barrel 400 side and the camera body 300 side are the same.
- a terminal (not shown) that contacts the connection terminal is provided.
- the main circuit board of the camera processing unit 200 and the hot shoe 310 of the camera body 300 also have a configuration in which a plurality of contacts (not shown) are provided on both.
- connection method there is a form in which the sub circuit board of the stereoscopic image capturing unit 100 and the main circuit board of the camera processing unit 200 are connected by an optical fiber by providing a light transmission / reception terminal. These optical contacts and terminals connected via an optical cable enable optical communication together with a serial interface described later.
- an appropriate item is selected from a liquid crystal panel for displaying a through image or a menu image of a subject, a display button for switching the liquid crystal panel ON / OFF, and a menu image displayed on the liquid crystal panel.
- a cross operation button to be operated at the time of selection, an execution button to be pressed when displaying or confirming a menu screen, and the like (not shown) are provided.
- an appropriate item is selected from a liquid crystal panel for displaying a through image of a subject and a menu image, a display button for switching the liquid crystal panel on and off, and a menu image displayed on the liquid crystal panel.
- a display button for switching the liquid crystal panel on and off
- a menu image displayed on the liquid crystal panel There is a form including a cross operation button that is operated when selecting, an execution button that is pressed when a menu screen is displayed or confirmed, and the like (not shown).
- the camera body 300 can be sequentially switched to a still image shooting mode, a moving image shooting mode, a night shooting mode, a flash shooting mode, and the like.
- the stereoscopic image capturing unit 100 can be operated from the camera main body unit 300 or can be operated from the camera processing unit 200.
- the stereoscopic image capturing unit 100 is detachably attached to the front end of the lens barrel 400.
- the stereoscopic image capturing unit 100 is attached to the front end portion of the lens barrel 400 with high accuracy via a bayonet mechanism.
- the bayonet mechanism is a socket-type simple joining mechanism that connects and disengages by inserting a claw into a groove and twisting it.
- a connection terminal is provided on the outer periphery of the front end of the lens barrel 400, and an output terminal (transmitter) of the stereoscopic image capturing unit 100 is in contact with the connection terminal.
- the transmitter may be a wireless communication interface having a modulation unit that modulates data to form a transmission signal and an antenna that transmits the transmission signal, instead of the output terminal.
- the stereoscopic image capturing unit 100 and the camera processing unit 200 are mounted on the non-stereo camera NS, it is possible to simultaneously perform stereoscopic image shooting with the stereo camera SC and normal shooting with the non-stereo camera NS. When not mounted, normal shooting is performed by the non-stereo camera NS.
- the stereoscopic image capturing adapter of this embodiment can be used alone as a camera dedicated to stereoscopic image capturing.
- FIG. 1 An example of a specific configuration of the stereo camera SC including the stereoscopic image capturing unit 100 and the camera processing unit 200 is shown in the block diagram of FIG. Another form is shown in FIG.
- the non-stereo camera NS held by the photographer is a single-lens camera with interchangeable lenses, and is in the form of an interchangeable lens type stereo camera SC * in which the right-eye imaging unit 1R and the left-eye imaging unit 1L are combined. It is possible to take a stereoscopic image with the interchangeable lens type stereo camera SC * SC mounted on the lens mount of the non-stereo camera NS.
- the stereoscopic image capturing apparatus of the present invention in the moving image capturing mode, in the case of the MOS sensor for the rolling shutter operation, when performing the high-speed electronic shutter for several line period exposure, 2 in the vertical direction. Even if the relative optical axes of the two image pickup units are deviated, a stereoscopic moving image shot with appropriate exposure for the same time can be obtained.
- the border is not blurred, and the horizontal parallax can be easily adjusted by image processing, resulting in a sharp stereoscopic effect It is done.
- mounting variations of a plurality of imaging units can be allowed by digital processing, and the lens barrel adjustment mechanism can be simplified and the cost can be reduced.
- this invention is not limited only to said each Example, In the range which does not change a summary, it can implement suitably.
- the present invention provides a stereoscopic image capturing apparatus, a stereoscopic image capturing system, and the like that can perform simultaneous imaging of the left-eye and right-eye imaging units through electronic timing adjustment regardless of the vertical optical axis shift of the left-eye and right-eye imaging units. And display / recording parallax images in which synchronism between the left-eye and right-eye captured images without spatially vertical deviation is ensured, and a high-quality stereoscopic image is obtained. Useful as technology.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Studio Devices (AREA)
- Stereoscopic And Panoramic Photography (AREA)
Abstract
Description
左目用イメージセンサ2Lを有する左目用撮像部1Lと、
右目用イメージセンサ2Rを有する右目用撮像部1Rと、
前記左目用、右目用イメージセンサ2L、2Rをフレーム同期して駆動制御するフレーム同期駆動制御部3と、
前記左目用撮像部1Lで取得される左目用撮影画像PLと、前記右目用撮像部1Rで取得される右目用撮影画像PRとに画像処理を行って、前記左目用撮影画像PLと前記右目用撮影画像PRとの視差画像を生成するカメラ処理部4と、
前記左目用撮影画像PLにおける被写体像のフレーミング位置と前記右目用撮影画像PRにおける被写体像のフレーミング位置との間の垂直方向ずれ量がゼロに近づくように、前記フレーム同期駆動制御部3において前記左目用、右目用イメージセンサ2L、2Rの駆動タイミングを調整する垂直タイミング調整器5と、
を備える。
左目用イメージセンサ2Lを有する左目用撮像部1Lで得られる左目用撮影画像PLにおける被写体像のフレーミング位置と、右目用イメージセンサ2Rを有する右目用撮像部1Rで得られる右目用撮影画像PRにおける被写体像のフレーミング位置との間の垂直方向ずれ量を求める第1のステップと、
前記垂直方向ずれ量Δhに基づいて、前記左目用、右目用イメージセンサ2L、2Rの駆動タイミングを調整する第2のステップと、
調整後の前記駆動タイミングに従って駆動する前記左目用、右目用イメージセンサ2L、2Rで取得された露光終了後の画像データをライン単位で順次に読み出す第3のステップと、
を含む。
前記駆動タイミングの調整を前記垂直タイミング調整器5に指示するためのずれ調整指示信号を、当該立体画像撮影装置の操作者による手動操作に基づいて生成する操作器を、
さらに有し、
前記垂直タイミング調整器5は、前記ずれ調整指示信号に基づいて前記駆動タイミングを前記フレーム同期駆動制御部3において調整するように構成されている。
垂直タイミング調整器5は、
前記左目用撮影画像PLにおける左側注目ポイントQLと前記右目用撮影画像PRにおける右側注目ポイントQRとをそれぞれ抽出したうえで、前記左側、右側注目ポイントQL,QRそれぞれの垂直方向座標を割り出す左右の注目ポイント座標算出器7L,7Rと、
前記左側注目ポイントQLの垂直方向座標YLと前記右側注目ポイントQRの垂直方向座標YRとの差分から前記垂直方向ずれ量Δhを算出する垂直方向ずれ量算出器8と、
垂直方向ずれ量算出器8による垂直方向ずれ量Δhを時間を単位とする遅延量Δtに換算する遅延量算出器9と、
前記駆動タイミングの遅延量Δtを加味した駆動タイミングをフレーム同期駆動制御部3に設定する駆動タイミング設定器10と、
を備えている、という態様である。
前記第1のステップは、
前記左目用撮影画像PLと前記右側撮像画像PRとを同一画面において左右に並べて表示する第1-1のステップと、
前記同一画面に表示された前記左目用、右目用撮影画像PL、PRそれぞれにおいて、同時刻の同一被写体の相対位置関係が前記左目用撮影画像PLと前記右目用撮影画像PRとで同じになるように設定されたトリミングエリアTL,TRを設定する第1-2のステップと、
前記左目用、右目用撮影画像それぞれの前記トリミングエリアTL,TRにおいて注目ポイントQL,QRを抽出する第1-3のステップと、
前記左目用撮影画像PLの前記注目ポイントQLと前記右目用撮影画像PRの前記注目ポイントQRとの間で前記垂直方向ずれ量Δhを検出する第1-4のステップと、
を備えている、という態様である。
前記第1-4のステップは、前記左目用撮影画像PLの前記トリミングエリアTLにおける前記注目ポイントQLが所属するラインの番号と、前記右目用撮影画像PRの前記トリミングエリアTRにおける前記注目ポイントQRが所属するラインのライン番号との間の差分を算出することで前記垂直方向ずれ量Δhを検出する、という態様である。
前記左目用、右目用撮影画像QL、PRにおける被写体の顔領域を検出する顔領域検出器をさらに備え、
前記注目ポイント座標算出器7L,7Rは、前記顔領域検出器によって検出された前記顔領域内の特定部位を前記左側、右側注目ポイントQL、PRに設定する、のが好ましい1態様である。
前記左目用、右目用撮像部1L、1Rは、変倍レンズを有する光学系をそれぞれ備え、前記左目用撮像部1Lの前記変倍レンズと、前記右目用撮像部1Rの前記変倍レンズとは、互いに連動するように構成されている、という態様である。
√n:n倍当たりの1辺の変化量
√b:倍率bでの1辺の変化量
と考えて(√nはnの平方根を意味する)、
Lb =[(Lt -Lw )/√n]×√b +Lw
となる。これは倍率bに対応したずれ量ライン数であるが、さらにこれから上記の駆動タイミングの遅延量Δtを算出し、フィードバックする。
前記左目用、右目用撮影画像PL、PRの撮影を開始する第4のステップと、
前記左目用、右目用撮影画像PL、PRを読み出す第5のステップと、
前記左目用、右目用撮影画像PL、PRにおいてトリミングエリアTL,TRをそれぞれ初期設定する第6のステップと、
前記左目用、右目用撮影画像PL、PRのトリミングエリアTL,TRそれぞれにおいて注目ポイントQL、QRを抽出する第7のステップと、
前記左目用撮影画像PLの前記注目ポイントQLと、前記右目用撮影画像PRの前記注目ポイントQRとの間で垂直方向ずれの有無を確認する第8のステップと、
前記垂直方向ずれがあるときに、前記左目用撮影画像PLのトリミングエリアTLと前記右目用撮影画像PRのトリミングエリアTRとのうちの一方の位置を垂直方向で調整する第9のステップと、
前記垂直方向ずれのずれ量Δhがゼロに収束するまで、前記第7の~第9のステップを繰り返す第10のステップと、
前記垂直方向ずれのずれ量Δhがゼロになったときの前記第9のステップにおける前記一方のトリミングエリアの垂直方向の調整量を、前記第8のステップで検出した前記垂直方向ずれのずれ量Δhに対応する駆動タイミングの遅延量として保存する第11のステップと、
さらに含む。
前記左目用、右目用イメージセンサ2L、2Rが有する電子シャッタがローリングシャッタであるか否かを判定する第12のステップと、
前記第12のステップで前記電子シャッタが前記ローリングシャッタであると判定すると、前記ずれ量Δhに対応した駆動タイミングの遅延量を、前記第12のステップにおいて前記電子シャッタが前記ローリングシャッタであると判定した前記左目用、右目用イメージセンサ2L、2Rに設定する第13のステップと、
さらに含む。
Δt=T2-T1
である。
Δt+T1=T2
であり、右目用撮影画像PRにおけるお父さん像の帽子の頂点の読み出しタイミングT2と同じタイミングとなる。
T2′=T2+Δtα
になったと考えられる。これと連動して、遅延量Δt′も静止状態の場合のΔtが、
Δt′=Δt+Δtα
になる。上式の(Δt+T1=T2)にあてはめ、ΔtをΔt′で置き換えると、
Δt′+T1=Δt+Δtα+T1
=Δt+(T2′-T2)+T1
=(T2-T1)+(T2′-T2)+T1
=T2′
つまり、
Δt′+T1=T2′
であり、静止状態の場合と同様に、右目用撮影画像PRにおけるお父さん像の帽子の頂点の読み出しタイミングT2′と同じタイミングとなる。
上記では、左目用、右目用撮影画像PL,PRの垂直方向ずれの確認の仕方については、特に詳しくは説明していなかった。そこで、以下では、左目用、右目用撮影画像PL,PRの垂直方向ずれの確認の具体例を説明する。調整はマニュアル方式である。
図13の(a)に示すように、左目用、右目用撮影画像PL,PRそれぞれにおいて顔領域KL,KRの検出を行う。検出した顔領域KL,KR内で注目ポイントQL,QRの抽出を行う。注目ポイントQL,QRとしては、眼や口や耳や鼻などが考えられる。抽出した注目ポイントQL,QRのY座標を求め、両Y座標の差分を垂直方向ずれ量Δhとする。垂直方向ずれ量Δhを遅延量Δtに換算し、遅延量Δtをフレーム同期駆動制御部3に送出する。遅延量Δtに基づいて被写体像フレーミング位置が上方側のイメージセンサの読み出しスタートパルスSPの出力タイミングを遅延量Δtだけ遅らせる。結果として、図13の(b)に示すように、視差画像としてマッチングのとれた左目用、右目用撮影画像PL,PRが得られる。
次に、撮影者の操作を介さず自動調整する方法について説明する。図14は上記構成の立体画像撮影装置の動作を示すフローチャートである。この制御の主体は、カメラ処理部200におけるCPU41およびその周辺部分である。処理の様子はモニタディスプレイに映出する必要はない。ただし、映出してもよい。
変倍レンズ11,21によるズーム比率が変化する場合、フレーム同期駆動制御部3での読み出しスタートパルスSPの遅延量Δtは、ズーム比率に比例して変動する。特に拡大ズームを実施した場合は、露光タイミングのずれが増大する。したがって、ズーム比率に連動して読み出しスタートパルスSPの遅延量Δtを調整する必要がある。
Lb =[(Lt -Lw )/√n]×√b +Lw
となる。これは倍率bに対応したずれ量ライン数であるが、さらにこれから上記の駆動タイミングの遅延量Δtを算出し、フィードバックする。この遅延量Δtは、光軸下方ずれで被写体像上方ずれの側のイメージセンサに設定されることになる。これにより、変倍レンズのズーム比率変化の如何にかかわりなく、左目用、右目用撮影画像PL,PR間で空間的に垂直方向のずれのない撮影の同時性が確保された表示用/記録用の視差画像が得られ、良品質な立体画像を得ることが可能となる。
カメラシステムとしては、図16に示すようなものが考えられる。これは、立体画像撮影用アダプタを含むハイブリッド撮影システムとなっている。このハイブリッド撮影システムは、立体画像撮影用のステレオカメラSCと、立体画像撮影用ではない通常撮影用の一般的なカメラである非ステレオカメラNSとの組み合わせから構成されている。
1R 右目用撮像部
2L 左目用イメージセンサ
2R 右目用イメージセンサ
3 フレーム同期駆動制御部
4 カメラ処理部
5 垂直タイミング調整器
6 操作器
7L,7R 注目ポイント座標算出器
8 垂直方向ずれ量算出器
9 遅延量算出器
10 駆動タイミング設定器
11,21 変倍レンズ
41 CPU
44 前処理部
45 メモリ制御部
46 画像メモリ
50 顔領域検出器
53 モニタインターフェース
54 操作パネル
100 立体画像撮像部
200 カメラ処理部
PL 左目用撮影画像
PR 右目用撮影画像
AL 左目側被写体像
AR 右目側被写体像
TL 左目用トリミングエリア
TR 右目用トリミングエリア
PTL 左目用トリミング画像
PTR 右目用トリミング画像
SP 読み出しスタートパルス
ESSP 電子シャッタスタートパルス(シャッタゲートパルス)
KL,KR 検出顔領域
QL,QR 注目ポイント
Δh 垂直方向ずれ量
Δt 遅延量
Claims (24)
- 左目用イメージセンサを有する左目用撮像部と、
右目用イメージセンサを有する右目用撮像部と、
前記左目用、右目用イメージセンサをフレーム同期して駆動制御するフレーム同期駆動制御部と、
前記左目用撮像部で取得される左目用撮影画像と、前記右目用撮像部で取得される右目用撮影画像とに画像処理を行って、前記左目用撮影画像と前記右目用撮影画像との視差画像を生成するカメラ処理部と、
前記左目用撮影画像における被写体像のフレーミング位置と前記右目用撮影画像における被写体像のフレーミング位置との間の垂直方向ずれ量がゼロに近づくように、前記フレーム同期駆動制御部において前記左目用、右目用イメージセンサの駆動タイミングを調整する垂直タイミング調整器と、
を備える立体画像撮影装置。 - 前記垂直タイミング調整器は、前記視差画像に基づいて、同時刻の同一被写体の相対位置関係が前記左目用撮影画像と前記右目用撮影画像とで同じになるように設定された左右一対のトリミングエリアの間のずれ量を求め、求めたずれ量に対応する駆動タイミングの遅延量を前記フレーム同期駆動制御部に設定するように構成されている、
請求項1に記載の立体画像撮影装置。 - 前記左目用、右目用イメージセンサは、露光制御のための電子シャッタとしてのローリングシャッタを有するMOS(Metal Oxide Semiconductor)イメージセンサである、
請求項1に記載の立体画像撮影装置。 - 前記駆動タイミングは、垂直同期信号の出力タイミングを含み、
前記垂直タイミング調整器は、前記左側、右目用撮影画像のうちで被写体像フレーミング位置が相対的に一方より上方側に位置する他方を取得する前記左目用イメージセンサまたは前記右目用イメージセンサにおいて、前記垂直同期信号の出力タイミングを、同期基準信号のタイミングから、前記垂直方向ずれ量に対応したライン数に相当する垂直走査期間分だけ遅延させるように構成されている、
請求項1に記載の立体画像撮影装置。 - 前記駆動タイミングは、前記ローリングシャッタのシャッタパルスと読み出しスタートパルスとの出力タイミングとを含み、
前記垂直タイミング調整器は、前記シャッタパルスと前記読み出しスタートパルスとを、前記左目用、右目用イメージセンサの垂直同期信号の遅延に並行して同一時間遅延させるように構成されている、
請求項3に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、前記垂直方向ずれ量をライン数ずれ量に換算したうえで、前記左目用、右目用撮影画像のうちで被写体像フレーミング位置が相対的に一方より上方側に位置する他方を取得する前記左目用イメージセンサまたは前記右目用イメージセンサに、前記ライン数ずれ量を前記駆動タイミングの遅延量として設定するように構成されている、
請求項1に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、前記駆動タイミングの調整量を、当該立体画像撮影装置の操作者による手動操作で前記フレーム同期駆動制御部に送出可能に構成されている、
請求項1に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、前記駆動タイミングの調整量を、自動的に前記フレーム同期駆動制御部に送出可能に構成されている、
請求項1に記載の立体画像撮影装置。 - 前記駆動タイミングの調整を前記垂直タイミング調整器に指示するためのずれ調整指示信号を、当該立体画像撮影装置の操作者による手動操作に基づいて生成する操作器を、
さらに有し、
前記垂直タイミング調整器は、前記ずれ調整指示信号に基づいて前記駆動タイミングを前記フレーム同期駆動制御部において調整する、
請求項1に記載の立体画像撮影装置。 - 前記第1、第2の撮影画像を表示するモニタをさらに備え、
前記カメラ処理部は、同時刻の同一被写体の相対位置関係が左右で同じになるように設定された左右一対のトリミングエリアに対応する左右一対のトリミング画像をモニタサイズに縮小リサイズし、縮小リサイズした前記左右一対のトリミング画像を前記モニタ上で左側と右側に並べて表示する機能を有している、
請求項9に記載の立体画像撮影装置。 - 前記第1、第2の撮影画像を表示するモニタをさらに備え、
前記カメラ処理部は、同時刻の同一被写体の相対位置関係が左右で同じになるように設定された左右一対のトリミングエリアのうちで左側トリミングエリアに対応する左側トリミング画像の左側半分に位置する左側1/2画像をモニタサイズの2分の1に縮小リサイズし、縮小リサイズした前記左側1/2画像を前記モニタ上で左側に表示するとともに、前記左右一対のトリミングエリアのうちで右側トリミングエリアに対応する右側トリミング画像の右側半分に位置する右側1/2画像をモニタサイズの2分の1に縮小リサイズし、縮小リサイズした前記右側1/2画像を前記モニタ上で右側に表示する機能を有している、
請求項9に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、前記駆動タイミングの遅延量として、前記左目用トリミングエリアのスタートラインの位置情報と前記右目用トリミングエリアのスタートラインの位置情報との差分を時間に換算して得られる遅延量を用いる、
請求項10に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、前記駆動タイミングの遅延量として、前記左目用トリミングエリアのスタートラインの位置情報と前記右目用トリミングエリアのスタートラインの位置情報との差分を時間に換算して得られる遅延量を用いる、
請求項11に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、
前記左目用撮影画像における左側注目ポイントと前記右目用撮影画像における右側注目ポイントとをそれぞれ抽出したうえで、前記左側、右側注目ポイントそれぞれの垂直方向座標を割り出す注目ポイント座標算出器と、
前記左側注目ポイントの垂直方向座標と前記右側注目ポイントの垂直方向座標との差分から前記垂直方向ずれ量を算出する垂直方向ずれ量算出器と、
前記垂直方向ずれ量を時間を単位とする遅延量に換算する遅延量算出器と、
前記遅延量を加味した前記駆動タイミングを前記フレーム同期駆動制御部に設定する駆動タイミング設定器と、
を備えている、
請求項1に記載の立体画像撮影装置。 - 前記左目用、右目用撮影画像における被写体の顔領域を検出する顔領域検出器をさらに備え、
前記注目ポイント座標算出器は、前記顔領域検出器によって検出された前記顔領域内の特定部位を前記左側、右側注目ポイントに設定する、
請求項14に記載の立体画像撮影装置。 - 前記左目用、右目用撮像部は、変倍レンズを有する光学系をそれぞれ備え、前記左目用撮像部の前記変倍レンズと、前記右目用撮像部の前記変倍レンズとは、互いに連動するように構成されている、
請求項1に記載の立体画像撮影装置。 - 前記垂直タイミング調整器は、前記変倍レンズのズーム比率の変化に応じて前記駆動タイミングの遅延量を可変する、
請求項16に記載の立体画像撮影装置。 - 左目用イメージセンサを有する左目用撮像部で得られる左目用撮影画像における被写体像のフレーミング位置と、右目用イメージセンサを有する右目用撮像部で得られる右目用撮影画像における被写体像のフレーミング位置との間の垂直方向ずれ量を求める第1のステップと、
前記垂直方向ずれ量に基づいて、前記左目用、右目用イメージセンサの駆動タイミングを調整する第2のステップと、
調整後の前記駆動タイミングに従って駆動する前記左目用、右目用イメージセンサで取得された露光終了後の画像データをライン単位で順次に読み出す第3のステップと、
を含む立体画像撮影方法。 - 前記第1のステップは、
前記左目用撮影画像と前記右側撮像画像とを同一画面において左右に並べて表示する第1-1のステップと、
前記同一画面に表示された前記左目用、右目用撮影画像それぞれにおいて、同時刻の同一被写体の相対位置関係が前記左目用撮影画像と前記右目用撮影画像で同じになるように設定されたトリミングエリアを設定する第1-2のステップと、
前記左目用、右目用撮影画像それぞれの前記トリミングエリアにおいて注目ポイントを抽出する第1-3のステップと、
前記左目用撮影画像の前記注目ポイントと前記右目用撮影画像の前記注目ポイントとの間で前記垂直方向ずれ量を検出する第1-4のステップと、
を含む、請求項18に記載の立体画像撮影方法。 - 前記第1-4のステップは、前記左目用撮影画像の前記トリミングエリアと、前記右目用撮影画像の前記トリミングエリアとのうちの一方の位置を、他方の位置に合わせ込むために、当該一方を垂直方向にシフトする際のシフト量を検出することで前記垂直方向ずれ量を検出する、
請求項19に記載の立体画像撮影方法。 - 前記第1-4のステップは、前記左目用撮影画像の前記トリミングエリアにおける前記注目ポイントが所属するラインのライン番号と、前記右目用撮影画像の前記トリミングエリアにおける前記注目ポイントが所属するラインのライン番号との間の差分を算出することで前記垂直方向ずれ量を検出する、
請求項18に記載の立体画像撮影方法。 - 前記注目ポイントとして、前記左目用、右目用撮影画像における被写体の顔領域の特定部位を用いる、
請求項19に記載の立体画像撮影方法。 - 前記左目用、右目用撮影画像の撮影を開始する第4のステップと、
前記左目用、右目用撮影画像を読み出す第5のステップと、
前記左目用、右目用撮影画像においてトリミングエリアをそれぞれ初期設定する第6のステップと、
前記左目用、右目用撮影画像のトリミングエリアそれぞれにおいて注目ポイントを抽出する第7のステップと、
前記左目用撮影画像の前記注目ポイントと、前記右目用撮影画像の前記注目ポイントとの間で垂直方向ずれの有無を確認する第8のステップと、
前記垂直方向ずれがあるときに、前記左目用撮影画像のトリミングエリアと前記右目用撮影画像のトリミングエリアとのうちの一方の位置を垂直方向で調整する第9のステップと、
前記垂直方向ずれのずれ量がゼロに収束するまで、前記第7の~第9のステップを繰り返す第10のステップと、
前記垂直方向ずれのずれ量がゼロになったときの前記第9のステップにおける前記一方のトリミングエリアの垂直方向の調整量を、前記第8のステップで検出した前記垂直方向ずれのずれ量に対応する駆動タイミングの遅延量として保存する第11のステップと、
さらに含む、
請求項18に記載の立体画像撮影方法。 - 前記左目用、右目用イメージセンサが有する電子シャッタがローリングシャッタであるか否かを判定する第12のステップと、
前記第12のステップで前記電子シャッタが前記ローリングシャッタであると判定すると、前記ずれ量に対応した駆動タイミングの遅延量を、前記第12のステップにおいて前記電子シャッタが前記ローリングシャッタであると判定した前記左目用、右目用イメージセンサに設定する第13のステップと、
さらに含む請求項23に記載の立体画像撮影方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2011800198816A CN102860016A (zh) | 2010-04-19 | 2011-03-24 | 立体图像拍摄装置以及立体图像拍摄方法 |
JP2012511522A JP5683025B2 (ja) | 2010-04-19 | 2011-03-24 | 立体画像撮影装置および立体画像撮影方法 |
US13/621,104 US9304388B2 (en) | 2010-04-19 | 2012-09-15 | Three-dimensional imaging device and three-dimensional imaging method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010-095732 | 2010-04-19 | ||
JP2010095732 | 2010-04-19 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/621,104 Continuation US9304388B2 (en) | 2010-04-19 | 2012-09-15 | Three-dimensional imaging device and three-dimensional imaging method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011132364A1 true WO2011132364A1 (ja) | 2011-10-27 |
Family
ID=44833909
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/001726 WO2011132364A1 (ja) | 2010-04-19 | 2011-03-24 | 立体画像撮影装置および立体画像撮影方法 |
Country Status (4)
Country | Link |
---|---|
US (1) | US9304388B2 (ja) |
JP (1) | JP5683025B2 (ja) |
CN (1) | CN102860016A (ja) |
WO (1) | WO2011132364A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019146510A1 (ja) * | 2018-01-26 | 2019-08-01 | 日立オートモティブシステムズ株式会社 | 画像処理装置 |
CN112203076A (zh) * | 2020-09-16 | 2021-01-08 | 青岛小鸟看看科技有限公司 | Vr系统中多摄像头曝光中心点的对齐方法、系统 |
US11546570B2 (en) | 2019-02-18 | 2023-01-03 | Canon Kabushiki Kaisha | Apparatus, apparatus control method, and recording medium, for synchronizing a plurality of imaging devices |
Families Citing this family (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2254091B1 (en) * | 2009-05-19 | 2020-03-25 | Veoneer Sweden AB | Vision system and method for a motor vehicle |
US9485495B2 (en) | 2010-08-09 | 2016-11-01 | Qualcomm Incorporated | Autofocus for stereo images |
KR101843450B1 (ko) * | 2011-08-23 | 2018-03-29 | 엘지전자 주식회사 | 이동 단말기 및 이동 단말기의 제어 방법 |
KR20130024504A (ko) * | 2011-08-31 | 2013-03-08 | 삼성전기주식회사 | 삼차원 카메라 시스템 및 주시각 조정 방법 |
US9438889B2 (en) | 2011-09-21 | 2016-09-06 | Qualcomm Incorporated | System and method for improving methods of manufacturing stereoscopic image sensors |
CN104247412B (zh) * | 2012-03-30 | 2016-08-24 | 富士胶片株式会社 | 图像处理装置、摄像装置、图像处理方法、记录介质以及程序 |
US9398264B2 (en) | 2012-10-19 | 2016-07-19 | Qualcomm Incorporated | Multi-camera system using folded optics |
WO2014111814A2 (en) | 2013-01-15 | 2014-07-24 | Mobileye Technologies Limited | Stereo assist with rolling shutters |
US9818040B2 (en) * | 2013-06-20 | 2017-11-14 | Thomson Licensing | Method and device for detecting an object |
US9955142B2 (en) * | 2013-07-05 | 2018-04-24 | Mediatek Inc. | On-line stereo camera calibration device and method for generating stereo camera parameters |
US10178373B2 (en) | 2013-08-16 | 2019-01-08 | Qualcomm Incorporated | Stereo yaw correction using autofocus feedback |
US8988509B1 (en) | 2014-03-20 | 2015-03-24 | Gopro, Inc. | Auto-alignment of image sensors in a multi-camera system |
US9197885B2 (en) | 2014-03-20 | 2015-11-24 | Gopro, Inc. | Target-less auto-alignment of image sensors in a multi-camera system |
EP3120542A4 (en) * | 2014-03-20 | 2017-03-01 | GoPro, Inc. | Auto-alignment of image sensors in a multi-camera system |
CN106416241B (zh) * | 2014-03-21 | 2019-07-26 | 欧姆龙株式会社 | 用于检测和减轻光学系统中的光学损伤的方法和装置 |
US9383550B2 (en) | 2014-04-04 | 2016-07-05 | Qualcomm Incorporated | Auto-focus in low-profile folded optics multi-camera system |
US9374516B2 (en) | 2014-04-04 | 2016-06-21 | Qualcomm Incorporated | Auto-focus in low-profile folded optics multi-camera system |
CN104977795A (zh) * | 2014-04-08 | 2015-10-14 | 立普思股份有限公司 | 立体摄影系统及其方法 |
US10013764B2 (en) | 2014-06-19 | 2018-07-03 | Qualcomm Incorporated | Local adaptive histogram equalization |
US9541740B2 (en) | 2014-06-20 | 2017-01-10 | Qualcomm Incorporated | Folded optic array camera using refractive prisms |
US9294672B2 (en) | 2014-06-20 | 2016-03-22 | Qualcomm Incorporated | Multi-camera system using folded optics free from parallax and tilt artifacts |
US9386222B2 (en) | 2014-06-20 | 2016-07-05 | Qualcomm Incorporated | Multi-camera system using folded optics free from parallax artifacts |
US9819863B2 (en) | 2014-06-20 | 2017-11-14 | Qualcomm Incorporated | Wide field of view array camera for hemispheric and spherical imaging |
US9549107B2 (en) | 2014-06-20 | 2017-01-17 | Qualcomm Incorporated | Autofocus for folded optic array cameras |
JP5847368B1 (ja) * | 2014-06-23 | 2016-01-20 | オリンパス株式会社 | 撮像装置および内視鏡装置 |
US9727967B2 (en) * | 2014-06-23 | 2017-08-08 | Samsung Electronics Co., Ltd. | Methods for determining estimated depth in an image and systems thereof |
US9832381B2 (en) | 2014-10-31 | 2017-11-28 | Qualcomm Incorporated | Optical image stabilization for thin cameras |
JP6307666B2 (ja) * | 2015-12-14 | 2018-04-04 | オリンパス株式会社 | 撮像装置 |
US20190349502A1 (en) * | 2016-11-29 | 2019-11-14 | Sony Corporation | Imaging device, imaging control method, and program |
US10498968B1 (en) | 2017-03-01 | 2019-12-03 | Altia Systems, Inc. | Intelligent zoom method and video system implementing same |
JP7000050B2 (ja) * | 2017-06-29 | 2022-01-19 | キヤノン株式会社 | 撮像制御装置及びその制御方法 |
JP2019114884A (ja) * | 2017-12-22 | 2019-07-11 | セイコーエプソン株式会社 | 検出装置、表示装置、検出方法 |
CN108259769B (zh) * | 2018-03-30 | 2020-08-14 | Oppo广东移动通信有限公司 | 图像处理方法、装置、存储介质及电子设备 |
CN108881737A (zh) * | 2018-07-06 | 2018-11-23 | 漳州高新区远见产业技术研究有限公司 | 一种应用于移动终端的vr成像方法 |
JP2020027957A (ja) * | 2018-08-09 | 2020-02-20 | オリンパス株式会社 | 画像処理装置、画像処理方法、及び画像処理プログラム |
CN109089103A (zh) * | 2018-10-24 | 2018-12-25 | 长沙智能驾驶研究院有限公司 | 双目相机姿态调整方法以及装置、计算机设备和存储介质 |
US10715783B1 (en) * | 2019-03-01 | 2020-07-14 | Adobe Inc. | Stereo-aware panorama conversion for immersive media |
DE102019207954B4 (de) * | 2019-05-29 | 2021-02-04 | Rodenstock Gmbh | Verfahren für ein Stereokamerasystem zur Erkennung von Abweichungen von einer Kalibrierung und Stereokamerasystem hierfür |
JP2020205498A (ja) * | 2019-06-14 | 2020-12-24 | マツダ株式会社 | 外部環境認識装置 |
CN110570368B (zh) * | 2019-08-21 | 2020-09-25 | 贝壳技术有限公司 | 深度图像的畸变矫正方法、装置、电子设备及存储介质 |
CN114450552A (zh) * | 2019-10-10 | 2022-05-06 | 松下知识产权经营株式会社 | 校正参数计算方法、位移量计算方法、校正参数计算装置及位移量计算装置 |
CN112326202B (zh) * | 2020-10-23 | 2022-12-09 | 歌尔光学科技有限公司 | 虚拟现实设备的双目视差测试方法、装置及工装 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0888868A (ja) * | 1994-09-20 | 1996-04-02 | Canon Inc | 複眼撮像装置 |
JPH1145334A (ja) * | 1997-05-29 | 1999-02-16 | Dainippon Screen Mfg Co Ltd | 画像の比較表示方法及び記録媒体 |
JP2004032244A (ja) * | 2002-06-25 | 2004-01-29 | Fuji Heavy Ind Ltd | ステレオ画像処理装置およびステレオ画像処理方法 |
JP2008252254A (ja) * | 2007-03-29 | 2008-10-16 | Fujifilm Corp | 立体撮影装置及び光軸調節方法 |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05130646A (ja) | 1991-11-08 | 1993-05-25 | Toshiba Corp | 立体カメラ装置 |
JP3266772B2 (ja) | 1995-11-10 | 2002-03-18 | 富士写真光機株式会社 | ズームレンズの駆動装置 |
JP4862129B2 (ja) * | 2005-12-02 | 2012-01-25 | 株式会社Jvcケンウッド | 車載カメラ装置を用いた運転支援装置 |
US20070165942A1 (en) * | 2006-01-18 | 2007-07-19 | Eastman Kodak Company | Method for rectifying stereoscopic display systems |
JP4867552B2 (ja) * | 2006-09-28 | 2012-02-01 | 株式会社ニコン | 撮像装置 |
JP2008061259A (ja) * | 2007-09-18 | 2008-03-13 | Fujifilm Corp | 撮影装置 |
JP2009094632A (ja) * | 2007-10-04 | 2009-04-30 | Panasonic Corp | 固体撮像装置 |
JP5109691B2 (ja) * | 2008-01-31 | 2012-12-26 | コニカミノルタホールディングス株式会社 | 解析装置 |
CN101505433A (zh) * | 2009-03-13 | 2009-08-12 | 四川大学 | 一种实拍实显多镜头数码立体系统 |
WO2012014355A1 (ja) * | 2010-07-27 | 2012-02-02 | パナソニック株式会社 | 撮像装置 |
US20130016186A1 (en) * | 2011-07-13 | 2013-01-17 | Qualcomm Incorporated | Method and apparatus for calibrating an imaging device |
-
2011
- 2011-03-24 WO PCT/JP2011/001726 patent/WO2011132364A1/ja active Application Filing
- 2011-03-24 JP JP2012511522A patent/JP5683025B2/ja active Active
- 2011-03-24 CN CN2011800198816A patent/CN102860016A/zh active Pending
-
2012
- 2012-09-15 US US13/621,104 patent/US9304388B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0888868A (ja) * | 1994-09-20 | 1996-04-02 | Canon Inc | 複眼撮像装置 |
JPH1145334A (ja) * | 1997-05-29 | 1999-02-16 | Dainippon Screen Mfg Co Ltd | 画像の比較表示方法及び記録媒体 |
JP2004032244A (ja) * | 2002-06-25 | 2004-01-29 | Fuji Heavy Ind Ltd | ステレオ画像処理装置およびステレオ画像処理方法 |
JP2008252254A (ja) * | 2007-03-29 | 2008-10-16 | Fujifilm Corp | 立体撮影装置及び光軸調節方法 |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019146510A1 (ja) * | 2018-01-26 | 2019-08-01 | 日立オートモティブシステムズ株式会社 | 画像処理装置 |
JP2019129482A (ja) * | 2018-01-26 | 2019-08-01 | 日立オートモティブシステムズ株式会社 | 画像処理装置 |
US11509813B2 (en) | 2018-01-26 | 2022-11-22 | Hitachi Astemo, Ltd. | Image processing device |
US11546570B2 (en) | 2019-02-18 | 2023-01-03 | Canon Kabushiki Kaisha | Apparatus, apparatus control method, and recording medium, for synchronizing a plurality of imaging devices |
CN112203076A (zh) * | 2020-09-16 | 2021-01-08 | 青岛小鸟看看科技有限公司 | Vr系统中多摄像头曝光中心点的对齐方法、系统 |
Also Published As
Publication number | Publication date |
---|---|
JP5683025B2 (ja) | 2015-03-11 |
CN102860016A (zh) | 2013-01-02 |
JPWO2011132364A1 (ja) | 2013-07-18 |
US20130010084A1 (en) | 2013-01-10 |
US9304388B2 (en) | 2016-04-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5683025B2 (ja) | 立体画像撮影装置および立体画像撮影方法 | |
US9077976B2 (en) | Single-eye stereoscopic image capturing device | |
EP2391119B1 (en) | 3d-image capturing device | |
US9521398B1 (en) | Modular configurable camera system | |
US8885026B2 (en) | Imaging device and imaging method | |
US20110280562A1 (en) | Camera body, imaging device, method for controlling camera body, program, and storage medium storing program | |
WO2015081870A1 (zh) | 一种图像处理方法、装置及终端 | |
WO2012086326A1 (ja) | 立体パノラマ画像作成装置、立体パノラマ画像作成方法及び立体パノラマ画像作成プログラム並びに立体パノラマ画像再生装置、立体パノラマ画像再生方法及び立体パノラマ画像再生プログラム、記録媒体 | |
WO2011121840A1 (ja) | 立体撮像装置 | |
JP5950678B2 (ja) | 撮像装置、制御方法、及びプログラム | |
JP2011259168A (ja) | 立体パノラマ画像撮影装置 | |
JP2011114547A (ja) | 立体画像表示装置、複眼撮像装置及び立体画像表示プログラム | |
JP5275789B2 (ja) | カメラ | |
US20110018978A1 (en) | 3d image display apparatus and 3d image display method | |
JP2011035642A (ja) | 多眼撮影装置およびその調整方法並びにプログラム | |
CN109151265B (zh) | 摄像设备、控制方法和存储介质 | |
JP5580486B2 (ja) | 画像出力装置、方法およびプログラム | |
CN116260956A (zh) | 一种虚拟现实拍摄方法及系统 | |
US20130076867A1 (en) | Imaging apparatus | |
JP2012124650A (ja) | 撮像装置および撮像方法 | |
WO2011101928A1 (ja) | 立体画像撮影用アダプタ、ハイブリッド撮影システムおよび電子カメラ | |
JP2010210691A (ja) | 立体撮像装置 | |
JP2012220603A (ja) | 3d映像信号撮影装置 | |
JP2013046343A (ja) | 画像撮像装置 | |
JP2010200024A (ja) | 立体画像表示装置および立体画像表示方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180019881.6 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11771708 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012511522 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11771708 Country of ref document: EP Kind code of ref document: A1 |