JPH08111878A - Hmd - Google Patents

Abstract

PURPOSE: To observe natural stereoscopic images by linking congestion setting and diopter setting in an HMD(head mounted display) for virtual image projecting left and right video images to both eyes by an eyepiece. CONSTITUTION: The congestion setting is performed by rotating a pair of virtual image display devices 201L and 201R on the left and right composed of display elements 202L and 202R and eyepiece optical systems 203L and 203R respectively to the left and right and changing the direction of virtual images to eyes EL and ER. The diopter setting is performed by changing the distance of the display elements 202L and 202R and the eyepiece optical systems 203L and 203R and changing the distance from the eyes to the virtual images. The cam grooves 209L, 209R, 211L and 211R of a prescribed shape are formed on a base plate 231 for mounting the virtual image display devices, pins 208L, 208R, 210L and 210R to be engaged with the cam grooves are provided on the virtual image display devices and the rotary movement of the virtual image display devices and the change of the distance of the display elements and the eyepiece optical systems are linked.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an HMD which projects a virtual image of an image display device on the retinas of both eyes to provide a stereoscopic image, and more particularly to setting of convergence and diopter of the HMD. is there.

[0002]

2. Description of the Related Art In a so-called HMD (head mounted display) which is mounted on the head and presents an image on a video display device in front of the eyes, a video display device and an eyepiece lens are provided for both left and right eyes. A type of projecting a virtual image with parallax on the eye has been proposed. For example, in U.S. Pat. No. 2,955,156, a virtual image of a television image is guided to the left and right eyes and can be mounted on the head.

In observing a stereoscopic image, it is important to set the binocular parallax, the convergence, that is, the angle formed by the left and right visual axes, and the diopter, that is, the distance from the eye to the virtual image. Of these, the binocular parallax is determined when the image to be displayed is captured.
Not adjusted with D. In the HMD disclosed in the above-mentioned US patent, the convergence and the diopter are adjustable, but they are configured to be adjusted independently. Also,
These adjustments are not made in conjunction with the distance of the stereoscopic image, but are made manually by the observer, and the convergence and diopter are kept in the set state unless the observer operates them.

Japanese Patent Laid-Open No. 1-277813 and US Pat. No. 5,034,
In No. 809, the convergence and the diopter are set in advance, and the virtual image is positioned at the intersection of the visual axes of both eyes. The convergence and diopter settings here are fixed and not adjustable.

Further, JP-A-62-115989 and JP-A-3-28979.
In No. 7 and Japanese Patent Publication No. 5-78013, an HMD that sets the distance from the eye to the virtual image according to the distance of the stereoscopic image being displayed.
Is disclosed.

Although not HMD, European Patent Application Publication No. 05
No. 27198 describes that in binoculars worn on the head, the objective lens is moved along the optical axis according to the distance to the object to be observed, and the optical axes of the left and right optical systems are crossed on the object. A configuration is disclosed. However, in this binocular, the light from the object passes through the objective lens and the eyepiece lens, and then becomes a parallel light beam and enters the eye. Therefore, regardless of the actual distance from the eye to the object, the eye is always focused on infinity, which causes a disagreement between the diopter and the parallax between the left and right images, which causes a sense of discomfort.

[0007]

In order to observe a natural stereoscopic image by the HMD, it is desirable that the convergence and the diopter are set so as to always correspond to the distance at which the stereoscopic image displayed is captured. In none of the above-mentioned conventional techniques, adjustment of the convergence and the diopter in association with each other is disclosed.
Also, in the HMD that performs convergence adjustment and diopter adjustment independently,
The adjustment mechanism had to be provided separately, and the configuration was complicated.

An object of the present invention is to realize an HMD that provides a stereoscopic image that is natural with respect to convergence and diopter and has a high sense of reality.

[0009]

In order to achieve the above object, according to the present invention, a virtual image display device which guides image light of a display element to an eye by an eyepiece optical system to give a virtual image of an image of the display element to the eye is provided. In one pair of HMDs, the left and right virtual image display devices are rotated left and right to perform convergence setting,
By setting the diopter by changing the distance between the left and right display elements and the eyepiece optical system, by linking the distance change between the left and right display elements and the left and right eyepiece optical system with the rotational movement of the left and right virtual image display device, the convergence The setting and the diopter setting are linked.

Further, a pair of left and right convergence setting means for changing the direction of the virtual image with respect to the eyes by rotationally moving the left and right virtual image display devices, and changing the distance between each of the left and right display elements and the eyepiece optical system. , A pair of left and right diopter setting means for changing the distance from the eye to the virtual image, and left and right convergence setting means and diopter setting means are interlocked to interlock the congestion setting and the diopter setting. And a driving means for giving a driving force to the convergence setting means and the diopter setting means, and a control means for controlling the driving means in accordance with the photographing distance of the image displayed on the display element.

Further, in the above-mentioned structure, the virtual image display device is rotationally moved by the left / right convergence setting means and the distance between the display element and the eyepiece is changed symmetrically by the left / right diopter setting means.

[0012]

In the HMD having a pair of left and right virtual image display devices for guiding the image light of the display device to the eye by the eyepiece optical system to give a virtual image of the image of the display device to the eyes, the left and right virtual image display devices are rotated left and right. Convergence setting is performed by changing the distance between the left and right display elements and the eyepiece optical system, and the diopter setting is performed, and the change in the distance between the left and right display elements and the left and right eyepiece optical system is rotated by the left and right virtual image display device. With the configuration in which the congestion setting and the diopter setting are linked by
Convergence and diopter are set for the left and right virtual image display devices, respectively, and further, settings are made so that there is no contradiction between congestion and diopter.

Further, a pair of left and right convergence setting means for changing the direction of the virtual image with respect to the eyes by rotating and moving the left and right virtual image display devices in the left and right direction, and changing the distance between each of the left and right display elements and the eyepiece optical system. , A pair of left and right diopter setting means for changing the distance from the eye to the virtual image, and left and right convergence setting means and diopter setting means are interlocked to interlock the congestion setting and the diopter setting. In the HMD having the configuration including the means, the convergence setting means, the driving means that applies the driving force to the diopter setting means, and the control means that controls the driving means according to the shooting distance of the image displayed on the display element, In the virtual image display device, the convergence and the diopter are set without contradiction, and the setting is automatically changed according to the distance when the displayed image is taken.

Further, the virtual image display device is rotated and moved by the left and right convergence setting means, and the distance between the display element and the eyepiece is changed symmetrically by the left and right diopter setting means so that the left and right display elements are displayed at the center. The image is always observed in front of the HMD wearer.

[0015]

EXAMPLE FIG. 23 shows an outline of a stereoscopic system which is a combination of a three-dimensional camera and an HMD (head mounted display). The three-dimensional camera 1 has two left and right cameras, and is installed near the subject O. The HMD 2 has a built-in left and right virtual image projection device that projects a virtual image on each of the left and right eyes. The subject image captured by the left camera of the three-dimensional camera 1 is used as the left eye, and the subject image captured by the right camera is used as the right eye. By projecting each virtual image, the stereoscopic image SV is provided to the observer wearing the HMD 2. HMD2 is 3D camera 1
The HMD wearer can observe an image at a place distant from the shooting site without having to place it near the. A controller 3 controls the three-dimensional camera 1 and the HMD 2. The controller 3 allows the HMD wearer to operate the HMD.
Placed near the.

In order to capture the left and right images of the observation object with the three-dimensional camera 1 and display these left and right images on the HMD 2 to reproduce a three-dimensional image having a natural three-dimensional effect, the following human vision is used. It is necessary to provide the features to the 3D camera 1 and the HMD 2.

In other words, human vision recognizes perspective and stereoscopic effect by adjusting the distance between the left and right eyes and the refractive power of the crystalline lens of each eyeball. Since the left and right eyes are apart, so-called parallax occurs in the images formed on the left and right retinas when observing the object. Due to this parallax, the relative perspective of the object is perceived. Further, when observing the object, the visual axes of both eyes point toward the object, and so-called convergence is set. The angle formed by both visual axes, that is, the vergence angle, becomes larger as the object is closer, and becomes smaller as the object is farther away, and the two visual axes become parallel at infinity.

Further, in order to clearly form an image of the object on the retina, the refractive power of the crystalline lens is changed according to the distance to the object to adjust the focus, that is, the diopter. The eye strain required for diopter adjustment is greater as the object is closer, and smaller as the object is farther away. From this tension, the distance to the object is recognized to some extent.

A three-dimensional camera for taking left and right images must be properly convergent with respect to an object, and must be properly focused in order to take clear images. Three
The congestion setting of the two-dimensional camera will be described with reference to FIG. In FIG. 24, A1, B1, and C1 indicate that the left camera 101L and the right camera 101R of the three-dimensional camera have constant congestion,
It shows a state in which the distance from the three-dimensional camera to the observation object O is different. The images captured by the left camera 101L and the right camera 101R in each state are A2, B2, and C2, respectively.
As shown in.

In the state of B1 in which the optical axes of the left and right cameras 101L and 101R intersect on the object O and the convergence is properly set, the left and right images are displayed at the centers of the left and right image frames FL and FR as shown in B2. To position. When these left and right images are observed to be fused during stereoscopic viewing, the left and right image frames FL and FR also coincide with each other, as in B3. Therefore, an image that matches the left and right sides including the background in the image frame is obtained, and a natural stereoscopic effect is reproduced.

On the other hand, in the state of A1 where the congestion is too large, the left and right images are divided into the left and right image frames F as shown in A2.
The image is taken outside the center of L and FR. On the contrary, when the image is captured in the state of C1 where the congestion is too small, the left and right images are located inside the center of the image frame like C2. A2 and C
When the left and right images of 2 are stereoscopically fused, a mismatch occurs between the left and right image frames FL and FR as shown in A3 and C3. The disagreement between the left and right image frames gives the observer a feeling of strangeness, but the sense of discomfort increases as the deviation of the vergence increases, and eventually the image cannot be viewed stereoscopically. On the other hand, when the congestion setting is not appropriate, when observing with the image frames FL and FR aligned, A2 and C2
As is clear from the above, the images of the object do not match on the left and right, and it is also difficult to stereoscopically view.

As described above, in order to capture left and right images that allow a natural stereoscopic image to be observed, the vergence of the three-dimensional camera is appropriately set according to the distance to the object to be observed (hereinafter referred to as the subject distance). Need to When the distance between the left and right cameras is constant, the convergence is uniquely determined according to the subject distance.
Further, the focus setting is uniquely determined corresponding to the congestion setting.

In order to observe the left and right images photographed by appropriately setting the convergence and focus of the three-dimensional camera by the HMD as a natural stereoscopic image, the HMD also needs to appropriately set the convergence and the diopter.

The HMD 2 is provided with a pair of left and right eyepieces each consisting of a display device and a convex lens as a virtual image projection device, and guides the image displayed on the display device to the observer's eye through the eyepiece lens. FIG. 21 schematically shows a state in which an image on the display device is observed through an eyepiece lens. Display device 202
When the eyepiece lens 203 is set so that is positioned between the eyepiece lens 203 and the front focus position thereof, the image light of the display device 202 is emitted from the surface PV farther than the display device 202, as shown in FIG. Perceived as
A virtual image is obtained on V. The position of the virtual image plane PV moves back and forth by moving the display device 202 back and forth with the eyepiece lens 203 fixed, or by moving the eyepiece lens 203 back and forth with the display device 202 fixed. . The diopter of the observer's eye E is adjusted so as to be in focus according to the distance to the virtual image plane PV.

FIG. 22 shows the left and right display devices 202L and 202R.
Shows a state in which the image of is observed with both eyes EL and ER. By inclining the left eye display device 202L and the eyepiece lens 203L, and the right eye display device 202R and the eyepiece lens 203L, respectively, it is perceived that an image exists at a position indicated by OP in the drawing. In the figure, β is the angle of convergence. If this vergence angle does not match the vergence angle of the three-dimensional camera at the time of shooting, the subject distance is not correctly reproduced and a natural stereoscopic effect cannot be obtained.

For example, consider the situation of FIG. In the figure, A represents a state in which the first observation target object O1 is photographed by the three-dimensional camera, and the second target object O2 exists farther than the target object O1. Left camera 101L and right camera 10
The optical axis of 1R intersects with the convergence angle θ on the first object O1 as shown by the solid line. A state in which the left and right images thus photographed are observed with the HMD in which the convergence is set to an infinite distance is shown in B. The left and right images of the first object O1 are located at the centers of the left and right display devices, respectively, as in B2 of FIG. 24, and the left and right optical axes 206L and 206R of the HMD are parallel to each other. At this time, the left and right images of the second object O2 are displayed outside the optical axes 206L and 206R. In order to stereoscopically view the image of the object O2 in this state, the visual axes EXL and E of the left and right eyeballs EL and ER are used.
Each XR must face outward. It is extremely difficult for an ordinary person to direct the visual axes of both eyes to the outside at the same time, and the observer cannot stereoscopically view the image.

As with the three-dimensional camera at the time of shooting, the vergence angle is θ
The state observed with the HMD set to is shown in C. In this case, the left and right visual axes EXL, EXR of the observer with respect to the left and right images of the second object O2 are directed inward, the convergence is appropriately reproduced, and the second object O2 is generated together with the first object O1. It is correctly observed as a stereoscopic image SV. In this state, the subject distance is accurately reproduced and a natural perspective is obtained.

As described above, it is necessary for the HMD to set the congestion in accordance with the congestion of the three-dimensional camera. In addition to this convergence setting, the distance to the virtual image plane PV is set to match the distance from the eye to the intersection of the left and right visual axes, and the left and right virtual images are positioned at the intersection of the visual axes, resulting in a more natural three-dimensional image. The image can be reproduced. Below, set the distance from the eye to the virtual image plane to H
The diopter setting of MD is different from the diopter adjustment for correcting myopia and hyperopia described below.

Generally, the human eye can clearly observe an object to be observed from a close range to infinity, but in myopia and hyperopia, the function of adjusting the refractive power of the crystalline lens is deteriorated, so that the long range or near range can be observed. The focus cannot be adjusted for the object. A method of correcting the HMD will be described with reference to FIG. In the figure, A is an example showing a state of image formation of an emmetropic eye, and the display device 202 is at the front focus position of the eyepiece lens 203. The image light passes through the eyepiece lens 203 to become parallel light, and enters the eye E as light from infinity. In an emmetropic eye, the refracting power is appropriately adjusted for this parallel light, and an image is formed on the retina.

A state of image observation with a myopic eye when the display device 202 and the eyepiece lens 203 are set in the same state is shown in B. In myopia, the refractive power cannot be adjusted for parallel light or light that is nearly parallel, and the image light intersects in front of the retina. Therefore, it does not form a clear image on the retina. in this case,
As shown in C, when the eyepiece lens 203 is moved slightly away from the eye, the image light intersects on the retina. This correction enables clear image observation. On the contrary, in the case of hyperopia, since the image light intersects behind the retina, the image light is corrected so as to intersect on the retina by bringing the eyepiece lens close to the eye.

The moving distance ΔMD of the eyepiece lens 203 required for this correction differs depending on the degree of myopia and hyperopia, and even the same observer has a difference between the left and right eyes. Therefore, HM
It is desirable that D is capable of right and left diopter correction. The diopter can be corrected by changing the position of the eyepiece lens 203 and moving the display device 202 forward or backward.

In the stereoscopic system of the present embodiment shown in FIG. 23, the three-dimensional camera 1 and the HMD 2 are constructed in consideration of the above human visual characteristics, and the three-dimensional camera 1 is not congested. The focus setting is variable, and the HMD 2 also has variable convergence and diopter settings, and can also perform diopter correction. Furthermore, the movement of the head of the observer wearing the HMD2 is detected by the HMD2, the orientation of the three-dimensional camera 1 is set to follow the orientation of the HMD2, and the observation target that the observer is trying to pay attention to is photographed. Further, the subject distance is detected based on the left and right images captured by the three-dimensional camera 1, and the convergence and focus of the three-dimensional camera 1 and the convergence and diopter of the HMD 2 are set according to the subject distance.

As described above, in the stereoscopic system of the present embodiment, the operation of the three-dimensional camera for photographing and the operation of the HMD for display are interlocked with each other, so that the object desired by the observer can be captured in real time and is clear. It provides a realistic 3D image.

A three-dimensional camera will be described with reference to FIGS. As shown in FIG. 1, two cameras 101L and 101R are installed on a base plate 131. Left camera 1
01L has a taking lens 102L, a CCD (charge coupled device) sensor 103L, a lens barrel 104L, and a lens holding lens barrel 105L. The CCD sensor 103L receives light transmitted through the taking lens 102L and converts the light into an electric signal, and is installed so as to be perpendicular to the optical axis 106L of the taking lens 102L and pass through the center of the optical axis 106L. The CCD sensor 103L is fixedly held by the lens barrel 104L, and the taking lens 102L is the lens holding lens barrel 10.
It is fixedly held at 5L.

The lens holding lens barrel 105L is a taking lens 10
The lens barrel 104 is movable so as to be movable along the optical axis 106L of 2L.
It is loaded into L, and it is possible to set the focus on subjects from the closest distance to infinity. FIG. 1 shows a state in which an object at infinity is in focus.
Distance L from D sensor 103L to taking lens 102L
₀ is almost the shortest within the movable range of the lens holding barrel 105L.

A pin 107L is provided on the outer surface of the lens barrel 104L located below the center of the light receiving surface of the CCD sensor 103L, and a hole 108L having a diameter slightly larger than the diameter of the pin 107L is formed in the base plate 131. ing. The lens barrel 104L is held by the base plate 131 by inserting the pin 107L into the hole 108L, and is rotatably arranged around the pin 107L. The pin 10 is also provided below the outer surface of the lens holding barrel 105L.
9L is provided, and the pin 109L is engaged with the cam groove 110L formed in the base plate 131. Left camera 101L
The pin 109L moves along the cam groove 110L and the lens holding barrel 105L moves along the optical axis 106L according to the rotation of the. Therefore, the amount of extension of the taking lens 102L also changes according to the rotation of the camera 101L.

Like the left camera 101L, the right camera 101R is also provided on the taking lens 102R, the CCD sensor 103R, the lens barrel 104R, the lens holding lens barrel 105R, the pin 107R provided on the lens barrel 104R and the lens holding lens barrel 105R. Has a pin 109R and a base plate 131 has a hole 1
08R and a cam groove 110R are formed. Right camera 10
In 1R, the pin 107R and the pin 109R engage with the hole 108R and the cam groove 110R, respectively, and rotate with respect to the base plate 131 about the pin 107R as an axis. In the three-dimensional camera 1, the distance between the pins 107L and 107R, that is, the distance between the centers of the light receiving surfaces of the left and right CCD sensors 103L and 103R is the baseline length. Optical axis 106 of left and right cameras 101L and 101R
Although L and 106R are parallel to each other in FIG. 1, the left and right cameras 101L and 1L are correspondingly arranged as the subject distance becomes shorter.
01R rotates inward, and the optical axis 106L,
106R intersects.

FIG. 2 shows the states of the left and right cameras 101L and 101R when photographing a subject located at a short distance. The left and right cameras 101L and 101R face inward so that the subject is located on the intersection of the optical axes 106L and 106R. The angle formed by the left and right optical axes 106L and 106R is the convergence angle of the three-dimensional camera 1.

Pins 109 of the left and right cameras 101L and 101R
Since L and 109R are engaged with the cam grooves 110L and 110R, the lens holding barrels 105L and 105R are the camera 101L,
The position is set according to the orientation of 101R. The distance between the CCD sensors 103L and 103R and the taking lenses 102L and 102R for an infinite subject was L ₀ , but for short-distance subjects, the lenses 102L and 102R were extended, and the distances between the sensors 103L and 103R and the lenses 102L and 102R were It becomes L ₁ . That is, a difference of L ₁ -L ₀ occurs in the focus position. In the present three-dimensional camera 1, the cam groove 1 is designed so that appropriate focusing is always performed according to the subject distance.
10L and 110R are formed in consideration of the rotation angles of the left and right cameras 101L and 101R and the moving amounts of the taking lenses 102L and 102R.

Since the pins 107L and 107R, which are the rotation axes, are provided below the center of the light receiving surfaces of the CCD sensors 103L and 103R as described above, the left and right sensors 103L and 103L are
The center-to-center distance of 103R, that is, the base line length, is always constant, and does not change during infinite subject shooting or short-distance subject shooting.

FIG. 3 shows the congestion setting mechanism of the left and right cameras 101L and 101R. In this figure, the left and right cameras 101L,
101R is a state in which an object at infinity is being photographed as in FIG. A gear 111L partially having teeth is fixedly attached to the pin 107L of the left camera 101L.
Gear 111R with teeth on part of pin 107R of 01R
Is fixedly installed. The gear 111L and the gear 111R mesh with each other, and the rotation of the gear 111L causes the left camera 10 to rotate.
1L rotates around the pin 107L, and the right camera 101
The R rotates about the pin 107R in the opposite direction to the left camera 101L by the same angle. The gear 111L also meshes with a gear 133 connected to a step motor 132 (not shown) described later. The rotation of the step motor 132 is performed by the gear 133,
111L to the left camera 101L, and the gear 111
It is transmitted to the right camera 101R via R, and the left and right cameras 10
1L and 101R rotate at the same angle in opposite directions at the same time.

A biasing spring 112L is provided between the gear 111L and the base plate 131, and biases the gear 111L counterclockwise. A biasing spring 112R is also provided between the gear 111R and the base plate 131, and the gear 111R is biased in the clockwise direction. Since the backlash between the gears 111L and 111R is moved in one direction by the biasing springs 112L and 112R, the left and right cameras 101L and
There is no error in the direction setting of 101R.

An electric switch 134 is provided on the base plate 131, and opens and closes by contact with and separation from an eccentric pin 135 provided on the gear 111L. The switch 134 is the left camera 101.
It is closed when L is facing the direction to shoot a subject at infinity, and is open otherwise. The open / close position can be adjusted by setting the left and right cameras 101L and 101R to the optical axes 106L and 1L with both terminals of the switch 134 opened.
06R is set to be parallel, and the eccentric pin 13
This is done precisely by rotating the eccentric pin 135 so that one terminal of the switch 134 pushed by 5 contacts the other terminal. The closing signal of the switch 134 is the left / right camera 1.
It is used as a signal indicating the reference position of rotation of 01L and 101R.

Further, in order to restrict the rotation range of the gear 111L, a cutout portion 136 is formed in the gear 111L, and a pin 137 is provided on the upper surface of the base plate 131 at a position corresponding to the cutout portion 136. . This prevents the inconvenience that the left and right cameras 101L and 101R are contacted and damaged due to the erroneous congestion setting.

FIG. 4 shows a state in which the convergence is set for a short-distance subject. The left and right cameras 10 are arranged so that the subject exists at the intersection of the extension lines of the optical axes 106L and 106R.
1L, 101R gear 13 by step motor 132
It is rotated symmetrically via 3, 111L, 111R.
As described above, the focus position setting is automatically performed along with the congestion setting. At this time, the switch 134 is open.

FIG. 5 shows the connection of the mechanical parts of the three-dimensional camera 1. The gear 133 is connected to the step motor 132 and meshes with the gear 111L. The pin 107L, which serves as the rotation axis of the left camera 101L and the gear 111L, engages with the hole 108L formed in the base plate 131. The base plate 131 is also formed with a hole 108R that engages with the pin 107R that is the rotating shaft of the right camera 101R and the gear 111R, but this is not shown in the figure. The pin 109L provided on the lower portion of the lens holding barrel 105L of the left camera 101L is inserted into the cam groove 110L, and the pin 109R of the right camera 101R is inserted into the cam groove 110R.

The rotation of the step motor 132 is performed by the gear 13
Left and right cameras 101L, 1 through 3 111L, 111R
01R, the left and right cameras 101L and 101R rotate, and the congestion is set. Left and right cameras 101L, 101R
The left and right pins 109L and 109R move along the cam grooves 110L and 110R, respectively, according to the rotation of the camera lens 1.
The feeding adjustment of 02L and 102R is performed. Therefore, the focus setting is also performed in conjunction with the congestion setting.

The play of the gears 111L and 111R is the biasing spring 1.
The left and right cameras 101L and 101R are rotated in one direction by the 12L and 112R, and the rotation range of the left and right cameras 101L and 101R is restricted by the pin 137 formed on the base plate 131 and the notch 136 of the gear 111L. In addition, the left and right cameras 101
It is determined whether L and 101R face infinity, that is, whether the left and right optical axes 106L and 106R are parallel.

In the three-dimensional camera 1 configured as described above, the optical axes 106L and 1L of the left and right cameras 101L and 101R are used.
The convergence setting is performed by rotating the step motor 132 so that the subject is located at the intersection on the extension line of 06R. Left and right cameras 101L and 101R are pins 109
Since L and 109R are engaged with the cam grooves 110L and 110R, the feeding amount is adjusted in conjunction with the convergence setting and the focus setting is automatically performed. Also, the left and right cameras 101L,
Since the baseline length of 101R is fixed, it is possible to capture left and right images that can always reproduce the correct sense of distance.

FIG. 6 shows the appearance of the three-dimensional camera 1. The three-dimensional camera 1 has a fixed base plate 138, and the fixed base plate 138 is installed in a horizontal place or set horizontally using a tripod or the like for use. On the fixed base plate 138, a rotary base plate 139 that can rotate horizontally 180 degrees each horizontally.
Is installed. The left and right cameras 101L and 101R and the base plate 131 are housed in the camera case 140 and installed on the rotary base plate 139.

The camera case 140 has a rotary shaft 141 on both left and right sides, and a rotary base plate 139 is provided with a pair of rotary shaft support arms 142 for supporting the rotary shaft 141. The camera case 140 has a rotary shaft 141 parallel to the rotary base plate 139 and is rotatably supported by the rotary support arm 1.
Supported by 42. As a result, the camera case 140 is held on the rotary base plate 139 and is vertically rotatable. The rotating shaft 141 is parallel to a straight line connecting the centers of the light receiving surfaces of the left and right CCD sensors 103L and 103R,
Moreover, it is set so as to be orthogonal to the central axis of rotation of the rotary base plate 139. The left and right cameras 101L and 101R are C
The centers of the light receiving surfaces of the CD sensors 103L and 103R are installed so as to be equidistant from the center of rotation of the rotary base plate 139.

Signals for controlling the movements of the left and right cameras 101L and 101R, video signals from the left and right cameras 101L and 101R, etc. are input and output via a signal cable 143. The camera case 140 is provided with a horizontal sensor 144 composed of a mercury switch, and detects the horizontal of the three-dimensional camera 1. The horizontal sensor 144 is closed when the camera is set horizontally, that is, when the vertical rotation about the rotation axis 141 is not performed. Δ in FIG.
BC indicates the distance between the centers of the light receiving surfaces of the left and right CCD sensors 103L and 103R, and is the baseline length of the above-described three-dimensional camera.

FIG. 7 shows a side view of a three-dimensional camera partially transparently. A gear 145 is provided at the end of the rotary shaft 141 of the camera case 140. The gear 145 meshes with the gear 147 connected to the step motor 146 in the rotation support arm 142, and the rotation of the step motor 146 is transmitted to the rotation shaft 141 to rotate the camera case 140 in the vertical direction. The left and right cameras 101L and 101R, the base plate 131 and the gears 111L and 111R are shown in perspective. Driver circuits 113L and 11 of CCD sensors 103L and 103R are provided behind the left and right cameras 101L and 101R.
3Rs are provided, and each has a flexible substrate 11
CCD sensor 103L, 103R by 4L, 114R
Is connected to

FIG. 8 shows a horizontal direction setting mechanism of the three-dimensional camera 1. The step motor 14 is attached to the fixed base plate 138.
8 is installed, and the step motor 148 has a gear 1
49 are connected. On the other hand, a rotary shaft 150 is provided at the center of the rotary base plate 139, and a gear 151 is coupled to the rotary shaft 150. The gear 149 and the gear 151 mesh with each other, and the rotation base plate 139 rotates with respect to the fixed base plate 138 by the rotation of the step motor 148.

A small conductive pattern 152 is formed on the surface of the gear 151, and an electric switch 153 is provided on the fixed base plate 138 so as to come into contact with the conductive pattern 152. The switch 153 is brought into conduction by contacting both terminals with the conductive pattern 152 at the same time. The conductive pattern 152 is arranged so as to conduct the switch 153 when the rotary base plate 139 faces a predetermined direction with respect to the fixed base plate 138, and is used for detecting the reference direction of the three-dimensional camera 1. The rotary base plate 139 rotates 180 ° in both left and right directions from the position where the switch 153 is closed.

As shown in FIG. 7, when the three-dimensional camera 1 is installed with the fixed base plate 138 horizontal, the rotary base plate 13
The rotation center of 9 is located ΔXC behind the light-receiving surfaces of the CCD sensors 103L and 103R, and the vertical rotation center of the camera case 140 is located ΔYC below the center of the light-receiving surfaces of the CCD sensors 103L and 103R.

Three-dimensional camera 1 constructed as described above
Can make a rotational movement similar to the rotation of a human head. That is, the human head is, as shown in FIG.
In the vertical direction, the point RP located at the back of the neck at the center of left and right
And rotates about an axis RV which is perpendicular to the plane of the drawing. The position of the rotation axis RV is ΔXH behind and ΔYH below the retina position of the eyeball E. In the horizontal direction, it rotates about an axis RH passing through the point RP.

FIG. 31 shows a state in which an image is observed by rotating the head in the vertical direction. The observation object O is located near the face of the observer and below the horizontal visual axis EX (A). In order to observe the object O, the head rotates downward around the point RP on the back of the neck (B). This is a natural movement of a human, and at this time, the visual axis EX forms an angle of α with the horizontal,
The distance between the eyeball E and the object O is Db. When the object O is observed by rotating the head downward about the eye E (C), the visual axis EX at that time forms an angle of β with the horizontal, and the distance between the eye E and the object O is Dc. Become. Further, even if the object O is observed by rotating the eyeball E downward without moving the head, the positional relationship between the object O and the eye E is the same as the motion of C, and the line of sight is between the horizontal angle and β. None, the distance between the eye E and the object O is Dc.

Here, β is larger than α and Dc is larger than Db. The motion of C that rotates the head about the eyeball E is extremely unrealistic, and the motion of moving only the eyeball E without moving the head is an unnatural motion that is intentionally performed. The difference between α and β and Db and Dc is small when the distance between the observer and the observation object is large, but becomes more remarkable as the distance becomes smaller.

Therefore, also in the three-dimensional camera, the setting position of the rotation axis is important for taking a natural image.
As shown in FIG. 7, the horizontal rotation center of the three-dimensional camera is ΔX from the light receiving surfaces of the CCD sensors 103L and 103R.
It is located at the rear of C, and the center of rotation in the vertical direction is located ΔYC below the center of the light receiving surface of the CCD sensors 103L and 103R. This is the binocular and rotation axis Δ of the above human head.
This is the same as the relationship between XH and ΔYH, and reproduces the natural human motion shown in FIG. 31B.

The distance ΔBH between both eyes shown in FIG. 30 is the basis for distance recognition by human vision, and corresponds to the base line length ΔBC in a three-dimensional camera. In order to take a stereoscopic image naturally, in addition to the appropriate setting of the vertical and horizontal rotation center positions of the three-dimensional camera, the baseline length Δ
BC also needs to be set correctly in consideration of the distance from the light receiving surfaces of the CCD sensors 103L and 103R to the center of rotation. In the present embodiment, the three-dimensional camera and the rotation of the human head have a similar relationship, and a natural three-dimensional image is captured, so that the image on the HMD is more realistic. Specifically, the above-mentioned ratios of ΔXC and ΔXH, ΔYC and ΔYH, and ΔBC and ΔBH are set to be constant.
A three-dimensional camera is similar to a human head.

FIG. 27 shows the second structure of the three-dimensional camera. Left and right cameras 171L and 171R are symmetrically installed on the base plate 191. The CCD sensor 173L of the left camera 171L is fixedly installed vertically to the base plate 191. In front of the CCD sensor 173L, a left photographing lens 172L fixedly installed in the lens holding barrel 175L is arranged. A pair of pins 177L and 178L are provided below the outer surface of the lens holding barrel 175L and engage with cam grooves 179L and 180L formed in the base plate 191 respectively. Further, the lens holding barrel 175L is provided with an arm 182L via a pin 181L. The arm 182L is rotatable around the pin 181L.

The right camera 171R is also constructed similarly to the left camera 171L, and the left and right CCD sensors 173L, 1
73R is arranged so that the light receiving surface is located on the same plane. A step motor 192 is fixedly installed on the base plate 191 between the left and right CCD sensors 173L and 173R.
It is coupled to a rotary shaft 194 supported by a shaft support frame 193. A mover 195 that is movable in the axial direction is provided in contact with the rotating shaft 194. A male screw is cut on the outer periphery of the rotary shaft 194, and a female screw that meshes with the mover 195 is cut. The mover 195 moves back and forth along the rotary shaft 194 by the rotation of the step motor 192.
An arm 196 extending forward is fixed to the mover 195, and a front end of the arm 196 has left and right lens holding lens barrels 1.
The ends of the arms 182L and 182R connected to 75L and 175R are connected via a pin 197. Left and right arm 1
82L and 182R rotate about the pin 197 with respect to the arm 196. With this configuration, the step motor 192
Is rotated by a moving element 195, arms 196, 182L, 18
It is transmitted to the left and right lens holding barrels 175L and 175R via 2R.

In FIG. 27, the left and right cameras 171L, 171R
Is a state in which convergence is set to infinity, and the optical axis 17
6L and 176R are parallel. Step motor 19
The left and right lens holding lens barrels 175L and 175R are driven by the rotation of 2 to set the convergence on the near object.
8 shows. Even at this time, the left and right photographing lenses 172L, 172R
The optical axes 176L and 176R of are kept parallel. Left CC
The subject is located on the intersection of a line 183L connecting the center of the light receiving surface of the D sensor 173L and the center of the left lens 172L and a line 183R connecting the center of the light receiving surface of the right CCD sensor 173R and the center of the right lens 172R.

The distance from the taking lenses 172L and 172R to the light receiving surfaces of the CCD sensors 173L and 173R was L ₀ when the convergence was set to the infinite subject, but became L ₁ when the convergence was set to the short-distance subject, which was long. ing. By properly performing this distance adjustment, the correct focus setting according to the subject distance is performed. Cam groove 17 of base plate 191
The shapes of 9L, 180L, 179R, and 180R are set so that the focus is properly set according to the change in congestion. Furthermore, so that the left and right optical axes 176L, 176R are always kept parallel, the cam grooves 179L, 180L, 179R,
180R is formed.

The detection of the subject distance in the present stereoscopic system will be described. FIG. 19 shows a state in which the congestion is set large (A), properly (B), and small (C) with respect to an object located at a constant distance from the three-dimensional camera, and the left and right CCD sensors 103L in each state. The image captured by 103R is shown. Left and right cameras 101L, 101R
Is directed to the subject and the convergence is properly set (B), the subject is imaged in the left-right center in both the left CCD sensor 103L and the right CCD sensor 103R. The distance P ₀ between the left and right subject images at this time is
The left and right cameras 101 like the three-dimensional camera of this embodiment
L and 101R are CCD sensors 103L and 10 respectively
When rotating around the center of the light receiving surface of 3R, the value becomes a constant value peculiar to the camera regardless of the distance from the camera to the subject. The distance P ₀ between the left and right images is called the reference image distance. When the vergence angle is too large (A), the subject image is closer to the outside than the center of the light receiving surface, and when the vergence angle is too small (C), the subject image is closer to the inside than the center. The subject distance can be obtained from the distance P between the left and right subject images and the congestion of the three-dimensional camera set at that time.

The interval P between the left and right subject images is detected as follows. Now, a horizontally long rectangular distance measuring area MA is provided in the central portion of the light receiving surfaces of the left and right CCD sensors 103L and 103R in the vertical direction.
Consider L and MAR. Left and right distance measurement areas MA of A in FIG.
FIG. 20 is an enlarged view of L and MAR. It is assumed that each ranging area includes n pixels in the horizontal direction and m pixels in the vertical direction, and the data included in each pixel in the left ranging area MAL is XL (i,
j), the data contained in each pixel of the right distance measuring area MAR is XR
It is represented by (i, j) (i = 1, n, j = 1, m). First, the pixel data of the left and right distance measurement areas MAL and MAR are added in the vertical direction and converted into one-dimensional data strings YL (i) and YR (i) in the horizontal direction, respectively.

[0068]

[Equation 1]

[0069]

[Equation 2]

Subsequently, detection blocks B1, B2, B3 are set in the left distance measuring area MAL with the block size bn. B1 = YL (i) (i = k1, k1 + bn-1) B2 = YL (i) (i = k2, k2 + bn-1) B3 = YL (i) (i = k3, k3 + bn-1) )

For each of these detection blocks B1, B2, B3, a one-dimensional data string YR in the right distance measuring area MAL
Perform correlation calculation with (i).

[0072]

(Equation 3)

[0073]

[Equation 4]

[0074]

(Equation 5)

By performing the above calculation, Z1 (ks), Z2 (ks), Z
For 3 (ks), P is obtained by finding the minimum ks. FIG. 20 shows a state where P is obtained in the detection block B1. When the distance P between the left and right subject images is obtained in this way, the distance from the three-dimensional camera to the subject is calculated based on the baseline length of the three-dimensional camera and the convergence angle of the left and right cameras at that time.

Although the number of detected blocks is 3 here, the number of blocks is set so that all the elements of the one-dimensional data string YL (i) are included in any of the blocks. Further, the block size bn is also set according to the number of blocks, but adjacent blocks may be partially overlapped so that the elements of the one-dimensional data string YL (i) belong to two or more blocks, Each element of the data string YL (i) may be set so as to belong to only a single block.

The structure of the HMD of this embodiment is shown in FIG. H
Inside the MD2, a pair of left and right virtual image projection devices 201L and 201R are installed on a base plate 231. The left virtual image projection device 201L includes a video display device 202L and a video display device 202.
Eyepiece 2 consisting of a convex lens that guides the L image to the left eye EL
03L, a display device holding lens barrel 204L that holds the image display device 202L, a lens holding lens barrel 205L that internally houses and holds the eyepiece lens 203L, and a diopter correction lens barrel 207L that holds the lens holding lens barrel 205L movably back and forth. Have. An LCD (liquid crystal display device) is used as the image display device 202L, and is set so as to be perpendicular to the optical axis 206L of the eyepiece lens 203L and the optical axis 206L passes through the center of the display surface.

The inner size of the display device holding lens barrel 204L is formed slightly larger than the outer size of the diopter correction lens barrel 207L, and the rear inner surface of the display device holding lens barrel 204L and the front of the diopter correction lens barrel 207L are formed. It is set to be in sliding contact with the outer surface.

The lens-holding lens barrel 205L has a cylindrical shape and has an external thread formed on the outer periphery thereof, and an internal thread that meshes with the internal surface of the diopter correction lens barrel 207L is formed. By rotating the lens holding barrel 205L around the optical axis 206L, the lens holding barrel 205L moves back and forth along the optical axis 206L together with the eyepiece lens 203L. As a result, the distance between the eyepiece lens 203L and the image display device 202L changes, and the diopter correction can be performed according to the individual difference of the observer. The right virtual image projection device 201R is also the left virtual image projection device 2
The configuration is similar to that of 01L, and the left and right diopter corrections are performed independently.

Pins 208L and 208R are provided on the lower portions of the outer surfaces of the diopter correction lens barrels 207L and 207R, respectively.
It is inserted in the cam grooves 209L and 209R formed in No. 1. Further, pins 210L and 210R are formed on the lower portion of the outer surface of the display device holding lens barrels 204L and 204R, respectively.
The cam grooves 211L and 211R formed in 31 are respectively inserted. A step motor 232 is fixedly installed on a base plate 231 between the left and right eyepieces 203L and 203R, and is connected to a rotary shaft 234 supported by a shaft support frame 233. A male screw is cut on the outer periphery of the rotary shaft 234, and the male screw 235 meshes with the mover 235.
The mover 235 moves back and forth along the rotation shaft 234 by the rotation of the step motor 232.

The mover 235 is provided with a pin 236,
Two arms 212 that are rotatable around a pin 236
L and 212R are attached. A pin 213L is provided on the lower portion of the outer surface of the left display device holding lens barrel 204L,
It is rotatably connected to the tip of the arm 212L.
Similarly, the right display device holding lens barrel 201R is also connected to the arm 212R via the pin 213R. Arm 212L, 2
Pins 215L and 215R are provided in the middle of 12R, respectively, and cam grooves 216L and 2L provided on the base plate 231 are provided.
It is inserted in each 16R. An electric switch 237 is installed on the base plate 231 near the tip of the rotary shaft 234, and is set to close when the mover 235 is located at the tip of the rotary shaft 234 and one terminal of the switch 237 is pushed. Has been done. The left and right virtual image projection devices 201L and 201R and the arms 212L and 212R are arranged symmetrically with respect to the rotation axis 234.

FIG. 12 shows a perspective view near the step motor 232. Rotation of the rotary shaft 2 by rotation of the step motor 232.
34 rotates, and the mover 235 moves in the left-right direction in the figure, that is, H.
Move in the front-back direction of MD2. The left and right arms 212L and 212R also move accordingly. In FIG. 12, the switch 237
Is open, but the mover 235 is near the left end of the rotation axis in the figure,
When the switch 237 is moved to the vicinity of the front end of the rotary shaft 234 of the HMD 2, the switch 237 is closed by the mover 235. That is, the switch 237 is provided at a position to be closed when the convergence is set to infinity, and the left and right virtual image projection device 2
It is used as a reference for rotation of 01L and 201R.

In such a structure, when the mover 235 moves back and forth due to the rotation of the step motor 232, the arms 212L and 212R move to the pin 21 in the middle of the arm.
5L and 215R move along the cam grooves 216L and 216R, and the left and right virtual image projection devices 201L and 201R also move to the cam grooves 20.
It moves according to the shapes of 9L, 211L, 209R, and 211R.
FIG. 9 shows a state in which an image of a subject at infinity is displayed and observed. The mover 235 exists near the tip of the rotary shaft 234, and the switch 237 is closed. At this time, the optical axes 206L of the left and right eyepieces 203L and 203R,
206R are parallel, and the left and right eyeballs EL,
Pass through the center of ER. In addition, the left and right image display devices 202L,
The display surface of 202R exists on the same plane.

FIG. 10 shows a state of the HMD 2 when the step motor 232 is rotated and the mover 235 is moved to the motor 232 side. This corresponds to a state of displaying and observing an image of a short-distance subject. At this time, the optical axes 206L and 206R of the eyepieces 203L and 203R are tilted inward from the parallel, and the vergence angle is set so that the observation object exists on the intersection of the extension lines of the optical axes 206L and 206R. Optical axes 206L and 206R are image display devices 202L and 2
Passing through the center of 02R is maintained as in FIG.

The left and right lens holding lens barrels 205L and 205R are arranged in the cam groove 209 together with the diopter correction lens barrels 207L and 207R.
It is moving inward along L and 209R. The left and right display device holding lens barrels 204L and 204R are also cam grooves 211L and 211R.
It is moving inward along, but is approaching the eyepieces 203L and 203R with it. Lens holding lens barrel 2
This movement of the 05L, 205R and the display device holding lens barrels 204L, 204R is caused by the cam grooves 209L, 211L, 209R, 21R.
This is due to the 1R shape. Cam grooves 209L, 209R
Is formed in an arc shape centering on the eyes EL and ER of the observer, and the left and right virtual image projectors 201L and 201R rotate symmetrically about the left and right eyes EL and ER.

In FIG. 9, the convergence is set for the image of the subject at infinity, and the distance between the eyepieces 203L and 203R and the image display devices 202L and 202R is S ₀ . On the other hand, when the convergence is set for the image of the short-distance subject as shown in FIG. 10, S ₁ is set, and the distance between the eyepieces 203L and 203R and the image display devices 202L and 202R is linked with the expansion of the convergence angle. It becomes smaller, and the diopter is set corresponding to the subject distance. As a result, the sense of distance due to the convergence and the sense of distance due to the diopter are interlocked, and a more realistic image can be obtained. The diopter setting linked with the convergence is performed independently of the correction of the diopter deviation peculiar to the observer, which has already been described. Note that the switch 237 is open when the short distance is supported.

A perspective view of the HMD 2 is shown in FIG. As described above, the driving force of the step motor 232 causes the arm 2 to move.
The left and right virtual image projection devices 201L,
It is transmitted to 201R, and congestion and diopter are set. The HMD 2 configured in this way is covered with a case (not shown), and the case is equipped with a mounting member for mounting on the head.

In this stereoscopic vision system, the orientation of the three-dimensional camera is changed in accordance with the movement of the head of the observer wearing the HMD, the observation target noticed by the observer is captured by the three-dimensional camera, and the image is displayed on the HMD. It is displayed and it is necessary to detect the movement of the observer's head. Therefore, as shown in FIG. 11, a vertical angular velocity sensor 238 for detecting a vertical rotational angular velocity and a horizontal angular velocity sensor 239 for detecting a horizontal rotational angular velocity are provided on the HMD base plate 231. is set up. In this embodiment, a vibration gyro using piezoelectric ceramics is used as the angular velocity sensors 238 and 239.

The piezoelectric vibrating gyro detects rotational movement,
A voltage linearly corresponding to the angular velocity is output. The direction of the rotational movement can be known from the sign of the difference between the output voltage when the piezoelectric vibrating gyroscope performs the rotational movement and the output voltage when the piezoelectric vibrating gyroscope is at rest, and the magnitude of the angular velocity can be known from the absolute value of the difference. Therefore, by continuing to detect the output voltage of the piezoelectric vibrating gyro, the azimuth of the HMD equipped with this can be known.

A method of detecting the HMD azimuth in this embodiment will be described with reference to the example shown in FIG. 32A is a graph showing the output of the piezoelectric vibration gyro with respect to time,
The DC reference output is an output voltage when the HMD wearer keeps his / her head stationary. The difference between this DC reference voltage and the detected output voltage is proportional to the magnitude of the angular velocity. Therefore, in the figure, the area of the portion Sa surrounded by the curve of the DC reference voltage and the output voltage located above it represents the rotation angle in one direction, and is surrounded by the curve of the DC reference voltage and the output voltage located below it. The area of the broken portion Sb represents the rotation angle in the opposite direction. If the rotation angle with respect to time is represented by a graph, it will be B.

The output voltage of the piezoelectric vibrating gyro drifts while maintaining a linear relationship with the angular velocity due to changes in environmental conditions such as temperature. That is, the graph of A in FIG. 32 is translated in the vertical axis direction. Therefore, in order to correctly obtain the angular velocity by the piezoelectric vibration gyro, it is necessary to set the DC reference output that matches the environmental conditions. The DC reference output also differs depending on the individual piezoelectric vibration gyro. Therefore, the DC reference output must be set for each HMD.

In this embodiment, at the time of starting the stereoscopic system or at any time after starting, the HMD wearer keeps his / her head stationary for a certain period of time, detects the output voltage of the piezoelectric vibrating gyro during that time, and calculates the average value. Take and use as DC reference output. As a result, the fluctuation of the output voltage due to the above-mentioned environmental conditions such as temperature and the individual difference of the piezoelectric vibration gyro is corrected,
The MD rotation speed can always be detected correctly.

The azimuth change amount of the HMD can be strictly obtained by integrating the detected angular velocity with respect to time. However, since the configuration of the integrating circuit becomes complicated, the angular velocity is detected in a minute time period in this embodiment, and the product of the angular velocity and the period time is calculated by assuming that the angular velocity is constant during one period, and this is approximated. Specifically, it is the amount of change in the HMD direction. The shorter the angular velocity detection cycle, the higher the accuracy of the calculated azimuth change amount. In this embodiment, the angular velocity detection cycle is set to 1 msec, and the azimuth of the HMD can be obtained with sufficient accuracy. Further, since the HMD is provided with the vertical angular velocity sensor 238 and the horizontal angular velocity sensor 239, it is possible to detect rotation of the head of the HMD wearer in any direction,
You can know the direction of the HMD correctly.

FIG. 13 is a block diagram showing the functions of the stereoscopic system of this embodiment. The three-dimensional camera 1 has a taking lens 10
The left and right cameras 101L and 101R having 2L and 102R and image pickup elements 103L and 103R are provided. The image of the object O is captured by the taking lenses 102L and 102R.
The image is formed on L and 103R. The outputs of the image pickup devices 103L and 103R are electric signals and are left and right image processing units 115L and 115R.
The image processing is performed. In addition, the image sensor 103
L and 103R are driven by driver circuits 113L and 113R included in the video processing units 115L and 115R. The signals from the image processing units 115L and 115R are transmitted to the HMD 2 and also to the controller 3 that controls the three-dimensional camera 1 and the HMD 2.

The HMD 2 is provided with left and right virtual image projectors 201L and 201R having display elements 202L and 202R and eyepieces 203L and 203R, and a subject image photographed by the three-dimensional camera 1 is observed by the observer's left and right eyeballs EL and ER. To serve.
As described above, this embodiment is a lightweight LCD suitable for HMDs.
Is used as a display element. The video signal of the left camera 101L is input to the drive circuit 214L of the display element 202L for the left eye, and the video signal of the right camera 101R is input to the drive circuit 214R of the display element 202R for the right eye. Drive circuit 2
Display elements 202L, 2L by signals from 14L, 214R
A subject image is displayed on 02R. Left and right display elements 20
The images displayed on 2L and 202R are the eyepieces 203L,
203R generates virtual images on the left and right eyeballs EL and ER of the observer. In addition, the HMD 2 is provided with the azimuth detecting device 240 including the vertical and horizontal angular velocity sensors 238 and 239, and the azimuth of the head of the observer wearing the HMD 2 is detected. The output of the azimuth detecting device 240 is sent to the controller 3.

In the controller 3, the left and right image processing units 115
The subject distance is detected based on the signals from L and 115R. The controller 3 determines the pins 109L and 109R and the cam groove 110 for the three-dimensional camera 1 according to the detected distance.
Convergence and focus setting unit 154 including L, 110R, step motor 132, etc. is controlled to set congestion and focus, and HM
Pins 208L, 210L, 208R, 21 for D2
0R, the cam grooves 209L, 211L, 209R, 211R, and the convergence / diopter setting unit 241 including the step motor 232 are controlled to set the convergence and diopter of the left and right virtual image projectors 201L, 201R. The controller 3 also controls the step motor 14 according to the output of the azimuth detecting device 240 of the HMD 2.
The camera azimuth setting mechanism 155 of the three-dimensional camera 1 including 6, 148 and the like is controlled to set the azimuth of the three-dimensional camera 1 in association with the orientation of the observer's head.

In the present embodiment, the controller 3 performs distance measurement, camera direction setting, camera convergence, focus setting, HMD convergence, and diopter setting.
The control device for performing these controls is the three-dimensional camera 1 and H.
You may make it the structure each built into MD2. Also, 3
It is also possible to record the distance information together with the left and right images of the two-dimensional camera on a video tape or the like and observe the reproduced image on the HMD. At this time, since the HMD can set the convergence and diopter according to the recorded distance information, in addition to real-time observation, it is possible to observe an image having a natural stereoscopic effect at any time after photographing.

FIG. 14 is a block diagram showing the electric circuit of the stereoscopic system of this embodiment. The three-dimensional camera 1 and the HMD 2 are controlled by the controller 3. A step motor 132 for setting convergence and focus of the three-dimensional camera is connected to the controller 3 via a motor drive circuit 156. The switch 134 is the one shown in FIG. 3, and when the convergence is set for an infinitely distant subject, the signal is used as a reference for the convergence setting, and the optical axes 106L and 106R of the left and right cameras are outside the parallel. The drive of the step motor 132 is controlled so as not to face the. The step motor 232 for setting the convergence and diopter of the HMD is connected to the controller 3 via the motor drive circuit 242. The switch 237 shown in FIG. 9 is closed when the left and right optical axes 206L and 206R of the HMD are set in parallel and is used as a reference for the convergence setting.
The drive of the step motor 232 is limited so as to prevent the 206R from facing outward.

The step motor 148 and the switch 153 for rotating the three-dimensional camera in the horizontal direction are those shown in FIG. The step motor 148 is a motor drive circuit 1
It is connected to the controller 3 via 57, and the point where the switch 153 is closed is used as the reference position for rotation. A step motor 146 for rotating the three-dimensional camera in the vertical direction is connected to the controller 3 via a motor drive circuit 158. For the vertical rotation, the signal from the horizontal sensor 144 shown in FIG. 6 is used as a reference. The outputs of the horizontal angular velocity sensor 239 and the vertical angular velocity sensor 238 installed on the HMD are the A / D conversion circuit 24.
After that, it is sent to the controller 3 as a digital signal.

Reference numeral 301 denotes a switch for generating a signal for starting the operation of this system, which is operated by an observer wearing the HMD. A switch 302 generates a signal for requesting the initial setting of the orientation of the three-dimensional camera and the HMD, which is also operated by the HMD wearer. Further, a buzzer 303 is provided to send a signal to the HMD wearer at the time of initial setting. The system start operation and initial setting will be described later.

Left and right CCD sensors 10 of the three-dimensional camera
3L and 103R are driver circuits 113L and 11 respectively.
Driven by 3R. Driver circuits 113L and 113
R is connected to a CCD drive circuit 116 that generates the timing for driving the CCD. Therefore, the left and right CCD sensors 103L and 103R are driven at the same timing. The output of the left CCD sensor 103L is processed by the left image processing circuit 117L and sent to the left LCD drive circuit 214L of the HMD. The LCD drive circuit 214L receives the video signal from the video processing circuit 117L and displays the video on the LCD 202L which is the left display element.

The video processing circuit 117L is the A / D conversion circuit 1
It is connected to the field memory circuit 119L via 18L. The video signal is converted into a digital value for one field by the A / D conversion circuit 118L and stored in the field memory circuit 119L. The field memory circuit 119L is connected to the controller 3 and can transfer digital data to the controller 3.

The output of the right CCD sensor 103R is processed by the completely same structure. That is, the signal processed by the video processing circuit 117R is displayed on the LCD 202R as an image via the LCD driving circuit 214R on the one hand and stored in the field memory circuit 119R via the A / D conversion circuit 118R on the other hand. Video processing circuit 117L, 117
The R, A / D conversion circuits 118L and 118R and the field memory circuits 119L and 119R are connected to the CCD drive circuit 116 for timing.

Next, the control process of the stereoscopic system of the present embodiment will be described with reference to the flow charts of FIGS. After the three-dimensional camera is installed with the fixed base plate horizontal, the power is turned on. The controller is configured to reset automatically when the power is turned on.
The process starts from step # 105 of FIG. First,
In step # 110, processing such as port initialization and internal RAM initialization is performed, and then in step # 115, the motor, CCD,
Turn on the power supply to drive the LCD.

Then, the convergence of the HMD and the three-dimensional camera is set to the reference position of infinity. HM in step # 120
The convergence and diopter setting motor 232 of D is driven until the switch 237 is closed, the convergence and diopter of the HMD are set to infinity, and the congestion and focus setting motor 132 of the three-dimensional camera is closed by the switch 134 at # 125. Drive to set the convergence and focus of the 3D camera to infinity.

Next, the azimuth of the three-dimensional camera is set to the reference position. In step # 130, the vertical drive step motor 146 is driven until the horizontal switch 144 is closed,
It is assumed that no rotation is made in the vertical direction. As a result, the three-dimensional camera is set horizontally. # 135
Then, the horizontal drive motor 148 is driven until the switch 153 is closed, so that the motor is not rotated in the horizontal direction. As a result, the three-dimensional camera faces the reference azimuth and is set at a position where it can rotate 180 ° in both left and right directions.

Furthermore, the direction of the observer's head, that is, HM
Initialize the parameter indicating the direction of D. Step # 14
At 0, the parameter XH1 indicating the horizontal direction and the parameter YH1 indicating the vertical direction are both set to 0.

After the system initialization is completed by the above process, it is then determined in step # 145 whether or not a system activation request has been issued. When the switch 301 is closed and an activation request is made by the HMD wearer, the process proceeds to the activation routine # 205 and the activation process is started. If the switch 301 is open and the activation request is not issued, the power source is turned off in # 150, and the state is maintained until the activation request is made in the sleep mode of # 155 which is the low power consumption mode.

16 and 17 show the starting process. First, in step # 210, the power source is turned on, and in # 215 and # 220, the three-dimensional camera is set to the reference direction. Next, in # 225, the orientation parameters XH1 and YH1 of the HMD are set to 0 and initialized. Then, check the HMD congestion and diopter # 230.
In step # 235, the distance is once set to infinity and then set again in correspondence with the predetermined distance. The convergence and focus of the three-dimensional camera are once set to infinity at # 240, and then set to the same predetermined distance as the HMD at step # 245.

As the above-mentioned predetermined distance, the most general subject distance of the subject observed using this stereoscopic vision system is selected. As a result, 3D camera and HMD
Will be set near the subject distance to be actually observed, and the time required for setting the three-dimensional camera and HMD after activation will be shortened. Therefore, the image observation can be performed immediately after the start-up. In the above operation, the distance is set to infinity, which is the reference position, and then the predetermined distance is set, so that the three-dimensional camera and the HMD can accurately perform the convergence, focus, and diopter setting. Instead of the most general subject distance, for example, three representative values corresponding to a long distance, a medium distance, and a short distance are stored and held, and an arbitrary one is selected by a switch operation to select a three-dimensional camera. May be set, convergence of focus and HMD, and diopter may be set.

After the three-dimensional camera and the HMD are set, the CCD sensor is driven in step # 250 in FIG. 17 to start the photographing by the three-dimensional camera. After the start of photographing, whether or not there is a request for initialization of the orientation of the HMD and the three-dimensional camera is determined in # 255. The orientation initialization request is made by the HMD wearer operating the switch 302.

When the switch 302 is off, initialization is not requested, and the process advances to # 260 to permit the timer interrupt for performing the process of setting the orientation of the three-dimensional camera every 1 msec. Next, in step # 265, the image information of the three-dimensional camera is A / D converted, and it is determined whether or not it is stored in one field. If it is not stored in the memory, the determination is repeated and waits until the memory is completed.

After the memory is completed, the digital data of the right distance measuring area and the left distance measuring area are input to the controller in # 270 and # 275, and in step # 280, the distance measuring operation is performed by the above-described method and the distance P between the left and right images is calculated. Ask for. Next, in # 285, it is determined whether or not the distance measurement has been correctly performed. When the reliability of the distance measurement is low due to the low contrast of the subject or the like, it is determined that the distance measurement is impossible, and the process branches to step # 315. If the reliability of distance measurement is high, the image distance P is compared with the reference image distance P _{0 in} # 290 to determine whether or not the congestion of the three-dimensional camera is set appropriately. When the image interval P matches the reference image interval P ₀ within the predetermined allowable range ΔP, it is determined that the congestion of the three-dimensional camera is correctly set, and the process proceeds to step # 315. When the image distance P exceeds the allowable range ΔP and deviates from the reference image distance P ₀ , it is determined that the congestion of the three-dimensional camera is not set correctly, and the congestion setting of the three-dimensional camera and the HMD is performed as follows. I do.

First, in step # 295, the convergence angle to be set is calculated. From the absolute value of the difference between the image interval P and the reference image interval P ₀ and the sign, the magnitude and direction of the deviation of the image interval are obtained, and the convergence angle to be set is calculated based on the convergence angle set at that time. To be done. Next, at # 300, the number of drive pulses of the convergence and focus setting step motor of the three-dimensional camera is obtained. In the stereoscopic system of the present embodiment, the congestion of the HMD is set in accordance with the congestion of the three-dimensional camera, so # 305
Then, the number of driving pulses of the HMD convergence and diopter setting step motor is also obtained. Then, in step # 310, each step motor is driven in the appropriate direction by the calculated number of pulses, and 3
The congestion of the two-dimensional camera and the HMD is set at the same time.

If the permissible range ΔP of the image interval is too large, even if the congestion setting is inadequate, it is determined that the congestion setting is correct, and the congestion is not reset. Conversely, it is too small. And congestion will be set more frequently than necessary. Therefore, the permissible range ΔP is set to a large value within a range that does not cause any inconvenience in convergence and focus setting in actual shooting and image observation.

At step # 315, the activation request switch 3
The setting status of 01 is determined. If the switch 301 remains on, the process returns to # 255 and the control process is continued. If the switch 301 is turned off, it is required to stop the system operation. In # 320, the timer interrupt is prohibited, in # 325, the power supply is turned off, and then the process goes to sleep in # 330. To do.

If it is determined in step # 255 that the direction initialization request has been made, the process branches to step # 405 to initialize the direction of the three-dimensional camera and the direction of the HMD. This is to match the orientation of the three-dimensional camera with the orientation of the head of the HMD wearer at the start of use of the stereoscopic system. This is for re-matching the two-dimensional camera and the HMD when they do not match each other.

First, at # 410, the interruption of the 1 msec timer for performing the azimuth setting process of the three-dimensional camera is prohibited. Then, in # 415, the three-dimensional camera is set horizontally, and # 420
Use to return the horizontal rotation to the reference position. After that, a buzzer sounds at # 425 to notify the HMD wearer that the three-dimensional camera has been set to the initial orientation. HMD wearers should look straight ahead and do not move their heads. In this state, # 430 detects the horizontal rotation speed of the HMD, and # 435 detects the vertical rotation speed. The rotation speed detection of the HMD in this stationary state continues for 10 seconds,
If it is determined that 10 seconds have passed in # 440, # 44
At 5, horizontal and vertical rotation speed averaging processing is performed. Also, the buzzer is turned off at # 450, and HM
The D wearer is released from the state in which the head is stationary.

The human head slightly oscillates even at rest, and the average of the angular velocities in the horizontal and vertical directions can be obtained by the above method. The average value thus obtained can be regarded as the output of the angular velocity sensor substantially when the head of the HMD wearer is stationary. Step # 455
Then, these values are used to set the DC reference output of the piezoelectric vibration gyro that is the angular velocity sensor. Next, in # 460, the direction parameters XH1 and YH1 of the HMD are initialized to 0,
The control proceeds to step # 260 described above.

FIG. 18 shows a 1 msec timer interrupt process for matching the orientation of the three-dimensional camera with the orientation of the HMD. In this stereoscopic system, the control processing shifts to step # 505 at a cycle of 1 msec and the following processing is performed.
First, in step # 510, the horizontal rotation speed of the HMD is detected by the horizontal angular velocity sensor, and in step # 515, the vertical rotation speed is detected by the vertical angular velocity sensor.
Next, in # 520, the horizontal direction XH1 and the vertical direction YH1 are calculated. Specifically, these calculations are performed according to the following two equations. XH1 = XH1 + (XAS-XAD), ka YH1 = YH1 + (YAS-YAD), ka

Here, XAS is the horizontal rotation speed detected this time, and XAD is the horizontal rotation speed at rest determined in # 445 of FIG. 17, that is, the horizontal DC reference output. Similarly, YAS is the vertical rotation speed detected this time, and YAD is the vertical rotation speed at rest, that is, the vertical DC reference output. Also, ka is a constant for converting a voltage representing an angular velocity into an angle. Right side first
The terms XH1 and YH1 are the horizontal and vertical azimuths set last time, respectively, and the second term (XAS- XA
D) ・ ka and (YAS-YAD) ・ ka are the amount of change in the horizontal and vertical angles from the previous time to the current time, respectively.

Next, the direction XH of the HMD thus obtained
1, YH1 and azimuth XH set by 3D camera
2, YH2 are compared, and the azimuth of the camera is set if necessary. In step # 525, it is determined whether or not the absolute value of the difference between the horizontal direction XH1 of the HMD and the horizontal direction XH2 of the three-dimensional camera is within a predetermined allowable range ΔX, and if it exceeds the allowable range ΔX # Branch to 535. If it is within the allowable range ΔX, the process proceeds to # 530 and the vertical direction is compared. When the absolute value of the difference between the HMD vertical direction YH1 and the three-dimensional camera vertical direction YH2 exceeds the predetermined allowable range ΔY, the process proceeds to # 535. If it is within the allowable range ΔY, it means that the azimuths of the HMD and the three-dimensional camera are the same in the horizontal direction and the vertical direction, and the interrupt processing ends in # 555.

If it is determined in steps # 525 and # 530 that the HMD and the three-dimensional camera have azimuths in a direction exceeding the allowable range in either the horizontal direction or the vertical direction,
The process proceeds to step # 535, and processing for matching the orientation of the three-dimensional camera with the orientation of the HMD is performed. In # 540, the driving pulse numbers of the step motor for setting the horizontal direction and the step motor for setting the vertical direction of the three-dimensional camera are calculated, and these step motors are driven in # 545 to set the orientation of the three-dimensional camera. In addition, the parameter XH2 and YH2 representing the orientation of the three-dimensional camera in # 550 is the parameter X representing the orientation of the HMD.
Replace with H1 and YH1. After that, the process proceeds to # 555 to end the interrupt process and restart the process performed before the interrupt.

The permissible ranges ΔX and ΔY between the HMD azimuth and the three-dimensional camera azimuth are set to a level at which the observer does not feel discomfort even if the three-dimensional camera does not move following the movement of the head. do it. Also, the horizontal allowable range Δ
The allowable range ΔY in the vertical direction with respect to X may be set to the same value or different values.

By the above interruption processing, the heading of the HMD wearer is detected in a cycle of 1 msec, and when the deviation from the heading of the three-dimensional camera becomes a predetermined value or more, the heading of the three-dimensional camera becomes the heading of the HMD wearer. Is set to match the direction of. If the deviation is within a predetermined value, the orientation of the 3D camera is not changed, so there is no unnecessary fine orientation setting of the 3D camera linked to an unintentional fine movement of the head, and a stable stereoscopic image with no discomfort can be obtained. It can be provided to an observer wearing the HMD.

In the stereoscopic system of the present embodiment configured and controlled as described above, the orientation of the three-dimensional camera for shooting is linked to the orientation of the HMD for display,
It is possible to automatically photograph and display the observation object desired by the observer. Moreover, the convergence of the three-dimensional camera, the focus setting, the convergence of the HMD, and the diopter setting are appropriately and automatically set according to the distance from the three-dimensional camera to the object.

[0127]

According to the HMD of the present invention, the vergence setting and the diopter setting are made for each of the left and right virtual image display devices. Moreover, since the congestion setting and the diopter setting are interlocked with each other, the device configuration and the operation are simpler than the method of independently setting the congestion and the diopter. In addition, it can be set with high accuracy, and there is no risk of inconsistency between congestion and diopter. Therefore, when observing the left and right images with parallax, it is possible to perform a natural and realistic stereoscopic image observation.

According to the structure of claim 2, the convergence and the diopter can be set without any contradiction, and the setting of the diopter and the diopter is always changed by the control means in accordance with the distance to the object at the time of photographing. It is possible to perform natural and realistic stereoscopic image observation. Further, the wearer does not need to perform operations for setting the convergence and the diopter, and the HMD can be easily used, and anyone can be provided with an image having the same stereoscopic effect.

According to the HMD of claim 3, since the left and right interlocking means can be configured symmetrically, the device does not become complicated. Further, since the vergence is set symmetrically and the diopter is also symmetrically set, the HMD wearer can always observe the image displayed in the center of the display element directly in front.

[Brief description of drawings]

FIG. 1 is a plan view showing a configuration of a three-dimensional camera when capturing an object at infinity.

FIG. 2 is a plan view showing a configuration of a three-dimensional camera when photographing a short-distance subject.

FIG. 3 is a plan view showing a convergence and focus setting mechanism of a three-dimensional camera when photographing an infinite object.

FIG. 4 is a plan view showing a convergence and focus setting mechanism of a three-dimensional camera when photographing a short-distance subject.

FIG. 5 is an exploded perspective view of a three-dimensional camera.

FIG. 6 is a perspective projection view of a three-dimensional camera.

FIG. 7 is a perspective side view showing a vertical direction setting mechanism of a three-dimensional camera.

FIG. 8 is a plan view showing a horizontal direction setting mechanism of the three-dimensional camera.

FIG. 9 is a plan view showing an infinitely distant subject showing the configuration of the HMD.

FIG. 10 is a plan view showing the configuration of an HMD when a short-distance subject is displayed.

FIG. 11 is a perspective projection view of a main part of the HMD.

FIG. 12 is a perspective view of an essential part showing a convergence and diopter setting mechanism of the HMD.

FIG. 13 is a block diagram showing functions of the stereoscopic system.

FIG. 14 is a block diagram showing an electric circuit of the stereoscopic system.

FIG. 15 is a flowchart showing a reset routine of the stereoscopic system.

FIG. 16 is a flowchart showing a startup routine of the stereoscopic system.

FIG. 17 is a flowchart showing a startup routine of the stereoscopic system.

FIG. 18 is a flowchart showing an interrupt routine of the stereoscopic system.

FIG. 19 is a diagram showing a relationship between a congestion setting of a three-dimensional camera and a left-right image interval.

FIG. 20 is a diagram showing a distance measuring area and left and right images of a three-dimensional camera.

FIG. 21 is a diagram showing diopter of HMD.

FIG. 22 is a diagram showing HMD congestion.

FIG. 23 is a diagram showing an outline of the configuration of a stereoscopic system.

FIG. 24 is a diagram showing a relationship between a convergence setting of a three-dimensional camera and fusion of left and right images.

FIG. 25 is a diagram showing a relationship between congestion settings of a 3D camera and an HMD.

FIG. 26 is a diagram showing diopter correction of an HMD.

FIG. 27 is a plan view showing the second configuration of the three-dimensional camera when photographing an infinite object.

FIG. 28 is a plan view showing a second configuration of the three-dimensional camera when photographing a short-distance subject.

FIG. 29 is a side view showing the rotation of the human head.

FIG. 30 is a front view showing the human eye width.

FIG. 31 is a side view showing the vertical rotation of the human head.

FIG. 32 is a diagram showing detection of HMD azimuth by an angular velocity sensor.

[Explanation of symbols]

1 3D camera 2 HMD 3 controller (control means) 101L left camera 101R right camera 102L left shooting lens 102R right shooting lens 103L left CCD sensor 103R right CCD sensor 132 step motor 138 fixed base plate 139 rotating base plate 144 horizontal sensor 146, 148 step motor 171L, 171R camera 172L, 172R shooting lens 173L, 173R CCD sensor 192 step motor 201L left virtual image projection device (left virtual image display device) 201R right virtual image projection device (right virtual image display device) 202L left image display device (left display) Element) 202R Right image display device (right display element) 203L Left eyepiece (left eyepiece optical system) 203R Right eyepiece (right eyepiece optical system) 204L Left display device holding lens barrel 204R Right display device holding lens barrel 205L Left lens Lens holding lens barrel 205R right lens holding lens barrel 206L left eyepiece lens optical axis 206R right eyepiece lens optical axis 207L left diopter correction lens barrel 207R right diopter correction lens barrel 208L, 208R pin (convergence setting means, interlocking means) 209L, 209R cam groove (convergence setting means, interlocking means) 210L, 210R pin (diopter setting means, interlocking means) 211L, 211R cam groove (diopter setting means, interlocking means) 212L, 212R arm (convergence setting means, diopter setting means) Means, interlocking means) 231 base plate 232 step motor (driving means) 234 rotating shaft 235 mover 237 switch 238 vertical angular velocity sensor 239 horizontal angular velocity sensor

Claims (3)

[Claims] 1. An HMD having a pair of left and right virtual image display devices for guiding image light of a display device to an eye by an eyepiece optical system to give a virtual image of the image of the display device to the eye. Convergence setting is performed by moving the diopter setting by changing the distance between the left and right display elements and the eyepiece optical system, and the left and right virtual image display of the distance change between the left and right display elements and the left and right eyepiece optical system. By interlocking with the rotational movement of the device,
HM characterized by interlocking congestion setting and diopter setting
D. 2. An HMD having a pair of left and right virtual image display devices for guiding image light of a display device to an eye by an eyepiece optical system to give a virtual image of an image of the display device to the eye. And a pair of left and right convergence setting means for changing the direction of the virtual image with respect to the eye by rotating the display element and the distance between the left and right display elements and the eyepiece optical system to change the distance from the eye to the virtual image. Driving power for the diopter setting means, a pair of left and right interlocking means for interlocking the congestion setting and the diopter setting by interlocking the left and right congestion setting means and the diopter setting means, and the driving force for the congestion setting means and the diopter setting means. HM, which has a driving means for providing the driving means, and a control means for controlling the driving means according to a photographing distance of an image displayed on the display element.
D. 3. The horizontal movement of the virtual image display device by the left / right convergence setting means, and the change of the distance between the display element and the eyepiece lens by the left / right diopter setting means are performed symmetrically. HMD described.