JP2004233909A - Head-mounted display - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a head-mounted display (HMD) whose specified operation such as sound output and picture output is performed in a hands-free state as intended by a user and which hardly causes malfunction. <P>SOLUTION: The head-mounted display having the picture output part to output a picture and mounted on the head has an instruction input means to detect user's specified action and accept user's instructions, and a control means to make the head-mounted display perform the specified operation according to the instruction inputted by the instruction input means. The instruction input means has a movement detecting function to detect the movement of the user's head. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a head mounted display that outputs an image so that a user can view an image while being mounted on the head of the user.
[0002]
[Prior art]
A head-mounted display (hereinafter abbreviated as HMD) is mounted on the head of a user and displays an image to the user. Since the field of view of such an HMD is obstructed by an image when worn, it is difficult to manually operate an external switch or the like to change audio output, image output, and the like. Therefore, an HMD that can be operated hands-free is required.
[0003]
[Problems to be solved by the invention]
The present invention has been made in order to solve the above-described problem, and it is an object of the present invention to provide an HMD that performs a specific operation of the HMD such as audio output and image output in a hands-free manner and in which a malfunction does not easily occur according to a user's intention. Make it an issue.
[0004]
[Means for Solving the Problems]
In order to achieve the above object, a head-mounted display according to the invention according to claim 1 has an image output unit that outputs an image, and is a head-mounted display that can be mounted on a head, and detects a specific action of a user. An instruction input unit that receives an instruction of the user, and a control unit that performs a specific operation of the head mounted display in accordance with the instruction input by the instruction input unit,
The instruction input means is a head mounted display having a motion detection function for detecting a motion of a user's head.
[0005]
A head-mounted display according to a second aspect of the present invention is the head-mounted display according to the first aspect, wherein the motion detection function includes at least a motion of a head around an axis in a body axis direction and a motion perpendicular to the body axis. The control means detects the movement of the head around the axis in the body axis direction detected by the movement detection means, and the head around the axis orthogonal to the body axis. The head-mounted display is characterized in that control is performed in accordance with a combination with the movement of the head-mounted display.
[0006]
A head-mounted display according to a third aspect of the present invention is the head-mounted display according to any one of the first and second aspects, wherein the instruction input unit has a line-of-sight detection function for detecting a line-of-sight movement of the user. The head-mounted display further includes a control unit, wherein the control unit performs control in accordance with a motion detection result by the motion detection function and a visual line detection result by the visual line detection function.
[0007]
A head-mounted display according to a fourth aspect of the present invention is the head-mounted display according to the third aspect, wherein the control unit is configured to perform the control based on one of the gaze detection result and the motion detection result. Control is performed so that an instruction to execute a specific operation of the head-mounted display is received, and according to one of the results of the line-of-sight detection result or the motion detection result, before the instruction is executed, the received instruction is executed. A head-mounted display, wherein control is performed so as to request confirmation from the user.
[0008]
A head-mounted display according to a fifth aspect of the present invention is the head-mounted display according to any one of the first to fourth aspects, wherein the instruction input unit has a voice detection function for detecting a voice uttered by the user. The head-mounted display further comprises a head-mounted display that performs control in accordance with at least a motion detection result by the motion detection function and a voice detection result by the voice detection function.
[0009]
A head-mounted display according to a sixth aspect of the present invention is the head-mounted display according to any one of the first and second aspects, wherein the instruction input unit includes a line-of-sight detection function for detecting a line-of-sight movement of the user. And a voice detection function of detecting a voice uttered by the user, wherein the control unit causes the display output unit to display a display screen including a plurality of options according to a voice detection result by the voice detection function. Control is performed so as to accept selection from the plurality of options according to the gaze detection result, and confirm the content of the acceptance performed according to the head movement detection result. The head mounted display is characterized by being controlled as described above.
[0010]
A head-mounted display according to a seventh aspect of the present invention is the head-mounted display according to any one of the first to sixth aspects, wherein the instruction input unit detects at least two user actions performed at different timings. A head mounted display having a function.
[0011]
The head-mounted display according to the invention according to claim 8 is the head-mounted display according to any one of claims 1 to 6, wherein the instruction input unit detects at least two user actions performed in parallel. A head mounted display having a function.
[0012]
The head-mounted display according to the ninth aspect is the head-mounted display according to any one of the first to eighth aspects, wherein the specific operation of the head-mounted display is at least one of an audio output and an image output. A head mounted display characterized by a change.
[0013]
A head-mounted display according to a tenth aspect is the head-mounted display according to any one of the first to ninth aspects, wherein the specific operation of the head-mounted display is one of turning on and off a power supply. A head mounted display characterized by the above.
[0014]
A head-mounted display according to an eleventh aspect of the present invention is the head-mounted display according to any one of the first to tenth aspects, wherein the image output unit is positioned in front of a user and retracted from the user. The state further comprises means for switching by power, the specific operation of the head-mounted display, the image output unit, in a state of being located in front of the user, and in a state of being retracted from the user's eyes and retracted And a head mounted display characterized by switching.
[0015]
A head-mounted display according to a twelfth aspect of the present invention is the head-mounted display according to any of the first to eleventh aspects, wherein the form of the user's specific action and the specification of the head-mounted display to be performed in accordance therewith are specified. And a means for registering and storing a set of actions corresponding to the actions of the user, wherein the control means is configured to store a specific action of the user detected by the instruction input means and a specific action of the stored user. And if the form is determined to match, the head mount is controlled to perform a specific operation of the head mounted display to be performed in accordance with the stored action corresponding to the action. It is a display.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(1st Embodiment)
1 to 3 show an example of a head mounted display according to the first embodiment of the present invention. In the head mounted display according to the present embodiment, the image output unit 100 can be displaced in a state where the image output unit 100 is positioned in front of the user (see FIG. 2) and in a state where the image output unit 100 is separated from the user and retracted (see FIG. 3). It has a structure that can be retracted backward from the user's eye position when not in use.
[0017]
As shown in FIGS. 1 to 3, a head-mounted display (hereinafter abbreviated as HMD) includes an image output unit 100 having a function of outputting an image to a user U, a sound, music, sound effect, and the like. The sound output unit 200 having the function of generating the sound, the image output unit 100, the sound output unit 200, and the sound detection device 470 are integrally connected, and these are detachably attached to the head 1 of the user U. And a support portion 300. Further, it has a head movement detecting device 700 that detects the movement of the user's head, a gaze detecting device 900 that detects the movement of the user's gaze, and a voice detecting device 470 that detects a voice emitted by the user. I have.
[0018]
The HMD is mounted on the head 1 of the user U so as to sandwich the head 1 as shown in FIGS. That is, in a state where the sound output unit 200 is in contact with the ear of the user U, the support unit 300 is attached to the head 1 by sandwiching the head of the user U by the elastic force of the support unit 300 itself. The HMD has a configuration in which an information processing device D that outputs an image display signal, an audio output signal, and the like is provided separately. The information processing device D is connected to each output unit, each detection device, and each drive device via a cable 800. The information processing device D includes a power supply, a circuit that generates and outputs an image signal, and a circuit that generates and outputs an audio signal. A plug 810 is provided at the end of the cable 800 and is detachably connected to the connector CN of the information processing device D. Therefore, by attaching the part integrally connected to the support part 300 of the HMD to the head 1, the user U can obtain an image formed by the image display signal and the sound output signal supplied from the information processing device D. Sound can be viewed via the HMD.
[0019]
Next, each component will be described.
[0020]
The image output unit 100 includes an electro-optical converter that converts an image display signal representing an image into an optical signal to form an image, and an optical system that enlarges the formed image. In the electro-optical converter, for example, a screen can be formed like a display screen of an information processing device. For example, as shown in FIG. 9, an option selection window or the like can be displayed. Of course, it is also possible to display a moving image.
[0021]
The sound output unit 200 converts a sound output signal into sound. In the present embodiment, a left sound output unit 200L and a right sound output unit 200R are provided. The sound output unit 200 is provided with a sound converter that converts an electric signal into sound. In addition, in the present embodiment, the sound output unit 200 also functions as a contact part with the ear when the HMD is mounted on the head 1 of the user U, together with a support unit 300 described later, due to its structure.
[0022]
In the present embodiment, a connection unit that connects the cable 800 and the internal wiring is provided on the sound output unit 200L side.
[0023]
As illustrated in FIGS. 1 to 3, the support unit 300 includes a band 310 connecting the left and right sound output units 200 </ b> L and 200 </ b> R, an image output unit support arm 320 provided at one end of the band 310, and a sound detection device. And a support arm 333.
[0024]
The band 310 has a curved shape so that the head 1 of the user U can be sandwiched from behind. The sound output units 200L and 200R are connected and fixed to both ends. The band 310 includes a main body 311 and an image output unit support arm housing 312 that houses the image output unit support arm 320 so that the image output unit support arm 320 can be retracted. The band 310 is provided with wiring (not shown) connected to the cable 800 from the information processing device D. Some of these wirings are connected to the image output unit 100 and the sound detection device 470, and others are connected to the left and right sound output units 200L and 200R. Further, there are also wirings leading to a gaze detection device 900 and a head movement detection device 700 described later.
[0025]
In the present embodiment, an opening 312b is provided in the image output unit support arm housing 312 adjacent to the sound output unit 200L. The image output section support arm 320 can protrude and retract from the opening 312b. That is, the image output unit 100 is connected to the band 310 by a slide mechanism including the image output unit support arm 320 and the image output unit support arm housing 312. Although not shown, the opening 312b is provided with a stopper (not shown) for preventing the image output unit support arm 320 from dropping out when pulled out. In addition, a guide rail (not shown) is provided in the image output unit support arm storage unit 312 in order to smoothly move the image output unit support arm 320 in and out. As a result, the image output section support arm 320 slides smoothly.
[0026]
A motor 391 that electrically slides and accommodates the image output unit support arm 320 is attached to the image output unit support arm accommodation unit 312 of the band 310.
[0027]
FIG. 4 is a sectional view of the motor unit 390 to which the motor 391 is attached.
[0028]
A roller 393 is attached to a motor shaft 392 of the motor 391, and the roller 393 rotates around the motor shaft 392 in conjunction with the rotation of the motor 391.
[0029]
The roller 393 is in contact with the side surface of the image output unit support arm 320, and when the roller 393 rotates, the image output unit support arm 320 moves due to frictional force. In FIG. 2, when the roller 393 rotates counterclockwise, the image output unit support arm 320 is pushed into the image output unit support arm accommodating unit 312 to be in the state of FIG.
Conversely, when the roller 393 rotates clockwise, the image output unit support arm 320 is pulled out from the image output unit support arm housing 312, and the state shown in FIG. 2 is obtained.
[0030]
The material of the friction surface of the roller 393 may be any material that generates a predetermined frictional force, and is preferably an elastic body such as rubber. However, if the frictional force is too large, the user cannot manually put the image output unit support arm 320 in and out, which is not preferable.
(Head motion detector)
The head movement detecting device 700 is attached to the center of the band 310, that is, a position that comes to the position of the back of the head when worn by the user.
[0031]
A vertical gyro sensor and a horizontal gyro sensor for detecting vertical and horizontal accelerations, angular velocities, or angular accelerations are mounted on a circuit board of the head movement detecting device 700. With this attachment, the head movement (vertical swing) in the body axis direction and the head movement (lateral swing) in the direction orthogonal to the body axis can be detected. As a gyro, an optical gyro that utilizes the fact that the laser beam circulates with a mirror or fiber inside the casing and the timing of receiving light changes from the emission of the laser circling inside when the casing changes direction; A mechanical gyro that detects angular displacement through application; a vibrating method that utilizes the fact that when an angular velocity is applied to a vibrating (primary vibration) mass, vibration (secondary vibration) also occurs in a direction perpendicular to it due to the "Coriolis" effect Gyro; etc. can be used. The detection result of the head movement detection device 700 is sent to the central control device 854 of the information processing device D.
(Gaze detection device)
The gaze detection device 900 is attached to a surface of the image output unit 100 facing the user. As shown in FIG. 5, the gaze detection device 900 includes an infrared light irradiating unit 901 that irradiates the user's eye 2 with infrared light, a dichroic mirror 904 that reflects infrared light out of light reflected by the user's cornea, It comprises a condenser lens 903 for condensing light reflected from the dichroic mirror 904 and a visibility sensor 902 for detecting condensed infrared light. The gaze detection device 900 configured in this manner illuminates the user's eye 2 with infrared light, and determines the center position of the pupil and the position of the reflected light from the infrared light reflected by the cornea and the image of the entire eyeball, The method employs a method of calculating the rotation angle of the eyes and calculating and finding where in the image the user is looking.
(Voice detection device)
The voice detection device 470 is connected to the band 310 via a voice detection device support arm 333. The voice detection device 470 is a device that converts a voice signal emitted by a user into an electric signal. For example, a ribbon-type microphone that generates power using the movement of a ribbon-shaped foil in a magnetic field; a dynamic-type (moving-coil-type) microphone that moves and generates power by moving a coil in a magnetic field; A condenser microphone that converts a sound pressure into an electric signal by using a capacitance; a carbon microphone that picks up sound by changing a contact resistance between carbon particles by a sound pressure can be used.
(Information processing device)
As shown in FIG. 6, the information processing device D sends an image output signal to the image output unit 100, and sends an image output signal processing device 851 that processes the image display signal and an audio output signal to the audio output unit 200. Output signal processing device 852 for managing the information processing device, a central control device 854 for controlling the entire information processing device, and a power supply control circuit 856 for controlling power on / off and power saving modes, and charging the battery 857. Further, it has an input / output interface 858 for exchanging information signals with the HMD main body.
Further, it has a main switch SW for turning on and off the power supply. Further, a connector CN for connecting a plug 810 of the cable 800 for exchanging signals with the HMD is provided. A rechargeable battery 857 is prepared as a power supply.
[0032]
The information processing device D is provided with a storage device 853 for storing image information, sound information, and the like that are the basis for generating the image display signal and the sound output signal. In addition, a communication device for accepting image information and audio information from outside may be provided as necessary.
(Storage device)
The storage device 853 stores the following information in addition to the image information and the sound information.
[0033]
The storage device 853 stores a standard pattern of a vertical or horizontal acceleration change detected when the user shakes the head vertically or horizontally. This pattern is used in motion input processing 8542 performed by the central control device 854 shown in FIG. 7 to determine whether the user has swung the head vertically or horizontally. In addition, the manner in which the user's head is swung is not always constant but has a certain range.
Therefore, a threshold value is provided, and if the deviation from the pattern is within the threshold value, it is considered that the pattern matches the pattern. This threshold is also stored in the storage device 853.
[0034]
Further, information on a specific operation of the HMD to be performed when it is determined that the head is swung vertically or horizontally is stored. That is, a specific action form of the user and a specific operation of the head mounted display to be performed in accordance with the specific action form are stored as a set. For example, when the head is swung vertically, a specific menu selection window is output as an image.
[0035]
Further, the storage device 853 stores an acoustic model for voice recognition.
This acoustic model is used in the voice input processing 8544 performed by the central control device 854. The acoustic model is a so-called speech word dictionary created based on average pronunciation data. Speech recognition is performed by matching an acoustic model with a speech signal (waveform) input by a user. The details will be described together with the description of the voice input processing.
[0036]
Further, information on a specific operation of the HMD to be performed when the user utters a voice or a specific word is stored. For example, when the user says "volume", a volume adjustment menu is output as an image.
[0037]
In addition, a specific operation of the HMD according to a combination of a head movement, a voice input, and a gaze input is also stored. For example, when a voice input is performed after a head motion; when a head motion is performed after a voice input; when a voice input and a head motion are performed substantially simultaneously; The specific operation of the HMD to be performed is stored.
[0038]
(Central control unit)
The central control device 854 manages the entire information processing device D. In addition, as shown in FIG. 7, an input determination process 8541, a motion input process 8542, a line-of-sight input process 8543, and a voice input process 8544 are performed.
[0039]
The input determination process 8541 is a process of determining which method the user has input. The HMD of the present embodiment performs a specific operation according to a specific action of the user. The user can instruct the HMD by a method of moving the head, a method of moving the line of sight, or a method of emitting a voice. The motion of the head is detected by the motion detection device 700. The movement of the line of sight is detected by the line of sight detection device 900. The voice emitted by the user is detected by the voice detection device 470. Then, based on these detection results, the input determination processing 8541 determines how the user's action was performed.
[0040]
The motion input processing 8542 determines whether or not the user has shaken the head vertically or horizontally from the detection result of the head movement detection device 700. To the central control device 854, the detection results of the vertical acceleration change and the horizontal acceleration change of the user's head are sent from the head movement detecting device 700. If a vertical or horizontal acceleration change is detected, the motion input processing 8542 checks the acceleration change pattern against a standard vertical or horizontal swing pattern stored in the storage device 853. If they match or are within a predetermined threshold range, it is determined that the user has swung the head vertically or horizontally.
[0041]
Then, based on the determination result, each processing device is instructed to perform the operation of the HMD to be performed accordingly.
[0042]
In the eye gaze input processing 8543, it is determined where the user is looking on the screen based on the detection result from the eye gaze detecting device 900. The direction of the user's line of sight can be known from the detection result of the line of sight detection device 900. The position of the user's viewpoint on the screen is obtained from the positional relationship between the user's eyes and the image output unit and the direction of the line of sight.
[0043]
Further, when a screen listing options is output, if the viewpoint does not move from the position of a certain option for a predetermined time, it is determined that the option has been selected, and the process proceeds to the next process.
[0044]
In the voice input processing 8544, the voice generated by the user is analyzed based on the detection result of the voice detection device 470. It has a step of simply determining whether or not a voice is uttered, and a step of voice recognition for analyzing the meaning of the uttered word.
[0045]
Whether or not a sound is simply emitted can be determined based on whether or not the sound detection device 470 has detected a sound having a volume equal to or higher than a predetermined value. However, even if the user does not emit a voice due to external noise or the like, the user may reach the predetermined place, so that the predetermined value may be appropriately changed.
[0046]
The voice recognition is performed by matching (collating) a voice signal (waveform) emitted by the user with an acoustic model stored in the storage device 853. The matching is performed sequentially from the beginning of the word, usually in units of 10 to 20 milliseconds. For example, when recognizing the word "volume", the word "o" is recognized first. Matching candidates are narrowed down to words starting with “O” in the acoustic model. Next, when "on" is recognized, the candidates are narrowed down to music, sound, voice, syllable, etc., and when "onri" is recognized, the candidates are narrowed down to volume, etc., and finally the best matching is performed. The word is output.
[0047]
When the speech recognition is performed, each processing device is instructed to perform an HMD operation to be performed according to the meaning of the word.
(flowchart)
FIG. 8 is a diagram illustrating a flow of an operation when a head movement, a gaze movement, and a voice input are performed in the HMD according to the present embodiment.
[0048]
In step S100, the presence or absence of a voice input is determined by the voice detection device 470. If a voice input has been made, voice recognition is performed in step S101. Next, in step S102, what the user is instructing (commanding) is recognized based on the result of the voice recognition, and in step S103, each control device is instructed to execute the command. These processes are performed in the process of the voice input process 8544 of the central control device 854.
[0049]
Further, in step S200, it is determined whether or not there is a line-of-sight input. When the line of sight has been input, the gaze position of the user is detected in step S201. Next, in step S202, it is specified which option on the screen the user has selected. Subsequently, in step S203, each processing device is instructed to execute the content of the option. These processes are performed in the course of the line-of-sight input process 8543 of the central control device 854.
[0050]
In step S300, it is determined whether there is an input due to the movement of the head. If the input is made based on the movement of the head, in step S301, the type of the movement of the head (for example, vertical swing or horizontal swing) is recognized. Next, in step S302, the instruction content corresponding to the instruction is specified. Subsequently, in step S303, each control device is instructed to execute the instruction. These processes are performed in the process of the operation input process 8542 of the central control device 854.
(Example of Operation of First Embodiment)
An example of the operation of the HMD configured as described above will be described.
[0051]
The HMD according to the present invention is characterized in that it operates by reliably reflecting the user's intention by a combination of head movement detection, gaze detection, and voice detection.
[0052]
Hereinafter, an example will be described in which the intention of the user is reliably read, the image output is turned off, and the image output unit 100 is evacuated by a combination of head movement detection, gaze detection, and voice detection.
When the user turns off the image output and wants to evacuate the image display unit 100 from his / her eyes, the HMD according to the present embodiment can perform the following operation.
[0053]
When, for example, the word “OFF” is input from the user to the voice detection device 470, the voice signal is recognized as voice. As a result, it is recognized that the command is a command for requesting image output of the OFF target selection window 151. Then, execution of this command is instructed to the image output signal processing device 851.
[0054]
In response to this, the image output signal processing device 851 sends an image signal to the image output unit 100 to display the OFF target selection window 151. Then, the image output unit 100 outputs the off target selection window 151 as shown in FIG.
[0055]
This allows the user to observe the off target selection window 151 on the display screen 150 as shown in FIG. In the OFF target selection window 151, options are listed as to which of the power supply, the sound output, and the image output is turned off. Also, options for saving the image output unit are listed. The arrow icon 159 moves in conjunction with the user's viewpoint. The user moves the arrow icon 159 so that the one to be turned off is overlaid on the option. At this time, an audio guide such as “Please pay attention to options” may be output as audio. If there is a guide by voice output, the user can understand the operation method and can proceed smoothly and quickly to the next processing.
[0056]
The user adjusts the viewpoint to the option to be selected for a certain time.
[0057]
The gaze detection device 900 detects the gaze of the user and sends the detection result to the central control device 854. If the viewpoint does not move for a certain period of time or longer, the option at the position of the viewpoint on the screen is selected, and the option is specified.
[0058]
Next, as shown in FIG. 10, a confirmation screen 152 for confirming whether the option has been correctly selected is displayed. At this time, a voice guide such as "Please shake your head vertically if you like" may be output as voice. This informs the user what to do next, which makes the operation easier for the user and allows the user to proceed to the next process quickly and reliably. If the selection is correct, the user shakes his head vertically. If not, shake your head.
[0059]
At this time, if the head is swung vertically, the central controller 854 recognizes that the head is determined to be correct, and sets so as to proceed to the next processing. On the other hand, if the head is shaken sideways, it is recognized that the head is incorrect, and the processing is set so as to return to the off target selection window again.
[0060]
That is, when the user shakes his / her head vertically, it is recognized that the selection is correct, and the central control device 854 instructs each processing circuit to execute the content of the option.
[0061]
Here, for example, when the user selects the option of “image display off, evacuating the image output unit”, the central control device 854 turns off the image output, the motor 391 is driven, and the image output unit 100 Is instructed to evacuate from the user's eyes.
[0062]
As a result, the image output is turned off, and the image output unit 100 is separated from the user's eyes and retracted (see FIG. 3).
[0063]
As described above, according to the present embodiment, not only the movement of the head but also the detection of other actions of the user are combined, so that the intention of the user is reliably read, and the operation of the HMD is performed according to the intention of the user. You.
[0064]
In the above example, the voice detection result is assigned to the display of the display screen including a plurality of options, and the gaze detection result is assigned to the reception of an instruction to execute a specific operation of the head-mounted display, and the head movement Is assigned to confirm the received instruction content before executing the instruction content.
[0065]
The combination of user actions performed to instruct the HMD to perform a specific operation is not limited to the above.
[0066]
For example, a specific operation of the HMD may be performed according to a combination of two or more types of head movements, such as when the head is swung vertically and immediately swung horizontally. For example, in the case where the head is swung once in the vertical and horizontal directions and once in the vertical direction, the power may be turned off or the image output unit 100 may be evacuated from the user's eyes.
[0067]
Also, a combination of the same actions such as shaking the head twice vertically may be used, or a direction of the head movement such as shaking the head right first and then the head forward may be used.
[0068]
Further, a specific operation of the HMD may be performed according to the order of the user's actions. That is, a specific operation of the HMD may be performed by distinguishing whether two or more user actions are performed at different timings or in parallel. For example, a gaze input is performed after the head is moved; a head movement and a voice input are performed at the same time; a speech input or a gaze input is performed after the head is moved; In this case, the HMD may perform a specific operation.
[0069]
Further, it is preferable that the user can register what kind of action the HMD performs when the user performs the action.
[0070]
The registration can be performed by selecting an option on the screen with a line of sight and registering it. For example, an option listing the correspondence between the specific action form of the user and the specific operation of the HMD may be output to the screen and selected.
[0071]
Then, a specific action form of the user is associated with a specific operation of the head mounted display to be performed in accordance with the specific action form, and a set is stored in the storage device 853.
[0072]
Further, the specific operation of the HMD performed by the action of the user is not limited to, for example, turning off the sound and the image output. For example, when the user utters “soft”, as shown in FIG. 11, an application software screen 153 may be output as an image, and an operation such as opening a file in the computer may be performed on the screen. In this case, the arrow icon 159 is moved depending on the line of sight, and an operation target option, for example, “open file” is selected. After that, the screen may be shifted to a confirmation screen, and may be determined by swinging the head vertically. As described above, by instructing the user by a plurality of actions, the HMD can reliably read the user's intention and perform a specific operation without malfunction.
[0073]
The HMD of the embodiment described above is of a type in which the information processing device D is connected via a cable. However, the HMD to which the present invention is applied is not limited to this. For example, as shown in FIG. 12, the present invention can be applied to a device having a structure in which an information processing device D is mounted on an HMD. Further, the present invention can be applied to a configuration in which the information processing apparatus D and the HMD are wirelessly connected. In the latter case, a circuit and a power supply for wireless communication are mounted on the HMD. Also, a circuit for wireless communication is mounted on the information processing device side.
[0074]
In the above embodiment, an example having both the image output unit and the sound output unit has been described. The application of the present invention is not limited to this. For example, the present invention is also applicable to an HMD having only an image output unit.
[0075]
Further, in the above embodiment, the sound detection device 470 is arranged independently of the image output unit 100, but the sound detection device may be incorporated in the image output unit. In this case, the voice detection device support arm 333 becomes unnecessary, and the mechanism can be simplified. In this case, it is desirable to set the directivity of the sound detection device downward so that the sound can be reliably picked up.
[0076]
Further, the present invention is not limited to a single-eye HMD. The present invention is also applicable to a goggle type HMD that outputs an image to both eyes of a user.
[0077]
In the HMD of the present embodiment, since the image output unit support arm 320 can be accommodated in the image output unit support arm accommodation unit 312 of the support unit 300, the image output unit support arm 320 does not become an obstacle when not in use. It is convenient in such cases.
[0078]
【The invention's effect】
According to the present invention, there is provided an HMD that performs a specific operation of the HMD such as an audio output and an image output in a hands-free manner and with less malfunction according to a user's intention and reliably.
[Brief description of the drawings]
FIG. 1 is a perspective view showing an appearance of a head mounted display according to a first embodiment of the present invention.
FIG. 2 is a sectional view showing an outline of the structure of the head mounted display according to the first embodiment of the present invention.
FIG. 3 is a sectional view showing an outline of the structure of the head mounted display according to the first embodiment of the present invention.
FIG. 4 is a sectional view of a motor unit.
FIG. 5 is a diagram showing a structure of a visual line detection device.
FIG. 6 is a diagram showing components of a head mounted display and an information processing apparatus according to the first embodiment of the present invention.
FIG. 7 is a diagram showing components of an information processing apparatus applicable to the head mounted display according to the first embodiment of the present invention.
FIG. 8 is a flowchart showing a flow of processing according to a user action in the head mounted display according to the first embodiment of the present invention.
FIG. 9 is a view showing an example of a screen for setting an OFF operation mode;
FIG. 10 is an example of a confirmation screen.
FIG. 11 is a view showing a state where an option is selected with a line of sight on the screen of the application software.
FIG. 12 is a perspective view showing the appearance of a head mounted display integrated with an information processing apparatus according to the present invention.
[Explanation of symbols]
D: information processing device, CN: connector, SW: power switch,
U: User, 1: Head, 2: Eye
100 image output unit,
150 ... display screen, 152 ... off target selection window, 153 ... software application software screen, 159 ... arrow icon,
200 (200L, 200R) ... sound output unit,
Reference numeral 300: support portion, 310: band, 311: band body, 312: image output portion support arm accommodation portion, 312b: opening, 320: image output portion support arm,
333: voice detection device support arm,
390: motor unit, 391: motor, 392: motor shaft, 393: roller,
470 ... voice detection device,
700: motion detection device,
800 ... cable,
810: plug,
851 image output signal processing device, 852 audio output signal processing device, 853 storage device
854: central control device, 8541: input determination process, 8542: motion input process, 8543: gaze input process, 8544: voice input process,
856: power control circuit, 857: battery, 858: input / output interface,
900: line-of-sight detection device, 901: infrared light irradiation unit, 902: field-of-view sensor, 903: condenser lens, 904: dichroic mirror

Claims (12)

An image output unit that outputs an image, a head mounted display that can be mounted on the head,
An instruction input unit that detects a specific action of the user and receives an instruction from the user;
Control means for performing a specific operation of the head mounted display according to the instruction input by the instruction input means,
The head-mounted display, wherein the instruction input unit has a motion detecting function of detecting a motion of a user's head. The head mounted display according to claim 1, wherein
The motion detection function, at least, detects the movement of the head around the body axis direction, and the movement of the head around the axis orthogonal to the body axis,
The control means is controlled in accordance with a combination of the movement of the head around the axis in the body axis direction detected by the movement detection means and the movement of the head about an axis orthogonal to the body axis. A head mounted display characterized by the following. The head mounted display according to any one of claims 1 and 2,
The instruction input unit further has a line-of-sight detection function of detecting the movement of the line of sight of the user,
The head mounted display, wherein the control means performs control in accordance with a result of motion detection by the motion detection function and a result of visual line detection by the visual line detection function. The head mounted display according to claim 3, wherein
The control unit performs control such that an instruction to execute a specific operation of the head-mounted display is received in accordance with one of the result of the line-of-sight detection and the result of the movement detection. Alternatively, a head-mounted display is controlled so as to request the user to confirm a received instruction before executing the instruction in accordance with one of the results of the motion detection. The head mounted display according to any one of claims 1 to 4, wherein
The instruction input means further has a voice detection function of detecting a voice emitted by the user,
A head-mounted display that performs control according to at least a motion detection result by the motion detection function and a voice detection result by the voice detection function. The head mounted display according to any one of claims 1 and 2,
The instruction input unit further has a line-of-sight detection function for detecting the movement of the user's line of sight, and a sound detection function for detecting a sound emitted by the user,
The control means controls the display output unit to display a display screen including a plurality of options according to a sound detection result by the sound detection function,
In accordance with the gaze detection result, control is performed to accept a selection from the plurality of options,
A head-mounted display, wherein control is performed to confirm the content of the reception in accordance with a detection result of a head movement. The head mounted display according to any one of claims 1 to 6, wherein
The head mounted display, wherein the instruction input unit has a function of detecting at least two user actions performed at different timings. 7. The head mounted display according to claim 1, wherein said instruction input means has a function of detecting at least two user actions performed in parallel. The head mounted display according to any one of claims 1 to 8,
The specific operation of the head-mounted display is a change in at least one of an audio output and an image output. It is a head mounted display according to any one of claims 1 to 9,
The head mounted display according to claim 1, wherein the specific operation of the head mounted display is one of power on and off. The head mounted display according to any one of claims 1 to 10, wherein
The image output unit further includes a unit that can be switched between a state in which the user is positioned in front of the user and a state in which the image output unit is out of front and retracted by power,
The head mounted display according to claim 1, wherein the specific operation of the head mounted display switches between a state in which the image output unit is positioned in front of the user and a state in which the image output unit is separated from the user and retracted. The head mounted display according to any one of claims 1 to 11,
The apparatus further includes means for registering and storing a set of a specific action form of the user and a specific operation of the head mounted display to be performed in response thereto,
The control means, when it is determined that the specific action of the user detected by the instruction input means and the form of the stored specific action of the user match, the stored corresponding to the action A head-mounted display which controls so that a specific operation of the head-mounted display to be performed according to the action is performed.

Images