JP2001183968A - Pseudo experiencing device and pseudo experiencing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable an experiencing person to obtain a presence feel even if a ride means to be boarded by the experiencing person is oscillated in accordance with the movement over the entire part of videos. SOLUTION: This experiencing device has the ride means 7 which is arranged in front of a display means 3 and on which the experiencing person is placed, an oscillation means 8 which oscillates the ride means 8, an oscillation control means 8 which controls the drive of the oscillation means 8 and a moving vector detecting means 5 which detects the movement of the videos from the video signals of a video supply source 1. The oscillation control means 8 has the characteristics to impart the stimuli of the intensity meeting the movement of the videos detected by the moving vector detecting means 5 to the experiencing person and to gradually weaken the intensity of the stimuli when the intensity of the stimuli is maximum and continues for the prescribed time or longer.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an experience apparatus and a method for simulating a simulated experience by giving a human a shaking according to an image.

[0002]

2. Description of the Related Art In recent years, research and development of virtual reality technology have been actively conducted for the multimedia industry. Virtual reality is a technology that gives the illusion of being in a remote place even if it is in a remote place with video and audio, and is being used in various fields such as games, music, and movies.

[0003] The virtual experience device using the virtual reality technology is capable of realizing movement, vibration,
By reproducing software created to resemble video and audio, it is possible to provide a simulated experience to humans repeatedly and safely.

[0004] However, the software used in such a simulated experience device appeals to human sensibility, and therefore requires specialized know-how in its creation, and cannot be easily created.

[0005] Therefore, in software development for simulated experience, in order to reduce the man-hour and increase versatility, it is conventionally considered that the software is described in, for example, JP-A-11-153949.

The moving image detecting means detects a moving image which is a moving image of the image from a video signal, an audio detecting means detecting an audio corresponding to the moving image from an audio signal, and detecting the motion of the human. Motion detecting means, a motion generating means for generating respective motion signals from the motion video, the sound, and the motion of the human, and performing motion driving of a vehicle on which the human is mounted based on the motion signal. And a reproduction effect control unit for controlling a reproduction effect of the video signal and the audio signal based on the motion of the person.

With this configuration, since the motion drive control is performed based on actual measurement values such as motion video, audio, and human motion instead of software data dedicated to motion drive, the number of software development steps can be greatly reduced. In addition, since it does not depend on dedicated software, versatility can be improved.

[0008]

However, simply controlling the motion drive of the vehicle based on the motion of the video does not provide a realistic motion.

That is, in order for the user (passenger) to feel as if he or she is moving, it is obtained not by the movement of the subject at the center or the center of the image but by the movement of the scenery such as the surrounding part or the surrounding background. (The Journal of the Institute of Television Engineers of Japan, Vol. 50, No. 4, pp. 429-
435 (1996) "Reaction between image and body").

[0010] For this reason, even if the motion drive is simply controlled in accordance with the movement of the video, the sense of realism is not necessarily obtained for the user.

[0011] Further, if the stimulus is continuously given to the user, the user becomes accustomed to the stimulus, and the sensation of realism is reduced. Furthermore, the driving range of the vehicle is also limited, and if the vehicle is driven to the limit and then further driven, the vehicle cannot be driven, and the stimulus may not be given to the user.

The present invention has been made to solve the above-mentioned problem and to provide a more realistic experience.

[0013]

A simulated experience apparatus according to the present invention comprises a display means for displaying a video, a video signal supply means for supplying a video signal to the display means, and a video signal supplied from the video signal supply means. Motion detecting means for detecting the motion of the subject, and a pseudo effect means for giving a stimulus to the user, wherein the pseudo effect means generates the stimulus having a strength corresponding to the motion of the image detected by the motion detecting means. Give to the experienced person,
When the intensity of the stimulus is maximum and continues for a predetermined time or more, the intensity of the stimulus is gradually reduced.

[0014] The pseudo effect means is arranged in front of the display means, and ride means for placing an experienced person thereon, oscillating means for oscillating the ride means to stimulate the experienced person, Swing control means for controlling the drive of the swing means,
The swing control means drives the swing means in accordance with the movement of the image detected by the movement detection means, and when the maximum movement of the swing means continues for a predetermined time or more, the swing control means It is characterized in that the means is gradually returned to the center.

Further, when the movement of the image detected by the movement detecting means suddenly increases, the rocking control means determines whether or not the riding means is to be rocked in a direction corresponding to the movement of the image. It is characterized by slightly swinging in the opposite direction.

Further, in the simulated experience method according to the present invention, the step of detecting the movement of the image displayed on the display means, and the simulated effect means stimulates the user with a stimulus having an intensity corresponding to the movement of the image detected in the step. And when the intensity of the stimulus is maximum and continued for a predetermined time or more in the step,
And gradually decreasing the intensity of the stimulus.

Further, the step of providing the stimulus to the user by the pseudo effect means includes, when the movement of the image detected by the movement detecting means suddenly increases, the ride means in a direction corresponding to the movement of the image. The method includes a step of slightly swinging in the opposite direction to the above direction before swinging.

[0018] Further, the step of providing the stimulus to the user by the pseudo effect means includes, when the movement of the image detected by the movement detecting means suddenly increases, swinging the ride means in a direction corresponding to the movement of the image. The method includes a step of slightly swinging in the opposite direction to the above direction before moving.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIG.
A video supply source serving as a video signal supply means for a ROM, a TV broadcast, a TV game, or the like; a video capture means for capturing a video signal from the video supply source in at least one frame unit; Display means such as a liquid crystal display, 4
Reference numeral denotes display control means for displaying the video signal captured by the video capturing means 2 on the display means 3.

5 is a motion vector detecting means for detecting a motion vector from the video captured by the video capturing means 2, and 6 is a swing control means for generating a motion signal in accordance with the motion vector detected by the motion vector detecting means 5. , 7 have a chair portion 10 on which the experienced person 9 rides. For example, ride means for tilting the chair portion 10 forward, backward, left and right from a center position to swing the chair portion 10 so that the experienced person 9 can experience the body, and 8 is swing control means. The swing means swings the ride means 7 in response to the motion signal generated in step 6.

The motion vector detecting means 5 divides, for example, one screen of the display means 3 into a plurality of areas, detects a moving amount and a direction of luminance of each pixel in each area, and averages the moving amount in the area. And extract it as a motion vector.

FIG. 2 shows the swing control means 6 in detail. Reference numeral 61 denotes a central part of one screen displayed on the display means 3 for displaying the motion vector extracted by the motion vector detection means 5,
Classification unit for classifying into the peripheral part of the center part and the lower part on the screen, 6
Reference numeral 2 denotes a peripheral average motion vector calculation unit that calculates the average of the peripheral motion vectors classified by the classification unit 61. Reference numeral 63 denotes a central average motion vector that calculates the average of the central motion vectors classified by the classification unit 61. The calculation unit 64 is a classification unit 61
A lower-average-motion-vector calculator that calculates the average of the lower-motion vectors classified by the above; a peripheral-motion determiner 65 that determines whether the average calculated by the peripheral-average-motion-vector calculator 62 is smaller than a first predetermined value; 66 is a central motion determining unit that determines whether the average value calculated by the central average motion vector calculating unit 63 is smaller than a second predetermined value, 67 is an average value calculated by the lower average motion vector calculating unit 64 is a third predetermined value A lower motion determining unit 68 for determining whether the motion is smaller than the motion of the video, 68 is an irrelevant motion generator that generates motion irrelevant to the motion of the video when the motion of the video is entirely small, and 69 is an average motion vector calculator 62, 63. , 64 is an operation signal generation unit that generates a motion signal based on the average value calculated in 64.

The operation of the swing control means 6 is such that the motion vectors extracted by the motion vector detection means 5 are classified by the classification unit 61.
Is classified into each part, and the peripheral part average motion vector calculation part 6
2. The average motion vector is calculated by the center average motion vector calculation unit 63 and the lower average motion vector calculation unit 64.

First, the peripheral motion determining section 65 determines whether or not the peripheral motion vector is small. If the peripheral motion determining unit 65 determines that the distance is small, it issues a determination instruction to the central motion determining unit 66.
The average value calculated by the peripheral average motion vector calculation unit 62 is sent to the motion signal generation unit 69 to generate a motion signal.

The center motion judging section 66 makes a judgment in accordance with an instruction from the peripheral motion judging section 65. As a result of the determination by the central motion determining unit 66, if it is determined that the size is small, a determination instruction is issued to the lower motion determining unit 66, and if it is determined that the size is not small,
The average value calculated by the central average motion vector calculation unit 63 is sent to the motion signal generation unit 69, and a motion signal is generated.

The lower motion judging section 67 makes a judgment in accordance with an instruction from the central motion judging section 66. As a result of the determination by the lower motion determining unit 67, if it is determined that the value is smaller, an operation instruction is issued to the irrelevant motion generating unit 68. The signal is sent to the generator 69 to generate a motion signal.

Further, the unrelated motion generating section 68 generates a motion vector of a motion irrelevant to an image in accordance with an instruction from the lower motion determining section 67 and sends it to the motion signal generating section 69.
Generate a motion signal.

The operation of the swing control means 6 has been described above.

Reference numeral 11 denotes effect control means for controlling the driving of the effect device in accordance with the motion signal from the swing control means 6;
2 is a speaker 1 that outputs audio accompanying the video from the video source 1
Voice output means for outputting to 3; ride means 7;
A control signal from the effect control means 11, that is, a blowing means for controlling air blowing to the user in accordance with the movement of the image, in order to further enhance the sense of realism for the experienced person 9 who rides on the vehicle, and 15 is a control from the effect control means 11 A sound image control means 16 for controlling a sound image of a sound output from the speaker 13 in accordance with a signal, and a light means 16 for irradiating the user with light in accordance with a control signal from the effect control means 11 to enhance a sense of reality.

Note that the swing control means 6 responds to the motion vector detected by the motion vector detection means 5 by the ride means 7,
The user 9 feels a sense of reality through the speaker 13, the blowing means 14, and the light means 16, and gives a stimulus to enhance the realism. 11, the speaker 13, the blowing means 14, the sound image control means 15, and the light means 16 correspond to the pseudo effect means of the present invention.

The operation of the above configuration will be described with reference to FIG.

First, in step S1, the image capturing means 2 captures an image from the image supply source 1. In step S2, the motion vector detecting means 5
Extracts a motion vector. In step S3, the motion vector extracted in step S2 is taken into the swing control means 6, and the classification unit 61 classifies the motion vector into a central portion, a peripheral portion of the central portion, and a lower portion of the screen.

In step S4, an average peripheral motion vector is calculated by the peripheral average motion vector calculator 62.
In step S5, the central portion average motion vector is calculated by the central portion average motion vector calculation section 63. Step S6
Then, the average motion vector at the bottom of the screen is calculated by the lower average motion vector calculation unit 64.

In step S7, the peripheral motion determining section 65 determines whether or not the peripheral motion vector is smaller than a first predetermined value. If it is determined that it is not smaller, the process proceeds to step S8. In step S8, the surrounding motion determining unit 65 sends the average motion vector calculated by the surrounding average motion vector calculating unit 62 to the motion signal generating unit 69 to generate a motion signal and, based on the motion signal, the swing unit 8 Swings the ride means 7.

When the peripheral motion determining section 65 determines in step S7 that the motion vector of the peripheral portion is smaller than the first predetermined value, the process proceeds to step S9. In step S9, the central motion determining section 66 determines whether the central motion vector is smaller than a second predetermined value. If it is determined that it is not smaller, the process proceeds to step S10. In step S10, the central motion determining unit 66 determines that the central average motion vector calculating unit 63
Is sent to the motion signal generation unit 69 to generate a motion signal, and at step S8
Similarly, the rocking means 8 rocks the ride means 7 based on this motion signal.

When it is determined in step S9 that the center motion vector is smaller than the second predetermined value, the process proceeds to step S11. In step S11, the lower motion determiner 67 determines whether the motion vector at the lower part of the screen is smaller than a third predetermined value. If it is determined that it is not smaller, the process proceeds to step S12. Step S12
Then, the lower motion determining unit 67 calculates the average motion vector calculated by the lower average motion vector calculating unit 64 by using the motion signal generating unit 6.
9 to generate a motion signal, and, similarly to step S8, the rocking means 8 rocks the ride means 7 based on the motion signal.

When the lower motion judging section 67 judges in step S11 that the motion vector at the lower portion of the screen is smaller than the third predetermined value, the process proceeds to step S13. Step S13
Then, the motion vector for giving the periodic shake to the ride means 7 via the swing means 8 regardless of the video is generated by the unrelated motion generation unit 68. The motion signal generating section 69 generates a motion signal in accordance with the motion vector generated by the generating means 68, and based on the motion signal, the oscillating means 8
Swings the ride means 7.

In steps S8, S10 and S12 described above, the motion signal generated by the operation signal generating section 69 is also sent to the effect control means 11, and the effect control means 11 sends the motion signal to the blowing means 1 in accordance with the motion signal.
4. By controlling the sound image control means 15 and the light means 16,
The rider 7 is made to work on the rider 7 so as to enhance the sense of reality.

The operation signal generator 69 of the swing control means 6 in steps S8, S10 and S12 described above.
Will be further described with reference to FIG.

First, in step S20, it is determined whether or not the motion of the video has rapidly increased. If it is determined that the size has increased rapidly, the process proceeds to step S21. If it is determined that the size has not increased rapidly, the process proceeds to step S21.
Move to 23.

In step S21, the swing means 8 is controlled so as to slightly move the ride means 7 in the direction opposite to the direction of the movement detected in step S2. Here, "slightly" means
For example, the magnitude of the motion of the video detected in step S2 is 1
/ 10.

Next, in step S22, the ride means 7 is moved in the direction of the motion of the image detected in step S2 and in accordance with the magnitude of the motion obtained by adding the magnitude of the slight motion in step S21 to the magnitude of the motion. Oscillating means 8 to oscillate
To work.

In step S23, it is determined whether or not the ride means 7 is tilted from the center position to any of the front, rear, right and left positions. If it is determined that it is tilted, step S24
The process proceeds to step S2 if it is determined that the vehicle is not tilted.
Move to 6.

In the step S24, it is determined whether or not the inclination of the ride means 7 is kept the same for a predetermined time. If it is determined that the predetermined time has been maintained, the process proceeds to step S25. If it is determined that the predetermined time has not been maintained, the process proceeds to step S26. In step S25, the ride unit 7
Is maintained in the same inclined state, the rider 7 is often used to the experience of the state.
The swinging means 8 is controlled so as to gradually return to the center position.

In step S26, it is determined whether or not the inclination of the ride means 7 in the front, rear, left and right directions is at the limit position. If it is determined in this step that the limit is reached, the process proceeds to step S27. In step S27, if the ride means 7 is at the limit position, even if there is further movement of the image, it cannot be moved, so that the swing means 8 is controlled to gradually return the ride means 7 to the center position. In step S28, the swing control means 6 controls the ride means 7 to reproduce the movement other than the sudden movement among the movements detected in step S2.

By repeating steps S1 to S13 and steps S20 to S28 each time an image is captured in step S1, the user 9 can experience the experience.

In the above-described embodiment, the classifying unit 61 determines the motion vector extracted by the motion vector detecting means 5 in a central part, a peripheral part of the central part, and a lower part on the screen of one screen displayed on the display means 3. However, the present invention is not limited to this, and the subject and the background in the video may be classified. If the classifying unit 61 classifies the subject and the background of the image, the same configuration as described above can be realized by replacing the central part in the above-described embodiment with the subject and the surrounding part as the background.

In the above-described embodiment, the display means 3 has two
Although the configuration for displaying a three-dimensional video has been described, the present invention is not limited to this, and may be applied to a configuration in which a video signal obtained by converting a two-dimensional video into a three-dimensional video is displayed on the three-dimensional display unit 18.
This embodiment is shown in FIG. Note that the components denoted by the same reference numerals as those in FIG.

Reference numeral 17 denotes a 2D for generating a right-eye image and a left-eye image having a parallax amount corresponding to the motion vector extracted by the motion vector detecting means 5 based on the image captured by the image capturing means 2. Reference numeral 18 denotes a 3D display unit.

The motion vector detecting means 5 is always provided in order to use a motion vector when generating a three-dimensional image from a two-dimensional image. Is used as a control signal for swinging the ride means 7.

Further, as in the above-described embodiment, not only a configuration in which a video signal obtained by converting a two-dimensional video into a three-dimensional video is displayed on the display means 3, but also a three-dimensional video from the video source 1
That is, the left-eye image L and the right-eye image R may be output and displayed on the three-dimensional display unit 18 based on these signals.

Such a configuration will be described with reference to FIG. Note that, as in FIG. 5, the components denoted by the same reference numerals as those in FIG.

Reference numeral 19 denotes a subject specifying means for taking in the left-eye image L and the right-eye image R from the image supply source 1, distinguishing the subject from the background, and specifying the subject.

In this case, the image supply source 1 supplies a three-dimensional image, more specifically, supplies a left-eye image L and a right-eye image R. The display means 18 causes a three-dimensional display. The video capturing means 2 captures the left-eye video L and the right-eye video R, and based on the video captured by the video capturing means 2, the motion vector detecting means 5 The motion vector is detected in consideration of the specific result, and is sent to the swing control means 6 to drive the ride means 7.

[0055]

According to the present invention, it is possible to provide a simulated sensation device that can provide a more realistic sensation regardless of what kind of image is used.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a virtual experience apparatus of the present invention.

FIG. 2 is a detailed view of the swing control means in FIG.

FIG. 3 is a diagram illustrating an operation flowchart of the virtual experience apparatus.

FIG. 4 is a diagram illustrating an operation flowchart illustrating in detail the operation of an operation signal generation unit 69 in FIG. 2;

FIG. 5 is a diagram showing an embodiment of a pseudo experience device using a device for converting a two-dimensional image into a three-dimensional image.

FIG. 6 is a diagram showing an embodiment of a virtual experience apparatus when a three-dimensional image is supplied from an image supply source.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Video supply source 5 Motion vector detecting means 6 Swing control means 7 Ride means 8 Swing means 9 Experiencer

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Kenji Koyamada 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Sanyo Electric Co., Ltd. (72) Inventor Masahiro Seto 2-5-2 Keihanhondori, Moriguchi-shi, Osaka No. 5 Inside Sanyo Electric Software Co., Ltd.

Claims (6)

[Claims] A display means for displaying a video, a video signal supply means for supplying a video signal to the display means, a motion detection means for detecting a motion of a video from a video signal of the video signal supply means, Pseudo-effect means for providing a stimulus to the experienced person, wherein the pseudo-effect means gives the stimulus having a strength corresponding to the motion of the image detected by the motion detection means to the user, and A simulated experience device characterized in that the strength of the stimulus is gradually reduced when the stimulus is maximum and continues for a predetermined time or more. 2. The simulated effect means is arranged in front of the display means, and ride means for placing an experienced person, oscillating means for oscillating the ride means to stimulate the experienced person, Swing control means for controlling the drive of the swing means, wherein the swing control means drives the swing means in accordance with the motion of the image detected by the motion detection means, and 2. The simulated experience apparatus according to claim 1, wherein when the maximum movement of the means continues for a predetermined time or more, the swing means is gradually returned to the center. 3. The swing control means, when the movement of the image detected by the movement detecting means suddenly increases, before swinging the ride means in a direction corresponding to the movement of the image. 3. The simulated experience device according to claim 2, wherein the device is slightly swung in a reverse direction. 4. A step of detecting a movement of an image displayed on the display means, a step of giving a stimulus of an intensity corresponding to the movement of the image detected in the step by the pseudo effect means to the user, and a step of: A step of gradually reducing the intensity of the stimulus when the intensity of the stimulus is maximum and continues for a predetermined time or more. 5. The step of providing a stimulus to a user by the pseudo effect means, wherein the step of moving the ride means in a direction corresponding to the movement of the image when the movement of the image detected by the movement detecting means suddenly increases. The simulated experience method according to claim 4, further comprising a step of slightly swinging in a direction opposite to the direction before swinging. 6. The step of providing a stimulus to the user by the pseudo effect means, wherein the motion of the video detected by the motion detection means sharply increases, and the ride means is swung in a direction corresponding to the motion of the video. 6. The method according to claim 5, further comprising the step of slightly rocking in a direction opposite to the direction before moving.
Simulated experience method described in.