JP2012141826A - Program, information storage medium, image generation system and server system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program, an information storage medium, an image generation system and a server system which can realize appropriate timing determination processing in a stereoscopic viewing system.SOLUTION: An image generation system includes a moving object control unit for performing control of a moving object, a determination unit for performing timing determination processing using the moving object, and an image generation unit for generating a stereoscopic viewing image. The image generation unit generates a stereoscopic viewing image in which the moving object is displayed so as to make an image parallax of the moving object a prescribed parallax at a determination criteria timing used for the timing determination processing.

Description

  The present invention relates to a program, an information storage medium, an image generation system, a server system, and the like.

  2. Description of the Related Art In recent years, stereoscopic image generation systems have been spotlighted as systems for generating images that are more realistic in the fields of movies and games. For example, in a binocular stereoscopic image generation system which is one of the stereoscopic image generation systems, a left-eye image and a right-eye image are generated. Then, the player wears stereoscopic glasses, the left eye sees only the left eye image, and the right eye sees only the right eye image, thereby realizing stereoscopic vision. As a conventional technique of an image generation system that realizes such a stereoscopic view, there is a technique disclosed in Patent Document 1, for example. Further, as a stereoscopic viewing method, there is a naked eye type in addition to the glasses type. For example, a parallax barrier creates a state of “visible to the left eye but not to the right eye” or “visible to the right eye but not to the left eye” for each pixel. Visual system can be realized. Also, by utilizing the refraction of light by an optical element such as a lenticular, the direction of the light is controlled for each pixel, and a state similar to the above can be created to realize a stereoscopic vision system with naked eyes.

  On the other hand, in a music game or a ball game such as a baseball game, a moving object such as a musical note (indicating marker) or a ball is moved on the game screen, and the input timing of the player is determined according to a predetermined criterion. A timing determination process is performed to determine whether or not it is within.

  However, in a stereoscopic game realized by a stereoscopic system, when such timing determination processing is performed, an appropriate timing determination processing cannot be realized unless the display state of the moving object in the timing determination is an appropriate display state. There is a fear.

JP 2004-126902 A

  According to some aspects of the present invention, it is possible to provide a program, an information storage medium, an image generation system, a server system, and the like that can realize appropriate timing determination processing in a stereoscopic vision system.

  One aspect of the present invention includes a moving body control unit that controls a moving body, a determination unit that performs a timing determination process using the moving body, and an image generation unit that generates a stereoscopic image. The generation unit relates to an image generation system that generates a stereoscopic image on which the moving body is displayed so that the parallax of the image of the moving body becomes a predetermined parallax at a determination reference timing used for the timing determination process. . The present invention also relates to a program that causes a computer to function as each of the above-described units, or a computer-readable information storage medium that stores the program.

  According to one aspect of the present invention, the timing determination process is performed using the moving body controlled by the moving body control unit. Then, at the determination reference timing used for the timing determination process, the stereoscopic image is generated so that the parallax of the image of the moving object becomes a predetermined parallax. In this way, since the timing determination process is performed using a moving object that has a predetermined parallax at the determination reference timing, it is possible to realize an appropriate timing determination process in the stereoscopic system. become.

  Further, according to one aspect of the present invention, a timing data storage unit that stores data of the determination reference timing is included (the computer functions as the timing data storage unit), and the determination unit is stored in the timing data storage unit. The timing determination process may be performed based on the determination reference timing data and the player input timing.

  In this way, while performing the timing determination process using the data of the determination reference timing stored in the timing data storage unit, the stereoscopic image in which the parallax of the moving body image becomes the predetermined parallax at the determination reference timing is generated. become able to.

  In one embodiment of the present invention, a game calculation unit that performs a game calculation process is included (a computer is caused to function as the game calculation unit), and the determination unit determines whether the input timing and the determination reference timing are as the timing determination process. An evaluation process based on the timing relationship may be performed, and the game calculation unit may perform the game calculation process based on a result of the evaluation process.

  In this way, the timing determination process can be realized using a moving object in which the parallax of the image becomes a predetermined parallax at the determination reference timing, and the game is determined based on the result of the evaluation process based on the timing relationship between the input timing and the determination reference timing Arithmetic processing can be executed. Therefore, it is possible to reflect the result of the evaluation process of the timing determination process in the game calculation process while realizing an appropriate timing determination process in the stereoscopic vision system.

  In one aspect of the present invention, the game calculation unit performs at least one of a calculation process of a game result of the player, a display control process of the moving object, and a game effect process based on a result of the evaluation process. You may perform as arithmetic processing.

  In this way, the evaluation result of the timing determination process can be reflected in the game result of the player, the display control of the moving body, and the game effect process.

  In the aspect of the invention, the image generation unit may be used for stereoscopic viewing in which an image whose degree of parallax changes according to the evaluation result is displayed as an image of the evaluation display object that displays the result of the evaluation process. An image may be generated.

  In this way, the result of the timing determination evaluation process can be transmitted to the player by the change in the degree of parallax of the evaluation display object. Thus, the player can intuitively recognize the evaluation given to the player according to the degree of parallax of the evaluation display object.

  In one aspect of the present invention, the image generation unit generates a stereoscopic image on which the moving body is displayed so that the parallax of the image of the moving body is eliminated at the determination reference timing used in the timing determination process. May be.

  In this way, since the timing determination process is performed using a moving body that has no image parallax at the determination reference timing, it is possible to realize an appropriate timing determination process in the stereoscopic system.

  In the aspect of the invention, the image generation unit may generate a stereoscopic image in which parallax of the image of the moving body changes according to a timing difference between the input timing and the determination reference timing. .

  In this way, the timing difference between the input timing and the determination reference timing can be expressed by a change in parallax of the moving object image, and a timing determination process in which the player can easily grasp the timing can be realized. .

  In one aspect of the present invention, the image generation unit generates a stereoscopic image on which the moving body is displayed so that the parallax of the image of the moving body is eliminated at the determination reference timing used in the timing determination process. In addition, when it is determined in the timing determination process that the input timing of the player matches the determination reference timing and it is determined that the input of the player is successful, an image of the game effect display object is obtained. A stereoscopic image on which an image with parallax is displayed may be generated.

  In this way, even when the moving body is an image without parallax, the game effect effect can be improved by displaying the game effect effect display object as an image with parallax.

  In one aspect of the present invention, the moving body is an instruction marker that indicates a timing to be input by the player, and the determination unit is configured to input when the player inputs at the timing specified by the instruction marker. A match determination process between the input timing of the player and the determination reference timing may be performed as the timing determination process.

  In this way, when the player inputs an operation instructed by the instruction marker, a process for determining a match between the input timing and the determination reference timing is performed. Since the parallax of the image of the moving object becomes a predetermined parallax at the determination reference timing (for example, the parallax disappears), the player can easily input an operation instructed by the instruction marker at the determination reference timing.

  In the aspect of the invention, the image generation unit may display an input action display object indicating that an image having no parallax is displayed at the determination reference timing and the player input is performed for the instruction marker. May generate a stereoscopic image on which an image with parallax is displayed.

  In this way, the player can confirm that his / her input operation has been performed by looking at the input operation display object. Also, even if the instruction marker is an image without parallax, the input operation display object becomes an image with parallax so as to compensate for it, so it is as if the instruction marker is stereoscopically displayed as an image with parallax. The production effect that looks like this can be realized.

  In the aspect of the invention, the moving body control unit may perform control to move the moving body from the back side to the near side when viewed from the viewpoint side.

  In this way, the moving body is moved from the back side to the near side as viewed from the viewpoint side, and the stereoscopic image (for example, the parallax is such that the parallax of the moving body image becomes a predetermined parallax at the determination reference timing). By generating such a stereoscopic image that disappears, an appropriate timing determination process using a moving object can be realized.

  In the aspect of the invention, the moving body control unit moves the moving body so that the moving body passes through a determination reference area for informing a player of a timing to be input at the determination reference timing. Control may be performed.

  In this way, the moving body passes through the determination reference area at the determination reference timing, and the image of the moving body at that time becomes an image in which the parallax becomes a predetermined parallax (for example, an image having no parallax). Accordingly, for example, the player can perform an operation input while viewing an image of a moving object having a parallax of a predetermined parallax at the passage timing of the determination reference area, and therefore, an appropriate timing determination process in the stereoscopic system can be realized.

  In the aspect of the invention, the image generation unit may perform the projection of the moving object on the reference screen set at a position corresponding to the determination reference area, thereby drawing the determination reference timing. In the above, a stereoscopic image in which the parallax of the image of the moving body is eliminated may be generated.

  In this way, by performing perspective projection of the moving object on the reference screen, it is possible to generate an image of the moving object that has no parallax at the determination reference timing passing through the determination reference area.

  In the aspect of the invention, the image generation unit may generate a stereoscopic image in which at least one of a color, a shape, and a size of the moving body changes at the determination reference timing.

  In this way, the visibility of the moving object at the determination reference timing is improved, and timing determination processing and the like that can be easily grasped by the player can be realized.

  In one embodiment of the present invention, a sound generation unit that generates sound (a computer is caused to function as the sound generation unit), and the sound generation unit outputs a sound whose sound image localization changes according to parallax of the moving body. It may be generated.

  In this way, not only the parallax of the moving body but also the sound image localization linked to the parallax of the moving body can be used to make the player feel the stereoscopic effect of the moving body.

  In the aspect of the invention, the determination unit is a virtual display object corresponding to the moving object or the operation target for timing determination processing, and the display screen side space of the display unit is based on the stereoscopic image. A hit determination process between a virtual display object virtually displayed on the screen and a hit object in the real space may be performed.

  In this way, it is possible to perform hit determination processing between the virtual display object corresponding to the moving object or the operation target object and the hit object in the real space, and to reflect the hit determination result in the game result or the like. .

  In the aspect of the invention, the determination unit may perform a hit determination process between the virtual display object and the hit object based on imaging information from an imaging unit that images the player.

  In this way, it is possible to realize the hit determination process between the virtual display object and the hit object using the imaging information from the imaging unit.

  According to another aspect of the present invention, a moving body control unit that controls a moving body, a determination unit that performs timing determination processing using the moving body, and image generation data for generating a stereoscopic image are provided. An image generation data generation unit for generating the image, and the image generation data generation unit generates the parallax of the image of the mobile object at a predetermined parallax at a determination reference timing used for the timing determination process. The present invention relates to a server system that generates image generation data for generating a stereoscopic image on which is displayed.

1 is a configuration example of an image generation system according to the present embodiment. Explanatory drawing of the drawing position of the object in stereoscopic vision. FIG. 3A to FIG. 3C are explanatory diagrams of the method of this embodiment. 4A and 4B are explanatory diagrams of parallax of an image of a moving object. Explanatory drawing about a timing data storage part. FIG. 6A to FIG. 6E are explanatory diagrams of input timing evaluation processing. 7A and 7B are configuration examples of a portable game device to which the present embodiment is applied. The example of the image produced | generated when this embodiment is applied to a music game. The example of the image produced | generated when this embodiment is applied to a music game. FIG. 10A to FIG. 10C are explanatory diagrams of a timing determination method when the present embodiment is applied to a music game. FIGS. 11A to 11C are explanatory diagrams of an input timing evaluation method when the present embodiment is applied to a music game. 12A and 12B are explanatory diagrams of a display control method for an evaluation display object. FIGS. 13A and 13B are also explanatory diagrams of the display control method of the evaluation display object. FIGS. 14A and 14B are also explanatory diagrams of the display control method of the evaluation display object. FIG. 15A and FIG. 15B are explanatory diagrams of a display control method for a game effect display object. FIGS. 16A and 16B are explanatory diagrams of a display control method for an input action display object. FIGS. 17A and 17B are explanatory diagrams of a method of changing the color, shape, or size of the moving object at the determination reference timing. Explanatory drawing of the method of producing | generating the sound from which sound image localization changes according to the parallax of a moving body. 19A and 19B are explanatory diagrams of a technique for performing hit determination processing between a virtual display object and a hit target such as a finger of a player. The example of the image produced | generated when this embodiment is applied to a baseball game. The example of the image produced | generated when this embodiment is applied to a baseball game. The flowchart explaining the detailed process of this embodiment. The flowchart explaining the detailed process of this embodiment. The structural example of the server system of this embodiment.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. Configuration FIG. 1 shows an example of a block diagram of an image generation system (game system) of the present embodiment. Note that the configuration of the image generation system of the present embodiment is not limited to that shown in FIG. 1, and various modifications may be made such as omitting some of the components (each unit) or adding other components. .

  The operation unit 160 is for a player to input operation data, and functions thereof are direction instruction keys, operation buttons, analog sticks, levers, various sensors (such as an angular velocity sensor and an acceleration sensor), a microphone, or a touch panel type. This can be realized with a display.

  The storage unit 170 serves as a work area for the processing unit 100, the communication unit 196, and the like, and its function can be realized by a RAM (DRAM, VRAM) or the like. Then, the game program and game data necessary for executing the game program are held in the storage unit 170.

  The information storage medium 180 (a computer-readable medium) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), memory (ROM, etc.), etc. Can be realized. The processing unit 100 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 180. That is, in the information storage medium 180, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing the computer to execute processing of each unit). Is memorized.

  The display unit 190 outputs an image generated according to the present embodiment, and its function can be realized by an LCD, an organic EL display, a CRT, a touch panel display, an HMD (head mounted display), or the like. The sound output unit 192 outputs the sound generated by the present embodiment, and its function can be realized by a speaker, headphones, or the like.

  The auxiliary storage device 194 (auxiliary memory, secondary memory) is a storage device used to supplement the capacity of the storage unit 170, and can be realized by a memory card such as an SD memory card or a multimedia card.

  The communication unit 196 communicates with the outside (for example, another image generation system, a server, or a host device) via a wired or wireless network, and functions as a communication ASIC, a communication processor, or the like. It can be realized by hardware and communication firmware.

  Note that a program (data) for causing a computer to function as each unit of the present embodiment is obtained from an information storage medium of a server (host device) via an information storage medium 180 (or storage unit 170, auxiliary storage) via a network and communication unit 196. May be distributed to the device 194). Use of an information storage medium by such a server (host device) can also be included in the scope of the present invention.

  The processing unit 100 (processor) performs game processing, image generation processing, sound generation processing, and the like based on operation data from the operation unit 160, a program, and the like. The processing unit 100 performs various processes using the storage unit 170 as a work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, GPU, etc.), ASIC (gate array, etc.), and programs.

  The processing unit 100 includes a game calculation unit 102, an object space setting unit 104, a moving body control unit 106, a virtual camera control unit 108, a determination unit 110, an image generation unit 120, and a sound generation unit 130.

  The game calculation unit 102 performs game calculation processing. Here, as a game calculation, a process for starting a game when a game start condition is satisfied, a process for advancing the game, a process for calculating a game result, or a process for ending a game when a game end condition is satisfied and so on.

  The object space setting unit 104 performs an object space setting process in which a plurality of objects are arranged. For example, various objects (polygon, freedom, etc.) representing display objects such as characters (people, animals, robots, cars, ships, airplanes, etc.), pointing markers, balls, maps (terrain), buildings, courses (roads), trees, walls, etc. A process of placing and setting (objects configured with a primitive surface such as a curved surface or a subdivision surface) in the object space is performed. In other words, the position and rotation angle of the object in the world coordinate system (synonymous with direction and direction) are determined, and the rotation angle (rotation angle around the X, Y, and Z axes) is determined at that position (X, Y, Z). Arrange objects. Specifically, the object data storage unit 172 of the storage unit 170 stores object data such as the position, rotation angle, moving speed, moving direction, etc. of the object (part object) in association with the object number. . The object space setting unit 104 performs a process of updating the object data for each frame, for example.

  The moving body control unit 106 performs control processing for moving a moving body such as an instruction marker, a ball, and a character. Also, a control process for operating the moving body (moving body object) is performed. That is, based on operation data input by the player through the operation unit 160, a program (movement / motion algorithm), various data (motion data), etc., a moving body (object, model object) is moved in the object space, Performs control processing to move the moving body (motion, animation). Specifically, a simulation process for sequentially obtaining movement information (position, rotation angle, speed, or acceleration) and motion information (part object position or rotation angle) of a moving body for each frame (1/60 second). I do. A frame is a unit of time for performing a moving / movement process (simulation process) and an image generation process of a moving object.

  The virtual camera control unit 108 performs control processing of a virtual camera (viewpoint, reference virtual camera) for generating an image that can be seen from a given (arbitrary) viewpoint in the object space. Specifically, processing for controlling the position (X, Y, Z) or rotation angle (rotation angle about the X, Y, Z axis) of the virtual camera (processing for controlling the viewpoint position, the line-of-sight direction or the angle of view) I do.

  For example, when a character is photographed from behind using a virtual camera, the position (viewpoint position) and direction (gaze direction) of the virtual camera are controlled so that the virtual camera follows changes in the position or direction of the character. In this case, the virtual camera can be controlled based on information such as the position, direction, or speed of the character obtained by the moving body control unit 106. Alternatively, the virtual camera may be controlled to rotate at a predetermined rotation angle or to move along a predetermined movement path. In this case, the virtual camera is controlled based on virtual camera data for specifying the position (movement path) or direction of the virtual camera.

  The determination unit 110 performs various determination processes. Details of the determination process performed by the determination unit 110 will be described later.

  The image generation unit 120 performs drawing processing based on the results of various processing (game processing and simulation processing) performed by the processing unit 100, thereby generating an image and outputting the image to the display unit 190. Specifically, geometric processing such as coordinate transformation (world coordinate transformation, camera coordinate transformation), clipping processing, perspective transformation, or light source processing is performed. Based on the processing result, drawing data (the position of the vertex of the primitive surface) Coordinates, texture coordinates, color data, normal vector, α value, etc.) are created. Based on the drawing data (primitive surface data), the object (one or a plurality of primitive surfaces) after perspective transformation (after geometry processing) is converted into image information in units of pixels such as a drawing buffer 178 (frame buffer, work buffer, etc.). Draw in a buffer that can be stored. Thereby, an image that can be seen from the virtual camera (given viewpoint) in the object space is generated. Note that the rendering process can be realized by a vertex shader process, a pixel shader process, or the like.

  The sound generation unit 130 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.

  In this embodiment, the moving body control unit 106 controls moving bodies such as instruction markers (musical notes) and balls. For example, control is performed to move a moving object in the object space. Moreover, the determination part 110 performs the timing determination process using a moving body. For example, timing determination processing in various games such as a music game and a baseball game is performed based on the input timing of the player and the position of the moving body.

  Then, the image generation unit 120 generates a stereoscopic image. Specifically, the image generation unit 120 generates a stereoscopic image on which the moving body is displayed such that the parallax of the moving body image becomes a predetermined parallax at the determination reference timing used in the timing determination process. For example, the stereoscopic image on which the moving object is displayed is generated so that the parallax of the image of the moving object disappears at the determination reference timing.

  Here, the stereoscopic image is an image for the left eye and an image for the right eye, taking a two-eye separation spectacle system as an example. However, the stereoscopic image is not limited to the left-eye image and the right-eye image of the two-eye separation spectacle method, and may be a naked-eye type stereoscopic image. The naked-eye method includes a multi-eye method, an aerial image method, and the like in addition to the naked-eye binocular method. As the aerial image method, for example, the “fractional view method” has been proposed.

  The determination reference timing is a timing that is a reference for timing determination processing using a moving object, and is represented as, for example, a determination reference frame.

  In addition, the parallax of the moving body image being a predetermined parallax means that the parallax of the moving body image is a presumed parallax, for example, within a predetermined parallax range. Specifically, the first drawing position, which is the drawing position on the reference screen (perspective projection plane) of the object of the image for the left eye (first viewpoint image in a broad sense), and the image for the right eye (second viewpoint in a broad sense). The difference (difference) between the corresponding object (image) and the second drawing position, which is the drawing position on the reference screen, becomes a predetermined reference value or falls within a predetermined reference range. The data of the predetermined reference value and the predetermined reference range is stored in the storage unit 170, for example.

  Also, the loss of parallax of the moving object image means, for example, that the parallax of the moving object image becomes zero. Specifically, the first drawing position on the reference screen of the object of the left-eye image (first viewpoint image) and the second drawing position on the reference screen of the corresponding object of the right-eye image (second viewpoint image) Is a match. However, the parallax of the image of the moving body does not have to be completely zero, and the parallax value may be a predetermined threshold value or less. For example, the deviation (difference) between the first drawing position of the object for the left eye image and the second drawing position of the corresponding object of the right eye image may be less than or equal to a predetermined threshold (for example, one pixel or less). Further, the parallax (binocular parallax) in this case is a difference in the positions of the images seen by the left eye (first viewpoint) and the right eye (second viewpoint), and is called a binocular parallax. Is.

  For example, the timing data storage unit 174 of the storage unit 170 stores determination reference timing data. That is, a plurality of determination reference timings are stored as a plurality of determination reference frames, for example. Then, the determination unit 110 performs a timing determination process based on the determination reference timing data stored in the timing data storage unit 174 and the input timing of the player. For example, the timing determination process is realized by comparing the determination reference timing with the input timing of the player. Then, the image generation unit 120 displays the stereoscopic image (left-eye image, right-side image) on which the moving body is displayed so that the parallax of the moving body image becomes a predetermined parallax at the determination reference timing (for example, the parallax is eliminated). Eye image). For example, a moving body such as an instruction marker or a ball moves in the object space, for example, under the control of the moving body control unit 106. When the player performs an operation for timing determination using the operation unit 160, the input timing of the operation (the frame in which the operation is input) is acquired. Then, the determination unit 110 performs a match determination process (match determination process) between the player input timing and the determination reference timing as the timing determination process. In this case, the image generation unit 120 generates a stereoscopic image so that the moving object such as the pointing marker or the ball has a predetermined parallax (no parallax) at the determination reference timing. For example, the difference between the first drawing position of the moving object in the left-eye image and the second drawing position of the moving object in the right-eye image is within a predetermined reference value or a predetermined reference range at the determination reference timing. The left eye image and the right eye image are generated so that the first drawing position and the second drawing position match (including substantially the same) at the determination reference timing.

  Further, the determination unit 110 performs an evaluation process based on the timing relationship between the input timing and the determination reference timing as the timing determination process. For example, an evaluation process based on the timing difference (time difference, frame difference) between the input timing and the determination reference timing is performed. Then, the game calculation unit 102 performs a game calculation process based on the result of the evaluation process (whether good evaluation or bad evaluation). Specifically, at least one of the player's game result calculation process, the moving body display control process, and the game effect process is performed as a game calculation process based on the result of the evaluation process.

  Here, the game score calculation processing is calculation processing of player scores, points, and operation evaluations acquired by the game based on the timing determination processing. For example, the smaller the timing difference (time difference) between the input timing and the determination reference timing, the higher the score, the higher points, and the higher the operation evaluation are given to the player. The moving body display control process is a moving body display control process after the determination process based on the input timing and the determination reference timing, for example. For example, it is a process of changing the moving direction, moving speed, moving acceleration, or display mode of the moving body based on the evaluation result. The game effect process is an effect process for visually or audibly expressing the result of the timing determination process, and is a process for generating a video effect or a sound effect based on the evaluation result for the game effect.

  Further, the image generation unit 120 generates a stereoscopic image in which an image whose degree of parallax changes according to the evaluation result is displayed as an image of the evaluation display object that displays the result of the evaluation process. For example, when the evaluation result is the first result (for example, “good”), an evaluation display object having a parallax that can be seen in front of the reference screen when viewed from the viewpoint side is displayed. When the evaluation result is the second result (for example, “impossible”), for example, an evaluation display object having a parallax that can be seen on the back side of the reference screen when viewed from the viewpoint side is displayed. When the evaluation result is the third result (for example, “OK”), for example, an evaluation display object with zero parallax is displayed.

  In addition, the image generation unit 120 generates a stereoscopic image in which the parallax of the moving object image changes according to the timing difference between the input timing and the determination reference timing. For example, as the timing difference between the input timing and the determination reference timing increases, a stereoscopic image is generated in which the parallax of the moving object image increases or decreases. More specifically, the image generation unit 120 generates a stereoscopic image in which the parallax of the moving object image increases as the timing difference between the input timing and the determination reference timing increases. For example, when the timing difference between the input timing and the determination reference timing is large, a stereoscopic image that appears to protrude or retract from the reference screen in the stereoscopic view is generated.

  In addition, the image generation unit 120 generates a stereoscopic image in which the moving body is displayed so that the parallax of the image of the moving body is eliminated at the determination reference timing used for the timing determination process. In the timing determination process, when it is determined that the input timing of the player matches the determination reference timing and it is determined that the player input is successful, an image with parallax is displayed as the image of the game effect display object. The displayed stereoscopic image is generated. For example, processing for generating a game effect display object for an image with parallax is performed from the location of the moving object of the image without parallax. In this way, even if the moving body itself has no parallax, an effective game effect can be realized by using the game effect effect display object having the parallax.

  The moving body is, for example, an instruction marker that instructs the timing to be input by the player. Then, when the player inputs at the timing instructed by the instruction marker, the determination unit 110 performs a match determination process between the player input timing and the determination reference timing as the timing determination process. That is, the instruction marker which is a moving body moves under the control of the moving body control unit 106, and this instruction marker instructs the timing to be input by the player. When the player performs an operation input at the instructed timing, a match determination process (comparison process) between the input timing and the determination reference timing is performed. In this way, a game such as a music game using the instruction marker can be realized.

  In this case, the image generation unit 120 displays an image with no parallax at the determination reference timing for the instruction marker, and displays an image with parallax for the input action display object indicating that the player input has been performed. The generated stereoscopic image is generated. That is, at the determination reference timing, even when an image without parallax is displayed for the instruction marker, an image with parallax is displayed for the input action display object. By doing so, the player can easily grasp whether or not the input is properly performed by the input motion display object having parallax.

  The moving body control unit 106 performs control to move the moving body from the back side to the near side when viewed from the viewpoint side. For example, the moving body is moved so that the depth value of the moving body changes from the depth value indicating the back side when viewed from the viewpoint side to the depth value indicating the near side. Then, at the determination reference timing, an image is generated such that the parallax of the moving body becomes a predetermined parallax (an image where the parallax disappears). By doing so, it is possible to realize a determination process that is easy for the player to understand as the timing determination process using the moving object.

  The moving body may be a ball (an object representing a ball) in a ball game. Then, the determination unit 110 performs a hit determination process in which the ball is hit by the possession (bat, racket, etc.) or part (foot, hand, head, etc.) of the character appearing in the ball game as the timing determination process. As the ball game, various games such as a baseball game, a tennis game, and a soccer game can be assumed. Then, the moving body control unit 106 performs control to move the moving body so that the moving body passes through the determination reference area for informing the player of the timing to be input at the determination reference timing. For example, the moving body control unit 106 controls the movement of the moving body based on the determination reference timing data stored in the timing data storage unit 174 so that the moving body passes the determination reference area at the determination reference timing. By doing so, it is possible to generate a moving body image (a moving body image where the parallax is zero) such that the parallax becomes a predetermined parallax when passing through the determination reference area.

  The timing data storage unit 174 may store both the determination reference timing data and the display reference timing data. The timing determination process for the input timing of the player is performed using the determination reference timing data, and the movement control process for the moving body is performed using the display reference timing data. In this case, the determination reference timing and the display reference timing are data in which the timings are synchronized. By using such synchronized data, the parallax becomes a predetermined parallax when passing through the determination reference area. It is possible to generate an image of a moving body (moving body image with parallax being zero).

  In addition, the image generation unit 120 performs perspective projection of a moving object on a reference screen set at a position corresponding to the determination reference area and draws it, thereby eliminating the parallax of the moving object image at the determination reference timing. A visual image is generated. Specifically, a left-eye image is generated by perspectively drawing a moving object on the reference screen from the viewpoint of the left-eye virtual camera. Further, a right-eye image is generated by perspectively projecting and drawing a moving object on the reference screen from the viewpoint of the right-eye virtual camera.

  For example, in the case of the two-eye separation spectacle method, the image generation unit 120 includes a left-eye image generation unit (first viewpoint image generation unit in a broad sense) 122 and a right-eye image generation unit (second viewpoint image in a broad sense). Generating section) 124. Then, the left-eye image generation unit 122 generates a left-eye image (first viewpoint image) that is an image at the viewpoint (first viewpoint) of the left-eye virtual camera. The right-eye image generation unit 124 generates a right-eye image (second viewpoint image) that is an image at the viewpoint (second viewpoint) of the right-eye virtual camera.

  Note that the virtual camera control unit 108 controls a reference virtual camera serving as a reference for setting, for example, left-eye and right-eye virtual cameras. Then, based on the obtained position information and direction information of the reference virtual camera and information on the set inter-camera distance, position information (viewpoint position) and direction information (line-of-sight direction) of the left-eye and right-eye virtual cameras ) Note that the virtual camera control unit 108 may directly control the left-eye and right-eye virtual cameras.

  As a stereoscopic method, various methods such as a two-eye separation spectacle method, a parallax barrier, a lenticular, and a naked eye method using an optical element capable of controlling the direction of light can be assumed. Examples of the binocular separation glasses method include a polarization glasses method, a time separation method, and a color separation method. In the polarized glasses method, for example, the left-eye image and the right-eye image are alternately displayed on the odd-numbered and even-numbered lines of the display unit 190, and this is displayed on the polarized glasses (for example, the horizontal polarization filter for the left eye and the vertical for the right eye). Stereoscopic viewing is realized by viewing with a pair of spectacles with a directional polarizing filter. Alternatively, the stereoscopic image may be realized by projecting the image for the left eye and the image for the right eye with a projector having a special polarization filter and viewing the projection image with polarized glasses. In the continuous separation method (page-flip method), the left-eye image and the right-eye image are alternately displayed on the display unit 190 every predetermined period (for example, every 1/120 second or 1/60 second). In conjunction with the switching of the display, the left and right liquid crystal shutters of the glasses with the liquid crystal shutter are alternately opened and closed to realize stereoscopic viewing. In the color separation method, for example, an anaglyph image is generated and viewed with red-blue glasses or the like to realize stereoscopic viewing.

  In addition, the function of generating a stereoscopic image from the left-eye image and the right-eye image may be provided in the image generation unit 120 or may be provided in the display unit 190 (such as a television). For example, the image generation unit 120 outputs a side-by-side image signal. Then, the display unit 190 displays a field sequential image in which left-eye images and right-eye images are alternately assigned to odd lines and even lines based on the side-by-side image signal. Alternatively, a frame sequential image in which the left-eye image and the right-eye image are alternately switched every predetermined period is displayed. Alternatively, the image generation unit 120 may generate a field sequential method or frame sequential method image and output the generated image to the display unit 190.

  Further, the image generation unit 120 may generate a stereoscopic image in which at least one of the color, shape, and size of the moving object changes at the determination reference timing. For example, the color, shape, or size of the moving body is set to a color, shape, or size that is different from the timing other than the determination reference timing at the determination reference timing (just timing). By doing so, the visibility of the moving object at the determination reference timing is increased, and a timing determination process in which the player can easily grasp the timing can be realized.

  The sound generation unit 130 that generates sound such as game sound or sound effect may generate sound whose sound image localization changes according to the parallax of the moving body. That is, the sound image localization of the sound output from the sound output unit 192 is changed according to the degree of stereoscopic vision of the moving body. For example, in the case of parallax (stereoscopic image) where the moving body appears to jump out of the screen, the sound image localization of the sound is set to a position on the near side (player side) of the screen. On the other hand, in the case of parallax such that the moving body retracts to the back side of the screen, the sound image localization of the sound is set to a position on the back side of the screen. By doing so, the stereoscopic effect of the moving object in the stereoscopic image and the stereoscopic effect due to the sound image localization of the sound are linked, and the stereoscopic effect felt by the player can be improved.

  The determination unit 110 may perform a hit determination process between the virtual display object and the hit object in the real space. Then, for example, the game calculation unit 102 performs a game calculation process for calculating a game result such as a player's score or points based on the result of the hit determination process. Here, the virtual display object is a virtual display object corresponding to, for example, a moving object or an operation target object for timing determination processing, and is virtually displayed on the display screen side space of the display unit 190 based on the stereoscopic image. It is what is done. The operation target for the timing determination process is a display object that is the operation target of the player in the timing determination process using the moving body. The hit object is a physical object. More specifically, the determination unit 110 performs a hit determination process between the virtual display object and a hit object such as a player's part (finger, hand, etc.) or a pointing device, and the player (player's part or pointing device). This is performed based on imaging information from an imaging unit (camera) that performs imaging. By doing so, hit determination processing between the virtual display object and the hit object can be realized by simple processing using the imaging unit. In addition, you may perform the hit determination process of a virtual display thing and a to-be-hit object based on the sensor information from sensors (human sensor, an optical sensor, an infrared sensor, etc.) different from the imaging sensor of an imaging part.

  Note that the image generation system of the present embodiment may be realized by a server system. FIG. 24 shows a configuration example in the case of being realized by a server system.

  Server system 500 is communicatively connected to terminal apparatuses TM1 to TMn via network 510. For example, the server system 500 is a host, and the terminal devices TM1 to TMn are clients. The server system 500 can be realized by, for example, one or a plurality of servers (authentication server, game server, communication server, billing server, etc.). The network 510 (distribution network, communication line) is a communication path using, for example, the Internet or a wireless LAN. In addition to a LAN using a dedicated line (dedicated cable) or Ethernet (registered trademark) for direct connection, telephone communication is also possible. A communication network such as a network, a cable network, or a wireless LAN can be included. The communication method may be wired / wireless.

  The terminal devices TM1 to TMn are realized by, for example, a portable game device, a stationary home game device, an arcade game device, or the like. The portable game device may be a dedicated game device or a general-purpose device capable of executing a game program such as a mobile phone or a portable information terminal.

  The server system 500 includes a processing unit 600, a storage unit 670, an information storage medium 680, and a communication unit 696. The processing unit 600 includes a game calculation unit 602, an object space setting unit 604, a moving body control unit 606, a virtual camera control unit 608, a determination unit 610, an image generation data generation unit 620, and a sound generation data generation unit 630. The functions, operations, etc. of these units (blocks) are the same as those of the units (blocks) in FIG.

  For example, the moving body control unit 606 of the server system 500 controls the moving body, and the determination unit 610 performs timing determination processing using the moving body. Then, the image generation data generation unit 620 generates image generation data data for generating a stereoscopic image. The sound generation data generation unit 630 generates sound generation data for generating a sound.

  In this embodiment, the image generation data generation unit 620 displays the moving object so that the parallax of the image of the moving object becomes a predetermined parallax at the determination reference timing used in the timing determination process (for example, the parallax disappears). Image generation data for generating the stereoscopic image thus generated is generated.

  Here, the image generation data for generating the image is data for displaying the image generated by the method of the present embodiment on each of the terminal devices TM1 to TMn, and even the image data itself. It may be various data (object data, control result data, determination result data, display screen setting data, etc.) used by each terminal device to generate an image. For example, the server system 500 acquires operation information from the operation unit of each terminal device, performs various control processes and various determination processes, generates images, and distributes them to the terminal devices TM1 to TMn (stream distribution) Etc.), the image generation data described above is the image data itself. On the other hand, the server system 500 acquires operation information from the operation unit of each terminal device, performs various control processes and various determination processes, and the terminal devices TM1 to TMn are based on the control results and determination results. When an image is generated, the above-described image generation data is control result data, determination result data, object data, or the like. The same applies to the sound generation data generated by the sound generation data generation unit 630.

  It should be noted that the game calculation unit 602, the object space setting unit 604, the moving body control unit 606, the virtual camera control unit 608, the determination unit 610, the image generation data generation unit 620, the storage unit 670, the information storage medium 680, the communication unit 696, etc. Detailed functions and operations are the same as those described above with reference to FIG.

2. Next, the method of this embodiment will be described in detail.

2.1 Timing Determination Processing Using Determination Reference Timing First, view volume setting in stereoscopic viewing will be described before describing the method of the present embodiment. As shown in FIG. 2, in order to generate a stereoscopic image, a left-eye virtual camera VCL and a right-eye virtual camera VCR that are set at positions separated by a given inter-camera distance are used.

  Then, a left-eye view volume VVL (left-eye view frustum) is set corresponding to the left-eye virtual camera VCL, and a right-eye view volume VVR (right-eye view) corresponding to the right-eye virtual camera VCR. Frustum) is set. Specifically, the positions and directions of the left-eye and right-eye view volumes VVL and VVR are set based on the positions and directions of the left-eye and right-eye virtual cameras VCL and VCR.

  In this case, the left-eye image, which is an image that can be seen from the left-eye virtual camera VCL, is generated by perspectively projecting an object that exists in the left-eye view volume VVL onto the screen SC (reference screen). . Similarly, an image for the right eye, which is an image that can be viewed from the virtual camera for right eye VCR, is generated by perspectively projecting an object existing in the right-eye view volume VVR onto the screen SC and drawing it.

  In this case, since objects at positions that are not perspective-projected on the screen SC are not drawn, if perspective projection conversion processing is performed on these objects, processing is wasted. Therefore, as shown in FIG. 2, the left-eye and right-eye view volumes VVL are provided so that objects at positions that are not perspective-projected on the screen SC in the left-eye and right-eye virtual cameras do not enter the view volume. , VVR is set. In FIG. 2, CNL and CFL are the front clipping plane and the rear clipping plane of the left eye view volume VVL, respectively. CNR and CFR are the front clipping plane and the rear clipping plane of the right eye view volume VVR, respectively. It is.

  As described above, in the method of generating the left-eye image and the right-eye image to realize stereoscopic viewing, the sense of depth is recognized by the parallax between the left-eye image and the right-eye image.

  For example, in FIG. 2, the drawing position AL in the left-eye image and the drawing position AR in the right-eye image for the object located at the point A on the screen SC are the same position.

  On the other hand, the object located at the point B is drawn at the position of the point BL in the image for the left eye, and is drawn at the position of the point BR in the image for the right eye. Then, by providing a shift in the drawing positions of the left-eye image and the right-eye image in this way, it is possible to realize a stereoscopic expression in which the object located at the point B appears to jump out of the screen toward the user.

  Now, when timing determination processing such as a music game or a baseball game is performed in such a stereoscopic vision system, it has been found that it is difficult to determine the timing of the player unless any measures are taken.

  Therefore, in this embodiment, when the timing determination is performed using a moving object such as an instruction marker (musical note) in a music game or a ball in a baseball game, the parallax of the image of the moving object becomes a predetermined parallax at the determination reference timing of the timing determination. Thus, a technique for generating a stereoscopic image is employed. For example, the stereoscopic image is generated so that the parallax of the image of the moving object disappears at the determination reference timing. By doing in this way, for example, a stereoscopic image or the like in which the mobile body pops out or retracts at the determination reference timing is generated, and determination processing using this mobile body is performed. Therefore, for example, it is possible to realize timing determination such that the determination reference timing is the best timing, and it is possible to realize appropriate timing determination processing in the stereoscopic system.

  For example, in FIG. 3A, moving bodies MOB1, MOB2, and MOB3 such as instruction markers are moving. Specifically, for example, the moving bodies MOB1 to MOB3 move from the back side to the near side when viewed from the viewpoint side (virtual camera side) in the object space. In this case, the images of the mobile bodies MOB1 to MOB3 are parallax images different from the predetermined parallax. For example, the images of the mobile bodies MOB1 to MOB3 are images with parallax. Specifically, for example, a stereoscopic image that appears as if the moving bodies MOB1 to MOB3 are retracted on the back side with respect to the reference screen (perspective projection surface) is generated.

  In FIG. 3B, the stereoscopic image is generated so that the parallax of the image of the moving body MOB1 becomes a predetermined parallax at the determination reference timing TMB1. For example, the stereoscopic image is generated so that the parallax of the image of the moving body MOB1 is eliminated. At this time, the mobile bodies MOB2 and MOB3 are parallax images different from the predetermined parallax. For example, the image has parallax.

  Thereafter, the mobile bodies MOB1 to MOB3 continue to move, and as shown in FIG. 3C, at the determination reference timing TMB2, the stereoscopic image is now displayed so that the parallax of the image of the mobile body MOB2 becomes a predetermined parallax. Generated. For example, the stereoscopic image is generated so that the parallax of the image of the mobile object MOB2 is eliminated. At this time, the mobile bodies MOB1 and MOB3 have a parallax different from the predetermined parallax. For example, the image has parallax. Specifically, for example, a stereoscopic image that appears to pop out toward the near side with respect to the reference screen for the moving body MOB1 and that appears to retract toward the heel side with respect to the reference screen is generated for the moving body MOB3.

  Then, the player inputs a game operation so that the determination reference timing TMB1 in FIG. 3B matches the input timing. Taking a music game as an example, the mobile bodies MOB1 to MOB3 serve as instruction markers for instructing the player's operation input. The player performs an operation input instructed by the moving body MOB1 serving as an instruction marker at the determination reference timing TMB1. At this time, the moving body MOB1 becomes an image having a predetermined parallax (for example, an image having no parallax) at the determination reference timing TMB1.

  In addition, the player inputs a game operation so that the determination reference timing TMB2 in FIG. 3C matches the input timing. Taking a music game as an example, the player performs an operation input instructed by the moving body MOB2 serving as an instruction marker at the determination reference timing TMB2. At this time, the mobile unit MOB2 becomes an image with a predetermined parallax (for example, an image without parallax) at the determination reference timing TMB2.

  In this way, the player can perform an operation input in accordance with the determination reference timings TMB1 and TMB2 at which the images of the mobile bodies MOB1 and MOB2 are in a predetermined parallax state (a state without parallax). Therefore, the operation input just timing is in a state in which the images of the mobile bodies MOB1 and MOB2 having a predetermined parallax are displayed, so that the operation input of the player can be facilitated. That is, for example, when the predetermined parallax state is a state without parallax, the determination reference timings TMB1 and TMB2 are timings at which the moving body appears to pop out or retract in stereoscopic view. Therefore, the operation input by the player is facilitated by setting the determination reference timing to a timing determination just timing. Taking a music game as an example, at the determination reference timings TMB1 and TMB2, the moving bodies MOB1 and MOB2 become images having no parallax, and the player inputs the operation input instructed at the timing when the moving bodies MOB1 and MOB2 have no parallax. I do. Therefore, it is possible to realize a music game that is easy for the player to take timing. Note that the input timing is not limited to the timing at which the player actually operates the operation unit 160, but is the timing at which the player operates the operation unit 160 and is considered to be the input timing of the player in the timing determination process. May be. Taking a music game for playing an instrument as an example, when the instrument is played by operating the operation unit 160 by the player, the timing at which sound is actually output from the instrument may be used as the input timing of the player.

  4A and 4B are diagrams for describing parallax of an image in stereoscopic view. As shown in FIG. 4A, in the image with parallax, the drawing position of the object MOBL in the left-eye image is different from the drawing position of the object MOBR in the right-eye image. For example, the object at the position of the point B in FIG. 2 is drawn at the position of the point BL on the screen SC in the image for the left eye, and is drawn at the position of the point BR on the screen SC in the image for the right eye.

  On the other hand, as shown in FIG. 4B, in the image without parallax, the drawing position of the object MOBL in the left-eye image and the drawing position of the object MOBR in the right-eye image are the same. For example, the object at the position of the point A in FIG. 2 is drawn at the position of the point A on the screen SC in both the left-eye image and the right-eye image.

  FIG. 5 is a diagram for explaining timing data. For example, the timing data storage unit 174 in FIG. 1 stores data of the determination reference timings TMB1 to TMB5. Data of these determination reference timings TMB1 to TMB5 is stored in the timing data storage unit 174.

  Then, as shown in FIG. 5, the mobile unit MOB1 is controlled so that the parallax of the image becomes a predetermined parallax at the determination reference timing TMB1 (the parallax is controlled so that the parallax is eliminated). Similarly, the mobile bodies MOB2 to MOB5 are controlled so that the parallax of the image becomes a predetermined parallax at the determination reference timings TMB2 to TMB5 (the parallax is controlled so that the parallax is eliminated). Specifically, the mobile unit MOB1 is controlled to pass through the determination reference area at the determination reference timing TMB1. Similarly, the mobile units MOB2 to MOB5 are controlled to pass through the determination reference area at the determination reference timings TMB2 to TMB5.

  Here, the determination reference area is an area for notifying the player of the timing to be input. The determination reference area may be a two-dimensional graphic area or a three-dimensional graphic area. Alternatively, the determination reference area may be a position serving as a determination reference.

  For example, each of the mobile bodies MOB1 to MOB5 is controlled based on timing data stored in the timing data storage unit 174. That is, at each determination reference timing of TMB1 to TMB5, each mobile body of MOB1 to MOB5 is controlled to pass through the determination reference area. When each of the mobile bodies MOB1 to MOB5 passes through the determination reference area, a stereoscopic image is generated so that the parallax of the image becomes a predetermined parallax (no parallax). Thereby, at each determination reference timing of TMB1 to TMB5, an image in which the parallax of each moving body of MOB1 to MOB5 becomes a predetermined parallax (an image with no parallax) is generated.

  Note that the timing determination process of the player input timing is determined based on the determination reference timing data, and the movement control of each of the mobile bodies MOB1 to MOB5 is created in synchronization with the determination reference timing data. It is desirable to control based on the displayed display reference timing data. For example, the display reference timing data is data for controlling the mobile bodies MOB1 to MOB5 to pass, for example, a determination reference area (determination reference position) at the determination reference timings TMB1 to TMB5 in FIG. is there. In this case, the player input timing can be sampled multiple times within one frame. Accordingly, it is desirable that the determination reference timing data is, for example, data in msec units, while the display reference timing data is in frame units.

  Next, details of the timing determination process of the present embodiment will be described. In the present embodiment, as the timing determination process, an evaluation process based on the timing relationship between the player input timing and the determination reference timing is performed. In the following description, it is assumed that the state where the parallax of the moving body is the predetermined parallax is a state where there is no parallax.

  For example, in FIG. 6A, the input timing of the player coincides with the determination reference timing. For example, in FIG. 3B, at the determination reference timing TMB1 corresponding to the mobile object MOB1, the player has made an operation input, and it is determined that the determination reference timing TMB1 matches the input timing. At this timing, the image of the moving body MOB1 has no parallax.

  Specifically, the input timing of the player is sampled and acquired, for example, at a time interval in units of msec (a time interval shorter than the display reference timing). Then, it is searched whether or not the determination reference timing exists in the vicinity of the acquired input timing. Then, when a determination reference timing whose timing difference from the input timing is within, for example, one frame (or less than a predetermined threshold in a broad sense) is searched, it is determined that the determination reference timing matches the input timing. .

  In this embodiment, an evaluation process based on the timing relationship between the input timing and the determination reference timing is performed, and based on the result of the evaluation process, a player's game result calculation process, a moving body display control process, or a game Game calculation processing such as effect processing is performed.

  For example, in FIG. 6A, since it is evaluated that the input timing of the player and the determination reference timing coincide with each other, the game score (score, point) of the player is high. Taking a baseball game as an example, the moving direction of the ball, which is a moving body, is set to the center direction, which is a straight direction. In addition, a game effect process is performed to indicate the success of the player's input operation.

  On the other hand, in FIG. 6 (B) and FIG. 6 (C), the player input timing does not coincide with the determination reference timing, and in FIG. 6B, the input timing is earlier than the determination reference timing. In FIG. 6C, the input timing is later than the determination reference timing. Also in this case, the input timing is within the allowable period TR set with the determination reference timing as a reference (center). Therefore, it is determined that the player's input operation is successful, and an evaluation process for the player's input operation is performed. This evaluation process is performed based on a timing relationship such as a timing difference or a front-rear relationship between the input timing and the determination reference timing.

  For example, the shorter the timing difference (time difference) between the input timing and the determination reference timing, the higher the evaluation of the player's input operation, and the higher the game score given to the player. That is, when the timings coincide as shown in FIG. 6A, the player's score, points, and other game results become higher than those in FIGS. 6B and 6C. Moreover, you may perform display control of the moving body that the speed and acceleration of the moving body after a hit become high, so that a timing difference becomes short. Alternatively, the game effect may be enhanced as the timing difference becomes shorter. For example, when the input timing is just timing with respect to the determination reference timing as shown in FIG. 6A, an image or sound having a high game effect effect in timing determination is output. On the other hand, when there is a difference between the input timing and the determination reference timing as shown in FIGS. 6B and 6C, it is realized by a game effect display object, a game effect sound, or the like as compared with FIG. The game effect is reduced.

  Moreover, you may evaluate not only a timing difference but the back-and-forth relationship between input timing and determination reference timing. For example, the game results given to the player when the input timing is earlier than the determination reference timing as shown in FIG. 6B and when the input timing is later than the determination reference timing as shown in FIG. Or change the moving direction of the moving object after the hit. Or you may change the game production | presentation effect implement | achieved by the display thing for game productions, a game production sound, etc. by the case where input timing is earlier than the determination reference | standard timing.

  In the present embodiment, a stereoscopic image in which the parallax of the moving body image changes according to the timing difference between the input timing of the player and the determination reference timing is generated. Specifically, for example, as the timing difference between the player input timing and the determination reference timing increases, a stereoscopic image in which the parallax of the moving object image increases is generated. For example, in FIG. 6E, the timing difference between the input timing and the determination reference timing is larger than that in FIG. 6D, so that FIG. 6E moves more than FIG. 6D. The parallax of the body image increases.

  For example, as shown in FIG. 3B, when the player performs an operation input at the determination reference timing TMB1, the parallax of the image of the moving body MOB1 becomes zero. On the other hand, when the player performs an operation input with a time delay from the determination reference timing TMB1, the parallax of the image of the moving body MOB1 increases. That is, as the timing difference between the determination reference timing and the input timing increases, the parallax of the image of the moving body MOB1 increases accordingly.

  In this way, when the player performs an operation input at a timing when there is no parallax of the moving body image, the evaluation given to the operation of the player becomes high, and the parallax of the moving body image is reduced. When the player performs an operation input at the timing of the increased state, the evaluation or the like given to the player's operation becomes low. Accordingly, the parallax of the image of the moving object and the magnitude of the evaluation given to the operation of the player are linked, so that the input timing evaluation process that is easy for the player to understand and suitable for the stereoscopic system can be realized. .

2.2 Application Examples to Music Games Next, application examples of the method of the present embodiment to various games will be described. First, a configuration example of a portable game device to which the present embodiment is applied will be described with reference to FIGS. 7A and 7B. In the following description, the present embodiment is not limited to this, assuming that the state where the parallax of the moving body is the predetermined parallax is the state where the parallax of the moving body is not present.

  This portable game device has a main display portion 190M and a sub display portion 190S. The sub display unit 190S is realized by, for example, a touch panel type liquid crystal display, and is provided in the housing 10 of the game apparatus. The main display unit 190M is a display having, for example, a larger number of pixels than the sub display unit 190S, and is realized by, for example, a liquid crystal display. The main display unit 190M is a display that can display, for example, a stereoscopic image with naked eyes, and the game image is displayed in stereoscopic view.

  A case 10 and a case 20 of the portable game device are provided so as to be freely rotatable, and the case 10 is provided with a direction instruction key 12, an analog stick (joystick) 14, and operation buttons 16. Further, as shown in FIG. 7B, first and second cameras CM1 and CM2 are provided on the rear side of the housing 20 (the side opposite to the main display portion 190M). By photographing the subject using the first and second cameras CM1 and CM2, it is possible to obtain a left-eye image and a right-eye image with parallax, and stereoscopic display is possible.

  In addition, a camera CM3 (an imaging unit in a broad sense) is provided on the front side (the main display unit 190 side) of the housing 20. By providing such a camera CM3, it is possible to take an image of a player who is playing with a portable game device.

  Further, the portable game device has a built-in motion sensor (6-axis sensor) not shown. When the player moves the portable game device with his / her hand, using this motion sensor, acceleration in the X-axis, Y-axis, and Z-axis directions in FIG. Angular velocities around the Y axis and Z axis can be detected.

  Next, an application example of the method of the present embodiment to a music game will be described. 8 and 9 are examples of images generated when the method of the present embodiment is applied to a music game. In the following, a drum playing game will be described as an example of the music game, but the field of the music game to which the method of the present embodiment can be applied is not limited to this.

  In FIGS. 8 and 9, the indication markers MK1 to MK5, the determination reference area ARB, the drum character CHT, the evaluation display object SOB, and the gauge GOB are displayed on the main display portion 190M. The sub display unit 190S displays an image of the drum TOB that the player strikes.

  The instruction markers MK1 to MK5 (moving bodies in a broad sense), for example, move from the back side to the near side when viewed from the viewpoint side (virtual camera side) and pass through the determination reference area ARB. Specifically, each of the indication markers MK1 to MK5 passes through the determination reference area ARB at each determination reference timing of TMB1 to TMB5 as described with reference to FIG. For example, in FIG. 8, the instruction marker MK1 passes the determination reference ARB at the determination reference timing TMB1. In FIG. 9, the instruction marker MK2 passes the determination reference ARB at the determination reference timing TMB2. The player touches the screen of the sub display unit 190S with the touch pen 30 (pointing device) at each determination reference timing, which is a timing at which each of these instruction markers passes the determination reference ARB.

  Specifically, at the determination reference timing TMB1 at which the instruction marker MK1 for instructing the “don” operation passes through the determination reference area ARB, the player, among the screens of the sub-display unit 190S, as shown at B1 in FIG. Then, the image portion of the drum TOB is touched with the touch pen 30. Thereby, it is possible to make the player simulate the performance of playing a drum that hits the middle part of the drum and produces a “don” sound.

  On the other hand, at the determination reference timing TMB2, which is the timing at which the indication marker MK2 displayed as “Katsu” passes through the determination reference area ARB, as shown in B2 of FIG. A part other than the image is touched with the touch pen 30. As a result, the player can simulate the performance of a drum that strikes the drum frame and produces a “cut” sound. Each time the player's drumming operation is successful, the player's earned points represented by gauge GOB increase.

  In this embodiment, at each determination reference timing at which each indication marker passes through the determination reference area ARB, the image of each indication marker is an image having no parallax (an image with a predetermined parallax in a broad sense).

  For example, in FIG. 10A, since all of the instruction markers MK1 to MK3 do not pass through the determination reference area ARB and are located on the back side of the determination reference area ARB, MK1 to MK3 are parallax images. ing.

  On the other hand, in FIG. 10B, the instruction marker MK1 passes through the determination reference area ARB at the determination reference timing TMB1, and the image of MK1 is an image having no parallax. At this time, the instruction markers MK2 and MK3 do not pass through the determination reference area ARB, and are positioned on the back side of the determination reference area ARB, so that the images have parallax.

  In FIG. 10C, the instruction marker MK2 passes the determination reference area ARB at the determination reference timing TMB2, and the image of MK2 is an image having no parallax. At this time, since the instruction marker MK1 is located on the front side of the determination reference area ARB and the instruction marker MK3 is located on the back side of the ARB, MK1 and MK3 are images having parallax. Specifically, the instruction marker MK1 is an image that appears to jump out of the determination reference area ARB in stereoscopic view, and the instruction marker MK3 is an image that appears to be retracted from the determination reference area ARB.

  By doing as described above, it appears to the player's eyes that each indication marker jumps to the near side of the screen from the state of being retracted to the far side of the screen, with each judgment reference timing as a boundary. Therefore, the player can recognize that the state without parallax of the image of the indication marker is the just timing, and the timing for performing the operation (“don”, “cut” operation) indicated by the indication marker is easy. It becomes possible to grasp. Thereby, the operation interface environment of the player can be improved.

  In FIG. 8 to FIG. 10C, the determination reference area ARB has a two-dimensional square shape, but the present embodiment is not limited to this. For example, the determination reference area ARB may have a shape other than a square shape or a three-dimensional shape.

  Also, for example, in FIGS. 10A to 10C, lane display objects (display objects representing lines through which instruction markers such as musical notes flow) along the movement path of the instruction markers MK1 to MK3 are displayed. May be. In this case, it is desirable to change the degree of stereoscopic vision between the lane display object and the instruction markers MK1 to MK3 (moving bodies). For example, as the determination reference timing (determination reference area) approaches, each of the indication markers MK1 to MK3 may be displayed in a stereoscopic manner so as to appear to rise from the lane display object.

  Next, details of the timing determination process in the application example to the music game in FIGS. 8 to 10C will be described.

  In FIG. 11A, the input timing TMI of the player coincides with the determination reference timing TMB of the instruction marker MK. For example, in FIG. 8, at the same timing as the determination reference timing TMB (TMB1) at which the instruction marker MK (MK1) for instructing the “don” operation passes through the determination reference area ARB, the player uses the touch pen 30 to play the drum of the sub display unit 190S. Touching the TOB image. At this timing, the image of the instruction marker MK is an image having no parallax.

  When it is determined that the input timing of the player matches the determination reference timing in this way, a very high evaluation is given to the input operation of the player, and the evaluation of the input operation is notified as “good”. An evaluation display object SOB is displayed.

  For example, in FIG. 11B and FIG. 11C, the input timing of the player and the determination reference timing TMB at which the instruction marker MK passes the determination reference area ARB do not match. In FIG. This is earlier than the determination reference timing, and in FIG. 11C, the input timing is later than the determination reference timing. Also in this case, the input timing is within the allowable period TR set with the determination reference timing as a reference (center). Therefore, it is determined that the player's input operation has been successful to some extent, and the evaluation display object SOB is displayed informing that the evaluation of the input operation is “OK”.

  By doing as described above, it is possible to give the player an appropriate evaluation according to the timing relationship between the input timing and the determination reference timing in the music game. In this embodiment, when the player inputs an operation at the determination reference timing TMB as shown in FIG. 11A, the player is highly evaluated, and the image of the instruction marker MK at this time has no parallax. Become an image. Therefore, the player only has to determine the timing at which the parallax of the image of the instruction marker MK disappears and perform an operation input, so that it is possible to provide an operation interface environment that the player can easily grasp intuitively.

2.3 Display Control of Various Objects In this embodiment, as shown in FIGS. 8 and 9, an evaluation display object SOB for displaying the result of the operation input evaluation process is displayed. Specifically, an image in which the degree of parallax changes according to the evaluation result is displayed as the image of the evaluation display object SOB.

  For example, in FIG. 12A, the instruction marker MK passes the determination reference area ARB at the determination reference timing TMB, and the player sets the “don” at the determination reference timing TMB at which the instruction marker MK passes the determination reference area ARB. An operation input for instructing the operation is performed. Accordingly, as shown in FIG. 12B, the input timing TMI and the judgment reference timing TMB coincide with each other, and the player is given a “good” evaluation, which is a high evaluation result. Therefore, the evaluation display object SOB for informing the player of the evaluation result of “good” is displayed.

  In this case, as shown in FIG. 12B, the evaluation display object SOB is arranged in front of the reference screen SC (perspective projection plane) corresponding to the determination reference area ARB. Accordingly, as shown in FIG. 12A, a stereoscopic image in which the evaluation display object SOB appears to pop out from the screen is generated and displayed on the player. That is, the instruction marker MK is at the position of the reference screen SC (judgment reference area) and has no parallax, while the evaluation display object SOB has an image with parallax. The image appears to jump out from the front.

  On the other hand, in FIG. 13A and FIG. 13B, the player performs an operation input instructing an operation of “don” after the instruction marker MK passes the determination reference area ARB, and the input timing of the player The TMI is later than the determination reference timing TMB. Since the input timing TMI is within the allowable period TR set based on the determination reference timing TMB, the player is given an evaluation of “OK”. For this reason, the evaluation display object SOB for informing the player of the evaluation result of “OK” is displayed.

  Then, the evaluation display object SOB representing “OK” is arranged at the position of the reference screen SC corresponding to the determination reference area ARB, as shown in FIG. 13B. Therefore, the evaluation display object SOB becomes an image having no parallax. Moreover, since the instruction marker MK passes through the determination reference area ARB, the image of the instruction marker MK is an image that appears to jump out of the screen.

  In FIGS. 14A and 14B, the player input timing TMI is later than the determination reference timing TMB, and the input timing TMI is not within the allowable period TR. Therefore, an evaluation of “impossible” is given to the player, and an evaluation display object SOB for notifying the player of the evaluation result of “impossible” is displayed.

  Then, the evaluation display object SOB representing “impossible” is arranged at a position on the back side of the reference screen SC as shown in FIG. Accordingly, a stereoscopic image in which the evaluation display object SOB appears to be retracted to the back side of the screen is generated and displayed on the player.

  As described above, in FIGS. 12A to 14B, the evaluation display object SOB is an image in which the degree of parallax changes according to the evaluation result. That is, when the evaluation result is “good” as shown in FIGS. 12A and 12B, the evaluation display object SOB becomes an image that appears to protrude from the screen to the near side, and FIG. When the evaluation result is “OK” as shown in FIG. 13B, the image has no parallax. When the evaluation result is “impossible” as shown in FIGS. 14A and 14B, the evaluation display object SOB becomes an image that is seen to be retracted from the screen.

  Therefore, the player can intuitively recognize the evaluation given to the player according to the degree of parallax of the evaluation display object SOB. In other words, the player recognizes that the operation input is a good evaluation when the evaluation display object SOB appears to pop out from the screen, and the player input recognizes that the operation input is bad when the evaluation display object SOB appears to retract from the screen. Can be recognized. Therefore, the evaluation result can be transmitted to the player intuitively and easily.

  In this embodiment, when it is determined that the player's input is successful, an image of the game effect effect display object is displayed, and an image with parallax is displayed as the game effect effect display object. Yes.

  For example, in FIG. 15A, the drum character CHT is instructed to continuously strike the drum. When the operation input for the continuous hitting is successful, the images of the game effect display objects EOB1, EOB2, and EOB3 as shown in FIG. 15A are displayed. Specifically, as shown in FIG. 15B, the game effect display objects EOB1, EOB2, and EOB3 move so as to be emitted toward the front side of the screen from the position of the instruction marker MK. Therefore, while the instruction marker MK is an image having no parallax, for the game effect display objects EOB1, EOB2, and EOB3, a stereoscopic image that appears to pop out one after another on the front side of the screen is generated. become. For this reason, when the instruction marker MK is an image having no parallax, the game effect display objects EOB1, EOB2, and EOB3 appear to jump out of the screen stereoscopically, so that these objects EOB1, The effects of EOB2 and EOB3 can be greatly improved.

  In the present embodiment, for the instruction marker, an image having no parallax is displayed at the determination reference timing, and for the input action display object indicating that the player input has been performed, an image with parallax is displayed. I have to.

  For example, in FIG. 16A, as described with reference to FIG. 8, the “don” operation is performed by touching the image portion of the drum TOB of the sub display unit 190 </ b> S with the touch pen 30. In this case, an input action display object IOB indicating that such a “don” operation has been performed is displayed. Specifically, as shown in FIG. 16A, an image of the input action display object IOB is displayed around the instruction marker MK. By displaying such an image of the input operation display object IOB, the player can confirm that he / she has normally performed the input operation.

  In this case, as shown in FIG. 16 (B), the instruction marker MK is at the position of the reference screen SC (determination reference area) at the determination reference timing, and thus has no parallax. On the other hand, the input action display object IOB is arranged, for example, on the near side as viewed from the viewpoint side with respect to the reference screen SC.

  By doing in this way, the image of the input action display object IOB around the instruction marker MK appears to jump out of the screen, and the effect of production can be improved. That is, even if the instruction marker MK is an image without parallax, the input operation display object IOB becomes an image with parallax so as to compensate for the effect, so that the instruction marker MK can be projected from the screen. Can be realized.

  In the present embodiment, a stereoscopic image in which at least one of the color, shape, and size of a moving object such as an instruction marker changes may be generated at the determination reference timing.

  For example, in FIG. 17A, the instruction marker MK is moving toward the determination reference area ARB. Then, at the determination reference timing TM in FIG. 17B, the color, shape, size, or the like of the instruction marker MK changes. That is, in FIG. 17B, the color, shape, size, or the like of the instruction marker MK is different from that in FIG.

  In this way, the visibility of the instruction marker MK is increased by changing the color, shape, size, or the like of the instruction marker MK at the determination reference timing, which is just timing of the timing determination processing, for example. As a result, the player determines that the timing of FIG. 17B is the determination reference timing not only for the parallax state (predetermined parallax state, no parallax state) of the indication marker MK but also for the color, shape, or size. It becomes possible to grasp even by change. Therefore, it is possible to realize a timing determination process in which the player can easily grasp the timing.

  Note that the change in the image of the instruction marker MK (moving body) may be a change in display mode other than the change in color, shape, and size, as long as the visibility of the determination reference timing can be improved. .

  Moreover, in this embodiment, you may produce | generate the sound from which sound image localization changes according to the parallax of a moving body. That is, a sound is generated so that the sound output of the stereophonic sound matches the position of the stereoscopic view.

  For example, in FIG. 18, sound output portions SP1 and SP2 (speakers) are provided on the main display portion 190M side of the housing 20 of the portable game device of FIG. In addition, a virtual instruction marker MKV (virtual display object in a broad sense) corresponding to the instruction marker MK (moving body) is stereoscopically displayed in the display screen side space of the main display portion 190M (display section in a broad sense). The virtual instruction marker MKV does not actually exist in the display screen side space, but based on the stereoscopic image (left-eye image, right-eye image) of the instruction marker MK, is the player present at that position? The virtual display object is recognized as follows.

  The sound output units SP1 and SP2 output sound image localization sound (stereo sound) that is linked to the display position (virtual display position) of the virtual instruction marker MKV. For example, when the parallax of the indication marker MK is such that the parallax of the main display unit 190M is on the near side of the screen, and the virtual indication marker MKV is virtually displayed on the near side of the screen, the sound output units SP1 and SP2 The sound image localization formed by the sound is also set at a position on the near side of the screen.

  On the other hand, when the parallax of the indication marker MK is parallax on the heel side of the screen and the virtual indication marker MKV is virtually displayed on the heel side of the screen, the parallax is formed by sound from the sound output units SP1 and SP2. The sound image localization is also set to a position on the back side of the screen.

  In this way, for example, in a situation where the pointing marker (virtual pointing marker) appears to jump out from the screen, the sound image is localized so that it sounds like it pops out from the screen. On the other hand, in a situation where the indication marker appears to be retracted from the screen, the sound image is localized so that it sounds like it is retracted from the screen. Thereby, it becomes possible to emphasize the effect of a stereoscopic vision more, and the game effect effect etc. can be improved.

  Various known methods can be adopted as a method for setting the sound image localization of the sound. For example, it is possible to control stereophonic sound such as sound image localization by changing the volume, time difference, frequency characteristic, phase, or reverberation. When the volume difference is used, the sound image localization of the sound is controlled by, for example, the attenuation of the volume due to the distance or the intensity difference between both ears. When using the time difference, the sound image localization of the sound may be controlled by the time difference that the sound wave reaches. In the case of using a change in frequency characteristics, the sound image localization of the sound may be controlled by a change in the frequency characteristics due to transmission or interruption of sound waves. In the case of using a change in phase, the sound image localization of the sound may be controlled by a change in phase due to transmission or interruption of sound waves. In the case of using reverberation change, the sound image localization of the sound may be controlled by reproducing the sound field of the surrounding environment based on the reverberation characteristics.

  In the present embodiment, a virtual display object corresponding to a moving object or an operation target object for timing determination processing, which is virtually displayed in the display screen side space of the display unit based on the stereoscopic image, The hit determination process with the hit object in the real space may be performed. For example, a hit determination process between a virtual display object and a hit target that is a part of a player or a pointing device is performed based on imaging information from an imaging unit that images the player.

  For example, in FIG. 19A, the player PL can be photographed by a camera CM3 (imaging unit) provided on the main display unit 190M side of the portable game device. Specifically, when the player PL performs an operation of bringing a finger close to the screen of the main display unit 190M, the operation is detected based on the shooting information from the camera CM3.

  In FIG. 19 (B), a virtual instruction marker MKV (virtual display object in a broad sense) is virtually displayed in the display screen side space of the main display portion 190M, as described with reference to FIG. That is, based on the stereoscopic image (left-eye image, right-eye image) of the indication marker MK, a stereoscopic display that appears as if the virtual indication marker MKV exists at the position of FIG. Done.

  In FIG. 19B, hit determination processing is performed between the virtual instruction marker MKV and the finger of the player PL that is a hit body in the real space. For example, when the camera CM 3 detects that the finger of the player PL (a hit object in a broad sense) is present near the screen at the determination reference timing (display timing) of the virtual instruction marker MKV, the virtual instruction marker MKV It is determined that the player PL's finger hits. In this case, for example, distance information from the screen of the virtual instruction marker MKV may be specified from the parallax information of the image of the instruction marker MK, and the hit determination process may be performed using this distance information. Alternatively, when the camera CM 3 is a depth camera or the like that can detect depth information, position information (X, Y coordinates) in the screen of the finger of the player PL is specified and specified based on the depth information. The hit determination process may be performed using the position information. For example, when it is determined at the determination reference timing that the finger of the player PL and the virtual instruction marker MK are hit, a game calculation process such as adding the player's score or points is performed.

  In this way, it is possible to perform timing determination processing using a moving object such as an instruction marker by effectively using stereoscopic vision, and it is possible to realize timing determination processing based on hit determination processing that has not existed before. become.

  In FIG. 19A and FIG. 19B, hit determination processing is performed between a virtual display object corresponding to a moving body such as an instruction marker and a hit body such as a player's part. Is not limited to this. For example, separately from a moving object such as an instruction marker, an operation target for timing determination processing may be stereoscopically displayed on the screen. Thereby, a virtual display object corresponding to the operation target object for the timing determination process is virtually displayed in the display screen side space of the display unit. For example, the player performs an operation of touching a virtual display object corresponding to the operation target for the timing determination process with a hit target such as a finger or a pointing device at the determination reference timing of the moving object such as the pointing marker. Do. When it is determined that the hit object hits the virtual display object corresponding to the operation target (when it is determined that the hit object has been touched), a game calculation process such as adding a player's score, points, or the like is performed. Will be done.

2.4 Example of application to baseball game Next, an example of application to a baseball game will be described. 20 and 21 are examples of images generated when the method of this embodiment is applied to a baseball game. In FIG. 20, an image of a scene in which the pitcher character CHP throws the ball BL (moving body in a broad sense) against the batter character CHB is displayed on the main display portion 190M. The main display portion 190M also displays a determination criterion area ARB, and the sub display portion 190S displays an operation input area ARI corresponding to the determination criterion area ARB. The player performs a batting operation of hitting the ball BL by touching the position of the course in which the ball BL is predicted to fly in the operation input area ARI with the touch pen 30 (pointing device in a broad sense).

  The determination reference area ARB is set, for example, on the home base, and each pointing position of the operation input area ARI pointed by the touch pen 30 corresponds to each position of the determination reference area ARB. For example, when the upper right position of the operation input area ARI is touched, the upper right position of the determination reference area ARB is instructed, and when the lower left position of the operation input area ARI is touched, the lower left position of the determination reference area ARB is instructed.

  When the ball BL passes through the determination reference area ARB as shown in FIG. 20, the image of the ball BL is an image with zero parallax. That is, the ball BL is controlled so as to pass through the determination reference area ARB at the determination reference timing, and the image of the ball BL becomes an image with zero parallax at that time. For this reason, it appears to the player's eyes that the ball BL jumps to the front side of the screen from the state of being retracted to the back side of the screen, with the determination reference area ARB as a boundary. Therefore, the player can recognize that the state without parallax in the image of the ball BL is the just timing, and can easily grasp the timing of hitting the ball BL. Thereby, the operation interface environment of the player can be improved.

  In FIG. 20, the determination reference area ARB and the operation input area ARI are substantially the same size, but the present embodiment is not limited to this. For example, the difficulty level of the player's operation input may be set by changing the size of the operation input area ARI. For example, by making the size of the operation input area ARI smaller than the determination reference area ARB, the difficulty level of the player's operation input is set to a high difficulty level.

  In FIG. 21, the player expects the course of the ball BL to have a higher inner angle, and uses the touch pen 30 (or finger) to position the upper right position of the operation input area ARI of the sub display unit 190S as indicated by A1. Touching. In FIG. 21, an image after the ball BL has passed is displayed on the main display portion 190M, and the player's designated position display object DOB and the ball position display object POB are displayed.

  The designated position display object DOB is an object for informing the player's hit designated position, and is displayed at the position of the determination reference area ARB. That is, the indication position display object DOB is arranged and displayed at the same depth position (same Z value) as the determination reference area ARB. In FIG. 21, the player touches the upper right position of the operation input area ARI of the sub display unit 190S with the touch pen 30. Therefore, in the main display portion 190M, the indicated position display object DOB is arranged and displayed at the upper right position of the determination reference area ARB. Thus, the player can later confirm which course he / she touched when passing the ball BL by looking at the indicated position display object DOB.

  The ball position display object POB (moving body position display object in a broad sense) is an object for informing the player of the position of the ball BL at the input timing when the player has indicated the hit instruction position. For example, the ball position display object POB is arranged and displayed at the position of the ball BL at the input timing of the player.

  In FIG. 21, the course of the ball BL is in the middle at the input timing when the player designates the hit instruction position with the touch pen 30, and the ball BL passes through the middle of the determination reference area ARB. Accordingly, the ball position display object POB is arranged and displayed near the center. As a result, the player can later confirm the position of the ball BL at the timing when the player inputs the operation and swings the bat.

  The player can confirm the shift between the hit position and the ball position at the input timing by looking at the positional relationship between the designated position display object DOB and the ball position display object POB. For example, in FIG. 21, the displacement between the designated position display object DOB and the ball position display object POB is relatively large. Therefore, in FIG. 21, the player can later confirm that his / her designated position does not match the course of the ball BL. Thereby, an interface environment suitable for a baseball game can be provided to the player.

  In FIG. 21, the hit determination volume HV including the determination reference area ARB is set. When the ball BL is positioned in the hit determination volume HV at the input timing when the player touches the operation input area ARI with the touch pen 30, it is determined that the batting is successful and the ball hits the bat. The hit strength and hit direction of the ball BL are set based on the position of the ball BL at the input timing and the relationship between the touch position in the operation input area ARI and the passing position of the ball BL.

2.5 Detailed Processing Example Next, a detailed processing example of the present embodiment will be described with reference to the flowcharts of FIGS.

  FIG. 22 is a flowchart for explaining the details of the overall processing of this embodiment. First, data of determination reference timings TMB1 to TMBN as described with reference to FIG. 5 is read from the timing data storage unit 174 (step S1). Then, it is determined whether or not it is a frame update timing (step S2).

  If it is the frame update timing, the moving process of the moving body such as the pointing marker and the ball is performed (step S3). For example, a process of moving the moving body from the back side to the near side when viewed from the viewpoint side is performed. Also, the input timing TMI is acquired (step S4). Taking FIG. 8 as an example, the timing at which the player touches the sub display unit 190S with the touch pen 30 or the finger is acquired.

  Next, the judgment reference timing TMBm (1 ≦ m ≦ N) is compared with the input timing TMI (step S5). It is then determined whether TD = | TMBm−TMI | ≦ TR / 2 (step S6). That is, as described with reference to FIGS. 6A to 6C and the like, it is determined whether or not the input timing TMI is within the allowable period TR based on the determination reference timing TMBm. When TD = | TMBm−TMI | ≦ TR / 2, it is determined that the input timing TMI is within the allowable period TR, it is determined that the input is successful, the game result calculation process, Game calculation processes such as a display control process and a game effect process are executed (step S7).

  Next, an image for the left eye that can be seen from the virtual camera for the left eye is generated by perspective-projecting and drawing the object on the reference screen (step S8). In addition, the right-eye image that can be seen from the virtual camera for the right eye is generated by perspective-projecting and drawing the object on the reference screen (step S9). Thereby, a left-eye image and a right-eye image that are stereoscopic images are generated. For example, the moving object is controlled so as to pass through the reference screen at the determination reference timing, and the object of the moving object is perspective-projected and drawn on the reference screen so that the parallax of the image of the moving object is determined at the determination reference timing. A stereoscopic image that disappears is generated.

  FIG. 23 is a flowchart for explaining the details of the evaluation process when the method of this embodiment is applied to a music game.

  First, it is determined whether TD = | TMBm−TMI | ≦ TTH (step S11). Here, TTH is a threshold value of the evaluation process, and when TD = | TMBm−TMI | ≦ TTH, it is considered that the determination reference timing TMBm and the input timing TMI coincide with each other. If TD = | TMBm−TMI | ≦ TTH, it is determined that the input evaluation result is “good” (step S12). Then, as described with reference to FIGS. 12A and 12B, the evaluation result display object is arranged in front of the reference screen (step S13). As a result, a stereoscopic image in which an evaluation display object representing the evaluation result of “good” appears to pop out from the screen is generated.

  On the other hand, when TD = | TMBm−TMI | ≦ TTH is not satisfied, it is determined whether TD = | TMBm−TMI | ≦ TR / 2 (step S14). Here, TR is an allowable period for determining whether or not the operation input is successful. If TD = | TMBm−TMI | ≦ TR / 2, since the input timing TMI is within the allowable period TR, it is determined that the input evaluation result is “possible” (step S15). Then, as described in FIGS. 13A and 13B, the evaluation result display object is arranged at the position of the reference screen (step S16).

  On the other hand, if TD = | TMBm−TMI | ≦ TR / 2 is not satisfied, it is determined that the input evaluation result is “impossible” (step S17). Then, as described with reference to FIGS. 14A and 14B, an evaluation result display object is arranged on the back side of the reference screen (step S18). As a result, a stereoscopic image in which the evaluation display object representing the evaluation result of “impossible” appears to be pulled back from the screen is generated.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term (an image without parallax, an image without parallax, left) described in the specification or drawings at least once with a broader or synonymous different term (an image with a predetermined parallax, a first viewpoint image, a second viewpoint image, a moving object, etc.) Ophthalmic images, right-eye images, pointing markers / balls, etc.) can be replaced by the different terms anywhere in the specification or drawings. Further, the timing determination method, the determination reference timing setting method, the stereoscopic image generation method, the moving object movement control method, and the like are not limited to those described in the present embodiment, and methods equivalent to these are also included in the present invention. Included in the range. The present invention can be applied to various games. Further, the present invention is applied to various image generation systems such as a business game system, a home game system, a large attraction system in which a large number of players participate, a simulator, a multimedia terminal, a system board for generating a game image, and a mobile phone. it can.

TMB1-TMB5 judgment reference timing, MOB, MOB1-MOB3 mobile,
MK, MK1 to MK5 indicating marker, SOB evaluation display object,
GOB gauge CHT drum character, TOB drum,
ARB criteria area, ARI operation input area,
CHP pitcher character, CHB batter character, BL ball,
DOB indication position display object, POB ball position display object,
HV hit determination volume,
VCL virtual camera for left eye, virtual camera for VCR right eye,
SC screen (reference screen), VVL left eye view volume,
VVR Right eye view volume, CNL, CNR, CFL, CFR clipping plane,
CM1, CM2, CM3 camera,
MKV virtual instruction marker, PL, player, SP1, SP2 sound output unit,
10 housing, 12 direction instruction keys, 14 analog stick, 16 operation buttons,
20 housing, 30 touch pen, 190M main display section, 190S sub display section,
100 processing unit, 102 game calculation unit, 104 object space setting unit,
106 moving body control unit, 108 virtual camera control unit, 110 determination unit,
120 image generation units, 122 left-eye image generation units, 124 right-eye image generation units,
170 storage unit, 172 object data storage unit,
174 Timing data storage unit, 178 drawing buffer, 180 information storage medium,
190 display unit, 192 sound output unit, 194 auxiliary storage device, 196 communication unit,
500 server system, 510 network, TM1-TMn terminal device,
600 processing unit, 602 game calculation unit, 604 object space setting unit,
606 moving body control unit, 608 virtual camera control unit, 610 determination unit,
620 Image generation data generation unit, 630 Sound generation data generation unit

Claims (20)

A moving body control unit for controlling the moving body;
A determination unit that performs a timing determination process using the moving body;
As an image generation unit that generates a stereoscopic image,
Make the computer work,
The image generation unit
A program for generating a stereoscopic image on which the moving body is displayed so that a parallax of an image of the moving body becomes a predetermined parallax at a determination reference timing used in the timing determination process. In claim 1,
As a timing data storage unit for storing data of the determination reference timing,
Make the computer work,
The determination unit
A program for performing the timing determination process based on data of the determination reference timing stored in the timing data storage unit and an input timing of a player. In claim 2,
As a game calculation unit for performing game calculation processing,
Make the computer work,
The determination unit
As the timing determination process, an evaluation process based on a timing relationship between the input timing and the determination reference timing is performed,
The game calculation unit
A program for performing the game calculation process based on a result of the evaluation process. In claim 3,
The game calculation unit
A program characterized in that at least one of a calculation process of game results of the player, a display control process of the moving body, and a game effect process is performed as the game calculation process based on the result of the evaluation process. In claim 3 or 4,
The image generation unit
A program for generating a stereoscopic image in which an image in which a degree of parallax is changed according to the evaluation result is generated as an image of an evaluation display object for displaying the result of the evaluation process. In any one of Claims 1 thru | or 5,
The image generation unit
A program for generating a stereoscopic image on which a moving object is displayed so that parallax of the image of the moving object is eliminated at a determination reference timing used in the timing determination process. In any one of Claims 1 thru | or 6.
The image generation unit
A program for generating a stereoscopic image in which a parallax of an image of the moving object changes according to a timing difference between the input timing and the determination reference timing. In any one of Claims 1 thru | or 7,
The image generation unit
Generating a stereoscopic image on which the moving body is displayed so that there is no parallax in the image of the moving body at the determination reference timing used in the timing determination process;
If it is determined in the timing determination process that the input timing of the player matches the determination reference timing and it is determined that the input of the player is successful, there is a parallax as an image of the game effect display object A program for generating a stereoscopic image on which an image is displayed. In any one of Claims 1 thru | or 8.
The moving body is an instruction marker that indicates the timing to be input by the player,
The determination unit
A program characterized in that, when the player performs an input at a timing indicated by the indication marker, a match determination process between the input timing of the player and the determination reference timing is performed as the timing determination process. In claim 9,
The image generation unit
For the indication marker, an image without parallax is displayed at the determination reference timing, and for an input action display object indicating that an input has been made by the player, a stereoscopic image in which an image with parallax is displayed A program characterized by generating In any one of Claims 1 thru | or 10.
The mobile control unit is
A program for performing control to move the moving body from the back side to the near side when viewed from the viewpoint side. In any one of Claims 1 thru | or 11,
The mobile control unit is
In the determination reference timing, a program for performing control for moving the moving body so that the moving body passes through a determination reference area for notifying a player of a timing to be input. In claim 12,
The image generation unit
A stereoscopic image in which parallax of the image of the moving object is eliminated at the determination reference timing by perspectively projecting and drawing the object of the moving object on a reference screen set at a position corresponding to the determination reference area A program characterized by generating In any one of Claims 1 thru | or 13.
The image generation unit
A program for generating a stereoscopic image in which at least one of a color, a shape, and a size of the moving body changes at the determination reference timing. In any one of Claims 1 thru | or 14.
As a sound generator that generates sound,
Make the computer work,
The sound generator is
A program for generating a sound whose sound image localization changes according to the parallax of the moving body. In any one of Claims 1 thru | or 15,
The determination unit
A virtual display object corresponding to the moving object or the operation target object for timing determination processing, which is virtually displayed on the display screen side space of the display unit based on the stereoscopic image, and a real space A program characterized by performing hit determination processing with a hit object. In claim 16,
The determination unit
A program for performing a hit determination process between the virtual display object and the hit object based on imaging information from an imaging unit that images a player.   A computer-readable information storage medium, wherein the program according to any one of claims 1 to 17 is stored. A moving body control unit for controlling the moving body;
A determination unit that performs a timing determination process using the moving body;
An image generator for generating a stereoscopic image;
Including
The image generation unit
An image generation system for generating a stereoscopic image on which the moving body is displayed so that parallax of the image of the moving body is eliminated at a determination reference timing used in the timing determination process. A moving body control unit for controlling the moving body;
A determination unit that performs a timing determination process using the moving body;
An image generation data generation unit for generating image generation data for generating a stereoscopic image;
Including
The image generation data generation unit includes:
Generating image generation data for generating a stereoscopic image on which the moving object is displayed so that parallax of the image of the moving object is eliminated at a determination reference timing used in the timing determination process. Server system.

Images