JP5912289B2 - Information processing program, information processing apparatus, information processing system, and information processing method - Google Patents

Description

  The present invention relates to an information processing program, an information processing apparatus, an information processing system, and an information processing method, and in particular, for example, an information processing program, an information processing apparatus, and an information processing system for performing processing based on the attitude and / or movement of a display device And an information processing method.

  Conventionally, there is a game in which a user grips and operates a mobile portable terminal (portable game device), and an event is executed according to the posture and position of the mobile portable terminal in real space (see, for example, Patent Document 1). . The mobile portable terminal described in Patent Document 1 includes a sensor that detects the position and orientation of the mobile mobile terminal in real space, and the user of the terminal moves the mobile mobile terminal itself or moves the mobile mobile terminal itself. Proceed with the game while changing the attitude. For example, the mobile mobile terminal scrolls the image displayed on the display screen of the mobile mobile terminal according to the attitude of the mobile mobile terminal in real space. Then, when a scope is displayed in the center of the display screen of the mobile portable terminal and a predetermined button of the mobile portable terminal is pressed in a state where a virtual object (for example, insect object) is entered in the scope, the virtual object is displayed in the scope. Is considered captured.

Japanese Unexamined Patent Publication No. 2011-1981

  However, the mobile portable terminal disclosed in Patent Document 1 scrolls an image according to the attitude of the mobile portable terminal itself, and puts a virtual object in a scope fixedly displayed at the center of the display screen of the mobile portable terminal. Operate as follows. Therefore, since the positional relationship between the virtual world and the scope always changes due to the scrolling of the image, the user may scroll the image according to the attitude of the mobile mobile terminal itself while maintaining the positional relationship between the virtual world and the scope. Can not. In other words, when the user tries to look in the other direction of the virtual world, the scope always moves, so that the operation becomes uniform, and the reality and fun of the operation may be reduced.

  Therefore, an object of the present invention is to control an object appearing in a virtual world and a virtual camera that generates an image of the virtual world by controlling the attitude and / or movement of a display device that displays the image generated by the virtual camera. Another object of the present invention is to provide an information processing program, an information processing system, and an information processing method capable of diversifying user operations depending on the posture and / or movement of a display device.

  In order to achieve the above object, the present invention may employ the following configuration, for example. When interpreting the description of the claims, it is understood that the scope should be interpreted only by the description of the claims, and the description of the claims and the description in this column are contradictory. In that case, priority is given to the claims.

  One configuration example of the information processing program of the present invention is an information processing apparatus capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body. Run on computer. The information processing program causes the computer to function as direction determination means, object motion control means, first virtual camera motion control means, and display control means. The direction determining means determines whether the direction of the predetermined axis set in the portable display device is within a predetermined range based on data output from the portable display device. When it is determined that the direction of the predetermined axis is within the predetermined range, the object movement control unit controls the movement of the object arranged in the virtual world according to the direction of the predetermined axis. The first virtual camera operation control means controls the operation of the first virtual camera for generating an image of the virtual world according to the direction of the predetermined axis when the direction of the predetermined axis is within the predetermined range or outside the predetermined range. . The display control means displays a first image indicating the virtual world viewed from the first virtual camera on the portable display device.

  The information processing apparatus may be an apparatus that executes a game process and generates an image based on the game process, or may be a versatile apparatus such as a general personal computer. The portable display device may be a size that can be carried by a user, and is typically a display device that allows a user to visually recognize an image displayed on the portable display device in a posture held by both hands. Also good. Further, the portable display device is a device other than a means for outputting at least data corresponding to the attitude and / or movement of the portable display device body and a means for displaying the first image, like a terminal device in an embodiment described later. Other configurations may or may not be provided.

  According to the above, when the user performs an operation using the portable display device within a predetermined range, the operations of the object and the first virtual camera are controlled according to the direction of the predetermined axis of the portable display device. . Therefore, when the user performs an operation using the portable display device within a predetermined range, not only the line-of-sight direction of the first virtual camera for generating a virtual world to be displayed on the portable display device by the operation, Since the movement of the object arranged in the virtual world can also be controlled, this is a suitable operation for changing the movement of the object or changing the display range displayed on the LCD 61. On the other hand, when the user performs an operation using the portable display device outside the predetermined range, only the line-of-sight direction of the first virtual camera is changed according to the posture and movement of the portable display device. Therefore, the operation using the portable display device outside the predetermined range is a preferable operation for changing only the display range displayed on the portable display device while keeping the movement of the object as it is. In this way, by setting at least one predetermined range, the user can perform various operations by the posture and movement of one device. Further, when the user performs an operation using the portable display device outside the predetermined range, only the line-of-sight direction of the first virtual camera changes, and therefore the operation using the portable display device cannot move the object. The user can grasp that the range has been deviated, and the range for operating the object can also be implicitly indicated to the user.

  The direction determination unit may include a left / right direction determination unit. The left-right direction determination means determines whether the direction of the predetermined axis in the left-right direction of the real space is within the first range set in the left-right direction based on the data output from the portable display device. judge. In this case, when it is determined that the direction of the predetermined axis in the left-right direction is within the first range, the object motion control means, based on the direction of the predetermined axis in the left-right direction, The movement of the object may be controlled by rotating and / or moving the object.

  According to the above, it is possible to set a predetermined range in which the object is operated by rotating (yawing) the portable display device to the left and right.

  The direction determination unit may further include a vertical direction determination unit. The vertical direction determination means determines whether the direction of the predetermined axis in the vertical direction of the real space is within the second range set in the vertical direction based on the data output from the portable display device. judge. In this case, when it is determined that the direction of the predetermined axis in the vertical direction is within the second range, the object motion control unit is configured to move the virtual world in the vertical direction based on the direction of the predetermined axis in the vertical direction. The movement of the object may be controlled by rotating and / or moving the object.

  According to the above, it is possible to set a predetermined range in which the object is moved depending on whether the portable display device is rotated left and right (yaw) or rotated up and down (pitch).

  The first range may have an angle range with a predetermined size. The second range may have an angle range having a size different from that of the first range.

  According to the above, the predetermined range for moving the object can be set in different angular ranges depending on whether the portable display device is rotated left and right (yaw) or rotated up and down (pitch). An appropriate range can be set in accordance with the operation environment and the game environment when holding the portable display device.

  Further, the first range may be a range having the same angle range on the left and right with a reference direction in the real space as a center. The second range may be a range having different angular ranges in the vertical direction with respect to the horizontal direction in the real space.

  According to the above, the center of the predetermined range in which the object is operated can be set in different directions depending on whether the portable display device is rotated left and right (yaw) or vertically (pitch). Can set the range at an appropriate position according to the operating environment and the game environment when holding the portable display device.

  Further, the first virtual camera motion control means, when it is determined that the direction of the predetermined axis is within the predetermined range, the attitude of the first virtual camera so that the same direction as the direction of the object in the virtual world is the line-of-sight direction. May be controlled.

  According to the above, if the direction of the predetermined axis of the portable display device is within the predetermined range, the line-of-sight direction of the first virtual camera also changes according to the change in the direction of the object. Accordingly, as a result, the direction of the object and the line-of-sight direction of the first virtual camera are controlled based on the attitude and / or movement of the portable display device, so that the portable display device can be viewed in the direction desired by the user. It is also possible to provide the user with an image that looks into the virtual world through the portable display device and whether the object is in the virtual world. It is also possible to give such a feeling. In addition, the direction of the object can be set by the attitude and / or movement of the portable display device, and the virtual world viewed from the direction is displayed on the portable display device, so that the operation for setting the direction of the object is intuitive. This makes it easy to match the direction to the direction desired by the user.

  The object motion control means may control the motion of the object based on a change in the direction of a predetermined axis. The first virtual camera operation control means may control the operation of the first virtual camera based on a change in the direction of a predetermined axis.

  Based on the above, the user can control the operation of the object and the operation of the first virtual camera by changing the posture and / or position of the portable display device.

  The object motion control means may control the motion of the object by moving the object more greatly as the change in the direction of the predetermined axis is larger. The first virtual camera operation control means may control the operation of the first virtual camera by largely moving the first virtual camera as the change in the direction of the predetermined axis is larger.

  Based on the above, the user can control the amount of movement (including the amount of rotational movement) of the object and the first virtual camera by the magnitude of changing the posture and / or position of the portable display device.

  The object motion control means may control the motion of the object by rotating the object according to an angle of a predetermined axis. The first virtual camera operation control means may control the operation of the virtual camera by rotating the first virtual camera according to an angle of a predetermined axis.

  Based on the above, the user can control the operation of the object and the operation of the first virtual camera according to the angle of the portable display device.

  The information processing program may further cause the computer to function as posture movement calculation means. The posture movement calculation means calculates the posture and / or movement of the portable display device based on the data output from the portable display device. In this case, the direction determination means may determine whether the direction of the predetermined axis is within a predetermined range based on the attitude and / or movement of the portable display device.

  According to the above, the orientation and / or movement of the portable display device is calculated using the data output from the portable display device, and the direction of the predetermined axis is based on the posture and / or movement of the portable display device. It is possible to control the determination and the operation of the object and the first virtual camera.

  In addition, the posture movement calculation unit may calculate the posture and / or movement of the portable display device based on a predetermined direction in the real space. The object motion control means controls the motion of the object based on the direction corresponding to the predetermined direction set in the virtual world based on the attitude and / or movement of the portable display device based on the predetermined direction in the real space. May be. The first virtual camera operation control means is based on the direction corresponding to the predetermined direction set in the virtual world based on the attitude and / or movement of the portable display device with reference to the predetermined direction in the real space. The operation of the virtual camera may be controlled.

  According to the above, operation control based on a predetermined direction of the virtual world corresponding to the predetermined direction can be performed by the attitude and / or movement of the portable display device based on the predetermined direction of the real space.

  Further, the posture movement calculating means may calculate the posture and / or movement of the portable display device with the gravity direction of the real space as a predetermined direction, with the gravity direction as a reference. The object motion control means may control the motion of the object based on the gravity direction set in the virtual world based on the attitude and / or movement of the portable display device based on the gravity direction of the real space. . The first virtual camera motion control means is configured to operate the first virtual camera based on the gravity direction set in the virtual world based on the attitude and / or movement of the portable display device based on the gravity direction of the real space. May be controlled.

  According to the above, operation control based on the same direction of gravity can be performed by the posture and / or movement of the portable display device based on the direction of gravity.

  In addition, the posture movement calculation unit may calculate at least a posture and / or movement of the portable display device that rotates around the direction of gravity in real space. The direction determination means sets a range around the gravity direction of the real space as at least a predetermined range, and the direction of the predetermined axis is determined based on the posture and / or movement of the portable display device rotating around the gravity direction of the real space. You may determine whether it exists in the said predetermined range. The object motion control means controls the motion by rotating the object around the gravity direction set in the virtual world based on the attitude and / or movement of the portable display device rotating around the gravity direction in the real space. Also good. The first virtual camera operation control means rotates the first virtual camera around the gravity direction set in the virtual world based on the posture and / or movement of the portable display device around the gravity direction of the real space. The operation may be controlled.

  According to the above, by controlling the portable display device to the left and right in the real space, it is possible to control the operation of the object and the first virtual camera toward the left and right in the virtual world.

  Further, the posture movement calculation means may calculate at least a posture and / or movement of the portable display device that swings up and down around a horizontal direction perpendicular to the gravity direction of the real space. The direction determining means sets the range around the horizontal direction of the real space as at least the predetermined range, and the direction of the predetermined axis is based on the posture and / or movement of the portable display device swinging up and down around the horizontal direction. It may be determined whether or not it is within a predetermined range. The object motion control means swings the object up and down around the horizontal direction set in the virtual world corresponding to the horizontal direction based on the posture and / or movement of the portable display device swinging up and down around the horizontal direction. The operation may be controlled. The first virtual camera operation control means is configured to first move around a horizontal direction set in a virtual world corresponding to the horizontal direction based on a posture and / or movement of the portable display device swinging up and down around the horizontal direction. The operation may be controlled by shaking the virtual camera up and down.

  Based on the above, by controlling the portable display device up and down in the real space, the direction of the object and the first virtual camera can be controlled up and down in the virtual world.

  Further, the posture movement calculating means is configured to rotate around a first axis and a second axis perpendicular to the depth direction of the display screen perpendicular to the display screen of the portable display device on which the first image is displayed, and / or Alternatively, at least the movement may be calculated. The direction determination means sets the ranges around the first axis and the second axis as predetermined ranges, and based on the posture and / or movement of the portable display device rotating around the first axis and the second axis, respectively. It may be determined whether or not the direction of the predetermined axis is within the predetermined range. When it is determined that the direction of the predetermined axis is within a predetermined range around the first axis, the object motion control means is configured to change the portable display device according to the posture and / or movement of the portable display device around the first axis. The movement may be controlled by rotating the object around the axis set in the virtual world corresponding to the first axis. When it is determined that the direction of the predetermined axis is within a predetermined range around the second axis, the object motion control unit is configured to change the portable display device according to the posture and / or movement of the portable display device around the second axis. The movement may be controlled by rotating the object around the axis set in the virtual world corresponding to the second axis. The first virtual camera motion control means is orthogonal to the line-of-sight direction according to the attitude and / or movement of the portable display device around the first axis and the second axis, respectively, and the first axis and the second axis. You may control operation | movement of a 1st virtual camera by rotating around the axis | shaft corresponding to each.

  According to the above, in the real space, by moving the portable display device around two axes orthogonal to the depth direction of the display screen of the portable display device, the direction of the object or the first virtual camera is moved up and down. The operation can be controlled to the left and right.

  In addition, the posture motion calculation unit may calculate at least a posture and / or a motion that rotate around the horizontal axis and the vertical axis of the display screen of the portable display device orthogonal to the depth direction. The direction determining means sets the respective ranges around the horizontal axis and the vertical axis as predetermined ranges, and a posture in which the portable display device rotates around the horizontal axis and the vertical axis, respectively. Based on the movement, it may be determined whether the direction of the predetermined axis is within the predetermined range. If the direction of the predetermined axis is determined to be within a predetermined range around the horizontal axis, the object motion control means is responsive to the posture and / or movement of the portable display device rotating around the horizontal axis. The operation may be controlled by rotating the object around the horizontal axis set in the virtual world corresponding to the horizontal axis. The object motion control means, when it is determined that the direction of the predetermined axis is within a predetermined range around the vertical axis, according to the posture and / or movement of the portable display device rotating around the vertical axis The operation may be controlled by rotating the object around the vertical axis set in the virtual world corresponding to the vertical axis. The first virtual camera motion control means rotates around the horizontal axis of the virtual world perpendicular to the line-of-sight direction according to the attitude and / or movement of the portable display device about the horizontal axis. The operation of the first virtual camera may be controlled by rotating around the vertical axis of the virtual world in accordance with the attitude and / or movement of the portable display device rotating around the direction axis.

  According to the above, in the real space, by moving the portable display device around the display screen in the vertical and horizontal directions, the direction of the object and the first virtual camera is directed vertically and horizontally. Operation control.

  The posture motion calculation means may include a real space reference direction setting means and an angle difference calculation means. The real space reference direction setting means sets the reference direction of the portable display device in the real space. Based on the data output from the portable display device, the angle difference calculation means calculates the difference between the left and right angle around the vertical direction in the real space between the current direction of the predetermined axis and the reference direction, and the current direction of the predetermined axis and the actual direction. The vertical angle difference around the horizontal direction in the space is calculated. The direction determination unit may include a left / right direction determination unit and a vertical direction determination unit. The left-right direction determination means determines whether the left-right angle difference is within a first range set around the vertical direction of the real space. The vertical direction determination means determines whether the vertical angle difference is within a second range set around the horizontal direction of the real space. In this case, when it is determined that the left-right angle difference is within the first range, the object motion control means rotates and / or moves the object in the left-right direction of the virtual world based on the left-right angle difference. The movement of the object may be controlled. When it is determined that the vertical angle difference is within the second range, the object motion control means rotates and / or moves the object in the vertical direction of the virtual world based on the vertical angle difference, and The operation may be controlled. The first virtual camera operation control means rotates and / or moves the first virtual camera in the left-right direction of the virtual world based on the left-right angle difference, and based on the vertical angle difference, The operation of the first virtual camera may be controlled by rotating and / or moving the first virtual camera in the left-right direction.

  Based on the above, it is possible to control the operation of the object and the first virtual camera based on the reference direction and the horizontal direction set in the real space and the virtual world, respectively.

  The display control means may output image data indicating the first image to the portable display device. The portable display device may include an image data acquisition unit and a display unit. The image data acquisition unit acquires image data output from the information processing apparatus. The display means displays the first image indicated by the image data acquired by the image data acquisition means.

  Based on the above, the portable display device can function as a so-called thin client terminal that does not execute information processing such as game processing.

  The information processing program may further cause the computer to function as compressed image generation means. The compressed image generation means generates compressed image data by compressing image data indicating the first image. In this case, the display control means may output the compressed image data generated by the compressed image generation means to the portable display device. The image data acquisition unit may acquire compressed image data output from the information processing apparatus. The portable display device may further include display image expansion means. The display image expansion means expands the compressed image data to obtain image data indicating the first image. In this case, the display means displays the first image indicated by the image data acquired by the image data acquisition means and expanded by the display image expansion means.

  According to the above, since the first image is compressed and output from the information processing device to the portable display device, the first image can be output at high speed, and the first image is generated after the first image is generated. Can be delayed until the image is displayed on the portable display device.

  The display control means may further display a second image indicating the virtual world viewed from the second virtual camera on another display device connected to the information processing device, separately from the first image.

  The other display device is a display device connected to the information processing device like the monitor 2 in an embodiment described later, and may be a separate body from the portable display device, and is generated by the information processing device. Any other image can be used as long as it can display the second image. For example, the another display device may be configured integrally with the information processing device (in one housing).

  According to the above, when processing based on an operation for changing or moving the attitude of the portable display device is performed, the processing result is not only transmitted to the portable display device but also to another display device connected to the information processing device. Can also be displayed. Therefore, for example, the user can use the images displayed on the two devices according to the operation status and preference, and can also view an image suitable for the user operation. Further, an image to be displayed on another display device connected to the information processing device can be used as an image for other people to view, for example, different from the user, and is also suitable when a plurality of people view the processing result. Can be a comfortable viewing environment.

  The information processing program may further cause the computer to function as compressed image generation means. The compressed image generation means generates compressed image data by compressing image data indicating the first image. In this case, the display control unit outputs the compressed image data generated by the compressed image generation unit to the portable display device, and separately from the compressed image data, the image data indicating the second image is not compressed. You may output to a display apparatus. The portable display device may include an image data acquisition unit, a display image expansion unit, and a display unit. The image data acquisition unit acquires the compressed image data output from the information processing apparatus. The display image expansion means expands the compressed image data to obtain image data indicating the first image. The display means displays the first image indicated by the image data acquired by the image data acquisition means and expanded by the display image expansion means.

  According to the above, since the first image is compressed and output from the information processing device to the portable display device, the first image can be output at high speed, and the first image is generated after the first image is generated. Can be delayed until the image is displayed on the portable display device.

  The information processing program may further cause the computer to function as second virtual camera operation control means. The second virtual camera motion control means sets the second virtual camera for generating an image of the virtual world based on the position of the object in the virtual world at a position different from the first virtual camera and the object is Set to be included in two images.

  According to the above, the same virtual world is displayed not only on the portable display device but also on another display device, and images of virtual worlds with different viewpoints are displayed. Therefore, the user can use the images displayed on the two devices according to the operation situation and preference when operating.

  The second virtual camera motion control means may set the second virtual camera at a position farther from the object than the distance from the object to the first virtual camera. The display control means displays a range wider than the range of the virtual world indicated by the first image on another display device as the second image.

  According to the above, the virtual world image having a wider display range than the virtual world image displayed on the portable display device is displayed on another display device connected to the information processing device. An image suitable for the user operation when presented to the user can be displayed on each display device.

  Further, the second virtual camera operation control means may set the second virtual camera at a position where the object is bird's-eye view in the virtual world. The display control means may display an image in which a bird's eye view of an object placed in the virtual world is displayed on another display device as a second image.

  According to the above, an image of the virtual world corresponding to the attitude and / or movement of the portable display device is displayed on the portable display device, and the virtual world is displayed on another display device connected to the information processing device. Since the bird's-eye view image is displayed, for example, when presenting the state of the virtual world to the user, an image suitable for the user operation can be displayed on each display device.

  The portable display device may include at least one of a gyro sensor and an acceleration sensor. The direction determination means may determine whether the direction of the predetermined axis is within a predetermined range based on data output from at least one of the gyro sensor and the acceleration sensor.

  According to the above, by using the data indicating the angular velocity generated in the portable display device output from the gyro sensor and / or the data indicating the acceleration generated in the portable display device output from the acceleration sensor, The attitude and movement of the portable display device can be accurately calculated.

  In addition, the present invention may be implemented in the form of an information processing apparatus and an information processing system including the above-described units and an information processing method including operations performed by the above units.

  According to the present invention, by setting at least one predetermined range, the user can perform various operations by the posture and movement of one device.

1 is an external view showing an example of a game system 1 according to an embodiment of the present invention. Functional block diagram showing an example of the game apparatus body 5 of FIG. The figure which shows an example of an external appearance structure of the terminal device 6 of FIG. The figure which shows an example of a mode that the user hold | gripped the terminal device 6. The block diagram which shows an example of an internal structure of the terminal device 6 of FIG. The perspective view which shows an example of the external appearance of the board type controller 9 of FIG. The figure which shows an example of the AA sectional drawing of the board-type controller 9 shown in FIG. 6, and an example by which the part of the corner where the load sensor 94 is arrange | positioned is enlargedly displayed. FIG. 6 is a block diagram showing an example of an electrical configuration of the board type controller 9 of FIG. The figure which shows an example of the mode of the user who operates using the terminal device 6 and the board type controller 9 The figure which shows an example of the image displayed on LCD61 of the terminal device 6 The figure which shows an example of the image displayed on the monitor 2 The figure which shows an example which rotated the terminal device 6 right and left, and an example of the image displayed on LCD61 The figure for demonstrating an example of the relationship between the terminal device depth direction projected on the horizontal surface of real space, and the operation instruction direction projected on the horizontal surface of the virtual world, and the player object Po controlled to the direction based on the said operation instruction direction The figure for demonstrating an example with the operation instruction | indication direction at the time of rotating the terminal device 6 right and left, and the player object Po controlled to the direction based on the said operation instruction | indication direction The figure for demonstrating an example of a turret left-right operation range and a virtual camera left-right operation range The figure for demonstrating an example of a turret vertical operation range and a virtual camera vertical operation range The figure which shows an example of main data and a program memorize | stored in the main memory of the game device main body 5 of FIG. The flowchart which shows an example of the process performed in the game device main body 5 of FIG. A subroutine showing an example of the game control process in step 44 in FIG. A subroutine showing an example of the player object setting process in step 83 in FIG. Subroutine showing an example of the operation instruction direction calculation process in step 121 in FIG. The figure for demonstrating an example of the movement vectors Vw1-Vw15 set for every injection | emission object W1-W15 which moves the virtual world.

  With reference to FIG. 1, an information processing apparatus that executes an information processing program according to an embodiment of the present invention and an information processing system including the information processing apparatus will be described. Hereinafter, for the sake of specific explanation, a stationary game apparatus body 5 will be used as an example of the information processing apparatus, and a game system including the game apparatus body 5 will be described. FIG. 1 is an external view showing an example of a game system 1 including a stationary game apparatus 3. FIG. 2 is a block diagram illustrating an example of the game apparatus body 5. Hereinafter, the game system 1 will be described.

  In FIG. 1, a game system 1 includes a home television receiver (hereinafter referred to as a monitor) 2 which is an example of display means, and a stationary game apparatus 3 connected to the monitor 2 via a connection cord. Composed. The monitor 2 includes a speaker 2a for outputting the audio signal output from the game apparatus 3 as audio. In addition, the game apparatus 3 executes an optical disc 4 on which a program (for example, a game program) as an example of the information processing program of the present invention is recorded, and executes the program on the optical disc 4 to display and output a game screen on the monitor 2. A game apparatus body 5 equipped with a computer, a terminal device 6, a controller 7 for giving the game apparatus body 5 operation information necessary for operating an object or the like displayed on the display screen, and a board-type controller 9 including. The game system 1 executes a game process in the game apparatus body 5 based on a game operation using at least one of the terminal device 6, the controller 7, and the board type controller 9, and monitors a game image obtained by the game process 2 and / or the terminal device 6. Note that the game apparatus body 5, the terminal device 6, the controller 7, and the board-type controller 9 are connected so as to be capable of wireless communication by radio. For example, the wireless communication is executed in accordance with the Bluetooth (registered trademark) standard or the IEEE 802.11n standard, but may be executed in accordance with other standards such as infrared rays.

  An optical disc 4, which is an example of an information storage medium that can be used interchangeably with respect to the game device main body 5, is detachably inserted into the game device main body 5. The optical disc 4 stores an information processing program (typically a game program) to be executed in the game apparatus body 5. An insertion slot for the optical disc 4 is provided on the front surface of the game apparatus body 5. The game apparatus body 5 executes information processing by reading and executing an information processing program stored in the optical disc 4 inserted into the insertion slot.

  The monitor 2 is connected to the game apparatus body 5 via a connection cord. The monitor 2 displays a game image obtained by a game process executed in the game apparatus body 5. The monitor 2 has a speaker 2a, and the speaker 2a outputs game sound obtained as a result of the game processing. In other embodiments, the game apparatus body 5 and the stationary display apparatus may be integrated. Communication between the game apparatus body 5 and the monitor 2 may be wireless communication.

  The game apparatus body 5 is equipped with a flash memory 17 (see FIG. 2) that functions as a backup memory that stores data such as save data in a fixed manner. The game apparatus main body 5 displays the result as a game image on the monitor 2 and / or the terminal device 6 by executing a game program or the like stored on the optical disc 4. The game program or the like is not limited to the optical disc 4 and may be executed in advance recorded in the flash memory 17. Further, the game apparatus body 5 can reproduce the game state executed in the past by using the save data stored in the flash memory 17 and display the game image on the monitor 2 and / or the terminal device 6. . Then, the user of the game device 3 operates at least one of the terminal device 6, the controller 7, and the board type controller 9 while watching the game image displayed on the monitor 2 and / or the terminal device 6, thereby playing the game You can enjoy the progress.

  The controller 7 and the board-type controller 9 wirelessly transmit transmission data such as operation information to the game apparatus body 5 incorporating the controller communication module 19 by using, for example, Bluetooth technology. The controller 7 is an operation means for mainly selecting an option displayed on the display screen of the monitor 2. The controller 7 is provided with a housing large enough to be held with one hand, and a plurality of operation buttons (including a cross key and the like) exposed on the surface of the housing. Further, as will be apparent from the description below, the controller 7 includes an imaging information calculation unit that captures an image viewed from the controller 7. Then, as an example of the imaging target of the imaging information calculation unit, two LED modules (hereinafter referred to as markers) 8L and 8R are installed near the display screen of the monitor 2 (upper side of the screen in FIG. 1). Although details will be described later, the user (player) can perform a game operation to move the controller 7, and the marker 8 is used by the game apparatus body 5 to calculate the movement, position, posture, and the like of the controller 7. The marker 8 includes two markers 8L and 8R at both ends thereof. The marker 8R (same for the marker 8L) is specifically one or more infrared LEDs (Light Emitting Diodes), and outputs infrared light toward the front of the monitor 2. The marker 8 is connected to the game apparatus body 5, and the game apparatus body 5 can control lighting of each infrared LED included in the marker 8. The marker 8 is portable, and the user can install the marker 8 at a free position. Although FIG. 1 shows a mode in which the marker 8 is installed on the monitor 2, the position and orientation in which the marker 8 is installed are arbitrary. In addition, the controller 7 can receive transmission data wirelessly transmitted from the controller communication module 19 of the game apparatus body 5 by the communication unit, and can generate sound and vibration corresponding to the transmission data.

  In other embodiments, the controller 7 and / or the board-type controller 9 and the game apparatus body 5 may be connected by wire. In this embodiment, the game system 1 includes one controller 7 and one board-type controller 9, but the game apparatus body 5 can communicate with a plurality of controllers 7 and a plurality of board-type controllers 9. By using the number of controllers 7 and the board type controller 9 at the same time, it is possible to play a game with a plurality of people.

  The controller 7 has a housing formed by plastic molding, for example, and a plurality of operation portions (operation buttons) are provided in the housing. Then, the controller 7 transmits operation data indicating an input state to the operation unit (whether or not each operation button is pressed) to the game apparatus body 5.

  In addition, the controller 7 has an imaging information calculation unit that analyzes the image data captured by the imaging unit and determines a region having high luminance in the region, thereby calculating the position of the center of gravity, the size, and the like of the region. For example, the imaging information calculation unit includes imaging means fixed to the housing of the controller 7 and targets a marker that outputs infrared light such as the marker unit 65 and / or the marker 8 of the terminal device 6 as an imaging target. Then, the imaging information calculation unit calculates the position of the imaging target in the captured image captured by the imaging unit, and transmits marker coordinate data indicating the calculated position to the game apparatus body 5. Since this marker coordinate data changes corresponding to the direction (inclination angle) and position of the controller 7 itself, the game apparatus body 5 can calculate the direction and position of the controller 7 using this marker coordinate data.

  Further, the controller 7 includes an acceleration sensor and / or a gyro sensor. The acceleration sensor detects acceleration (including gravitational acceleration) generated in the controller 7 and transmits acceleration data indicating the detected acceleration to the game apparatus body 5. Since the acceleration detected by the acceleration sensor changes in accordance with the direction (tilt angle) and movement of the controller 7 itself, the game apparatus body 5 calculates the direction and movement of the controller 7 using the acquired acceleration data. Can do. The gyro sensor detects angular velocities around the three axes set in the controller 7 and transmits angular velocity data indicating the detected angular velocities to the game apparatus body 5. Since the angular velocity detected by the gyro sensor changes in accordance with the direction (tilt angle) and movement of the controller 7 itself, the game apparatus body 5 calculates the direction and movement of the controller 7 using the acquired angular velocity data. Can do. In this way, the user can perform a game operation by pressing an operation unit provided in the controller 7 and moving the controller 7 itself to change its position and posture (tilt).

  The controller 7 is provided with a speaker and a vibrator. The controller 7 processes the sound data transmitted from the game apparatus body 5 and outputs sound corresponding to the sound data from the speaker. Further, the controller 7 processes the vibration data transmitted from the game apparatus body 5 and activates the vibrator according to the vibration data to generate vibration. In the embodiment of the present invention to be described later, it is possible to play a game without using the controller 7. The detailed configuration of the board type controller 9 will be described later.

  The terminal device 6 is large enough to be gripped by the user, and the user can use the terminal device 6 by holding it in his hand or by placing the terminal device 6 in a free position. This is a portable device. Although the detailed configuration will be described later, the terminal device 6 includes an LCD (Liquid Crystal Display) 61 that is a display means, and input means (a touch panel 62, a gyro sensor 604, etc., which will be described later). The terminal device 6 and the game apparatus body 5 (terminal communication module 28 (see FIG. 2)) can communicate wirelessly (may be wired). The terminal device 6 receives data of an image (for example, a game image) generated by the game apparatus body 5 from the game apparatus body 5 and displays an image indicated by the data on the LCD 61. In this embodiment, an LCD is used as the display device. However, the terminal device 6 may have any other display device such as a display device using EL (Electro Luminescence). . In addition, the terminal device 6 transmits operation data representing the content of the operation performed on the own device to the game apparatus body 5 including the terminal communication module 28.

  Next, the internal configuration of the game apparatus body 5 will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of the configuration of the game apparatus body 5. The game apparatus body 5 includes a CPU (Central Processing Unit) 10, a system LSI (Large Scale Integration) 11, an external main memory 12, a ROM / RTC (Read Only Memory / Real Time Clock) 13, a disk drive 14, and an AV-IC. (Audio Video-Integrated Circuit) 15 and the like.

  The CPU 10 executes processing by executing a program stored on the optical disc 4 and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI 11 will be described later. The volatile external main memory 12 stores a program read from the optical disk 4, a program read from the flash memory 17, and various data, and stores a work area and a buffer of the CPU 10. Used as a region. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting up the game apparatus body 5 is incorporated, and a clock circuit (RTC) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to the internal main memory 35 or the external main memory 12 described later.

  The system LSI 11 includes an input / output processor (I / O processor) 31, a GPU (Graphics Processor Unit) 32, a DSP (Digital Signal Processor) 33, a VRAM (Video RAM) 34, and an internal main memory 35. Although not shown, these components 31 to 35 are connected to each other by an internal bus.

  The GPU 32 forms part of the drawing means and generates an image in accordance with a graphics command (drawing command) from the CPU 10. The VRAM 34 stores data (data such as polygon data and texture data) necessary for the GPU 32 to execute the graphics command. When an image is generated, the GPU 32 creates image data using data stored in the VRAM 34. In the present embodiment, the game apparatus body 5 may generate both a game image to be displayed on the monitor 2 and a game image to be displayed on the terminal device 6. Hereinafter, a game image displayed on the monitor 2 may be referred to as a “monitor game image”, and a game image displayed on the terminal device 6 may be referred to as a “terminal game image”.

  The DSP 33 functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 35 and the external main memory 12. In the present embodiment, both game sound output from the speaker 2a of the monitor 2 and game sound output from the speaker of the terminal device 6 may be generated for the game sound as well as the game image. Hereinafter, the game sound output from the monitor 2 may be referred to as “monitor game sound”, and the game sound output from the terminal device 6 may be referred to as “terminal game sound”.

  Of the images and sounds generated in the game apparatus body 5 as described above, the image data and sound data output to the monitor 2 are read out by the AV-IC 15. The AV-IC 15 outputs the read image data to the monitor 2 via the AV connector 16 and outputs the read audio data to the speaker 2 a built in the monitor 2. As a result, an image is displayed on the monitor 2 and a sound is output from the speaker 2a.

  Of the images and sounds generated in the game apparatus body 5, image data and sound data output to the terminal device 6 are transmitted to the terminal device 6 by the input / output processor 31 or the like. Data transmission to the terminal device 6 by the input / output processor 31 and the like will be described later.

  The input / output processor 31 transmits / receives data to / from components connected to the input / output processor 31 or downloads data from an external device. The input / output processor 31 is connected to the flash memory 17, network communication module 18, controller communication module 19, expansion connector 20, memory card connector 21, and codec LSI 27. An antenna 22 is connected to the network communication module 18. An antenna 23 is connected to the controller communication module 19. The codec LSI 27 is connected to a terminal communication module 28, and an antenna 29 is connected to the terminal communication module 28.

  The game apparatus body 5 can connect to a network such as the Internet and communicate with an external information processing apparatus (for example, another game apparatus or various servers). That is, the input / output processor 31 is connected to the network via the network communication module 18 and the antenna 22 and can communicate with an external information processing apparatus connected to the network. The input / output processor 31 periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. If the data is present, the data is sent via the network communication module 18 and the antenna 22. To the network. Further, the input / output processor 31 receives data transmitted from the external information processing apparatus or data downloaded from the download server via the network, the antenna 22 and the network communication module 18, and receives the received data in the flash memory 17. To remember. By executing the program, the CPU 10 reads out the data stored in the flash memory 17 and uses it in the program. In the flash memory 17, in addition to data transmitted and received between the game apparatus body 5 and the external information processing apparatus, save data (process result data or intermediate data) of a game played using the game apparatus body 5 is stored. It may be stored. The flash memory 17 may store a program such as a game program.

  Further, the game apparatus body 5 can receive operation data from the controller 7 and / or the board type controller 9. That is, the input / output processor 31 receives operation data and the like transmitted from the controller 7 and / or the board-type controller 9 via the antenna 23 and the controller communication module 19 and buffers the internal main memory 35 or the external main memory 12. Store in the area (temporary storage). As in the case of the external main memory 12, the internal main memory 35 may store a program read from the optical disc 4, a program read from the flash memory 17, or various data. The CPU 10 may be used as a work area or a buffer area.

  Further, the game apparatus body 5 can transmit and receive data such as images and sounds to and from the terminal device 6. When the game image (terminal game image) is transmitted to the terminal device 6, the input / output processor 31 outputs the game image data generated by the GPU 32 to the codec LSI 27. The codec LSI 27 performs predetermined compression processing on the image data from the input / output processor 31. The terminal communication module 28 performs wireless communication with the terminal device 6. Therefore, the image data compressed by the codec LSI 27 is transmitted to the terminal device 6 via the antenna 29 by the terminal communication module 28. In the present embodiment, the image data transmitted from the game apparatus body 5 to the terminal device 6 is used for the game. In the game, if the displayed image is delayed, the operability of the game is adversely affected. For this reason, it is preferable that the transmission of image data from the game apparatus body 5 to the terminal device 6 should be as delayless as possible. Therefore, in this embodiment, the codec LSI 27 is, for example, H.264. The image data is compressed using a highly efficient compression technique such as H.264 standard. Other compression techniques may be used, and when the communication speed is sufficient, the image data may be transmitted without compression. The terminal communication module 28 is a communication module that has received, for example, Wi-Fi authentication. For example, the terminal communication module 28 uses a MIMO (Multiple Input Multiple Output) technology adopted in the IEEE 802.11n standard, for example. Wireless communication may be performed at high speed, or another communication method may be used.

  In addition to the image data, the game apparatus body 5 transmits audio data to the terminal device 6. That is, the input / output processor 31 outputs the audio data generated by the DSP 33 to the terminal communication module 28 via the codec LSI 27. The codec LSI 27 performs compression processing on the audio data in the same manner as the image data. The compression method for the audio data may be any method, but a method with a high compression rate and less deterioration of the sound is preferable. In other embodiments, audio data may be transmitted without being compressed. The terminal communication module 28 transmits the compressed image data and audio data to the terminal device 6 via the antenna 29.

  Furthermore, the game apparatus body 5 transmits various control data to the terminal device 6 as needed in addition to the image data and the sound data. The control data is data representing a control instruction for the constituent elements included in the terminal device 6, for example, an instruction for controlling lighting of the marker unit (marker unit 65 shown in FIG. 5) or a camera (camera 66 shown in FIG. 5). Indicates an instruction to control imaging. The input / output processor 31 transmits control data to the terminal device 6 in accordance with an instruction from the CPU 10. With respect to this control data, the codec LSI 27 does not perform data compression processing in the present embodiment, but may perform compression processing in other embodiments. Note that the above-described data transmitted from the game apparatus body 5 to the terminal device 6 may or may not be encrypted as necessary.

  Further, the game apparatus body 5 can receive various data from the terminal device 6. Although details will be described later, in the present embodiment, the terminal device 6 transmits operation data, image data, and audio data. Each data transmitted from the terminal device 6 is received by the terminal communication module 28 via the antenna 29. Here, the image data and audio data from the terminal device 6 are subjected to the same compression processing as the image data and audio data from the game apparatus body 5 to the terminal device 6. Therefore, these image data and audio data are sent from the terminal communication module 28 to the codec LSI 27, subjected to expansion processing by the codec LSI 27, and output to the input / output processor 31. On the other hand, the operation data from the terminal device 6 has a smaller data amount than images and sounds, and therefore may not be subjected to compression processing. Further, encryption may or may not be performed as necessary. Therefore, the operation data is received by the terminal communication module 28 and then output to the input / output processor 31 via the codec LSI 27. The input / output processor 31 stores (temporarily stores) the data received from the terminal device 6 in the buffer area of the internal main memory 35 or the external main memory 12.

  Further, the game apparatus body 5 can be connected to other devices and external storage media. That is, the expansion connector 20 and the memory card connector 21 are connected to the input / output processor 31. The expansion connector 20 is a connector for an interface such as USB or SCSI, and connects a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. By connecting, communication with the network can be performed instead of the network communication module 18. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 31 can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data or read data.

  The game apparatus body 5 (for example, the front main surface) is provided with a power button 24, a reset button 25, an insertion port for attaching / detaching the optical disk 4, an eject button 26 for removing the optical disk 4 from the insertion port of the game apparatus body 5, and the like. It has been. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, power is supplied to each component of the game apparatus body 5. When the reset button 25 is pressed, the system LSI 11 restarts the startup program of the game apparatus body 5. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

  In other embodiments, some of the components included in the game apparatus body 5 may be configured as expansion devices that are separate from the game apparatus body 5. At this time, the expansion device may be connected to the game apparatus body 5 via the expansion connector 20, for example. Specifically, the expansion device includes, for example, the codec LSI 27, the terminal communication module 28, and the antenna 29, and may be detachable from the expansion connector 20. According to this, the said game device main body can be set as the structure which can communicate with the terminal device 6 by connecting the said extended apparatus with respect to the game device main body which is not provided with said each component.

  Next, the configuration of the terminal device 6 will be described with reference to FIGS. FIG. 3 is a diagram illustrating an example of an external configuration of the terminal device 6. 3A is a front view of the terminal device 6, FIG. 3B is a top view, FIG. 3C is a right side view, and FIG. 3D is a bottom view. FIG. 4 is a diagram illustrating an example of a state in which the user holds the terminal device 6.

  As shown in FIG. 3, the terminal device 6 includes a housing 60 that is roughly a horizontally-long rectangular plate shape. The housing 60 is large enough to be gripped by the user. Accordingly, the user can move the terminal device 6 or change the arrangement position of the terminal device 6.

  The terminal device 6 has an LCD 61 on the surface of the housing 60. The LCD 61 is provided near the center of the surface of the housing 60. Therefore, the user can move the terminal device 6 while looking at the screen of the LCD 61 by holding the housings 60 on both sides of the LCD 61 as shown in FIG. FIG. 4 shows an example in which the user holds the terminal device 6 sideways by holding the housings 60 on both the left and right sides of the LCD 61 (that is, the terminal device 6 is held with the long side direction in the horizontal direction). However, it is also possible to hold the terminal device 6 vertically (that is, hold the terminal device 6 with the long side direction vertical).

  As illustrated in part (a) of FIG. 3, the terminal device 6 includes a touch panel 62 on the screen of the LCD 61 as an operation unit. In the present embodiment, the touch panel 62 is a resistive film type touch panel. However, the touch panel 62 is not limited to the resistive film type, and any type of touch panel such as a capacitance type can be used. The touch panel 62 may be a single touch method or a multi touch method. In this embodiment, a touch panel 62 having the same resolution (detection accuracy) as the resolution of the LCD 61 is used. However, the resolution of the touch panel 62 and the resolution of the LCD 61 are not necessarily matched. The input to the touch panel 62 is normally performed using a touch pen. However, the input to the touch panel 62 is not limited to the touch pen and can be performed by a user's finger. The housing 60 may be provided with a storage hole for storing a touch pen used for performing an operation on the touch panel 62. Thus, since the terminal device 6 includes the touch panel 62, the user can operate the touch panel 62 while moving the terminal device 6. That is, the user can directly input (by the touch panel 62) to the screen while moving the screen of the LCD 61.

  As shown in FIG. 3, the terminal device 6 includes two analog sticks 63A and 63B and a plurality of operation buttons 64A to 64L as operation means. Each analog stick 63A and 63B is a device that indicates a direction. Each analog stick 63A and 63B is configured such that the stick portion operated by the user's finger can slide or tilt in any direction (any angle in the up / down / left / right and diagonal directions) with respect to the surface of the housing 60. Has been. The left analog stick 63A is provided on the left side of the screen of the LCD 61, and the right analog stick 63B is provided on the right side of the screen of the LCD 61. Therefore, the user can input with the left or right hand to instruct the direction using the analog stick 63A or 63B. Further, as shown in FIG. 4, the analog sticks 63 </ b> A and 63 </ b> B are provided at positions where the user can operate while holding the left and right portions of the terminal device 6. Also, the analog sticks 63A and 63B can be easily operated.

  The operation buttons 64A to 64L are operation means for performing predetermined inputs. As shown below, each operation button 64A-64L is provided in the position which a user can operate in the state which hold | gripped the right-and-left part of the terminal device 6 (refer FIG. 4). Therefore, the user can easily operate these operation means even when the user moves the terminal device 6.

  As shown in part (a) of FIG. 3, among the operation buttons 64A to 64L, a cross button (direction input button) 64A and operation buttons 64B to 64H are provided on the surface of the housing 60. These operation buttons 64A to 64G are arranged at positions where they can be operated with the thumb of the user (see FIG. 4).

  The cross button 64A is provided on the left side of the LCD 61 and below the left analog stick 63A. That is, the cross button 64A is arranged at a position where it can be operated with the left hand of the user. The cross button 64 </ b> A has a cross shape and is a button capable of instructing the vertical and horizontal directions. The operation buttons 64B to 64D are provided below the LCD 61. These three operation buttons 64B to 64D are arranged at positions that can be operated by both the left and right hands. The four operation buttons 64E to 64H are provided on the right side of the LCD 61 and below the right analog stick 63B. That is, the four operation buttons 64E to 64H are arranged at positions that can be operated with the user's right hand. Further, the four operation buttons 64E to 64H are arranged so as to have a vertical / left / right positional relationship (relative to the center position of the four operation buttons 64E to 64H). Therefore, the terminal device 6 can also function the four operation buttons 64E to 64H as buttons for instructing the user in the up / down / left / right directions.

  Further, as shown in FIGS. 3A, 3B, and 3C, the first L button 64I and the first R button 64J are provided on an oblique upper portion (upper left portion and upper right portion) of the housing 60. Provided. Specifically, the first L button 64I is provided at the left end of the upper side surface of the plate-like housing 60, and protrudes from the upper and left side surfaces. The first R button 64J is provided at the right end of the upper side surface of the housing 60, and protrudes from the upper and right side surfaces. In this way, the first L button 64I is disposed at a position operable with the user's left index finger, and the first R button 64J is disposed at a position operable with the user's right hand index finger (see FIG. 4).

  As shown in FIGS. 3B and 3C, the second L button 64K and the second R button 64L are provided on the back surface of the plate-like housing 60 (that is, the surface opposite to the surface on which the LCD 61 is provided). Are protruded from the leg portions 68A and 68B. Specifically, the second L button 64K is provided slightly above the left side (left side when viewed from the front side) of the housing 60, and the second R button 64L is provided on the right side (from the front side) of the back side of the housing 60. It is provided slightly above the right side when viewed. In other words, the second L button 64K is provided at a position substantially opposite to the left analog stick 63A provided on the front surface, and the second R button 64L is provided at a position substantially opposite to the right analog stick 63B provided on the front surface. It is done. The second L button 64K is disposed at a position that can be operated by the user's left hand middle finger, and the second R button 64L is disposed at a position that can be operated by the user's right hand middle finger (see FIG. 4). Further, as shown in FIG. 3C, the second L button 64K and the second R button 64L are provided on the diagonally upward surfaces of the feet 68A and 68B, and have button surfaces that are diagonally upward. When the user grips the terminal device 6, it is considered that the middle finger moves in the vertical direction. Therefore, the user can easily press the second L button 64K and the second R button 64L by turning the button surface upward. Further, by providing the foot portions 68A and 68B on the back surface of the housing 60, the user can easily grip the housing 60, and the operation buttons are provided on the foot portions 68A and 68B, so that the housing 60 is held. It becomes easy to operate.

  3, since the second L button 64K and the second R button 64L are provided on the back surface, the terminal device 6 is placed with the screen of the LCD 61 (the surface of the housing 60) facing upward. The screen may not be completely horizontal. Therefore, in other embodiments, three or more legs may be formed on the back surface of the housing 60. According to this, since the terminal device 6 can be placed so that three or more feet are in contact with the floor surface when the screen of the LCD 61 is facing upward, the terminal device 6 is placed so that the screen is horizontal. can do. Alternatively, the terminal device 6 may be placed horizontally by attaching a detachable foot.

  Functions corresponding to the game program are appropriately assigned to the operation buttons 64A to 64L. For example, the cross button 64A and the operation buttons 64E to 64H may be used for a direction instruction operation or a selection operation, and the operation buttons 64B to 64E may be used for a determination operation or a cancel operation.

  Although not shown, the terminal device 6 has a power button for turning on / off the terminal device 6. Further, the terminal device 6 has an operation button for turning on / off the screen display of the LCD 61, an operation button for setting connection (pairing) with the game apparatus body 5, and a speaker (a speaker 607 shown in FIG. 5). ) May have an operation button for adjusting the volume.

  As shown in part (a) of FIG. 3, the terminal device 6 includes a marker part (marker part 65 shown in FIG. 5) including a marker 65 </ b> A and a marker 65 </ b> B on the surface of the housing 60. As an example, the marker unit 65 is provided on the upper side of the LCD 61. Like the markers 8L and 8R of the marker 8, the marker 65A and the marker 65B are each composed of one or more infrared LEDs. Similar to the marker 8 described above, the marker unit 65 is used by the game apparatus body 5 to calculate the movement of the controller 7 with respect to the marker unit 65. Further, the game apparatus body 5 can control lighting of each infrared LED included in the marker unit 65.

  The terminal device 6 includes a camera 66 that is an imaging unit. The camera 66 includes an imaging device (for example, a CCD image sensor or a CMOS image sensor) having a predetermined resolution, and a lens. For example, the camera 66 is provided on the surface of the housing 60. Therefore, the camera 66 can take an image of the face of the user who has the terminal device 6, and can take an image of the user who is playing the game while viewing the LCD 61 as an example.

  The terminal device 6 includes a microphone (a microphone 609 shown in FIG. 5) that is a voice input unit. A microphone hole 60 b is provided on the surface of the housing 60. The microphone 609 is provided inside the housing 60 that is inside the microphone hole 60b. The microphone 609 detects sounds around the terminal device 6 such as a user's voice.

  The terminal device 6 includes a speaker (speaker 607 shown in FIG. 5) that is an audio output means. As shown in part (d) of FIG. 3, a speaker hole 60 a is provided on the lower side surface of the housing 60. The output sound of the speaker 607 is output from the speaker hole 60a. In the present embodiment, the terminal device 6 includes two speakers, and a speaker hole 60a is provided at each position of the left speaker and the right speaker.

  The terminal device 6 includes an expansion connector 67 for connecting other devices to the terminal device 6. In the present embodiment, the extension connector 67 is provided on the lower side surface of the housing 60 as shown in FIG. Note that any other device connected to the expansion connector 67 may be used. For example, a controller (such as a gun-type controller) used for a specific game or an input device such as a keyboard may be used. If it is not necessary to connect another device, the expansion connector 67 may not be provided.

  In addition, regarding the terminal device 6 shown in FIG. 3, the shape of each operation button and the housing 60, the number of components, the installation position, and the like are merely examples, and other shapes, numbers, and installation positions are included. Also good.

  Next, the internal configuration of the terminal device 6 will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of the internal configuration of the terminal device 6. As shown in FIG. 5, in addition to the configuration shown in FIG. 3, the terminal device 6 includes a touch panel controller 601, a magnetic sensor 602, an acceleration sensor 603, a gyro sensor 604, a user interface controller (UI controller) 605, a codec LSI 606, a speaker. 607, a sound IC 608, a microphone 609, a wireless module 610, an antenna 611, an infrared communication module 612, a flash memory 613, a power supply IC 614, a battery 615, and a vibrator 619. These electronic components are mounted on an electronic circuit board and stored in the housing 60.

  The UI controller 605 is a circuit for controlling data input / output with respect to various input / output units. The UI controller 605 includes a touch panel controller 601, an analog stick 63 (analog sticks 63A and 63B), operation buttons 64 (operation buttons 64A to 64L), a marker unit 65, a magnetic sensor 602, an acceleration sensor 603, a gyro sensor 604, and a vibrator. 619. The UI controller 605 is connected to the codec LSI 606 and the expansion connector 67. Further, a power supply IC 614 is connected to the UI controller 605, and power is supplied to each unit via the UI controller 605. A built-in battery 615 is connected to the power supply IC 614 to supply power. The power supply IC 614 can be connected to a charger 616 or a cable that can acquire power from an external power source via a connector or the like. Power supply and charging can be performed. The terminal device 6 may be charged by attaching the terminal device 6 to a cradle having a charging function (not shown).

  The touch panel controller 601 is a circuit that is connected to the touch panel 62 and controls the touch panel 62. The touch panel controller 601 generates touch position data in a predetermined format based on a signal from the touch panel 62 and outputs it to the UI controller 605. The touch position data represents the coordinates of the position where the input is performed on the input surface of the touch panel 62. The touch panel controller 601 reads signals from the touch panel 62 and generates touch position data at a rate of once per predetermined time. Various control instructions for the touch panel 62 are output from the UI controller 605 to the touch panel controller 601.

  The analog stick 63 outputs to the UI controller 605 stick data representing the direction and amount in which the stick unit operated by the user's finger has slid (or tilted). In addition, the operation button 64 outputs operation button data representing the input status (whether or not it has been pressed) to each of the operation buttons 64 </ b> A to 64 </ b> L to the UI controller 605.

  The magnetic sensor 602 detects the direction by detecting the magnitude and direction of the magnetic field. The azimuth data indicating the detected azimuth is output to the UI controller 605. Also, a control instruction for the magnetic sensor 602 is output from the UI controller 605 to the magnetic sensor 602. Regarding the magnetic sensor 602, an MI (magnetic impedance) element, a flux gate sensor, a Hall element, a GMR (giant magnetoresistance) element, a TMR (tunnel magnetoresistance) element, an AMR (anisotropic magnetoresistance) element, or the like was used. Although there is a sensor, any sensor may be used as long as it can detect the direction. Strictly speaking, in a place where a magnetic field is generated other than the geomagnetism, the obtained azimuth data does not indicate the azimuth, but even in such a case, the terminal device 6 moves. Since the azimuth data changes, the change in the attitude of the terminal device 6 can be calculated.

  The acceleration sensor 603 is provided inside the housing 60 and detects the magnitude of linear acceleration along the direction of the three axes (the xyz axis shown in FIG. 3A). Specifically, the acceleration sensor 603 sets the long side direction of the housing 60 to the x axis (in the state in which the marker unit 65 is arranged on the upper side of the LCD 61, the right direction along the long side direction toward the display screen of the LCD 61). x-axis positive direction), the short side direction of the housing 60 is the y-axis (the marker unit 65 is arranged on the upper side of the LCD 61, and the upward direction along the short side direction toward the display screen of the LCD 61 is the y-axis. The magnitude of the linear acceleration of each axis is detected with the direction perpendicular to the surface of the housing 60 as the z-axis (the display screen depth direction of the LCD 61 is the z-axis positive direction). Acceleration data representing the detected acceleration is output to the UI controller 605. Further, a control instruction for the acceleration sensor 603 is output from the UI controller 605 to the acceleration sensor 603. The acceleration sensor 603 is, for example, a capacitive MEMS acceleration sensor in the present embodiment, but other types of acceleration sensors may be used in other embodiments. Further, the acceleration sensor 603 may be an acceleration sensor that detects a uniaxial or biaxial direction.

  The gyro sensor 604 is provided inside the housing 60 and detects angular velocities around the three axes of the x-axis, the y-axis, and the z-axis. Angular velocity data representing the detected angular velocity is output to the UI controller 605. Further, a control instruction for the gyro sensor 604 is output from the UI controller 605 to the gyro sensor 604. Any number and combination of gyro sensors may be used for detecting the triaxial angular velocity, and the gyro sensor 604 may be composed of a two-axis gyro sensor and a one-axis gyro sensor. Good. Further, the gyro sensor 604 may be a gyro sensor that detects a uniaxial or biaxial direction.

  The vibrator 619 is, for example, a vibration motor or a solenoid, and is connected to the UI controller 605. In response to a control instruction output from the UI controller 605 to the vibrator 619, the vibrator 619 operates to generate vibration in the terminal device 6. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the user's hand holding the terminal device 6 can be realized.

  The UI controller 605 outputs operation data including touch position data, stick data, operation button data, azimuth data, acceleration data, and angular velocity data received from each of the above components to the codec LSI 606. When another device is connected to the terminal device 6 via the expansion connector 67, the operation data may further include data representing an operation on the other device.

  The codec LSI 606 is a circuit that performs compression processing on data transmitted to the game apparatus body 5 and expansion processing on data transmitted from the game apparatus body 5. Connected to the codec LSI 606 are an LCD 61, a camera 66, a sound IC 608, a wireless module 610, a flash memory 613, and an infrared communication module 612. The codec LSI 606 includes a CPU 617 and an internal memory 618. For example, although the terminal device 6 is configured not to perform the game process itself, it is necessary to execute a minimum program for management and communication of the terminal device 6. As an example, when the power is turned on, a program stored in the flash memory 613 is read into the internal memory 618 and executed by the CPU 617, whereby the terminal device 6 is activated. A part of the internal memory 618 is used as a VRAM for the LCD 61.

  The camera 66 captures an image in accordance with an instruction from the game apparatus body 5 and outputs the captured image data to the codec LSI 606. Control instructions for the camera 66 such as an image capturing instruction are output from the codec LSI 606 to the camera 66. Note that the camera 66 can also capture moving images. That is, the camera 66 can repeatedly capture images and repeatedly output image data to the codec LSI 606.

  The sound IC 608 is a circuit that is connected to the speaker 607 and the microphone 609 and controls input / output of audio data to and from the speaker 607 and the microphone 609. That is, when audio data is received from the codec LSI 606, the sound IC 608 outputs an audio signal obtained by performing D / A conversion on the audio data to the speaker 607, and outputs sound from the speaker 607. The microphone 609 detects sound (such as user's voice) transmitted to the terminal device 6 and outputs a sound signal indicating the sound to the sound IC 608. The sound IC 608 performs A / D conversion on the audio signal from the microphone 609 and outputs audio data in a predetermined format to the codec LSI 606.

  The codec LSI 606 transmits image data from the camera 66, audio data from the microphone 609, and operation data from the UI controller 605 to the game apparatus body 5 via the wireless module 610 as terminal operation data. In the present embodiment, the codec LSI 606 performs the same compression processing as the codec LSI 27 on the image data and audio data. The compressed image data and audio data and the terminal operation data are output to the wireless module 610 as transmission data. An antenna 611 is connected to the wireless module 610, and the wireless module 610 transmits the transmission data to the game apparatus body 5 via the antenna 611. The wireless module 610 has the same function as the terminal communication module 28 of the game apparatus body 5. That is, the wireless module 610 has a function of connecting to a wireless LAN by a method compliant with, for example, the IEEE 802.11n standard. Data transmitted from the wireless module 610 may or may not be encrypted as necessary.

  As described above, the transmission data transmitted from the terminal device 6 to the game apparatus body 5 includes operation data (terminal operation data), image data, and audio data. When another device is connected to the terminal device 6 via the extension connector 67, the data received from the other device may be further included in the transmission data. In addition, the infrared communication module 612 performs infrared communication with other devices in accordance with, for example, the IRDA standard. The codec LSI 606 may transmit the data received by infrared communication to the game apparatus body 5 by including the data in the transmission data as necessary.

  Further, as described above, compressed image data and audio data are transmitted from the game apparatus body 5 to the terminal device 6. These data are received by the codec LSI 606 via the antenna 611 and the wireless module 610. The codec LSI 606 decompresses the received image data and audio data. The expanded image data is output to the LCD 61, and an image corresponding to the image data is displayed on the LCD 61. The expanded audio data is output to the sound IC 608, and a sound corresponding to the audio data is output from the speaker 607.

  When the control data is included in the data received from the game apparatus body 5, the codec LSI 606 and the UI controller 605 issue a control instruction to each unit according to the control data. As described above, the control data is for each component included in the terminal device 6 (in this embodiment, the camera 66, the touch panel controller 601, the marker unit 65, the sensors 602 to 604, the vibrator 619, and the infrared communication module 612). Data representing a control instruction. In the present embodiment, as the control instruction represented by the control data, an instruction to operate each of the above components or to stop (stop) the operation can be considered. In other words, components that are not used in the game may be paused in order to reduce power consumption. In this case, the transmission data transmitted from the terminal device 6 to the game device body 5 includes data from the paused components. Do not include. In addition, since the marker unit 65 is an infrared LED, the control may simply be ON / OFF of power supply.

  As described above, the terminal device 6 includes operation means such as the touch panel 62, the analog stick 63, and the operation buttons 64. However, in other embodiments, instead of these operation means or together with these operation means. The configuration may include other operation means.

  Further, the terminal device 6 includes a magnetic sensor 602, an acceleration sensor 603, and a gyro sensor 604 as sensors for calculating the movement of the terminal device 6 (including changes in position and orientation, or position and orientation). In other embodiments, the configuration may include only one or two of these sensors. Moreover, in other embodiment, it may replace with these sensors or the structure provided with another sensor with these sensors may be sufficient.

  Moreover, although the terminal device 6 is a structure provided with the camera 66 and the microphone 609, in other embodiment, it does not need to be provided with the camera 66 and the microphone 609, and may be provided only with either one. Good.

  Further, the terminal device 6 is configured to include a marker unit 65 as a configuration for calculating the positional relationship between the terminal device 6 and the controller 7 (the position and / or orientation of the terminal device 6 viewed from the controller 7). In other embodiments, the marker unit 65 may not be provided. Moreover, in other embodiment, the terminal device 6 may be provided with another means as a structure for calculating the said positional relationship. For example, in another embodiment, the controller 7 may include a marker unit, and the terminal device 6 may include an imaging element. Furthermore, in this case, the marker 8 may be configured to include an imaging element instead of the infrared LED.

  Next, the configuration of the board type controller 9 will be described with reference to FIGS. FIG. 6 is a perspective view showing an example of the appearance of the board-type controller 9 shown in FIG. As shown in FIG. 6, the board-type controller 9 includes a base 9a on which the user rides (a user's foot is placed), and at least four load sensors 94a to 94d for detecting a load applied to the base 9a. . Each of the load sensors 94a to 94d is included in the base 9a (see FIG. 7), and in FIG. 6, their arrangement positions are indicated by broken lines. In the following description, the four load sensors 94a to 94d may be described as the load sensor 94 when collectively described.

  The base 9a is formed in a substantially rectangular parallelepiped and has a substantially rectangular shape when viewed from above. For example, the base 9a has a rectangular short side set to about 30 cm and a long side set to about 50 cm. The upper surface of the base 9a is formed flat, and a pair of surfaces are set for the user to put on both feet and ride. Specifically, on the upper surface of the base 9a, a surface on which the user places the left foot (a region surrounded by a double line on the left back side in FIG. 6) and a surface on which the left foot is placed (the right front side in FIG. 6). (Regions surrounded by double lines) are set. And the side surfaces of the four corners of the base 9a are formed, for example, so as to partially protrude in a columnar shape.

  In the table 9a, the four load sensors 94a to 94d are arranged at a predetermined interval. In this embodiment, the four load sensors 94a to 94d are arranged at the peripheral edge of the base 9a, specifically at the four corners. The interval between the load sensors 94a to 94d is set to an appropriate value so that the intention of the game operation according to the user's load applied to the table 9a can be detected with higher accuracy.

  FIG. 7 shows an example of an AA sectional view of the board-type controller 9 shown in FIG. 6 and an example in which a corner portion where the load sensor 94 is arranged is enlarged. In FIG. 7, the base 9 a includes a support plate 90 and legs 92 for the user to ride. The load sensors 94a to 94d are respectively provided at locations where the legs 92 are disposed. In this embodiment, since the four legs 92 are provided at the four corners, the four load sensors 94a to 94d are respectively arranged at the four corners. The leg 92 is formed in a substantially bottomed cylindrical shape by plastic molding, for example, and the load sensor 94 is disposed on a spherical component 92 a provided on the bottom surface in the leg 92. The support plate 90 is supported by the legs 92 via the load sensor 94.

  The support plate 90 includes an upper layer plate 90a that forms the upper surface and the upper side surface, a lower layer plate 90b that forms the lower surface and the lower side surface, and a middle layer plate 90c provided between the upper layer plate 90a and the lower layer plate 90b. The upper layer plate 90a and the lower layer plate 90b are formed by plastic molding, for example, and are integrated by adhesion or the like. The middle layer plate 90c is formed by press molding of one metal plate, for example. The middle layer plate 90c is fixed on the four load sensors 94a to 94d. The upper layer plate 90a has lattice-like ribs (not shown) on the lower surface thereof, and is supported by the middle layer plate 90c via the ribs. Therefore, when the user gets on the base 9a, the load is transmitted to the four legs 92 via the support plate 90 and the load sensors 94a to 94d. As indicated by arrows in FIG. 7, the reaction from the floor caused by the input load is transmitted from the leg 92 to the upper layer plate 90a via the spherical component 92a, the load sensors 94a to 94d, and the middle layer plate 90c. To do.

  The load sensor 94 is, for example, a strain gauge (strain sensor) type load cell, and is a load converter that converts an input load into an electric signal. In the load sensor 94, in response to the load input, the strain generating body 95 is deformed to cause distortion. This strain is converted into a change in electrical resistance by a strain sensor 96 attached to the strain generating body 95 and further converted into a change in voltage. Therefore, the load sensor 94 can output a voltage signal indicating the input load from the output terminal.

  The load sensor 94 may be another type of load sensor such as a tuning fork vibration type, a string vibration type, a capacitance type, a piezoelectric type, a magnetostrictive type, or a gyro type.

  Returning to FIG. 6, the board type controller 9 is further provided with a power button 9c. When the power button 9c is operated (for example, the power button 9c is pressed) while the board type controller 9 is not activated, power is supplied to each circuit component (see FIG. 8) of the board type controller 9. However, the board-type controller 9 may be turned on in accordance with an instruction from the game apparatus body 5 to start supplying power to each circuit component. The board-type controller 9 may be automatically turned off when the state in which the user is not on continues for a certain time (for example, 30 seconds) or longer. Further, when the power button 9c is operated again while the board-type controller 9 is activated, the power supply may be turned off and the power supply to each circuit component may be stopped.

  FIG. 8 is a block diagram showing an example of the electrical configuration of the board-type controller 9. In FIG. 8, the flow of signals and data is indicated by solid arrows, and the supply of power is indicated by broken arrows.

  In FIG. 8, the board type controller 9 includes a microcomputer 100 for controlling the operation thereof. The microcomputer 100 includes a CPU, a ROM, a RAM, and the like (not shown), and the CPU controls the operation of the board type controller 9 according to a program stored in the ROM.

  The microcomputer 100 is connected to the power button 9c, the AD converter 102, the DC-DC converter 104, and the wireless module 106. Further, an antenna 106 a is connected to the wireless module 106. The four load sensors 94a to 94d are connected to the AD converter 102 via the amplifier 108, respectively.

  Further, the board-type controller 9 accommodates a battery 110 for supplying power to each circuit component. In another embodiment, an AC adapter may be connected to the board type controller 9 instead of the battery 110 so that commercial power is supplied to each circuit component. In this case, instead of the DC-DC converter 104, it is necessary to provide in the board type controller 9 a power supply circuit that converts alternating current into direct current and steps down and rectifies the direct current voltage. In this embodiment, power supply to the microcomputer 100 and the wireless module 106 is performed directly from the battery 110. That is, power from the battery 110 is always supplied to some components (CPU) and the wireless module 106 in the microcomputer 100, and whether the power button 9c is turned on or not is turned on from the game apparatus body 5. It is detected whether or not a command instructing is transmitted. On the other hand, power from the battery 110 is supplied to the load sensors 94 a to 94 d, the AD converter 102, and the amplifier 108 via the DC-DC converter 104. The DC-DC converter 104 converts the voltage value of the direct current from the battery 110 into a different voltage value and supplies the voltage value to the load sensors 94 a to 94 d, the AD converter 102, and the amplifier 108.

  The power supply to the load sensors 94a to 94d, the AD converter 102, and the amplifier 108 may be performed as necessary under the control of the DC-DC converter 104 by the microcomputer 100. In other words, the microcomputer 100 controls the DC-DC converter 104 to operate the load sensors 94a to 94d to detect the load, thereby controlling the load sensors 94a to 94d, the AD converter 102, and Power may be supplied to the amplifier 108.

  When power is supplied, the load sensors 94a to 94d each output a signal indicating the input load. These signals are amplified by the amplifiers 108, converted from analog signals to digital data by the AD converter 102, and input to the microcomputer 100. Identification information of the load sensors 94a to 94d is given to the detection values of the load sensors 94a to 94d so that the detection values of the load sensors 94a to 94d can be identified. In this way, the microcomputer 100 can acquire data indicating the load detection values of the four load sensors 94a to 94d at the same time.

  On the other hand, when it is determined that the load sensors 94a to 94d do not need to be operated, that is, when the load detection timing is not reached, the microcomputer 100 controls the DC-DC converter 104 to load sensors 94a to 94d and the AD converter 102. , And the power supply to the amplifier 108 is stopped. Thus, since the board type controller 9 can detect the load and the distance by operating the load sensors 94a to 94d only when necessary, the power consumption for load detection can be suppressed. .

  The case where load detection is necessary is typically when the game apparatus body 5 (FIG. 1) wants to acquire load data. For example, when the game apparatus body 5 requires load information, the game apparatus body 5 transmits an information acquisition command to the board-type controller 9. When the microcomputer 100 receives an information acquisition command from the game apparatus body 5, the microcomputer 100 controls the DC-DC converter 104 to supply power to the load sensors 94 a to 94 d and detect the load. On the other hand, when the microcomputer 100 has not received the information acquisition command from the game apparatus body 5, the microcomputer 100 controls the DC-DC converter 104 to stop the power supply to the load sensors 94 a to 94 d and the like.

  Note that the microcomputer 100 may control the DC-DC converter 104 by determining that the load detection timing comes at regular intervals. When such periodic load detection is performed, the period information may be given to the microcomputer 100 of the board-type controller 9 from the game apparatus body 5 at the start of the game, for example, or stored in the microcomputer 100 in advance. May be installed.

  Data indicating detection values from the load sensors 94a to 94d is transmitted from the microcomputer 100 to the game apparatus body 5 via the wireless module 106 and the antenna 106a as board operation data (input data) of the board type controller 9. For example, when a load is detected in response to a command from the game apparatus body 5, the microcomputer 100 receives the detected value data of the load sensors 94 a to 94 d from the AD converter 102 and receives the detected value data as a game. It transmits to the apparatus main body 5. Note that the microcomputer 100 may transmit the detection value data to the game apparatus body 5 at regular intervals. When the transmission cycle is longer than the load detection cycle, data including load values at a plurality of detection timings detected up to the transmission timing may be transmitted.

  The wireless module 106 can communicate with the same wireless standard (Bluetooth, wireless LAN, etc.) as the controller communication module 19 of the game apparatus body 5. Therefore, the CPU 10 of the game apparatus body 5 can transmit an information acquisition command to the board-type controller 9 via the controller communication module 19 or the like. Thus, the board type controller 9 can receive commands from the game apparatus body 5 via the wireless module 106 and the antenna 106a. Further, the board-type controller 9 can transmit board operation data including load detection values (or load calculation values) of the load sensors 94 a to 94 d to the game apparatus body 5.

  For example, in the case of a game that is executed using a mere total value of four load values detected by the four load sensors 94a to 94d, the user uses the four load sensors 94a to 94d of the board-type controller 9. Any position can be taken. That is, the user can play the game by riding in any direction on any position on the table 9a. However, depending on the type of game, it is necessary to perform processing by identifying which direction the load value detected by the four load sensors 94 is viewed from the user. That is, it is necessary to grasp the positional relationship between the four load sensors 94 of the board-type controller 9 and the user. In this case, for example, it may be assumed that the positional relationship between the four load sensors 94 and the user is defined in advance, and the user gets on the table 9a so as to obtain the predetermined positional relationship. Typically, a positional relationship in which there are two load sensors 94a to 94d on each of the front, rear, left and right of the user who rides on the center of the base 9a, that is, the user gets on the center of the base 9a of the board type controller 9 A positional relationship is defined. In this case, in this embodiment, the base 9a of the board-type controller 9 is formed in a rectangular shape in plan view, and the power button 9c is provided on one side (long side) of the rectangle. As a mark, the user decides in advance that the user should get on the base 9a so that the long side provided with the power button 9c is in a predetermined direction (front, back, left or right). . In this way, the load values detected by the load sensors 94a to 94d are load values in a predetermined direction (right front, left front, right rear, and left rear) as viewed from the user. Therefore, the board-type controller 9 and the game apparatus body 5 use the identification information of the load sensor 94 included in the load detection value data and the arrangement data indicating the position or direction of the load sensor 94 set (stored) with respect to the user. Based on this, it is possible to grasp which direction each load detection value corresponds to when viewed from the user. Thereby, for example, it is possible to grasp the intention of the game operation by the user, such as the front / rear / left / right operation direction or the distinction of the raised foot.

  Next, before describing specific processing performed by the game apparatus body 5, an outline of information processing performed by the game apparatus body 5 will be described with reference to the drawings. FIG. 9 is a diagram showing an example of a state of a user who operates using the terminal device 6 and the board type controller 9. FIG. 10A is a diagram illustrating an example of an image displayed on the LCD 61 of the terminal device 6. FIG. 10B is a diagram illustrating an example of an image displayed on the monitor 2. FIG. 11 is a diagram illustrating an example in which the terminal device 6 is rotated left and right (yaw) and an example of an image displayed on the LCD 61. FIG. 12 illustrates an example of the relationship between the depth direction of the terminal device projected onto the horizontal plane of the real space and the operation instruction direction projected onto the horizontal plane of the virtual world, and the player object Po controlled in the direction based on the operation instruction direction. It is a figure for doing. FIG. 13 is a diagram for explaining an example of the operation instruction direction when the terminal device 6 is rotated left and right (yaw) and the player object Po controlled in a direction based on the operation instruction direction. FIG. 14A is a diagram for describing an example of a turret left-right operation range and a virtual camera left-right operation range set in the left-right direction of the virtual world (real space). FIG. 14B is a diagram for explaining an example of a turret vertical operation range and a virtual camera vertical operation range set in the vertical direction of the virtual world (real space).

  As shown in FIG. 9, the user performs an operation using the terminal device 6 and the board type controller 9 to change the posture and direction of the terminal device 6 and an operation to change the load applied to the board type controller 9. Do. Specifically, the user puts one foot on the board-type controller 9 while holding the terminal device 6. Then, the user operates on the board type controller 9 while watching the image displayed on the monitor 2 or the image displayed on the LCD 61 of the terminal device 6 (for example, the user steps on one foot on the board type controller 9). By performing the operation, an operation to increase or decrease the weight applied to one foot on the board-type controller 9) and an operation to move the terminal device 6 itself are performed. The player object Po operates in the virtual world on the LCD 61 and the monitor 2 of the terminal device 6 in accordance with the direction and orientation of the terminal device 6 held by the user and the user operation on the board type controller 9 (for example, A game image in which the orientation of the virtual camera set in the virtual world is changed according to the direction of the player object Po is expressed.

  As shown in FIG. 10A, the LCD 61 of the terminal device 6 displays a state in which the player object Po is firing a water gun in the virtual world from the subjective viewpoint of the player object Po. In the example shown in FIG. 10A, a virtual world in a subjective view including the tip of the water gun operated by the player object Po (the tip of the turret) is displayed. A state of erupting is displayed. Also, a plurality of enemy objects Eo are arranged in the virtual world, and a state in which one of the enemy objects Eo is throwing an enemy bullet B toward the player object Po is also displayed. Thus, by displaying the virtual world based on the subjective viewpoint of the player object Po on the LCD 61, the user who holds the terminal device 6 and looks at the display on the LCD 61 can play the game from the same viewpoint as the player object Po. It can give a sense of presence in the virtual world.

  Further, as shown in FIG. 10B, the same virtual world as the virtual world displayed on the LCD 61 is also displayed on the monitor 2. In the example shown in FIG. 10B, the state in the virtual world as viewed from the upper rear and far behind the player object Po operating the water gun is displayed together with the player object Po. In this way, by displaying on the monitor 2 the situation in the virtual world as viewed from the upper rear of the player object Po, the user can easily grasp the situation around the player object Po and the positional relationship with the enemy object Eo. At the same time, other people watching the user playing the game can enjoy watching the attacking action of the player object Po.

  In the example shown in FIG. 10B, the state in the virtual world as viewed from the far rear upper side of the player object Po is displayed on the monitor 2, but the virtual world of another viewpoint may be displayed on the monitor 2. Absent. By displaying the same virtual world not only on the terminal device 6 but also on the monitor 2 and displaying images of the virtual worlds with different viewpoints, the images displayed on the two display devices when the user operates can be changed to It becomes possible to use properly according to preference. For example, with respect to the subjective viewpoint image of the player object Po displayed on the terminal device 6, a virtual camera (second virtual camera) for displaying the virtual world on the monitor 2 is set at a position away from the player object Po. If a range wider than the range of the virtual world indicated by the terminal device 6 is displayed on the monitor 2, the position of the virtual camera may not be far from the upper back of the player object Po. Specifically, a virtual camera for displaying the virtual world on the monitor 2 may be set at a position where the player object Po is bird's-eye view or bird's-eye view.

  As an example, when the user performs an action of putting weight on the board-type controller 9 with one foot, the player object Po performs an action of injecting water W from the water gun. At this time, the larger the load applied to the board type controller 9 by the user, the greater the amount of water W to be ejected and the faster the injection speed at which the water W is ejected. Moreover, when the variation | change_quantity to which the load applied on the board-type controller 9 becomes more than a predetermined threshold value, the injection object different from the water W (For example, the large ball which became a mass of water more than the water W), Ejected from a water gun. In addition, when the user performs an operation of weakening (for example, setting to 0) the weight applied to the board-type controller 9, the player object Po performs an operation of stopping the water W ejected from the water gun. In this way, the user moves the player object Po by ejecting the injection object by the operation on the board-type controller 9 (that is, whether or not the injection object is injected, the injection amount and the injection speed at which the injection object is injected, the type of the injection object) ), And an analog operation is possible using the board-type controller 9.

  For example, as described above, the board type controller 9 outputs a load detection value corresponding to the user operation on the board type controller 9. If the load detection value is used, the total load applied to the board-type controller 9 can be calculated. If the total load is used, it is possible to estimate whether the user is putting weight on the board type controller 9 or weakening the weight on the board type controller 9. If the total load is used, the magnitude of the load applied to the board type controller 9 by the user and the change amount of the load applied to the board type controller 9 can be calculated. In accordance with such a user action estimated on the board-type controller 9, an action in which the player object Po injects an injection object is set.

  Further, the direction in which the player object Po looks in the virtual world (that is, the line-of-sight direction of the virtual camera placed at the subjective viewpoint of the player object Po) changes according to the attitude (direction) of the terminal device 6 held by the user. At the same time, the direction in which the player object Po injects an injection object (for example, water W) (the direction of the water gun turret) changes. For example, the water gun injects an injection object in response to the user turning the back surface of the terminal device 6 up, down, left, and right, that is, the z-axis positive direction that is the depth direction of the LCD 61 (terminal device depth direction). The direction also changes vertically and horizontally in the virtual world. In addition, as the user turns the terminal device depth direction of the terminal device 6 up, down, left, and right, the line of sight of the virtual camera also changes up, down, left, and right, so that the game image of the LCD 61 displayed from the subjective viewpoint of the player object Po Also changes according to the direction change. For example, as shown in FIG. 11, when the user changes the orientation of the terminal device 6 so that the depth direction of the terminal device is directed to the right, the turret direction of the water gun in the virtual world changes to the right, and the virtual in the virtual world The camera gaze direction also changes to the right by the same angle. As apparent from comparison with FIG. 10A, the virtual world that scrolls to the left is displayed on the LCD 61 as a result, and the turret of the water gun is displayed in a fixed position on the LCD 61 in the same posture. When the water W is continuously ejected from the turret of the water gun, the LCD 61 displays a state in which the water W meanders and is ejected in the virtual world in accordance with the direction change of the turret.

  12 and 13 respectively show the posture of the terminal device 6 overlooking the real space, and the posture of the player object Po and the virtual camera overlooking the virtual world. As shown in FIG. 12, the virtual camera (first virtual camera) for generating a virtual world to be displayed on the LCD 61 is arranged at the subjective viewpoint of the player object Po operating the water gun in the virtual world. Then, an operation instruction direction in which the direction of the terminal device depth direction (z-axis positive direction) of the terminal device 6 in the real space is reflected in the virtual world is calculated, and the turret direction is set in a direction that matches the operation instruction direction. The Further, the attitude of the virtual camera is controlled so that the direction (that is, the turret direction) that coincides with the operation instruction direction becomes the line-of-sight direction (Z direction in the drawing). In this way, by matching the operation instruction direction in which the depth direction of the terminal device is reflected in the virtual world and the line-of-sight direction of the virtual camera, the direction in which the terminal device 6 is directed vertically and horizontally and the virtual camera in the virtual world Since the directions in the vertical and horizontal directions coincide with each other, an image as if looking into the virtual world using the LCD 61 as a viewing window can be displayed on the LCD 61.

  Consider a case where the user changes the orientation of the terminal device 6 so that the depth direction of the terminal device is directed to the right (A direction shown in FIG. 12). For example, as shown in FIG. 13, consider a case where the orientation of the terminal device 6 is changed so that the depth direction of the terminal device is directed to the A direction by an angle B. In this case, the operation instruction direction changes in the A direction by an angle B in the virtual world as well as the change in the depth direction of the terminal device in the real space. The turret direction of the water gun operated by the player object Po is also a change in the operation instruction direction centering on a predetermined position in the virtual world (for example, the subjective viewpoint of the player object Po and the position where the virtual camera is arranged). It changes by the angle B in the same A direction. The visual line direction of the virtual camera also changes by an angle B in the same A direction as the change of the operation instruction direction, with a predetermined position in the virtual world (for example, the viewpoint position of the virtual camera) as the center.

  Here, the range in which the user can change the turret direction of the water gun may be limited to a predetermined range in advance. For example, as shown in FIG. 14A, the turret left-right operation range in which the turret can change the direction to the left and right in the virtual world is a predetermined angle range centered on the virtual world reference direction (for example, 45 ° left and right of the virtual world reference direction). And a total range of 180 ° of 90 ° to the left and right of the virtual world reference direction). The virtual world reference direction is a direction indicating the front direction in the virtual world corresponding to the front of the user in the real space (real space reference direction), and is set according to a user operation as an example. Further, as shown in FIG. 14B, the turret up / down operation range in which the turret can change the vertical direction in the virtual world is a predetermined angle range (for example, the virtual world) based on the virtual world horizontal direction (real space horizontal direction). 45 ° in the elevation direction from the horizontal direction and 10 ° in the depression direction, a total range of 55 °). When the operation instruction direction is set outside the turret left / right operation range and / or outside the turret vertical operation range, the turret direction is within the turret left / right operation range and / or the turret vertical operation range closest to the operation instruction direction. Is set.

  On the other hand, the range in which the user can change the viewing direction of the virtual camera may not be limited. For example, as shown in FIG. 14A, the virtual camera left / right operation range in which the viewing direction of the virtual camera can be changed to the left and right can be set in all directions. Further, as shown in FIG. 14B, the virtual camera up / down operation range in which the visual line direction of the virtual camera can be changed up and down can be set in all directions. Therefore, when the operation instruction direction is set outside the turret left / right operation range and / or outside the turret vertical operation range, the turret direction is set within the turret left / right operation range and / or the turret vertical operation range. Is set to the same direction as the operation instruction direction. That is, when the user points the terminal device 6 in a direction in which the operation instruction direction that is outside the turret left / right operation range and / or outside the turret vertical operation range is calculated, the visual line direction of the virtual camera differs from the turret direction. The resulting virtual world is displayed on the LCD 61.

  For example, the terminal device 6 outputs acceleration data and angular velocity data corresponding to the movement and posture change of the terminal device 6. And if the acceleration which the said acceleration data shows is used, since the direction of the gravitational acceleration which is acting on the terminal device 6 is computable, what kind of attitude | position is the terminal device 6 on the basis of the perpendicular direction in real space? Can be estimated. In addition, if the angular velocity indicated by the angular velocity data and / or the acceleration indicated by the acceleration data is used, the angular velocity and / or dynamic acceleration acting on the terminal device 6 can be known. By using this, it is possible to estimate the posture change (direction change) from the initial posture of the terminal device 6 in the real space. In accordance with the estimated posture change (direction change) of the terminal device 6 as described above, the action of the player object Po (turret direction) and the posture of the virtual camera (line-of-sight direction) are set.

  In this way, the user can change the action of the player object Po and the attitude of the virtual camera depending on the direction and attitude of the terminal device 6 to be held. For example, the turret direction of the player object Po changes according to the direction and orientation of the terminal device 6 held by the user, and the direction (injection direction) when the injection object emitted from the turret changes. As an example, when the user directs the terminal device 6 in the vertical and horizontal directions (that is, pitch and yaw), the turret direction changes in conjunction with the direction change, and the injection direction also changes. Specifically, when the user changes the direction of the terminal device 6 itself so that the back surface of the terminal device 6 faces upward (that is, when pitched in the elevation direction), the turret direction is virtually within the turret up / down operation range. Change upward in the world. In addition, when the user changes the direction of the terminal device 6 itself so that the back surface of the terminal device 6 is directed leftward (that is, when yawed to the left), the turret direction is left in the virtual world within the turret left-right operation range. Change direction. Thus, by linking the attitude and direction of the terminal device 6 with the turret direction, the user can perform a realistic operation as if the user is moving the water gun (turret) using the terminal device 6. Further, as described above, the virtual camera is set to the subjective viewpoint of the player object Po operating the water gun, and the line-of-sight direction of the virtual camera changes according to the direction and orientation of the terminal device 6 to be held. In this way, by linking the posture and orientation of the terminal device 6 with the posture and orientation of the virtual camera, the user can feel as if the user has become a player object Po operating the water gun, and the terminal device. You can feel as if you are looking into the virtual world via the LCD 61 of the six. In the above-described game example, an operation of injecting an injection object from the water gun is performed in response to the user applying a load to the board-type controller 9. The contents of the injection object to be injected (for example, whether or not the injection object is injected, the injection amount and injection speed at which the injection object is injected, the type of the injection object, etc.) are determined according to the load applied to the board type controller 9. Analog operation is possible. That is, in addition to giving the user a sense of being in the virtual world by the image displayed on the LCD 61, the user further operates the water gun in real space through an analog operation using the board type controller 9. As a feeling of operation is given, the feeling of being in a virtual world is further increased.

  Next, details of processing performed in the game system 1 will be described. First, main data used in the processing will be described with reference to FIG. FIG. 15 is a diagram showing an example of main data and programs stored in the main memory of the game apparatus body 5.

  As shown in FIG. 15, the data storage area of the main memory includes board operation data Da, terminal operation data Db, load value data Dc, terminal device direction / posture data Dd, operation direction data De, turret direction data Df, injection object Data Dg, virtual camera data Dh, image data Di, and the like are stored. In addition to the data included in the information shown in FIG. 15, the main memory appropriately stores data necessary for game processing such as image data of various objects displayed on the monitor 2 and the LCD 61 and sound data used in the game. Remembered. Various program groups Pa constituting the information processing program are stored in the program storage area of the main memory.

  The board operation data Da stores a series of operation information (board operation data) transmitted as transmission data from the board type controller 9 and is updated to the latest board operation data. For example, the board operation data Da includes load data Da1 and the like. The load data Da1 is data indicating load detection values detected by the load sensors 94a to 94d of the board type controller 9, respectively.

  The terminal operation data Db stores a series of operation information (terminal operation data) transmitted from the terminal device 6 as transmission data, and is updated to the latest terminal operation data. For example, the terminal operation data Db includes acceleration data Db1, angular velocity data Db2, and the like. The acceleration data Db1 is data representing the acceleration (acceleration vector) detected by the acceleration sensor 603. For example, the acceleration data Db1 represents a three-dimensional acceleration having acceleration components in the xyz three-axis directions shown in FIG. 3 as components, but in another example, acceleration data in any one or more directions is expressed. It may represent. The angular velocity data Db2 is data representing the angular velocity detected by the gyro sensor 604. For example, the angular velocity data Db2 represents the angular velocities around the three axes xyz shown in FIG. 3, but in other examples, the angular velocity data Db2 may represent angular velocities around any one or more axes. .

  Note that the game apparatus body 5 includes data (for example, load detection values) included in operation information transmitted at predetermined intervals (for example, every 1/200 second) from the controller 7, the board-type controller 9, and the terminal device 6, respectively. , Data indicating acceleration and angular velocity) are sequentially received. For example, the received data is sequentially stored in the main memory by the input / output processor 31. In the processing flow described later, the CPU 10 reads the latest board operation data and terminal operation data from the main memory every frame (for example, every 1/60 seconds), and updates the board operation data Da and the terminal operation data Db, respectively. An example is used.

  Further, the operation information transmitted from the controller 7, the board type controller 9, and the terminal device 6 at each predetermined period may be temporarily stored in a buffer (not shown) provided in the controller communication module 19, the terminal communication module 28, or the like. In this case, the data stored in the buffer is read for each frame, and the board operation data Da (for example, load data Da1) and terminal operation data Db (for example, acceleration data Db1 and angular velocity data Db2) of the main memory are read out. ) Is updated and used. At this time, since the cycle for receiving the operation information and the processing cycle are different, the operation information received at a plurality of times is described in the buffer. Of the operation information received at a plurality of times, The process may be executed using only the latest operation information, the process may be executed using representative values (for example, average values) of operation information received at a plurality of time points, Processing for the number of pieces of operation information received at the time may be executed.

  The load value data Dc is a set of data indicating the load value detected by the board type controller 9. For example, the load value data Dc is a set of data indicating the total value (total load value) of the loads detected by the load sensors 94a to 94d. Specifically, the load value data Dc is an array of data indicating the total load value within a predetermined period calculated in time series, and data indicating the total load value in each element of the array is time-series. Stored.

  The terminal device direction / attitude data Dd includes real space reference direction data Dd1, current direction data Dd2, and the like. The real space reference direction data Dd1 is data indicating the reference direction (posture; real space reference direction) of the terminal device 6 in the real space. The current direction data Dd2 is data indicating the current direction and orientation of the terminal device 6 in the real space. In this embodiment, various corrections are performed and the respective direction data are set. For example, real space reference direction data Dd1 and current direction data Dd2 are calculated based on acceleration data Db1 and angular velocity data Db2 included in terminal operation data Db. A method for calculating the real space reference direction and the current direction will be described later.

  The operation direction data De includes virtual world reference direction data De1, operation instruction direction data De2, and the like. The virtual world reference direction data De1 is data indicating a virtual world reference direction set in the virtual world. The operation instruction direction data De2 is data indicating the operation instruction direction currently instructed by the user with respect to the virtual world. A method for calculating the virtual world reference direction and the operation instruction direction will be described later.

  Turret direction data Df includes turret left-right direction data Df1, turret up-down direction data Df2, and the like. The turret left-right direction data Df1 is data indicating the left-right direction in the virtual world of the water gun turret. The turret vertical direction data Df2 is data indicating the vertical direction in the virtual world of the water gun turret.

  The injection object data Dg includes object type data Dg1, injection amount data Dg2, injection vector data Dg3, position data Dg4, and the like for each injection object existing in the virtual world. The object type data Dg1 is data indicating the type (for example, water W or large ball) of the injection object injected from the turret. The injection amount data Dg2 is data indicating the injection amount per unit time in the injection object injected from the turret. The ejection vector data Dg3 is data indicating the moving speed and the moving direction in the virtual world for each of the ejected objects ejected per unit time. The position data Dg4 is data indicating the position in the virtual world for each ejected object ejected per unit time.

  The virtual camera data Dh is data related to the virtual camera set in the virtual world. For example, the virtual camera data Dh includes data related to a first virtual camera for generating a game image to be displayed on the LCD 61 of the terminal device 6 and data related to a second virtual camera for generating a game image to be displayed on the monitor 2. including.

  The image data Di includes player object data Di1, emission object image data Di2, background image data Di3, and the like. The player object data Di1 is data for arranging a player object Po that operates a water gun in the virtual world and generating a game image. The ejected object image data Di2 is data for generating a game image by arranging an ejected object in the virtual world. The background image data Di3 is data for arranging a background in the virtual world and generating a game image.

  Next, details of processing performed in the game apparatus body 5 will be described with reference to FIGS. FIG. 16 is a flowchart showing an example of processing executed in the game apparatus body 5. FIG. 17 is a subroutine showing an example of the game control process of step 44 in FIG. FIG. 18 is a subroutine showing an example of the player object setting process in step 83 in FIG. FIG. 19 is a subroutine showing an example of the operation instruction direction calculation process in step 121 in FIG. FIG. 20 is a diagram for explaining an example of movement vectors Vw1 to Vw15 set for each of the emission objects W1 to W15 moving in the virtual world. Here, in the flowcharts shown in FIG. 16 to FIG. 19, the processing in which the player object Po is operated and displayed according to the user's operation using the terminal device 6 and the board type controller 9 is mainly included in the processing. Detailed explanations of other processes not described and directly related to the present invention will be omitted. 16 to 19, each step executed by the CPU 10 is abbreviated as “S”.

  When the power of the game apparatus body 5 is turned on, the CPU 10 of the game apparatus body 5 executes a startup program stored in the ROM / RTC 13, thereby initializing each unit such as the main memory. Then, the information processing program stored in the optical disc 4 is read into the main memory, and the CPU 10 starts executing the program. The flowchart shown in FIGS. 16-19 is a flowchart which shows the process performed after the above process is completed.

  In FIG. 16, the CPU 10 executes initial processing (step 40), and proceeds to the next step. For example, in the initial processing in step 40, the CPU 10 constructs a virtual world, places the player object Po and the virtual camera (first virtual camera and second virtual camera) in the virtual world at predetermined positions, Are set at initial positions, and initial values of various parameters used in the game process are set.

  Next, the CPU 10 sets a reference direction based on the data transmitted from the terminal device 6 (step 41), and proceeds to the next step. Hereinafter, an example in which the CPU 10 sets the reference direction will be described.

  The terminal device 6 repeatedly transmits data as described above to the game apparatus body 5. In the game apparatus body 5, the terminal communication module 28 sequentially receives the data, and the input / output processor 31 sequentially stores the terminal operation data, camera image data, and microphone sound data in the main memory. In step 41, the CPU 10 reads the latest terminal operation data from the main memory and updates the acceleration data Db1 and the angular velocity data Db2.

  Next, the CPU 10 calculates the direction and orientation of the terminal device 6 in real space. For example, the CPU 10 calculates the current direction and posture of the terminal device 6 as a reference direction (initial posture) in the real space based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2, and the calculated terminal device The real space reference direction data Dd1 is updated using the data indicating the six reference directions. For example, the CPU 10 can calculate the amount of rotation (direction change amount) in the real space per unit time of the terminal device 6 using the angular velocity indicated by the angular velocity data Db2. Further, in a state where the terminal device 6 is almost stationary in a real space (static state), the acceleration applied to the terminal device 6 is a gravitational acceleration, and therefore the terminal device 6 is accelerated by the acceleration indicated by the acceleration data Db1. The direction of gravity to be applied (that is, the attitude of the terminal device 6 with respect to the vertical direction in the real space) can be calculated. Therefore, the CPU 10 can calculate the initial posture of the terminal device 6 based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2. In the following description, when step 41 is executed, the actual direction is determined from the direction in which the back surface of the terminal device 6 faces in real space (the z-axis positive direction shown in FIG. 3 and the terminal device depth direction). Set the spatial reference direction.

  The initial posture of the terminal device 6 may be calculated based on the acceleration indicated by the acceleration data Db1, may be calculated based on the direction of magnetism detected by the magnetic sensor 602, or the terminal device 6 is specified. The specific posture at the time when the predetermined operation is performed may be used as the initial posture by causing the user to perform the predetermined operation in the state of the posture. When calculating the attitude of the terminal device 6 as an absolute attitude based on a predetermined direction in real space, it is necessary to calculate the initial attitude. The timing for setting the initial posture, that is, the timing for executing the step 41 may be automatically performed at the start of the game, or the user can use the terminal device 6 to perform a predetermined operation (for example, a predetermined operation button). The operation may be performed in response to the operation of pressing 64).

  In step 41, the real space reference direction is converted into the model coordinate system in the virtual world, and the virtual world reference direction data De1 is updated with the converted direction as the reference direction in the virtual world.

  In the reference direction setting process in step 41, the reference direction is set after various corrections are made to the attitude and direction of the terminal device 6, and the real space reference direction data Dd1 is obtained using the corrected reference direction. Updated. Further, the corrected real space reference direction is converted into the model coordinate system in the virtual world, and the virtual world reference direction data De1 is updated with the converted direction as the reference direction in the virtual world. A method for correcting the attitude and direction of the terminal device 6 will be described later.

  Following step 41, the process of step 42 is executed. Thereafter, a processing loop composed of a series of processes from step 42 to step 51 is repeatedly executed at a rate of once per predetermined time (one frame time).

  In step 42, the CPU 10 acquires board operation data transmitted from the board type controller 9, and proceeds to the next step. Here, the board-type controller 9 repeatedly transmits board operation data to the game apparatus body 5. Therefore, in the game apparatus body 5, the controller communication module 19 sequentially receives the board operation data, and the received board operation data is sequentially stored in the main memory by the input / output processor 31. The board operation data transmission interval of the board-type controller 9 is preferably shorter than the game processing time (one frame time), for example, 1/200 second. In step 42, the CPU 10 reads the latest board operation data from the main memory and updates the load data Da1. The board operation data includes data indicating identification information of the load sensors 94a to 94d and data indicating load detection values detected by the load sensors 94a to 94d, respectively. Is used to update the load data Da1.

  Next, the CPU 10 acquires various data transmitted from the terminal device 6 (step 43), and proceeds to the next step. Here, the terminal device 6 repeatedly transmits the data to the game apparatus body 5. Therefore, in the game apparatus body 5, the terminal communication module 28 sequentially receives the data, and the camera image data and the microphone sound data are sequentially subjected to expansion processing by the codec LSI 27. Then, the input / output processor 31 sequentially stores terminal operation data, camera image data, and microphone sound data in the main memory. In step 43, the CPU 10 reads the latest terminal operation data from the main memory and updates the acceleration data Db1 and the angular velocity data Db2.

  Next, the CPU 10 performs a game control process (step 44), and proceeds to the next step. The game control process is a process of executing a game by executing a process of operating a player object Po or a virtual camera in the virtual world according to a game operation by a user. In the game example, the user can play various games using the terminal device 6 and the board type controller 9. Hereinafter, the game control process in step 44 will be described with reference to FIG.

  In FIG. 17, the CPU 10 calculates the load value (step 81) and proceeds to the next step. For example, the CPU 10 calculates the total load value by adding the load detection values indicated by the load data Da1, and updates the latest data in the time series data array in the load value data Dc using the data indicating the total load value. To do. Specifically, since the load data Da1 indicates the latest load detection values detected by the load sensors 94a to 94d, the total load value is calculated by summing the load detection values. The total load value calculated in this way changes according to the user operation on the board-type controller 9 and weight shift (posture). As an example, when the user performs an operation of applying a load on the board type controller 9, the total load value increases according to the applied load.

  Next, the CPU 10 calculates the direction change and posture of the terminal device 6 (step 82), and proceeds to the next step. For example, the CPU 10 calculates the xyz-axis direction in the real space of the terminal device 6 based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2, and sets the calculated xyz-axis direction as the current direction. The current direction data Dd2 is updated using the data indicating the direction.

  Here, the CPU 10 can calculate the amount of rotation (direction change amount) in the real space per unit time of the terminal device 6 using the angular velocity indicated by the angular velocity data Db2. Further, in a state where the terminal device 6 is almost stationary in a real space (static state), the acceleration applied to the terminal device 6 is a gravitational acceleration. The direction of gravity to be applied (that is, the attitude of the terminal device 6 with respect to the vertical direction in the real space and the xyz-axis direction with respect to the vertical direction) can be calculated. Therefore, the CPU 10 can calculate the direction change and the posture of the terminal device 6 based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2.

  In the present embodiment, the direction change and orientation of the terminal device 6 are calculated based on the data indicating the acceleration and angular velocity detected by the terminal device 6, but in other embodiments, any one of the data or You may calculate the direction change and attitude | position of the terminal device 6 using three or more data. For example, the magnetic sensor 602 provided in the terminal device 6 detects the magnetism applied to the terminal device 6, and has a predetermined orientation (that is, a predetermined direction) with respect to the terminal device 6 from the direction of geomagnetism applied to the terminal device 6. The attitude of the terminal device 6 with respect to the azimuth can be calculated. Note that the amount of rotation of the terminal device 6 can be calculated even when a magnetic field other than geomagnetism is generated in the real space where the terminal device 6 is disposed. Therefore, the CPU 10 can calculate the direction change and the posture of the terminal device 6 by using at least one of data indicating acceleration, angular velocity, and magnetism detected by the terminal device 6.

  The specific calculation method of the attitude of the terminal device 6 may be any method. For example, the acceleration data Db1 indicates the attitude of the terminal device 6 calculated based on the angular velocity indicated by the angular velocity data Db2. A method of correcting using the indicated acceleration and the direction of magnetism detected by the magnetic sensor 602 is conceivable.

  Specifically, the CPU 10 first calculates the attitude of the terminal device 6 based on the angular velocity indicated by the angular velocity data Db2. Any method may be used to calculate the attitude of the terminal device 6 from the angular velocity, but as an example, the previous attitude (direction of the xyz axis calculated last time) and the current angular speed (in the current processing loop). And the angular velocity obtained in step 42). The CPU 10 calculates a new xyz-axis direction by rotating the previous xyz-axis direction by a unit time around each axis at the current angular velocity. The previous xyz-axis direction is represented by current direction data Dd2, and the current angular velocity is represented by angular velocity data Db2. Therefore, the CPU 10 reads the current direction data Dd2 and the angular velocity data Db2, and calculates the attitude of the terminal device 6 (new xyz axis direction). As described above, in step 41, the initial posture of the terminal device 6 is determined. Therefore, when calculating the attitude of the terminal device 6 from the angular velocity, the CPU 10 can calculate the current attitude of the terminal device 6 with reference to the initially calculated initial attitude of the terminal device 6.

  Next, CPU10 correct | amends the attitude | position (xyz-axis direction) of the terminal device 6 calculated based on angular velocity using the acceleration which acceleration data Db1 shows. Specifically, the CPU 10 calculates the attitude (xyz axis direction) of the terminal device 6 based on the acceleration indicated by the acceleration data Db1. Here, in a state where the terminal device 6 is substantially stationary, the acceleration applied to the terminal device 6 is a gravitational acceleration. Therefore, in this state, since the direction of gravity acceleration (gravity direction) can be calculated using the direction of acceleration indicated by the acceleration data Db1, the direction of the terminal device 6 with respect to the gravity direction (xyz with respect to the gravity direction). Axial direction) can be calculated.

  When the posture of the terminal device 6 based on the acceleration is calculated, the CPU 10 corrects the posture based on the angular velocity using the posture based on the acceleration. Specifically, the CPU 10 brings the attitude of the terminal device 6 calculated based on the angular velocity (in the xyz axis direction) closer to the attitude of the terminal device 6 calculated based on the acceleration (in the xyz axis direction) at a predetermined rate. Make corrections. The predetermined ratio may be a predetermined fixed value, or may be set according to an acceleration or the like indicated by the acceleration data Db1. In addition, regarding the attitude of the terminal device 6 calculated based on the acceleration, the attitude cannot be calculated for the rotation direction with the gravity direction as an axis, so the CPU 10 does not correct the rotation direction. Also good. When correcting the attitude of the terminal device 6 calculated based on the angular velocity based on the magnetic direction detected by the magnetic sensor 602, the CPU 10 determines the attitude of the terminal device 6 calculated based on the angular velocity. The terminal device 6 calculated based on the direction of magnetism may be approximated at a predetermined rate. Based on the above, the CPU 10 can accurately calculate the attitude of the terminal device 6.

  Next, the CPU 10 sets a player object Po (step 83), and proceeds to the next step. Hereinafter, the player object setting process in step 83 will be described with reference to FIG.

  In FIG. 18, the CPU 10 performs an operation instruction direction calculation process (step 121), and proceeds to the next step. Hereinafter, the operation instruction direction calculation process performed in step 121 will be described with reference to FIG.

  In FIG. 19, the CPU 10 corrects the vertical direction of the terminal device 6 (step 221), and proceeds to the next step. For example, the CPU 10 corrects the direction (posture) of the terminal device 6 so that a state of being directed downward by a predetermined angle (for example, 20 °) with respect to the horizontal direction of the real space is an instruction for the horizontal direction. Specifically, in step 82, the CPU 10 calculates the direction of the xyz axis in the real space of the terminal device 6 based on the acceleration indicated by the acceleration data Db1 and the angular velocity indicated by the angular velocity data Db2, and the calculated xyz. The current direction data Dd2 is updated with the axial direction as the current direction. In step 221, the CPU 10 uses the respective directions of the xyz axes indicated by the current direction data Dd2 so that the y-axis direction and the z-axis direction are directed upward by the predetermined angle with respect to the x-axis direction (that is, The y-axis direction and the z-axis direction are corrected by rotating the y-axis direction and the z-axis direction clockwise by a predetermined angle around the x-axis when viewed in the positive x-axis direction.

  Next, the CPU 10 corrects the inclination around the z axis (step 222), and proceeds to the next step. For example, the CPU 10 corrects the direction (attitude) of the terminal device 6 so that the x axis of the terminal device 6 is in the horizontal direction in the real space. Specifically, the CPU 10 forcibly corrects the x-axis direction in the horizontal direction of the real space by rotating the x-axis direction around the z-axis direction using the xyz-axis direction corrected in step 221 above. Then, a new z-axis direction is calculated based on the outer product of the corrected x-axis direction and the y-axis direction. Then, a new y-axis direction is calculated by the outer product of the newly calculated z-axis direction and the x-axis direction corrected in the horizontal direction, and the current direction data Dd2 is calculated using the newly calculated xyz-axis direction. Update.

  In the reference direction setting process in step 41, the xyz-axis direction of the terminal device 6 is corrected in the same manner as in steps 221 and 222, and the real-space reference is set with the corrected z-axis positive direction as the real-space reference direction. The direction data Dd1 is updated.

  Next, the CPU 10 calculates a horizontal angle difference between the real space reference direction and the current direction (step 223), and proceeds to the next step. Here, the horizontal angle difference is an angle difference obtained by projecting the angle difference between the real space reference direction in the real space (the z-axis positive direction at the time of initial setting) and the z-axis positive direction indicated by the current direction data Dd2 on the horizontal plane. And the angle at which the direction of the terminal device 6 (the direction in which the back surface of the terminal device 6 faces (the z-axis positive direction shown in FIG. 3)) changes from the initial posture of the terminal device 6 about the vertical direction of the real space. It is shown. For example, the CPU 10 calculates the horizontal angle difference using the real space reference direction indicated by the real space reference direction data Dd1 and the z-axis positive direction indicated by the current direction data Dd2.

  Next, the CPU 10 calculates the vertical angle difference between the horizontal direction of the real space and the current direction (step 224), and proceeds to the next step. For example, using the z-axis positive direction indicated by the current direction data Dd2, the CPU 10 calculates the angle difference between the horizontal direction in the real space and the z-axis positive direction as the vertical angle difference.

  Next, the CPU 10 calculates an operation instruction direction with respect to the virtual world reference direction according to the horizontal angle difference calculated in step 223 and the vertical angle difference calculated in step 224 (step 225). The process ends. For example, the CPU 10 uses the virtual world reference direction indicated by the virtual world reference direction data De1, and the angle difference generated when the virtual world reference direction and the operation instruction direction are projected onto the horizontal plane of the virtual world is the horizontal angle difference. And the same positional relationship (that is, if the z-axis positive direction is rotated counterclockwise with respect to the real space reference direction, the operation instruction direction is also rotated counterclockwise with respect to the virtual world reference direction). The operation instruction direction in the virtual world is calculated as follows. Further, the CPU 10 determines that the angle difference between the horizontal direction and the operation instruction direction in the virtual world is the above-described vertical angle difference and has the same positional relationship (that is, the z-axis positive direction is downward with respect to the horizontal direction of the real space). If so, the operation instruction direction in the virtual world is calculated so that the operation instruction direction is also downward with respect to the horizontal direction of the virtual world. Then, the CPU 10 updates the operation instruction direction data De2 using the calculated operation instruction direction.

  Returning to FIG. 18, after the operation instruction direction calculation process in step 121, the CPU 10 determines whether or not the operation instruction direction is within the turret left-right operation range (step 122). Then, when the operation instruction direction is within the turret left / right operation range, the CPU 10 proceeds to the next step 123. On the other hand, if the operation instruction direction is not within the turret left / right operation range, the CPU 10 proceeds to the next step 124. Here, as described with reference to FIG. 14A, the turret left / right operation range is a range in which the turret direction of the water gun can be changed to the left / right (horizontal direction) according to the operation instruction direction, and the virtual world reference direction is A predetermined angle range is set at the center. Then, in step 122, the CPU 10 uses the virtual world reference direction indicated by the virtual world reference direction data De1 and the operation indication direction indicated by the operation indication direction data De2 to set the virtual world reference direction and the operation indication direction to the virtual world. It is determined whether or not the angle difference that occurs when projected onto the horizontal plane is within the turret left-right operation range.

  In step 123, the CPU 10 sets the turret left-right direction and the first virtual camera left-right direction according to the operation instruction direction calculated in step 121, and proceeds to the next step 126. For example, the CPU 10 sets the direction in which the operation instruction direction indicated by the operation instruction direction data De2 is projected on the horizontal plane of the virtual world as it is to the left and right direction of the turret and the left and right direction of the first virtual camera. Data regarding the left-right direction of the first virtual camera in the turret left-right direction data Df1 and the virtual camera data Dh is updated using the left-right direction of the virtual camera.

  On the other hand, in step 124, the CPU 10 sets the turret left-right direction within the turret left-right operation range, and advances the processing to the next step. For example, the CPU 10 sets the turret left-right direction as the direction within the turret left-right operation range in the direction in which the operation instruction direction indicated by the operation instruction direction data De2 is projected onto the horizontal plane of the virtual world, and the set turret left-right direction Is used to update the turret left-right direction data Df1.

  Next, the CPU 10 sets the left-right direction of the first virtual camera according to the operation instruction direction calculated in step 121 (step 125), and proceeds to the next step 126. For example, the CPU 10 sets the direction in which the operation instruction direction indicated by the operation instruction direction data De2 is projected to the horizontal plane of the virtual world as it is in the left-right direction of the first virtual camera, and uses the set first virtual camera left-right direction for virtual Data regarding the left-right direction of the first virtual camera in the camera data Dh is updated.

  In step 126, the CPU 10 determines whether or not the operation instruction direction is within the turret vertical operation range. When the operation instruction direction is within the turret up / down operation range, the CPU 10 proceeds to the next step 127. On the other hand, if the operation instruction direction is not within the turret up / down operation range, the CPU 10 proceeds to the next step 128. Here, as described with reference to FIG. 14B, the turret vertical operation range is a range in which the turret direction of the water gun can be changed vertically (vertical direction) according to the operation instruction direction, and the horizontal direction of the virtual world is A predetermined angle range is set as a reference. Then, in step 126, the CPU 10 uses the operation instruction direction indicated by the operation instruction direction data De2 to determine whether the angle difference generated between the virtual world horizontal direction and the operation instruction direction is within the turret vertical operation range. Determine whether or not.

  In step 127, the CPU 10 sets the turret vertical direction and the first virtual camera vertical direction in accordance with the operation instruction direction calculated in step 121, and proceeds to the next step 130. For example, the CPU 10 sets the direction in which the operation instruction direction indicated by the operation instruction direction data De2 is projected onto the vertical plane of the virtual world as it is in the vertical direction of the turret and the vertical direction of the first virtual camera. The data regarding the vertical direction of the first virtual camera in the vertical direction data Df2 and the virtual camera data Dh is updated using the vertical direction of one virtual camera.

  On the other hand, in step 128, the CPU 10 sets the turret vertical direction within the range of the turret vertical operation, and proceeds to the next step. For example, the CPU 10 sets the direction within the turret up / down operation range closest to the direction in which the operation instruction direction indicated by the operation instruction direction data De2 is projected on the vertical plane of the virtual world as the turret up / down direction, and the set turret up / down The turret vertical direction data Df2 is updated using the direction.

  Next, the CPU 10 sets the first virtual camera vertical direction according to the operation instruction direction calculated in step 121 (step 129), and proceeds to the next step 130. For example, the CPU 10 sets the direction in which the operation instruction direction indicated by the operation instruction direction data De2 is projected on the vertical plane of the virtual world as the first virtual camera vertical direction, and uses the set first virtual camera vertical direction. Data regarding the vertical direction of the first virtual camera in the virtual camera data Dh is updated.

  In step 130, the CPU 10 determines whether or not there is an injection of the injection object. Then, if there is an injection of the injection object, the CPU 10 advances the process to the next step 131. On the other hand, if there is no injection of the injection object, the CPU 10 ends the processing by the subroutine. Here, the player object Po can inject an injection object such as water W using the operated water gun, and the board-type controller 9 applies a user's predetermined operation (for example, a load above a predetermined threshold). In accordance with the operation to be applied to the turret, the injection object is ejected from the turret in the set turret direction. And “injection” determined in step 130 means that the predetermined operation is being performed (the latest total load value indicated by the load value data Dc is equal to or greater than a predetermined threshold) and / or This shows a case where an injection object has been injected and is moving in the virtual world (when an injection vector whose size is not 0 is set in the injection vector data Dg3).

  In step 131, the CPU 10 determines whether or not the total load value is equal to or greater than a predetermined threshold value. Here, the threshold value used in step 131 is a predetermined value for determining whether or not the user is performing an injection operation using the board type controller 9, and is added to the board type controller 9. When the latest total load value is equal to or greater than the threshold value, it is determined that the injection operation is being performed. Then, the CPU 10 refers to the latest total load value indicated by the load value data Dc, and proceeds to the next step 132 when the total load value is equal to or greater than a predetermined threshold value. On the other hand, if the latest total load value is less than the predetermined threshold value, the CPU 10 advances the process to the next step 133.

  In step 132, the CPU 10 adds data related to the injection object (type of injection object, injection amount, and injection vector) based on the total load value and the turret direction, and proceeds to the next step 133. For example, when the above-described injection operation for injecting an injection object is performed, the CPU 10 sets the position of the injection object to be newly injected within the turret of the water gun and uses the turret direction of the water gun to indicate the injection vector of the injection object. Set to the direction. Further, the CPU 10 calculates the injection speed and the injection amount per unit time of a newly injected injection object according to the latest total load value indicated by the load value data Dc. Specifically, the CPU 10 sets the injection speed to be faster and the injection amount per unit time to be larger as the latest total load value is larger. Further, the CPU 10 refers to the history of the total load value indicated by the load value data Dc, and newly starts when the amount of change from the total load value calculated in the previous process to the latest total load value is less than a predetermined value. An injection object to be injected is set as a first injection object (for example, water W), and when the amount of change is equal to or greater than the predetermined value, an injection object to be newly injected is set to a second injection object (for example, more than water W). Set to a large ball of water. Then, the CPU 10 uses the object type data Dg1, the injection amount data Dg2, the injection vector data Dg3, and the position data Dg4 indicating the set type, injection amount, injection vector, and position of the set injection object as data relating to the new injection object. Is added to the injection object data Dg. Thus, by repeating the above step 132, data relating to a new injection object is added to the injection object data Dg.

  Note that step 132 is repeated for each processing cycle of the game apparatus body 5 when the latest total load value is equal to or greater than a predetermined threshold value. In this case, a new injection object is generated for each cycle. It will be. Therefore, the cycle in which the above step 132 is repeated may be appropriately set according to the timing at which a new ejection object is desired to be generated in the virtual world. In this case, the processing in step 131 is performed at each timing, and the processing in steps 131 and 132 is not performed at other timings.

  In step 133, the CPU 10 moves the injection object set in the injection object data Dg based on the injection vector, and ends the processing by the subroutine. For example, based on the injection vector set in the injection object data Dg, the CPU 10 moves the injection object in the virtual world to set the position of a new injection object, and uses the set position for the injection object The position data Dg4 is updated. Further, the CPU 10 corrects the injection vector of the injection object based on the environment (gravity, wind, influence from other objects, etc.) of the virtual world where the injection object is arranged, and uses the corrected injection vector. The injection vector data Dg3 of the injection object is updated. In addition, when the projecting object collides with another object due to the movement, the CPU 10 sets data indicating that the projecting object collides with the other object, and sets data regarding the ejected object (object type data Dg1, The injection amount data Dg2, the injection vector data Dg3, and the position data Dg4) are deleted from the injection object data Dg.

  In this way, by repeating the above step 132, new injection objects are repeatedly set in the turret of the water gun operated by the player object Po, and the turret direction is injected for each newly set injection object. An injection vector as a direction is set. Then, by repeating the above step 133, the position of the injection object set in the injection object data Dg is set by moving in the virtual world based on the set injection vector. For example, as shown in FIG. 20, the injection objects W1 to W15 have injection vectors Vw1 to Vw15 in which the turret direction at the time of injection is the vector direction, and the injection speed corresponding to the total load value at the time of injection is the magnitude of the vector. Based on the above, move in a connected state in the virtual world. Therefore, when the user performs an operation of changing the direction of the turret while performing an injection operation, an injection vector in a different direction is set for each of the injection objects injected from the turret. W15 will meander and move in a continuous state in the virtual world. Here, the position where the injection object reaches in the virtual world is determined in an analog manner based on the turret direction at the time of injection and the injection speed corresponding to the total load value at the time of injection. Therefore, it is difficult for the user who plays the game to predict the position where the injection object reaches. However, when the injection objects are continuously injected, the user predicts the arrival position of the injection object to be injected with reference to the arrival position of the injection object previously injected by moving the injection objects in succession. It becomes possible.

  Returning to FIG. 17, after the player object setting process in step 83, the CPU 10 sets parameters relating to the virtual camera (step 84), and ends the process of the subroutine. For example, the terminal game image and the monitor game image are respectively generated by a three-dimensional CG image obtained by arranging a virtual camera in the virtual world and calculating a game space viewed from the virtual camera. Specifically, the first virtual camera for generating the terminal game image is arranged at a position that is the subjective viewpoint of the player object Po in the virtual world. In the first virtual camera, the direction based on the left and right direction of the first virtual camera and the vertical direction of the first virtual camera set by the processing of Step 123, Step 125, Step 127, and Step 129 is the line-of-sight direction. The direction is set to be the horizontal direction of the virtual world. Further, the second virtual camera for generating the monitor game image is set in the same virtual world where the first virtual camera is set, and the player object Po arranged in the virtual world is viewed from a distance. It is fixedly set so that the situation in the virtual world is included. In this way, since the terminal game image and the monitor game image become virtual world game images viewed from different viewpoints, the virtual world game viewed from different viewpoints also on the LCD 61 and the monitor 2 on which they are displayed. An image will be displayed.

  Returning to FIG. 16, after the game control process in step 44, the CPU 10 and the GPU 32 generate a monitor game image to be displayed on the monitor 2 (step 45), and the process proceeds to the next step. For example, the CPU 10 and the GPU 32 read out each data representing the result of the game control process in step 44 from the main memory, read out data necessary for generating a monitor game image from the VRAM 34, generate a game image, and generate The monitored game image is stored in the VRAM 34. The monitor game image may be generated by any method as long as it represents the result of the game control process in step 44. For example, the monitor game image includes a player object that arranges the second virtual camera in the virtual world based on the parameters related to the second virtual camera indicated by the virtual camera data Dh, and operates the water gun in the virtual world based on the turret direction data Df. It is generated by a three-dimensional CG image obtained by arranging Po, arranging an injection object in the virtual world based on the injection object data Dg, and calculating the virtual world viewed from the second virtual camera. Specifically, the CPU 10 arranges the player object Po and the water gun in the virtual world so that the turret of the water gun operated by the player object Po faces the turret direction indicated by the turret direction data Df. Further, the CPU 10 determines the type and size of the injection object to be arranged based on the type and the injection amount of the object indicated by the injection object data Dg, and determines the determined injection object based on the position indicated by the injection object data Dg. Place each in a virtual world.

  Next, the CPU 10 and the GPU 32 generate a terminal game image to be displayed on the terminal device 6 (step 46), and proceed to the next step. For example, the CPU 10 and the GPU 32 read out each data representing the result of the game control process in step 44 from the main memory, read out data necessary for generating a terminal game image from the VRAM 34, generate a game image, and generate The monitored game image is stored in the VRAM 34. The terminal game image may be generated by any method as long as it represents the result of the game control process in step 44 as in the case of the monitor game image. Further, the terminal game image may be generated by the same method as the monitor game image or may be generated by a different method. For example, the game image for a terminal is a player object that arranges a first virtual camera in the virtual world based on parameters relating to the first virtual camera indicated by the virtual camera data Dh, and operates a water gun in the virtual world based on the turret direction data Df It is generated by a three-dimensional CG image obtained by arranging Po, arranging an injection object in the virtual world based on the emission object data Dg, and calculating the virtual world viewed from the first virtual camera.

  Next, the CPU 10 generates a monitor game sound to be output to the speaker 2a of the monitor 2 (step 47), and proceeds to the next step. For example, the CPU 10 causes the DSP 33 to generate a monitor game sound to be output from the speaker 2a in accordance with the result of the game control process in step 44. As an example, in the virtual world set in accordance with the result of the game control process in step 44 above, the CPU 10 sounds, motion sounds, and sound effects of each object assumed to be heard based on the position of the second virtual camera. The DSP 33 generates a monitor game sound in which BGM or the like to be output from the monitor 2 is added.

  Next, the CPU 10 generates a terminal game sound to be output to the speaker 607 of the terminal device 6 (step 48), and proceeds to the next step. For example, the CPU 10 causes the DSP 33 to generate terminal game sound to be output from the speaker 607 in accordance with the result of the game control process in step 44. As an example, in the virtual world set in accordance with the result of the game control process in step 44, the CPU 10 is assumed to be heard with reference to the position of the first virtual camera, the voice, action sound, and sound effect of each object. In addition, the DSP 33 generates a terminal game sound in which BGM or the like to be output from the terminal device 6 is added. The terminal game sound may be the same as or different from the monitor game sound. Further, the terminal game sound may be a part of the sound different from the monitor game sound (for example, sound effects are different but BGM is the same). If the monitor game sound and the terminal game sound are the same, the generation process of the terminal game sound may not be executed in step 48.

  Next, the CPU 10 outputs the monitor game image and the monitor game sound to the monitor 2 (step 49), and proceeds to the next step. For example, the CPU 10 sends the monitor game image data stored in the VRAM 34 and the monitor game sound data generated by the DSP 33 to the AV-IC 15. In response to this, the AV-IC 15 outputs data indicating the monitor game image and the monitor game sound to the monitor 2 via the AV connector 16. As a result, the monitor game image is displayed on the monitor 2 and the monitor game sound is output from the speaker 2a.

  Next, the CPU 10 transmits the terminal game image and the terminal game sound to the terminal device 6 (step 50), and proceeds to the next step. For example, the CPU 10 sends the terminal game image data stored in the VRAM 34 and the terminal game sound data generated by the DSP 33 to the codec LSI 27, and the codec LSI 27 performs a predetermined compression process. Further, the terminal game image data and the terminal game sound data subjected to the compression processing are transmitted to the terminal communication module 28 by the codec LSI 27 and transmitted to the terminal device 6 by the terminal communication module 28 via the antenna 29. The The terminal game image data and the terminal game sound data transmitted from the game apparatus body 5 are received by the wireless module 610 of the terminal apparatus 6 and subjected to predetermined decompression processing by the codec LSI 606. The terminal game image data that has undergone the decompression process is output to the LCD 61, and the data sound data of the terminal game sound that has undergone the decompression process is output to the sound IC 608. As a result, the terminal game image is displayed on the LCD 61 and the terminal game sound is output from the speaker 607.

  Next, the CPU 10 determines whether or not to end the game (step 51). As a condition for ending the game, for example, a condition that a game is over or a game is cleared is satisfied, or a user performs an operation to end the game. If the game is not finished, the CPU 10 returns to step 42 and repeats the process. If the game is finished, the process according to the flowchart is finished. Thereafter, the series of processing from step 42 to step 51 is repeatedly executed until it is determined in step 51 that the game is to be ended.

  As described above, according to the above-described processing, an image of the virtual world is displayed on the LCD 61 of the portable terminal device 6 that can be grasped and viewed by the user, and also according to the attitude and movement of the terminal device 6. Thus, an operation can be performed on the virtual world, and an operation using the board-type controller 9 can be performed in parallel with an operation using the terminal device 6. Here, the operation using the board-type controller 9 can be operated by the user's feet or operated by the user, and the attitude of the terminal device 6 constituted by a separate device can be used. It can be easily performed simultaneously with the operation of changing the movement.

  Further, according to the above-described processing, the operation using the terminal device 6 within the turret left-right operation range causes the sight line direction of the first virtual camera and the turret direction of the water gun to yaw to the left and right. Change left and right according to posture and movement. Further, the operation using the terminal device 6 within the turret up / down operation range also changes the line-of-sight direction of the first virtual camera and the turret direction of the water gun according to the attitude and movement of the terminal device 6 in the vertical direction. Let Therefore, the operation using the terminal device 6 within the turret left-right operation range and the turret vertical operation range is not only the line-of-sight direction of the virtual camera for generating the virtual world to be displayed on the terminal device 6 by the operation. Since the injection direction in which the injection object is injected into the virtual world can also be changed, the operation is suitable for adjusting the arrival position of the injection object and changing the display range displayed on the LCD 61. On the other hand, an operation using the terminal device 6 outside the turret left-right operation range or outside the turret vertical operation range is performed only in the direction of the line of sight of the first virtual camera. It changes according to the posture and movement to pitch. Therefore, the operation using the terminal device 6 outside the turret left / right operation range and outside the turret vertical operation range is a preferable operation for changing only the display range displayed on the LCD 61 while leaving the emission direction unchanged. Thus, by setting at least the turret left / right operation range and the turret up / down operation range, the user can perform various operations by the posture and movement of one device.

  In the above description, the range for determining the posture and movement direction of the terminal device 6 yawing to the left and right (turret left-right operation range) and the posture and movement of the terminal device 6 pitching up and down. Although the range for determining the direction (turret up / down operation range) is set, the range for determining the direction may be one, or may be three or more. For example, when the turret direction of the water gun changes only to the left and right of the virtual world, only the turret left and right operation range is set, and only the first virtual camera is always applied to the posture and movement in which the direction of the terminal device 6 is pitched up and down. You may change your posture. In addition, set a range where the turret direction of the water gun is changing relatively slowly (for example, a range adjacent to the left and right sides of the turret left / right operation range, and a range adjacent to the top and bottom sides of the turret vertical operation range). Thus, the turret direction may be controlled so that the speed at which the turret direction changes according to the direction of the terminal device 6 changes.

  In the above-described game example, control (position, direction, and orientation control) of the virtual camera (first virtual camera) for generating an image to be displayed on the LCD 61 based on the orientation of the terminal device 6 is performed. By such control, an image as if looking into the virtual world via the LCD 61 can be provided to the user, and the user can feel as if he is in the virtual world. Further, the operation using the attitude of the terminal device 6 includes a horizontal swing motion (yaw) around the vertical direction (for example, around the y-axis direction) and a vertical swing motion (pitch) around the horizontal direction (for example, around the x-axis direction). ), Which is suitable for controlling a virtual camera capable of moving in the virtual world in the same direction, and in the virtual world so that it has the same attitude as the terminal device 6 in the real space. By controlling the attitude of the virtual camera at, it is possible to provide an image that looks in the direction desired by the user in the virtual world. In addition to the rotation operation in the above two directions, the virtual camera is controlled to rotate around the line-of-sight direction according to the left-right rotational movement (roll) around the front-rear horizontal direction (for example, around the z-axis direction). Also good. By applying this control, a rotation operation in three directions becomes possible, and further, by controlling the posture of the virtual camera in the virtual world so as to be the same posture as the terminal device 6 in the real space, the user in the virtual world can Images that look in the desired direction can be provided.

  Further, in the above-described game example, the player object performs an action (for example, an injection action) in response to the user's action on the board type controller 9. In other words, the user is given a sense of being in the virtual world by the image displayed on the LCD 61, and is further given a sense of operation as if he / she became a player character in real space. The feeling of being in a virtual world will increase.

  In the above-described game example, the turret direction of the water gun operated by the player object Po displayed on the LCD 61 is controlled based on the attitude of the terminal device 6. By such control, it is possible to provide the user with an operating environment as if the terminal device 6 itself became a water gun, and to give the user a feeling that the user has become a virtual world player object Po. Further, the operation using the attitude of the terminal device 6 includes a horizontal swing motion (yaw) around the vertical direction (for example, around the y-axis direction) and a vertical swing motion (pitch) around the horizontal direction (for example, around the x-axis direction). ), And can be rotated in two directions, and is suitable for controlling the player object Po that can move in the virtual world. For example, in the case of the above-described game example, the left-right swing motion (yaw) around the vertical direction (y-axis direction) along the LCD 61 of the terminal device 6 is made to correspond to the left-right change (yaw) in the turret direction, and the left-right direction along the LCD 61 By making the vertical swing motion (pitch) around (x-axis direction) correspond to the vertical change (pitch) in the turret direction, it is possible to provide a shooting game that changes in the turret direction desired by the user in the virtual world.

  In the above-described game example, the action of the player object Po injecting the injection object according to the load value (total load value) applied to the board-type controller 9 (injection presence / absence of the injection object, the injection amount of the injection object, and the like) The injection speed and the type of injection object) are controlled. That is, the injection process according to the analog operation with respect to the board type controller 9 is possible. Therefore, the user operates the operation of one player object Po by a plurality of devices (terminal device 6 and board type controller 9), and an operation that has never been possible is possible, and the player object Po. An analog operation for this operation is also possible.

  In the above-described game example, the depth direction of the virtual world displayed on the LCD 61 of the terminal device 6 can be the turret direction of the water gun operated by the player object Po. Therefore, the user can set the turret direction according to the attitude of the terminal device 6, and the virtual world with the turret direction as the depth direction is displayed on the LCD 61. Therefore, the operation for setting the turret direction is intuitive and the turret direction Can be easily adjusted in the direction desired by the user.

  In the above-described game example, based on the attitude and movement of the terminal device 6, the attitude of the water gun operated by the player object Po (turret direction) and the first virtual camera for generating a virtual world image to be displayed on the LCD 61. The posture (gaze direction) is controlled. However, the player object Po may be moved or the first virtual camera may be moved based on the attitude or movement of the terminal device 6. For example, a movement angle and a movement distance are calculated from changes in the posture and movement of the terminal device 6, and the player object Po and / or the first virtual camera that operates the water gun according to the movement angle and the movement distance are moved in the virtual world. . Thus, even when the player object Po and / or the first virtual camera moves in the virtual world, the present invention can be similarly applied.

  In the above-described game example, the virtual world image of the subjective viewpoint viewed from the player object Po is displayed on the LCD 61. However, the virtual world image of another aspect may be displayed on the LCD 61. For example, an image of the virtual world including at least the player object Po may be displayed on the LCD 61 of the terminal device 6. As an example, a first virtual camera may be arranged near the back of the player object Po, and an image of the virtual world including at least the player object Po may be displayed on the LCD 61 of the terminal device 6. Further, even in the above-described game example, outside the turret left / right operation range or outside the turret vertical operation range, the turret direction is locked within the range, so that only the first virtual camera is in a state where the operation of the player object Po is stopped. May change posture. In this case, the line-of-sight direction of the first virtual camera may be set so as to look behind the player object Po with the position of the player object Po as a viewpoint, or the first virtual camera is moved in the vicinity of the front of the player object Po. The first virtual camera may be set so that at least a part of the player object Po is included in the view volume.

  In the above-described game example, when the virtual world image of the subjective viewpoint viewed from the player object Po is displayed on the LCD 61, a part of the player object Po (water gun turret operated by the player object Po) is displayed. However, the player object Po may not be displayed at all. For example, even if the player object Po is not displayed at all, an aim indicating the point where the injection object reaches in the virtual world may be displayed on the LCD 61, and when the injection object is injected as described above, You may display a mode that an injection | pouring object is inject | sprayed in order of injection | pouring.

  In the above description, as an example of the injection object, the large ball that is a mass of water W or a larger amount of water than the water W is used, but other objects may be used as the injection object. For example, the “eject object” used in the present specification is used as a term representing an object that the player object Po emits or fires to hit another object, and flames, bullets, Cannonballs, bombs, grenades, rockets, missiles, balls, arrows, beams, laser beams, etc.

  In the above-described game example, a processing example is used in which the control of the turret direction and the virtual camera posture is immediately linked in accordance with the operation instruction direction determined based on the posture of the terminal device 6. However, a process in which the control of the turret direction and / or the control of the attitude of the first virtual camera may be performed with a predetermined time delay according to the change in the operation instruction direction. In this case, after the turret direction changes, the attitude of the first virtual camera changes following the turret direction after a predetermined time delay, or the attitude change after a predetermined time delay after the attitude of the first virtual camera changes. It is also possible to display a virtual world in which the turret direction changes following this.

  In the above-described game example, the game image displayed on the LCD 61 of the terminal device 6 and the game image displayed on the monitor 2 are both images showing the same virtual world. It is an image with a different viewpoint and range. Therefore, the user can see the virtual world that is displayed in different views and display ranges displayed on the two display screens, and can view a suitable game image as appropriate according to the game situation and the like. In the above-described game example, the user holds and operates the terminal device 6 to change the turret direction according to the attitude and position of the terminal device 6 in the real space, and the image displayed on the LCD 61 also displays the turret direction. It can also be changed according to. Therefore, it is possible to give the user a sense of realism in the virtual world while viewing the image displayed on the LCD 61 while holding the terminal device 6. On the other hand, if only the image displayed on the LCD 61 is viewed, it may be difficult to grasp the position of the entire virtual world and the situation around the player object Po. By displaying the world, such a problem can be solved.

  In the above-described game example, the second virtual camera for generating an image of the virtual world displayed on the monitor 2 is fixed and set in the virtual world, but the second virtual camera is set according to the movement of the player object Po. You can change the position and orientation of the camera. As an example, the orientation of the second virtual camera is controlled and displayed on the monitor 2 so that the direction in which the line-of-sight direction is projected onto the horizontal plane of the virtual world matches the direction in which the turret direction or the operation instruction direction is projected onto the horizontal plane. You can generate a virtual world image.

  The game system 1 can play various games as described above using the terminal device 6 and the board-type controller 9 as operation means. While the terminal device 6 can be used as a portable display or a second display, the terminal device 6 can also be used as a controller that performs input by operation of the main body, touch operation, button operation, etc. According to the system 1, it becomes possible to implement a wide range of games. That is, since the terminal device 6 functions as a display device, a game in which the terminal device 6 is used as a display unit without using the monitor 2 or the controller 7 and the board type controller 9 is used as an operation unit. There can also be a system-like form. Further, since the terminal device 6 functions as a display device and also as an operation device, the terminal device 6 is used as a display means without using the monitor 2 and the controller 7, and the terminal device 6 and the board are used. There may be a form such as a game system using the mold controller 9 as an operation means. Furthermore, since the terminal device 6 functions as a display device and also as an operation device, the terminal device 6 is used as a display means without using the monitor 2, the controller 7, and the board type controller 9. There may be a game system that uses the terminal device 6 as an operation means.

  Moreover, in the said embodiment, the terminal device 6 functions as what is called a thin client terminal which does not perform a game process. However, in the terminal device 6, at least a part of the series of game processes executed by the game apparatus body 5 in the above embodiment may be executed by the terminal device 6. As an example, the terminal device 6 may execute a part of processing (for example, processing for generating a terminal game image). As another example, the terminal device 6 may execute all the series of game processes executed by the game apparatus body 5. In this case, since the terminal device 6 functions as a display device and also functions as a processing device that executes game processing, the terminal device 6 does not use the monitor 2, the game device body 5, and the controller 7. Can be used as a display means, a board type controller 9 is used as an operation means, and a terminal system 6 is used as a processing means. In the game system, only the terminal device 6 and the board type controller 9 are connected wirelessly or by wire, and various kinds of games can be performed by outputting board operation data from the board type controller 9 to the terminal device 6. Needless to say, when the board-type controller 9 is not used, the terminal device 6 may be used as a display unit, an operation unit, and a processing unit.

  In the above embodiment, posture data (for example, magnetic sensor 602, acceleration) used to calculate the posture and / or movement of the terminal device 6 (including the position and posture itself, or changes in the position and posture). At least one data output from the sensor 603 and the gyro sensor 604) is output from the terminal device 6 to the game apparatus body 5, and the attitude and / or movement of the terminal apparatus 6 is calculated by information processing in the game apparatus body 5. Yes. However, the attitude and / or movement of the terminal device 6 calculated in the game apparatus main body 5 may be calculated in the terminal device 6. In this case, data indicating the posture and / or movement of the terminal device 6 calculated by the terminal device 6 (that is, the position and posture of the terminal device 6 itself calculated using the posture data, or the position and posture of the terminal device 6). Data indicating a change) is output from the terminal device 6 to the game apparatus main body 5, and the data is used in information processing in the game apparatus main body 5.

  Further, in the above description, a mode in which the terminal device 6 and the game device body 5 and the board type controller 9 and the game device body 5 are connected by wireless communication is used. Wireless communication between them may be performed. As a first example, the terminal device 6 functions as a relay device for other wireless communication. In this case, the board operation data of the board type controller 9 is wirelessly transmitted to the terminal device 6, and the terminal device 6 wirelessly transmits the terminal operation data of the terminal device 6 to the game apparatus body 5 together with the received board operation data. In this case, the terminal device 6 and the game apparatus body 5 are directly connected by wireless communication, but the board-type controller 9 is connected to the game apparatus body 5 by wireless communication via the terminal device 6. . As a second example, the board type controller 9 functions as another wireless communication relay device. In this case, the terminal operation data of the terminal device 6 is wirelessly transmitted to the board type controller 9, and the board type controller 9 wirelessly transmits the board operation data of the board type controller 9 to the game apparatus body 5 together with the received terminal operation data. In this case, the board type controller 9 and the game apparatus body 5 are directly connected by wireless communication, but the terminal device 6 is connected to the game apparatus body 5 by wireless communication via the board type controller 9. Become. When another device relays and transmits operation data to the game apparatus main body 5, an electrical connection is established between the device that generates the operation data and another device that relays the operation data via a cable. It doesn't matter.

  Further, the terminal device 6 and / or the board-type controller 9 and the game apparatus body 5 may be electrically connected via a cable. In this case, the cable connected to the terminal device 6 and / or the board type controller 9 is connected to the connection terminal of the game apparatus body 5. As a first example, the terminal device 6 and the game apparatus body 5 are electrically connected via a first cable, and the board-type controller 9 and the game apparatus body 5 are electrically connected via a second cable. Connected. As a second example, the terminal device 6 and the game apparatus body 5 are electrically connected via a cable. In this case, the board operation data of the board type controller 9 may be transmitted wirelessly to the terminal device 6 and then transmitted to the game apparatus body 5 via the cable, or directly from the board type controller 9 to the game apparatus body 5. It may be transmitted wirelessly. As a third example, the board type controller 9 and the game apparatus body 5 are electrically connected via a cable. In this case, the terminal operation data of the terminal device 6 may be transmitted wirelessly to the board type controller 9 and then transmitted to the game apparatus body 5 via the cable, or directly from the terminal apparatus 6 to the game apparatus body 5. May be sent.

  In the above embodiment, the game system 1 has one terminal device 6 and one board-type controller 9, but the game system 1 has a plurality of sets of terminal devices 6 and a board-type controller 9. It may be. That is, the game apparatus main body 5 can wirelessly communicate with a plurality of sets of terminal devices 6 and board type controllers 9, and transmits game image data, game sound data, and control data to each terminal device 6. Operation data, camera image data, microphone sound data, and board operation data may be received from each terminal device 6 and the board type controller 9. The game apparatus body 5 performs wireless communication with each of the plurality of terminal apparatuses 6 and the board type controller 9. At this time, the game apparatus body 5 performs wireless communication with each terminal apparatus 6 and the board type controller 9. It may be performed by time division or may be performed by dividing the frequency band.

  As described above, when a plurality of sets of terminal devices 6 and the board-type controller 9 are provided, more types of games can be played using the game system 1. For example, when the game system 1 has two sets of terminal devices 6 and a board type controller 9, two users can play a game simultaneously. Further, when the game system 1 has two sets of terminal devices 6 and a board-type controller 9, the game system 1 has three display devices, so that game images for each of the three users can be displayed. It can be generated and displayed on each display device.

  In the above description, the board-type controller 9 is provided with a plurality of load sensors 94. However, since the information on the position of the center of gravity of the load applied to the board-type controller 9 is not necessary in the above processing, the load sensor 94 is used. At least one board type controller 9 may be provided.

  In the above embodiment, the stationary game apparatus 3 has been described. However, the information processing program of the present invention is executed by an information processing apparatus such as a portable game apparatus or a general personal computer. May be realized. In another embodiment, the present invention is not limited to a game device, and may be any portable electronic device such as a PDA (Personal Digital Assistant), a mobile phone, a personal computer, a camera, or the like. In any device, the present invention can be realized by connecting the device, the terminal device 6 and the board-type controller 9 wirelessly or by wire.

  In the above description, the example in which the information processing is performed by the game apparatus body 5 is used. However, at least a part of the processing steps in the information processing may be performed by another apparatus provided outside the game system 1. . For example, when the game apparatus body 5 is configured to be communicable with another apparatus (for example, a server or another game apparatus), the game apparatus body 5 and the other apparatus cooperate in the processing step in the information processing. It may be executed by doing. As an example, processing for setting a player object, a virtual world, and the like is performed in another device, and data related to the motion and posture of the player object is transmitted from the game device body 5 to the other device, and the above information processing is performed. It is possible. Then, image data indicating the virtual world generated by another device is transmitted to the game apparatus body 5 and the virtual world is displayed on the monitor 2 or the LCD 61. In this way, by performing at least a part of the processing steps in the information processing with another device, processing similar to the information processing described above can be performed. Note that at least a part of the processing steps in the information processing may be performed by the board-type controller 9 (microcomputer 100). Further, the above-described information processing can be executed by cooperation between one processor or a plurality of processors included in an information processing system including at least one information processing apparatus. In the above embodiment, the CPU 10 of the game apparatus body 5 executes the predetermined program, so that the process according to the above-described flowchart is performed. All may be done.

  Further, the shape of the game apparatus main body 5 described above, the shape of the terminal device 6, the controller 7, and the board-type controller 9, and the shapes, numbers, and installation positions of various operation buttons and sensors are merely examples. It goes without saying that the present invention can be realized even in the shape, number, and installation position. In addition, the processing order, setting value, display mode, value used for determination, and the like used in the information processing described above are merely examples, and the present invention can be realized even in other orders, display modes, and values. Needless to say.

  The information processing program (game program) may be supplied not only to the game apparatus body 5 through an external storage medium such as the optical disc 4 but also to the game apparatus body 5 through a wired or wireless communication line. Further, the information processing program may be recorded in advance in a nonvolatile storage device inside the game apparatus body 5. The information storage medium for storing the information processing program may be a CD-ROM, DVD, or an optical disk storage medium similar to them, a flexible disk, a hard disk, a magneto-optical disk, a magnetic tape, or the like. The information storage medium for storing the information processing program may be a nonvolatile semiconductor memory or a volatile memory. Such a storage medium can be referred to as a computer-readable recording medium. For example, the various functions described above can be provided by causing a computer or the like to read and execute the programs of these recording media.

  Although the present invention has been described in detail above, the above description is merely illustrative of the present invention in all respects and is not intended to limit the scope thereof. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. It is understood that the scope of the present invention should be construed only by the claims. Moreover, it is understood that those skilled in the art can implement an equivalent range from the description of the specific embodiments of the present invention based on the description of the present invention and the common general technical knowledge. In addition, it is to be understood that the terms used in the present specification are used in the meaning normally used in the art unless otherwise specified. Thus, unless defined otherwise, all technical and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In case of conflict, the present specification, including definitions, will control.

  An information processing program, an information processing apparatus, an information processing system, and an information processing method according to the present invention display an image of a virtual world on a display device that can be held by a user and view a screen, and the orientation of the display device When an operation on the virtual world is performed according to the movement, the information processing program, the information processing apparatus, and the information that can facilitate the operation performed in parallel with the operation and perform processing based on the operation It is useful as a processing system and an information processing method.

DESCRIPTION OF SYMBOLS 1 ... Game system 2 ... Monitor 3 ... Game device 4 ... Optical disk 5 ... Game device main body 10,617 ... CPU
11 ... System LSI
12 ... External main memory 13 ... ROM / RTC
14 ... Disk drive 15 ... AV-IC
16 ... AV connector 17 ... Flash memory 18 ... Network communication module 19 ... Controller communication module 20 ... Expansion connector 21 ... Memory card connectors 22, 23, 29, 611, 754 ... Antenna 24 ... Power button 25 ... Reset button 26 ... Eject Button 27 ... Codec LSI
28 ... Terminal communication module 31 ... Input / output processor 32 ... GPU
33 ... DSP
34 ... VRAM
35 ... Internal main memory 6 ... Terminal devices 60, 71 ... Housing 61 ... LCD
62 ... Touch panel 63 ... Analog stick 64 ... Operation button 65 ... Marker part 66 ... Camera 67 ... Expansion connector 68 ... Foot part 601 ... Touch panel controller 602 ... Magnetic sensors 603, 701 ... Acceleration sensors 604, 703 ... Gyro sensor 605 ... UI controller 606. Codec LSI
607, 706 ... Speaker 608 ... Sound IC
609 ... Microphone 612 ... Infrared communication module 613 ... Flash memory 614 ... Power supply IC
616 ... Charger 618 ... Internal memory 619, 704 ... Vibrator 7 ... Controller 72 ... Operation unit 73 ... Connector 74 ... Imaging information calculation unit 700 ... Board 702 ... LED
741 ... Infrared filter 742 ... Lens 743 ... Image sensor 744 ... Image processing circuit 75 ... Communication unit 752 ... Memory 8 ... Marker 9 ... Board type controller 90 ... Support plate 92 ... Leg 94 ... Load sensor 95 ... Strain body 96 ... Strain Sensor 102 ... AD converter 104 ... DC-DC converter 106, 610, 753 ... Wireless module 108 ... Amplifier 100, 751 ... Microcomputer 110, 615 ... Battery

Claims (28)

An information processing program executed by a computer of an information processing device capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body,
The computer,
Direction determining means for determining whether or not a direction of a predetermined axis set in the portable display device is within a predetermined range based on data output from the portable display device;
When it is determined that the direction of the predetermined axis is within the predetermined range, the movement of the object arranged in the virtual world is controlled according to the direction of the predetermined axis, and the direction of the predetermined axis is outside the predetermined range. If it is determined that there is an object motion control means that does not perform motion control of the object according to the direction of the predetermined axis ;
Even if the direction of the predetermined axis is in any of the outside of the predetermined range and the predetermined range, the operation of the first virtual camera for generating an image of the virtual world in accordance with the direction of the predetermined axis First virtual camera operation control means for controlling;
An information processing program for causing a first image indicating a virtual world viewed from the first virtual camera to function as display control means for displaying on the portable display device. The direction determination means determines whether the direction of the predetermined axis in the left-right direction of the real space is within a first range set in the left-right direction based on data output from the portable display device. Including left and right direction determination means for determining
When it is determined that the direction of the predetermined axis in the left-right direction is within the first range, the object motion control unit is configured to change the left-right direction of the virtual world based on the direction of the predetermined axis in the left-right direction. The information processing program according to claim 1, wherein the information processing program controls the movement of the object by rotating and / or moving the object in a direction. The direction determination means determines whether or not the direction of the predetermined axis in the vertical direction of the real space is within the second range set in the vertical direction based on data output from the portable display device. Further includes a vertical direction determination means for determining whether or not
When it is determined that the direction of the predetermined axis in the up-down direction is within the second range, the object motion control unit is configured to move the virtual world up and down based on the direction of the predetermined axis in the up-down direction. The information processing program according to claim 2, wherein the movement of the object is controlled by rotating and / or moving the object in a direction. The first range has an angle range of a predetermined size,
The information processing program according to claim 3, wherein the second range has an angular range having a size different from an angular range included in the first range. The first range is a range having the same angular range left and right around a reference direction in real space;
5. The information processing program according to claim 3, wherein the second range is a range having different angular ranges in the vertical direction with respect to a horizontal direction in real space.   When it is determined that the direction of the predetermined axis is within the predetermined range, the first virtual camera motion control unit is configured to cause the same direction as the direction of the object in the virtual world to be the line-of-sight direction. The information processing program according to claim 1, wherein the information processing program controls the posture of the computer. The object motion control means controls the motion of the object based on a change in the direction of the predetermined axis,
The information processing program according to any one of claims 1 to 6, wherein the first virtual camera operation control means controls the operation of the first virtual camera based on a change in a direction of the predetermined axis. The object motion control means controls the motion of the object by largely moving the object as the change in the direction of the predetermined axis is larger.
The information processing according to claim 7, wherein the first virtual camera operation control unit controls the operation of the first virtual camera by moving the first virtual camera to a greater extent as the change in the direction of the predetermined axis is larger. program. The object motion control means controls the motion of the object by rotating the object according to the angle of the predetermined axis,
The said 1st virtual camera operation | movement control means controls the operation | movement of the said virtual camera by rotating the said 1st virtual camera according to the angle of the said predetermined axis | shaft. Information processing program. Based on the data output from the portable display device, the computer is further functioned as posture movement calculation means for calculating the posture and / or movement of the portable display device,
10. The direction determination unit according to claim 1, wherein the direction determination unit determines whether or not a direction of the predetermined axis is within the predetermined range based on an attitude and / or movement of the portable display device. Information processing program described in 1. The posture movement calculating means calculates the posture and / or movement of the portable display device based on a predetermined direction in real space,
The object motion control means is based on the direction corresponding to the predetermined direction set in the virtual world based on the attitude and / or movement of the portable display device with the predetermined direction in real space as a reference. Control the behavior of the object,
The first virtual camera operation control means determines a direction corresponding to the predetermined direction set in the virtual world based on an attitude and / or movement of the portable display device with reference to the predetermined direction in real space. The information processing program according to claim 10, wherein an operation of the first virtual camera is controlled as a reference. The posture movement calculating means calculates the posture and / or movement of the portable display device based on the direction of gravity with the direction of gravity in real space as the predetermined direction,
The object motion control means controls the motion of the object based on the gravity direction set in the virtual world based on the attitude and / or movement of the portable display device based on the gravity direction of the real space. ,
The first virtual camera motion control means is configured to use the first virtual camera based on the gravity direction set in the virtual world based on the attitude and / or movement of the portable display device based on the gravity direction of the real space. The information processing program according to claim 11, which controls the operation of the camera. The posture movement calculating means calculates at least a posture and / or movement of the portable display device rotating around a gravity direction in real space;
The direction determining means sets a range around the gravity direction of the real space as at least the predetermined range, and the predetermined axis is based on a posture and / or movement of the portable display device around the gravity direction of the real space. To determine whether the direction is within the predetermined range,
The object motion control means rotates the object around the gravity direction set in the virtual world based on the posture and / or movement of the portable display device rotating around the gravity direction of the real space. Control
The first virtual camera motion control means is configured to move the first virtual camera around the gravity direction set in the virtual world based on the attitude and / or movement of the portable display device around the gravity direction in real space. The information processing program according to claim 12, wherein the operation is controlled by rotating the. The posture movement calculating means calculates at least a posture and / or a movement in which the portable display device swings up and down around a horizontal direction perpendicular to the direction of gravity in real space,
The direction determination means sets a range around the horizontal direction of the real space as at least the predetermined range, and the predetermined axis is based on a posture and / or movement of the portable display device swinging up and down around the horizontal direction. To determine whether the direction is within the predetermined range,
The object motion control means is configured to move the object around a horizontal direction set in the virtual world corresponding to the horizontal direction based on a posture and / or movement of the portable display device swinging up and down around the horizontal direction. Shake up and down to control the operation,
The first virtual camera operation control means is configured to move around the horizontal direction set in the virtual world corresponding to the horizontal direction based on a posture and / or movement of the portable display device swinging up and down around the horizontal direction. The information processing program according to claim 12, wherein the operation is controlled by swinging the first virtual camera up and down. The posture movement calculating means is configured to rotate around a first axis and a second axis perpendicular to the depth direction of the display screen perpendicular to the display screen of the portable display device on which the first image is displayed, and / or Or calculate at least the movement,
The direction determining means sets the respective ranges around the first axis and the second axis as the predetermined ranges, and the posture in which the portable display device rotates around the first axis and the second axis, and / or Alternatively, based on the movement, it is determined whether the direction of the predetermined axis is within the predetermined range,
When the direction of the predetermined axis is determined to be within a predetermined range around the first axis, the object motion control means changes the posture and / or movement of the portable display device around the first axis. In response, the movement of the object around the axis set in the virtual world corresponding to the first axis is controlled,
When it is determined that the direction of the predetermined axis is within a predetermined range around the second axis, the object motion control means may change the posture and / or movement of the portable display device around the second axis. In response, the movement of the object around the axis set in the virtual world corresponding to the second axis is controlled,
The first virtual camera motion control means is orthogonal to the line-of-sight direction according to the attitude and / or movement of the portable display device around the first axis and the second axis, respectively. The information processing program according to any one of claims 10 to 12, wherein an operation of the first virtual camera is controlled by rotating around an axis corresponding to two axes. The posture movement calculating means calculates at least a posture and / or a movement that respectively rotate around a horizontal axis and a vertical axis of the display screen of the portable display device orthogonal to the depth direction,
The direction determining means sets the respective ranges around the horizontal axis and the vertical axis as the predetermined ranges, and the portable display device is respectively set around the horizontal axis and the vertical axis. Determining whether the direction of the predetermined axis is within the predetermined range based on the rotating posture and / or movement,
The object motion control means may be configured such that, when it is determined that the direction of the predetermined axis is within a predetermined range around the horizontal axis, the portable display device rotates around the horizontal axis and / or According to the movement, the movement is controlled by rotating the object around the horizontal axis set in the virtual world corresponding to the horizontal axis.
The object motion control means may be configured such that when the direction of the predetermined axis is determined to be within a predetermined range around the vertical axis, the portable display device rotates around the vertical axis and / or According to the movement, the movement is controlled by rotating the object around the vertical axis set in the virtual world corresponding to the vertical axis,
The first virtual camera motion control means rotates around the horizontal axis of the virtual world perpendicular to the line-of-sight direction according to the attitude and / or movement of the portable display device around the horizontal axis. Controlling the operation of the first virtual camera by rotating around the vertical axis of the virtual world according to the attitude and / or movement of the portable display device about the vertical axis. The information processing program according to claim 15. The posture movement calculating means includes:
Real space reference direction setting means for setting a reference direction of the portable display device in real space;
Based on the data output from the portable display device, the left-right angle difference around the vertical direction in the real space between the direction of the predetermined axis and the reference direction at the current time, the direction of the predetermined axis at the current time and the real space Angle difference calculating means for calculating a vertical angle difference around the horizontal direction with respect to the horizontal direction,
The direction determination means includes
Left-right direction determination means for determining whether the left-right angle difference is within a first range set around the vertical direction of real space;
Vertical direction determination means for determining whether or not the vertical angle difference is within a second range set around the horizontal direction of the real space,
When it is determined that the left-right angle difference is within the first range, the object motion control means rotates and / or moves the object in the left-right direction of the virtual world based on the left-right angle difference. Control the movement of the object,
When it is determined that the vertical angle difference is within the second range, the object motion control means rotates and / or moves the object in the vertical direction of the virtual world based on the vertical angle difference. Control the movement of the object,
The first virtual camera operation control means rotates and / or moves the first virtual camera in the horizontal direction of the virtual world based on the left-right angle difference, and is orthogonal to the line-of-sight direction based on the vertical angle difference. The information processing program according to claim 10, wherein the first virtual camera is rotated and / or moved in a horizontal direction of the virtual world to control an operation of the first virtual camera. The display control means outputs image data indicating the first image to the portable display device,
The portable display device is:
Image data acquisition means for acquiring image data output from the information processing apparatus;
The information processing program according to claim 1, further comprising: a display unit that displays the first image indicated by the image data acquired by the image data acquisition unit. And further causing the computer to function as compressed image generation means for generating compressed image data by compressing image data representing the first image,
The display control means outputs the compressed image data generated by the compressed image generation means to the portable display device,
The image data acquisition means acquires the compressed image data output from the information processing apparatus;
The portable display device further includes display image expansion means for expanding the compressed image data to obtain image data indicating the first image,
The information processing program according to claim 18, wherein the display means displays the first image indicated by the image data acquired by the image data acquisition means and expanded by the display image expansion means. The display control means further displays, separately from the first image, a second image showing the virtual world viewed from a second virtual camera on another display device connected to the information processing device. 18. An information processing program according to any one of 1 to 17. And further causing the computer to function as compressed image generation means for generating compressed image data by compressing image data representing the first image,
The display control means outputs the compressed image data generated by the compressed image generation means to the portable display device, and separately from the compressed image data, the separate image data indicating the second image is not compressed. Output to the display device
The portable display device is:
Image data acquisition means for acquiring compressed image data output from the information processing apparatus;
Display image decompression means for decompressing the compressed image data to obtain image data representing the first image;
The information processing program according to claim 20, further comprising: a display unit configured to display the first image indicated by the image data acquired by the image data acquisition unit and expanded by the display image expansion unit.   Based on the position of the object in the virtual world, the second virtual camera for generating an image of the virtual world is at a position different from the first virtual camera, and the object is included in the second image The information processing program according to claim 20 or 21, further causing the computer to function as second virtual camera operation control means for setting to be performed. The second virtual camera operation control means sets the second virtual camera at a position further away from the object than the distance from the object to the first virtual camera,
The information processing program according to claim 22, wherein the display control means displays a range wider than the range of the virtual world indicated by the first image as the second image on the other display device. The second virtual camera operation control means sets the second virtual camera at a position where the object is bird's-eye view in the virtual world,
The information processing program according to claim 22, wherein the display control unit displays an image obtained by bird's-eye view of the object arranged in the virtual world as the second image on the another display device. The portable display device includes at least one of a gyro sensor and an acceleration sensor,
25. The direction determination unit according to claim 1, wherein the direction determination unit determines whether or not a direction of the predetermined axis is within the predetermined range based on data output from at least one of the gyro sensor and the acceleration sensor. The information processing program as described in any one. An information processing apparatus capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device body,
Direction determining means for determining whether or not a direction of a predetermined axis set in the portable display device is within a predetermined range based on data output from the portable display device;
When it is determined that the direction of the predetermined axis is within the predetermined range, the movement of the object arranged in the virtual world is controlled according to the direction of the predetermined axis, and the direction of the predetermined axis is outside the predetermined range. If it is determined that there is an object motion control means that does not perform motion control of the object according to the direction of the predetermined axis ;
Even if the direction of the predetermined axis is in any of the outside of the predetermined range and the predetermined range, the operation of the first virtual camera for generating an image of the virtual world in accordance with the direction of the predetermined axis First virtual camera operation control means for controlling;
An information processing apparatus comprising: a display control unit configured to display a first image indicating a virtual world viewed from the first virtual camera on the portable display device. An information processing system capable of displaying an image on a portable display device configured to communicate with a plurality of devices and outputting at least data according to the attitude and / or movement of the portable display device main body. ,
Direction determining means for determining whether or not a direction of a predetermined axis set in the portable display device is within a predetermined range based on data output from the portable display device;
When it is determined that the direction of the predetermined axis is within the predetermined range, the movement of the object arranged in the virtual world is controlled according to the direction of the predetermined axis, and the direction of the predetermined axis is outside the predetermined range. If it is determined that there is an object motion control means that does not perform motion control of the object according to the direction of the predetermined axis ;
Even if the direction of the predetermined axis is in any of the outside of the predetermined range and the predetermined range, the operation of the first virtual camera for generating an image of the virtual world in accordance with the direction of the predetermined axis First virtual camera operation control means for controlling;
An information processing system comprising: display control means for displaying a first image showing a virtual world viewed from the first virtual camera on the portable display device. Included in an information system comprising at least one information processing device capable of displaying an image on the portable display device that outputs at least data according to the attitude and / or movement of the portable display device main body. An information processing method executed by cooperation between one processor or a plurality of processors,
A direction determining step for determining whether or not a direction of a predetermined axis set in the portable display device is within a predetermined range based on data output from the portable display device;
When it is determined that the direction of the predetermined axis is within the predetermined range, the movement of the object arranged in the virtual world is controlled according to the direction of the predetermined axis, and the direction of the predetermined axis is outside the predetermined range. If it is determined that there is an object motion control step that does not perform motion control of the object according to the direction of the predetermined axis ;
Even if the direction of the predetermined axis is in any of the outside of the predetermined range and the predetermined range, the operation of the first virtual camera for generating an image of the virtual world in accordance with the direction of the predetermined axis A virtual camera operation control step to control;
A display control step of displaying a first image showing a virtual world viewed from the first virtual camera on the portable display device.

Images