JP2005253871A - Communicative match type virtual reality tennis game system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communicative match type virtual reality tennis game system which can ensure reality even by a small communication volume. <P>SOLUTION: A VR tennis game system (100) connects VR tennis game devices (10) which are respectively set at two sites by the internet (54). A computer (42 in Fig.3, not shown) of each VR tennis game device displays a game picture including a ball shape (40 in Fig.2, not shown) on a screen (12) by a projector (14). A player (16) swings a racket (18) in an actual space in front of the screen. The computer prepares ball returning data in response to the swing (ball-returning motion) of the player. The ball returning data includes a ball shape position and a ball velocity at that time. Therefore, each computer calculates the track of the ball shape based on the ball returning data, and displays the flying ball shape on the screen conforming to the track. In this case, a motion image of the swing motion of a partner which has been stored in the memory (48 or 52) of the computer (42) in advance is displayed together on the game picture as well. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a virtual reality tennis game system, and in particular, a player progresses a tennis game by actually swinging a racket in a real space using two virtual reality tennis game devices connected via a network (communication line) such as the Internet. The present invention relates to a communication game type virtual reality (Virtual Reality: hereinafter referred to as “VR”) tennis game system.

  The present inventors have proposed a VR video tennis game system for rehabilitation in Patent Document 1. This Patent Document 1 proposes a technique for playing a tennis game with a computer installed at a remote place, particularly in FIGS. 2 and 6.

In Patent Document 1, the player is photographed with a camera and the photographed video is directly sent from the communication unit to the partner computer, or the photographed video is converted into CG (computer graphics) and sent to the partner computer. Like that.
JP2003-180896A [A63B 69/00 504 69/38 A63F 13/00]

  In the prior art of Patent Document 1, the amount of information to be communicated can be significantly reduced by transmitting only the movement parameter of the opponent player, but nothing is described specifically. Moreover, when displaying an opponent player by CG, reality is lacking.

  Therefore, the main object of the present invention is to provide a novel communication battle VR tennis game system.

  Another object of the present invention is to provide a communication battle type VR tennis game system capable of greatly reducing the amount of communication information while ensuring a certain degree of reality.

  According to the first aspect of the present invention, a virtual reality tennis game device is installed at each of two sites, and each virtual reality tennis game device is provided with a computer and display means for displaying a game screen according to image processing by the computer. Is a virtual reality reality game in which the game is progressed by swinging a racket in real space so as to hit a ball figure displayed on the display means, and the computers of each virtual reality tennis game device are connected by a communication line. This is a tennis game system, in which each virtual reality tennis game device is provided with a camera, and the camera captures the swing motion of the player, and the computer at least based on the camera image captured by the camera of the player's swing motion. Creates return ball data including ball position data and ball speed data, and sends the return ball data to the opponent computer. The computer also calculates and displays the trajectory of the ball figure based on the return ball data sent from the opponent computer. It is a communication battle type virtual reality tennis game system displayed on the means.

  According to the first aspect of the present invention, two VR tennis game devices (10: reference numerals indicating corresponding parts in the embodiment; hereinafter the same) are connected by the Internet (54). Each VR tennis game apparatus includes a computer (42) and display means (12, 14). A ball graphic is displayed on the game screen displayed by the display means, and the player of each site sees the ball graphic and actually swings the racket to hit it. The player's returning ball motion is photographed by a camera (24), for example. Specifically, by placing a marker (20) on the racket (18) and photographing the marker with a camera, the computer detects that a ball returning operation has been performed, and the ball figure position and ball speed at that time are detected. Create and send return ball data including.

  Each computer calculates the trajectory of the ball figure based on the ball position data and velocity data included in the return ball data, and displays the ball figure by the respective display means.

  According to the first aspect of the present invention, only the return ball data needs to be communicated between the two sites in real time, so that the amount of communication can be greatly reduced. Therefore, the competitive VR tennis game can be played smoothly without a sense of incongruity.

  According to a second aspect of the present invention, the computer displays the court figure fixedly on the display means, and displays the moving image of the opponent player so as to be superimposed on the court figure. It is a game system.

  In the invention of claim 2, the court figure (30), the net figure (32), etc. are fixedly displayed on the display means by the computer (42), and a moving image of the opponent player's return ball motion is also displayed. Therefore, the presence and reality can be further improved.

  According to a third aspect of the present invention, each virtual reality tennis game device further includes storage means, and the moving images of the opponent player are stored in the storage means, and the computer receives the return ball data from the opponent computer from the storage means. 3. The communication battle type virtual reality tennis game system according to claim 2, wherein a moving image of the opponent player is read and displayed on the display means.

  In the invention of claim 3, since the display data of the moving image of the opponent player is stored in the storage means, it is read out when necessary, that is, in accordance with the return motion of the opponent player, and the moving image of the opponent player is displayed on the game screen. can do.

  According to the fourth aspect of the present invention, the player's swing motion is photographed by the camera, thereby creating a moving image of the player, and before the game starts or after the game starts, the return computer receives the return ball data. 4. The communication battle type virtual reality tennis game system according to claim 3, wherein the moving image is transmitted to the opponent computer, whereby the moving image of the player is prestored in the storage means in each virtual reality tennis game device. .

  In the invention of claim 4, since it is not necessary to communicate the moving image data of the opponent player during the game, the moving image of the opponent player can be displayed with a very small amount of communication.

  The invention according to claim 5 shoots the player's swing motion with a camera, thereby creating a moving image of the player, and transmitting the moving image of the player to the opponent computer in real time. Reality tennis game system.

  In the invention of claim 5, the player moving image communicates in real time. However, in this case, for example, only the player image extracted by the background subtraction method may be transmitted.

  According to the sixth aspect of the present invention, the face image of the player is photographed by the camera, and the face image is transmitted to the opponent computer in real time, whereby the face image is displayed on the display means together with the moving image of the player in each virtual reality tennis game device. A communication battle type virtual reality tennis game system according to any one of claims 2 to 5, wherein the game system is a communication battle type virtual reality tennis game system.

  In the invention of claim 6, since the face image of the opponent player is displayed on the display means, the opponent's facial expression can also be seen.

  According to the seventh aspect of the present invention, virtual reality tennis game devices are installed at two sites, respectively, and each virtual reality tennis game device is provided with a computer and display means for displaying a game screen according to image processing by the computer. Is a virtual reality reality game in which the game is progressed by swinging a racket in real space so as to hit a ball figure displayed on the display means, and the computers of each virtual reality tennis game device are connected by a communication line. A control method in the tennis game system, in which each virtual reality tennis game device is provided with a camera, and a moving image of the player is captured by the camera, and the moving image of the player is sent to the opponent computer before the game starts or at the beginning of the game. Thus, in each virtual reality tennis game device, the player's moving images are stored in advance in the storage means, and in each virtual reality tennis game device, the computer reads out the player's moving images stored in the storage means as needed during the game. It is a control method of a communication battle type virtual reality tennis game system displayed on a display means.

  In the seventh aspect of the invention, as in the fourth aspect of the invention, the moving image of the opponent player can be displayed without communicating the moving image data of the opponent player during the game.

  According to the present invention, since only the return ball data is communicated between the sites, the amount of communication data can be greatly reduced. Therefore, even if a communication line that is not very fast is used, the communication battle VR tennis game can be smoothly played.

  If a tennis court figure is fixedly displayed on each VR tennis game apparatus and a player moving image is displayed on top of it, reality can be secured to some extent even if the communication amount is reduced.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  In one embodiment of the communication battle type VR tennis game system of the present invention, two VR tennis game apparatuses 10 shown in FIG. 1 are used. Therefore, first, the VR tennis game apparatus 10 of FIG. 1 will be described in detail.

  The VR tennis game apparatus 10 includes a screen 12, and a video for the VR tennis game is displayed on the screen 12 by the projector 14. The coil cup 12 and the projector 14 constitute a display device or display means. In this embodiment, the ceiling-mounted projector 14 is used in order to secure a sufficient space on the floor surface, but a floor-standing projector may be used. Furthermore, in this embodiment, a video display device comprising a combination of the screen 12 and the projector 14 is used, but other video display devices such as a large screen television may be used. However, since the game screen is displayed on the screen 12, the reference number “12” may indicate the game screen.

  The VR tennis game apparatus 10 of this embodiment can be used mainly for upper limb movements in rehabilitation, and the trainee actually swings the racket 18 in the real space in front of the screen 12 as the player 16. The racket 18 is provided with a light marker 20 such as a visible light emitting diode (LED) such as red.

  In order to provide such an optical marker 20, although not shown, it is necessary to provide the racket 18 with a switch for supplying a battery and power from the battery to the optical marker (LED) 20. However, as this switch, a switch that is turned on when the player holds the racket 18 and turned off when the player releases it can be used.

  On the other hand, one USB camera 22L and 22R as the first camera is provided on each of the left and right sides of the screen 12, and the optical marker 20 of the racket 18 is photographed by the USB cameras 22LR and 22L, thereby making the player as described below. It is detected whether 16 swings the racket 18 (whether it swings).

  Note that a USB camera is connected to a USB (Universal Serial Bus: an interface standard that can connect various peripheral devices with a common connector) terminal of a computer. For example, the horizontal angle is 50 degrees and the number of pixels is 25−. It is a relatively inexpensive camera called a Web camera with about 450,000 pixels. However, a camera with a larger number of pixels may be used. However, according to experiments by the inventors, the tennis game of the example was able to cope with such a number of pixels. Since the number of pixels is small, the amount of computer processing is small, and the system can be constructed inexpensively as a whole.

  Further, one USB camera 24 as a second camera is provided in the lower center of the screen 12. On the other hand, different colors, for example, yellow and blue color markers 26L and 26R are attached to the ankles of the left and right feet of the player 16. By photographing the two color markers 26L and 26R with the USB camera 24, the posture (returning ball direction) at the time of forehand or backhand is determined as will be described later.

  Note that the VR tennis game apparatus 10 of FIG. 1 is provided with stereo speakers 28R and 28L, and the speakers 28R and 28L generate a hitting sound, cheering of the audience, or other appropriate BGM.

  FIG. 2 shows a display example of a game screen on the screen 12 when a tennis game is executed by the VR tennis game apparatus 10 of FIG. 1 embodiment. That is, the tennis court figure 30, the net figure 32, and the audience figure 34 are fixedly displayed in the virtual space by the computer 42 (FIG. 3). The display data of the court figure 30, the net figure 32, and the spectator figure 34 that are fixedly displayed is stored in advance in a storage means such as an internal memory 48 or an external memory 52 described later shown in FIG. They can be read and displayed.

  The computer 42 further displays a racket figure 36 representing the racket 18 swung in the real space by the player 16 on the front side of the court, and displays the opponent player figure (an opponent player figure) 38 as a moving image on the opponent's court, Further, the tennis ball figure 40 is moved and displayed so as to fly between the racket figure 36 and the opponent player figure 38. The racket figure 36 can be displayed using the display data set in each storage means similarly to the display data of the fixed display figure.

  FIG. 3 shows a block diagram of the VR tennis game apparatus 10 of FIG. 1 embodiment. The VR tennis game apparatus 10 includes a computer 42 that receives camera signals from the USB cameras 22L and 22R and 24. Then, a video signal is supplied from the graphic board 44 to the projector 14 and an audio signal is supplied from the sound board 46 to the speakers 28R and 28L.

  The internal memory 48 of the computer 42 stores a program shown in a flowchart described later, and the internal memory 48 is also used as a working memory, a register, or the like. In particular, the internal memory 48 is a hue register for presetting hue values (yellow and blue in the embodiment) indicating colors (marker colors) to be extracted or acquired by the computer 42 in image processing. (Not shown) is formed.

  An external memory 52 is coupled to the computer 42 via a memory interface 50. The external memory 52 is an arbitrary recording medium or storage medium such as a hard disk, a disk such as a CD-RW or a DVD-RW, or a semiconductor memory. Here, these are referred to as “memory” for convenience. The external memory 52 stores or records master data including trainers (players) and their symptoms necessary for rehabilitation.

  In the VR tennis game system of the embodiment, as shown in FIG. 4, the VR tennis game device 10 shown in FIGS. 1 and 3 is used as a device at site A and a device at site B. Accordingly, the components shown in FIG. 1 and FIG. 3 are indicated by the same number, but when it is necessary to distinguish between them, “A” or “B” is added to the number to indicate whether it is that of Site A or Site B. To distinguish.

  In the VR tennis game system 100 of FIG. 4, the site A computer 42 </ b> A and the site B computer 42 </ b> B are set to be able to access the lobby server 56 through the Internet 54. A player at site A (not shown) and a player at site B (not shown) each use their own device 10 shown in FIG. 1 so that both players can play VR tennis as their own players. You can play games.

  In this VR tennis game system 100, when the IP address of the computer of the opponent player's apparatus 10 is known and can be connected to each other, the computers 42A and 42B are directly connected via the Internet 54. In this case, “YES” is determined in the first step S101 of FIG. 5, and in the next step S103, it is determined whether or not the own computer 42 is to function as a server. When the own computer 42A or 42B functions as a server, “YES” is determined in the step S103, and in step S105, reception of a connection request transmitted from the partner computer 42B or 42A functioning as a client is awaited. If it is determined in step S107 that a connection request from the counterpart computer has been received, "YES" is determined in this step S107, and the process proceeds to FIG.

  If it is determined in step S103 that the server is not to be a server, “NO” is determined in step S103, and in the subsequent step S109, the IP address of the partner computer to be the server is designated as a client, The connection request is transmitted to the partner computer until it is determined in step S111 that connection is possible. If it is determined in step S111 that connection is possible, the process proceeds to FIG.

  When the IP address of the opponent player is not known, “NO” is determined in step S101 in order to search for the opponent, and in the next step S113, the computer 42 on its own side passes through the Internet 54 shown in FIG. The lobby server 56 is accessed. The lobby server 56 operates a virtual waiting room for players who wish to compete in communication. Accordingly, in step S115, a list of waiting players for communication that are stored in a memory (not shown) of the lobby server 56 can be acquired.

  In the server 56, the computer of the player who has requested the server 56 to transmit the list is added to the list and registered. Therefore, the player who has accessed the server 56 is also a target for selection from other players.

  If there is an opponent who wants to compete by looking at the list, “YES” is determined in step S117, and a connection request is transmitted to the selected opponent in step S119 until connection permission is given in step S121. When the connection permission is given in step S121, the process proceeds to FIG.

  If “NO” is determined in the step S117, that is, if the player who wants to play is not in the list, it is determined whether or not a connection request has been transmitted from another player's site (computer) in the next step S123. To do. When responding to a connection request from another site, the battle is permitted in the next step S125, and the process proceeds to FIG.

  When “YES” is determined in any of steps S107, S111, S121, or S125 of FIG. 5, that is, when a communication battle tennis game is played with the opponent player, the opponent computer is connected in the first step S127 of FIG.

  In the next step S129, a swing pattern is created and transmitted to the counterpart computer. Specifically, this step S129 can be executed as a subroutine shown in FIG. However, this step S129 is not processed when real-time image transmission / reception described later is performed.

  In the first step S131 in FIG. 7, the player first takes a background image as shown in FIG. 8 (in the place where the player is actually going to swing) using, for example, the camera 24 shown in FIG. Subsequently, in step S133, the player starts a forehand swing, for example, with a racket in accordance with an instruction from a computer, for example, 42A. Then, in the next step S135, a player image as shown in FIG. As shown in FIG. 9, the player image is naturally taken together with the background image (FIG. 8).

  Thereafter, in step S137, the computer 42A extracts only the player image area using the background subtraction method. Specifically, by superimposing the image of FIG. 8 taken in step S131 and the image of FIG. 9 taken in step S135 and discarding regions (pixels) having substantially the same brightness, as a result, Only the player image remains.

  In subsequent step S139, the computer 42A adds the player image extracted in step S137 to the swing pattern file. Since the swing pattern video is a set of continuous still images, it is added to the pattern file one frame at a time. The other pattern file is stored in, for example, the internal memory 48 shown in FIG.

  Steps S133 to S139 are repeatedly executed until the completion of the swing is detected in Step S141. When “YES” is determined in Step S141, a large number of still image sets indicating, for example, forehand swing and backhand are included in the pattern file. Accumulated.

  In the last step S143 in FIG. 7, the computer 42A transmits the pattern file in the internal memory 48 to the counterpart computer 42B. At the same time, the background image shown in FIG. 8 taken in the previous step S131 is also sent to the partner computer 42B so that it can be used by the partner computer 42B.

  In the embodiment, Microsoft's DirectPlay (product name) is installed in each of the computers 42A and 42B. This application software called DirectPlay is a collection of APIs (application programming interfaces) that summarize functions related to communication, and enables description of network applications such as multiplayer games. DirectPlay (product name) performs all the difficult tasks related to player connection and session management, and once connected to each other, this DirectPlay is used to send data with or without guarantee to each other. be able to. In the embodiment, all the data transmissions in the previous step S143 and the like are executed using this function.

  Then, in the subsequent step S145 of FIG. 6, the swing pattern file of the opponent player transmitted from the opponent computer 42B is received. However, when transmitting and receiving a real-time image, this step S145 is omitted together with the previous step S129.

  The swing pattern file of the opponent player received in step S145 is accumulated and stored in an appropriate area of the internal memory 48 or the external memory 52 of FIG. Therefore, the computer 42 on its own side can draw out (read out) the swing pattern of the opponent player when necessary.

  The battle is started in step S147 following FIG. 6, and real-time image transmission is executed in the next step S149. However, step S149 (and step S163) is omitted in a system in which the swing pattern is transmitted and received prior to the start of the battle in steps S129 and S145.

  In other words, the system 100 of FIG. 4 is designed so that one of two types can be selectively set. One type uses the swing pattern of the opponent player when a relatively low-speed communication line is used. Is stored in the memory of the player in advance, and the swing pattern of the opponent player read from the memory is superimposed and displayed on the game screen on the player's side. Another type is that when a relatively high-speed communication line is used, the opponent player's swing pattern is received from the opponent computer each time, and the opponent player's swing pattern received in real time is displayed on the game screen on his / her side. This is a method of superimposing display.

  When the latter type is selected, real-time transmission of the swing pattern is executed in step S149. However, specifically, this step S149 is executed as a subroutine of FIG.

  First steps S151, S153, and S155 in FIG. 10 are the same as steps S131, S135, and S137 in FIG. 7 described above. That is, in step S151, the background shown in FIG. 8 is photographed. In step S153, the player shown in FIG. 11 is photographed. In step S155, the player area is extracted by the background subtraction method as shown in FIG. In this way, a moving image of the player is created.

  In the next step S157, the computer 42A extracts only the face area of the player as shown in FIG. For example, the height of the player image extracted in step S155 is measured, and an area centered on a specific proportion of heights that the face is expected to exist is extracted as the face area. In this way, the player's face image is generated.

  After that, in step S159, the moving image data of the player area and the face area is encoded. That is, the player moving image and the face image data are encoded. However, the compression rate is changed according to the communication environment so that it can be smoothly played back by the other computer, and compression coding is performed at a compression rate optimum for the communication state at that time. In step S161, the compression-encoded data (player moving image and face image) is transmitted to the counterpart computer 42B via the communication line.

  When the real-time image transmitted from the counterpart computer is received in step S163, the subroutine shown in FIG. 14 is actually executed. In the first step S165, code data transmitted from the partner computer is received and stored in the internal memory 48 (FIG. 3) as necessary. In the subsequent step S167, the encoded data is decoded, and the decoded data, that is, the moving image data of the opponent player is stored in another area of the internal memory 48. Then, in the last step S169, the reproduced video data of the opponent player is synthesized and displayed in the game screen 12 displayed on the own display as shown in FIG. That is, in the case of a real-time image transmission / reception type system, the game screen including the moving image of the opponent player shown in FIG. 15 can be displayed by executing steps S149 and S163 of FIG. At this time, not only the moving image of the opponent player but also the face image 39 is displayed on the screen, so that the player's own player's face at that time, for example, the face when the opponent player misses, You can see real-time faces and other things in real time, further improving the reality and fun of the game.

  In the subsequent step S171 in FIG. 6, the computer 42A checks whether any data is sent from the partner computer 42B. When using the above-mentioned DirectPlay (product name), the formation of data to be transmitted and received between computers is [Type, Option, Data ...].

  Here, a numerical value having a certain meaning is entered in Type and Option. In Type, enter the type of information to be sent and received. For example, Type1 = game start, Type2 = ball information, Type3 = point determination.

  Add the contents attached to Type to Option. For example, when Type = 2, Option0 = forehand, Option1 = backhand. When Type = 3, Option0 = points of own player, Option1 = points of opponent player).

  In addition, the numerical information necessary for Type is entered in Data. For example, when Type = 2, the ball position.

  For example, assume that the position of the ball immediately before hitting is X, Y, Z, and the ball speed is Vx, Vy, Vz. When the player hits the ball, the ball speeds Vx ′, Vy ′, and Vz ′ to return are obtained according to the swing direction of the racket detected by the method described later. At this time, if the ball is hit with the forehand, the data to be transmitted is [2, 0, X, Y, Z, Vx ′, Vy ′, Vz ′]. When this data is received by the communication partner, the ball positions X, Y, Z and the ball speeds Vx ', Vy', Vz 'are reflected in the game, and drawing is performed so that the opponent returns.

  Also, when points (scores) are generated, for example, if the player at site A swings away and the ball bounces two times, points are added to the opponent player at site A and [3, 1] is transmitted. When this data is received at the site B, it is determined that the opponent player at the site B has missed and points are added to the player. When the hit ball of the opponent player is out on site A, points are added to the player on site A and [3, 0] is transmitted. When this data is received at site B, it is determined that the hit ball of the player at site B is out, and points are added to the opponent player.

  More specifically, when reception of transmission data from the counterpart computer is detected in step S171 in FIG. 6, values of Type and Option included in the reception data are acquired in next step S173.

  In the next step S175, the computer 42A determines whether Type is “1”. If Type = 1, in next step S177, the game start determination is set to “OK”, and the game is initialized. Thereafter, in step S179, it is determined whether the start determination (flag) is OK. If “YES”, the process proceeds to step S181 in FIG.

  In step S181 in FIG. 16, it is determined whether or not the Type included in the transmission data from the partner site acquired in step S173 (FIG. 6) is “3”. If “YES” is determined in step S181, it is necessary to make a point determination next, as described above. Therefore, in the subsequent step S183, the value of Option subsequent to the Type is “1”. Or “0”. That is, it is determined whether or not the value of Option is “0” in step S183. If “YES”, in order to give points to the opponent player as described above, the next step S185 is executed, and the next step S187 is executed. Then, the start determination flag is set to “NG” and the game is initialized to prepare for the next rally.

  On the other hand, if “NO” is determined in the step S183, a score is added to the player in the next step S189. Then, after step S187 or S189, the process returns to step S149 of FIG.

  If “NO” is determined in the step S181 in FIG. 16, in the subsequent step S191, the computer 42A determines whether or not the value of Type is “2”. Type = 2 means that data indicating whether the opponent player's swing is a forehand or a backhand has been transmitted, and in the next step S193, a swing pattern reproduction step is executed.

  The swing pattern reproduction process is the internal memory 48 (FIG. 3) depending on whether the swing pattern of the opponent player previously received in step S163 in FIG. ) And display it on the game screen on the screen 12 of the site on its own site.

  Specifically, in step S195 of FIG. 17, first, it is determined whether the value of Option subsequent to Type = 2 is “0”. If “YES”, as described above, since the swing pattern of the opponent player is the forehand, the computer 42A reads the forehand pattern file from the internal memory 48 in the subsequent step S197. If “NO” in the step S193, since the swing pattern of the opponent player is a backhand, the computer 42A reads the pack hand pattern file from the internal memory 48 in the subsequent step S199.

  In any case, in step S201, the read pattern file is synthesized and displayed on the game screen. That is, at the timing of redrawing the game screen (30 times per second), the opponent player's swing image (still image) is read one by one from the file, and it is displayed in the game screen, for example, as shown in FIG. Overlay and display. That is, on the game screen 12 of FIG. 2, the tennis court figure 30, the net figure 32, and the audience figure 34 are fixedly displayed, and the opponent player figure 38 is displayed as a moving image in which a forehand or backhand swing is performed.

  Thereafter, in step S203, the ball position X, Y, Z and the ball speeds Vx ′, Vy ′, Vz ′ included in the return ball data of the opponent player received in the previous step S173 (FIG. 6) are shown in FIG. The ball figure 40 is drawn so that the ball figure 40 moves on the orbit.

  In subsequent step S205, the computer 42A determines whether or not the ball is present at a position where the player 16 (FIG. 1) can hit the ball. More specifically, a position or area where the player 16 can hit the ball graphic 40 (FIG. 2) is set in advance, and when the ball graphic reaches the position or area, “YES” is determined in this step S205. The

  If “YES” is determined in the step S205, the computer 42A determines the swing of the racket 18 in the next step S207. Specifically, this step S207 is executed as a subroutine of FIG.

  In the first step S11 of FIG. 18, the computer 42A captures image signals (first camera signals) from the two USB cameras 22L and 22R. However, since it is only necessary to detect the optical marker 20 (FIG. 1) provided on the racket 18 in the swing determination, the exposure times of the USB cameras 22L and 22R are set to be extremely short, and the USB cameras 22L and 22R are used. Input a monochrome image signal. That is, only the luminance signal is input from the USB cameras 22L and 22R.

  In the next step S13, the computer 42A binarizes the luminance signals from the USB cameras 22L and 22R so as to detect a brightness area that seems to be the optical marker 20.

  In the subsequent step S15, the computer 42A executes an expansion process (a process of filling a divided area or a holed area into a series of areas).

  In step S17, the computer 42A performs area filtering. That is, in this step S17, the computer 42A detects a high luminance area after binarization in step S13 and expansion processing in step S15, and determines and removes a small area as noise. A region having an area equal to or greater than a predetermined threshold is detected. And the area | region of the largest area is determined to be the optical marker 20 among the area | regions of the area beyond the threshold value.

  After detecting the position of the optical marker 20 in step S17, the barycentric coordinates are measured in step S19. That is, the computer 42A measures the barycentric coordinates on the image and sets the center of the image as the origin. However, the “center of gravity” refers to the center of mass distribution when the pixel value is regarded as density.

  After measuring the center-of-gravity coordinates in step S19, in step S21, the computer 42A calculates the movement amount of the center of gravity based on the center-of-gravity position of the previous (previous frame) and the center-of-gravity position of the current (current frame). Next, in step S23, the computer 42A determines whether the amount of movement is equal to or greater than a predetermined threshold value.

  If “NO” is determined in the step S23, that is, if the amount of movement from the center of gravity position of the previous frame is smaller than the predetermined value, the computer 42A determines that the player 16 is not swinging the racket 18, and step S25. , The swing flag SF is reset (SF = 0). However, this swing flag SF is not shown in the figure, but is provided in the internal memory 48 shown in FIG. 3 in the same manner as a forehand flag FHF, a backhand flag BHF, and a serve flag SVF which will be described later.

  If “YES” is determined in step S23, that is, if the amount of movement from the center of gravity position of the previous frame is equal to or greater than a predetermined value, the computer 42A determines that the player 16 swings the racket 18 in step S27. The swing flag SF is set (SF = 1).

  In this way, it is possible to detect whether or not the player 16 actually swings the racket 18 simply by photographing the optical marker 20 of the racket 18 with the cameras 22L and 22R and processing the camera signal. Therefore, the sensor used in the prior art becomes unnecessary, and it is not necessary to perform complicated signal processing. Therefore, it is possible to detect whether or not the player 16 swings the racket 18 with a simple configuration and simple processing.

  However, in the embodiment, two USB cameras 22L and 22R are used to increase detection accuracy, that is, to avoid detection errors as much as possible. However, only one USB camera may be used for racket swing detection.

  In the next step S29, the computer 42A determines whether or not the optical marker 20 (FIG. 3) has moved from right to left (toward the screen 12 in FIG. 1). That is, in step S21, the movement amount of the center of gravity of the optical marker 20 from the previous frame in the current frame is calculated. In this calculation process, has the center of gravity moved from right to left in the horizontal direction of the screen 12? , It is possible to determine whether it has moved from left to right. The fact that the light marker 20 has moved from right to left means that the racket 18 has moved from right to left. In this case, if the player 16 is shaking the racket 18 with the right hand, It can be determined as a forehand swing. Therefore, when “YES” is determined in the step S29, the computer 42A sets the forehand flag FHF (FHF = 1) in the subsequent step S31.

  If “NO” in the step S29, the computer 42A determines whether or not the light marker 20 (FIG. 3) has moved from the left to the right toward the screen 12 in the next step S33. Similar to step S29, step S33 can also be determined from the calculation result of step S21. For example, the fact that the optical marker 20 has moved from left to right means that the racket 18 has moved from left to right. In this case, if the racket is shaken with the right hand, the backhand swing is Can be judged. Therefore, when “YES” is determined in the step S33, the computer 42A sets the backhand flag BHF (BHF = 1) in the subsequent step S35.

  When “NO” is determined in the step S33, the computer 42A determines whether or not the optical marker 20 has moved from the top to the bottom of the screen 12 in the next step S37. As described above, the amount of movement of the center of gravity of the optical marker 20 from the previous frame is calculated in step S21. In this calculation process, whether or not the center of gravity has moved from top to bottom in the vertical direction of the screen 12 is also calculated. Can be determined. The fact that the optical marker 20 has moved from top to bottom means that the racket 18 has been swung down from top to bottom, regardless of whether it is swung with the right hand or with the left hand. It can be determined as a serve swing. Accordingly, when “YES” is determined in the step S37, the computer 42A sets the serve flag SVF (SVF = 1) in the subsequent step S39.

  There is no need to distinguish between right-handed and left-handed. That is, the forehand when the racket is shaken with the right hand becomes the backhand when the racket is shaken with the left hand, the backhand when the racket is shaken with the right hand becomes the forehand when the racket is shaken with the left hand, and the forehand in the former case The flag FHF is set. In the latter case, the backhand flag BHF is only set, and there is no need to change the subsequent processing.

  Thereafter, in step S209 (FIG. 16), it is determined whether or not the player swings based on the result of the previous swing determination. If it did not swing, it was missed and the process proceeds to the next step S217 (FIG. 19).

  When it is determined in step S209 that there is a swing at a predetermined timing, in the subsequent step S211 (FIG. 16), the computer 42A detects the body orientation of the player 16 (FIG. 1). This step S211 is shown in detail in FIG.

  In the first step S41 of FIG. 20, the computer 42A refers to the flag area of the internal memory 48 (FIG. 3) and determines whether or not the forehand flag FHF is “1”. If “YES”, in the next step S43, the computer 42A detects the left foot color marker 26L (not shown). That is, when it is determined that the racket swing direction by the player's right hand is a forehand, the front foot is a left foot, that is, the foot closer to the screen 12 is a left foot. The marker 26L (FIG. 1) is detected.

  For example, in the case of a forehand, assuming that the front foot, that is, the left foot is on the right side of the screen 12, the front of the body of the player 16 is considered to be directed rightward in that state. On the contrary, in the state where the left foot is on the left side in the forehand, the front of the body of the player 16 is considered to be directed leftward. In the state where the left foot is in the middle (center), it is considered that the body of the player 16 faces the screen 12.

  In the forehand, when the front of the body of the player 16 is directed to the right, in the actual tennis game, it is considered that the racket 18 has hit the ball with its right-facing posture. You may think that you are hit back to the right side of the opponent's court.

  On the other hand, when the front of the player 16 is facing left in the forehand, in an actual tennis game, it is considered that the racket 18 has hit the ball with a left-facing posture. You may think that you will be hit back to the left side of the opponent's court.

  When the body of the player 16 faces the screen 12 in the forehand, it may be considered that the ball is simply hit back to the center of the opponent's court in the actual tennis game.

  In this way, it is conceivable to change the direction of the ball that the player 16 strikes depending on whether the position of the left foot in the forehand or the case is right, left, or center. Therefore, in this embodiment, in the case of a forehand, in order to detect the position of the left foot, the left foot color marker 26L is detected in step S43.

  In order to detect the left foot color marker 26L, a subroutine shown in FIG. 21 is executed. That is, in the first step S51, the computer 42A takes in an image signal (second camera signal) from the USB camera 24 installed below the screen 12. In addition to the image of the left foot color marker 26L of the player 16, the image signal includes an image of the background of the player. Therefore, in the next step S53, the computer 42A masks an area other than the area where the left foot color marker 26L is considered to exist in order to remove such noise, and the masked area, that is, the area other than the left foot color marker existing area. The image signal in this area is excluded from the processing target to reduce the burden on the computer.

  In subsequent step S55, the computer 42A determines the hue value (0 to 255 as appropriate) set in the hue register (not shown) from the RGB signal of the image of the area exposed from the mask, that is, the left foot color marker existing area. A region (part) of a specific color indicated by (value) (in this embodiment, the left foot color marker 26L is yellow, so yellow) is extracted.

  In step S57, a binarization process is performed in order to detect an area of a color that seems to be a left marker from the specific color area acquired in step S55. In subsequent step S59, the computer 42A executes expansion processing.

  In step S61, the computer 42A performs area filter processing. That is, in this step S61, the computer 42A has a smaller area in the same or similar hue area as the left foot color marker 26L (FIG. 1) after binarization in step S57 and expansion processing in step S59. The region is determined as noise and removed, and a region having an area equal to or larger than a predetermined threshold is detected. And the area | region of the largest area among the area | regions of the area more than the threshold value is determined as a left foot color marker. After performing the area filtering process in step S61, the process returns to step S45 in FIG.

  In step S45 of FIG. 20, the barycentric coordinates are measured from the image signal after the area filtering process, as in step S29 (FIG. 5). The barycentric coordinate position is the position of the left foot color marker 26L, and in the next step S47, the detection result of that position (left, middle, right) is returned, and the process returns to step S213 (FIG. 16).

  If “NO” is determined in the step S41 in FIG. 20, the computer 42A determines whether or not the backhand flag BHF is “1” in a next step S49. When “YES” is determined in the step S49, the right foot color marker 26R (FIG. 1) is detected in the next step S63 by the same method as the left foot color marker is detected in the step S43 and FIG.

  That is, when it is determined that the direction of the racket swing by the player's right hand is a backhand, the front foot is a right foot, that is, the foot closer to the screen 12 is a right foot. The right foot color marker 26R is detected.

  For example, in the case of a backhand, assuming that the front foot, that is, the right foot is on the right side toward the screen 12, the body of the player 16 is considered to be on the right side with respect to the screen 12 in that state. On the contrary, in the state where the right foot is on the left side in the backhand, the body of the player 16 is considered to be on the left side toward the screen 12. In the state where the right foot is in the middle (center), the body of the player 16 is considered to be in the center toward the screen 12.

  In backhand, when the player 16 is on the right side, in the case of an actual tennis game, the ball may simply be hit back to the right side of the opponent's court.

  On the other hand, when the player 16's body is on the left side in the backhand, in an actual tennis game, it may be considered that the ball is hit back to the left side of the opponent's court.

  When the body of the player 16 is centered in the left-right direction on the screen 12 in the backhand, it may be considered that the ball is simply hit back to the center of the opponent's court in the actual tennis game.

  In this way, it is conceivable to change the direction of the ball that the player 16 strikes depending on whether the position of the right foot in the backhand is right, left, or center. Therefore, in this embodiment, in the case of backhand, in order to detect the position of the right foot, the right foot color marker 26R is detected in step S63.

  In order to detect the right foot color marker 26R, the subroutine shown in FIG. 21 is executed. Since the detection method is the same as that for the left color marker 26L, redundant description is omitted here. Thereafter, after executing steps S45 and S47, the detection result of the position (left, middle, right) of the right foot color marker 26R is returned, and the process returns to step S213 (FIG. 16).

  In step S213 of FIG. 16 returned from step S47 of FIG. 20, the computer 42A displays the racket figure 36 (FIG. 2) at the position of the forehand or the backhand determined in the swing determination step (S207) and the player 16 As a result of the hit ball, the ball figure 40 (FIG. 2) is returned in the direction of the position (right, left, center) determined in step S47.

  In this way, depending on whether it is a forehand or a backhand, by detecting the position of the front foot at that time with the camera, the return direction from the player 16 can be easily determined. Accordingly, complicated calculation processing such as the hitting angle between the racket and the ball and the trajectory calculation is not required.

  In the description of the embodiment, the cameras 22L and 22R for determining the swing of the racket 18 and the forehand or backhand determination and the camera 24 for detecting the positions of the color markers 26L and 26R are separately prepared. However, the racket swing and the position of the front foot may be detected or determined by the camera signal of one camera. However, even in this case, for convenience, a camera for swing determination, forehand or backhand determination is referred to as a first camera, and a camera for forefoot position detection is referred to as a second camera.

  The color markers 26L and 26R are attached to the ankle with a color band, but can be arbitrarily modified. For example, it functions as a color marker even if the shoe color of the player is different between the left and right.

  Further, in the above-described embodiment, the optical marker 20 is provided on the racket 18 and is photographed by the first camera 22 to detect the swing of the racket 18 and its direction. However, the light marker 20 may be replaced with a color marker similar to the player's foot. For example, it is possible to determine the swing of the racket 18 and distinguish the forehand or the backhand by providing color sheets on both sides of the gut of the racket 18 and photographing the color sheets with a camera.

  In the next step S215 in FIG. 16, the computer 42A transmits the return ball data [1,0, X, Y, X, Vx ′, Vy ′, Vz ′] as a result of the player's swing to the opponent computer 42B. . However, this transmission data shows a case where the player has made a forehand swing, but if the player is a backhand, the second number is changed to “1”.

  Thereafter, in step S217 in FIG. 19, it is determined whether or not the ball graphic 40 (FIG. 2) is in contact with the court ground. If “YES”, the ball graphic in contact with the court ground is returned from the opponent player in step S219. It is determined whether it is the ball that has been played or the ball that the player has returned. When “NO” is determined in these steps S217 or S219, the process returns to step S149 of FIG. 6 in the same manner as when “NO” is determined in step S179 of FIG.

  If “YES” is determined in the step S219, that is, if the opponent's return ball comes into contact with the court ground, in the next step S221, the computer 42A determines whether or not the bound number is “1”. If the number of bounds is “1”, it is determined in the next step S223 whether or not it has fallen out of the court. However, this determination of out / in is made only within the court on its own side, and the partner computer 42B judges within the opponent's court.

  If it falls within the court, it is a mistake of the player, so the process returns to step S149 in FIG. 6, but if it falls off the court, it is a mistake of the opponent player, so in the next step S225, a score is given to the player. In step S227, data [3, 1] is transmitted to the partner computer 42B. That is, the data of Option = 0 is transmitted with Type = 3 so as to indicate the point on its own side.

  If the number of bounces is not “1”, it is a mistake of the player in the next step S229. Therefore, in the next step S231, a score is added to the opponent, and the data [3, 0] is recorded in the opponent computer in step S233. To 42B. That is, the data of Option = 1 is transmitted with Type = 3 so as to indicate the point of the other party.

  After step S227 or S233, “NG” is set to the start determination flag in step S235 to initialize the game, and then the process returns to step S149 in FIG.

  In the communication battle type VR tennis game system 100, data delay and packet loss may occur depending on the state of the network 54 (FIG. 4). However, according to the experiment of the inventors, in the system of the embodiment, it is 200 ms or less. If it is a delay, it can be played without a sense of incongruity, and this condition can be sufficiently cleared in the case of a general high-speed line.

  However, when packet loss occurs, transmission / reception of important data may be hindered. For this reason, in order to reliably perform transmission / reception, guaranteed data transmission is normally performed. This is a method of retransmitting data until it is notified that the data has been received from the receiving side during data transmission. However, if data retransmission is repeated, a lot of time is required until the data is transferred. Therefore, in order to avoid delay due to packet loss, the system 100 according to the embodiment repeats data transmission without guarantee a plurality of times. There is no certainty in data transmission without guarantee, but high-speed communication is possible because there is no notification from the other party. In order to improve the certainty, a plurality of the same data is continuously transmitted in a short time, and if one of the plurality is received, it is determined that the data has been received.

  Furthermore, in the VR tennis game system 100 of this embodiment, the player who is actually shaking the racket is photographed by the camera, and the moving image data is transmitted to the computer 42B of the opponent player connected to the network. At this time, an image obtained by extracting only the player by image processing (background difference method or the like) is transmitted. The computer on the opponent player side synthesizes and draws the received moving image data on the VR tennis game screen. That is, a player who is actually playing tennis in a remote place is displayed on the opponent's court on the tennis court made by the computer.

  Further, in the system 100 of this embodiment, a window is provided on the game screen so that the expression of the opponent player can be confirmed, and only the player's face up is displayed there. At this time, the position of the face is automatically detected by image processing.

  In an environment where the network is not high-speed, even if moving image data is transmitted in real time, the amount of moving image data is large, so that communication data is lost, and smooth moving image reproduction is difficult. In addition, if large-capacity data communication is performed during a game, communication battle data may be lost. Therefore, a plurality of patterns (fore, back, etc.) of a video that is swinging in advance at site A (your site) are shot, and the swing pattern is recorded on computer 42B at site B (the other site) before the game starts, or In the initial stage of the game, for example, before the return ball data is transmitted from the opponent computer 42B, the data is transmitted in the background so that no data is lost. The swing pattern is stored in the computer at the site B, and the video on which the player at the site A is swinging is reproduced on the game screen on the site B side in accordance with the return ball of the opponent player. Thereby, the communication load during the game can be removed.

  In the above-described embodiment, the player face image is transmitted together only when the player moving image is transmitted in real time. However, as described above, even in the embodiment in which the player moving image is stored in the storage unit in advance, only the player face image may be transmitted in real time. If only the player face image is used, the amount of data does not increase so much, and even a relatively low-speed communication line does not hinder the progress of the game. If the player face image is transmitted in real time, the facial expression of the opponent player at that time can be seen, and the sense of reality, reality, and the fun of the game are further improved.

FIG. 1 is an illustrative view showing a VR tennis game apparatus that can be used in one embodiment of the present invention. FIG. 2 is an illustrative view showing a display example of a game screen displayed on the VR tennis game apparatus of FIG. FIG. 3 is a block diagram showing an electrical configuration of the VR tennis game apparatus of FIG. FIG. 4 is an illustrative view showing one embodiment of the present invention. FIG. 5 is a flowchart showing the operation of the own computer in this embodiment. FIG. 6 is a flowchart showing the operation following FIG. FIG. 7 is a flowchart showing the subroutine of step S129 of FIG. FIG. 8 is an illustrative view showing a background image taken in the operation of FIG. FIG. 9 is an illustrative view showing a background image and a player image taken in the operation of FIG. FIG. 10 is a flowchart showing the subroutine of step S149 of FIG. FIG. 11 is an illustrative view showing a background image taken in the operation of FIG. FIG. 12 is an illustrative view showing a background image and a player image taken in the operation of FIG. FIG. 13 is an illustrative view showing a face image extracted in the operation of FIG. 9. FIG. 14 is a flowchart showing the subroutine of step S163 in FIG. FIG. 15 is an illustrative view showing a state in which the opponent player image and the face image are combined and displayed on the game screen in the operation of FIG. FIG. 16 is a flowchart showing the operation of the own computer subsequent to FIG. FIG. 17 is a flowchart showing the subroutine of step S193 in FIG. FIG. 17 is a flowchart showing the subroutine of step S207 of FIG. FIG. 19 is a flowchart showing the operation of the own computer subsequent to FIG. FIG. 20 is a flowchart showing the subroutine of step S211 of FIG. FIG. 21 is a flowchart showing the subroutine of step S43 in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... VR type tennis game system 10 ... VR tennis game device 12 ... Screen 14 ... Projector 16 ... Player 18 ... Racket 20 ... Light marker 22L, 22R, 24 ... USB camera 42 ... Computer 48 ... Internal memory

Claims (7)

A virtual reality tennis game device is installed at each of the two sites, and each virtual reality tennis game device is provided with a computer and display means for displaying a game screen in accordance with image processing by the computer, and the player displays on the display means. A virtual reality reality tennis game system in which a game is progressed by swinging a racket in real space so as to hit a ball figure, and the computers of the virtual reality tennis game devices are connected to each other via a communication line Because
Each virtual reality tennis game device is provided with a camera, and the camera is used to photograph the swing motion of the player,
The computer creates return ball data including at least ball position data and ball speed data based on a camera image obtained by shooting the player's swing motion with the camera, and transmits the return ball data to the opponent computer.
The communication battle type virtual reality tennis game system, wherein the computer further calculates a trajectory of the ball figure based on return ball data transmitted from the opponent computer and displays the trajectory on the display means.   The communication battle type virtual reality tennis game system according to claim 1, wherein the computer displays a court figure fixedly on the display means and displays a moving image of an opponent player superimposed on the court figure. Each of the virtual reality tennis game devices further includes a storage unit, and the moving image of the opponent player is accumulated in the storage unit,
3. The communication battle type virtual reality tennis game system according to claim 2, wherein when the return ball data from the opponent computer is received, the computer reads out the moving image of the opponent player from the storage means and displays it on the display means.   The player's swing motion is photographed by the camera, thereby creating a moving image of the player, and before the game starts or after the game starts, the moving image of the player is displayed until the return ball data is transmitted from the opponent computer. 4. The communication battle type virtual reality tennis game system according to claim 3, wherein a moving image of the player is stored in the storage means in advance in each virtual reality tennis game device.   The communication battle type virtual reality tennis game system according to claim 2, wherein the swing motion of the player is photographed by the camera, thereby creating a moving image of the player, and transmitting the moving image of the player to the opponent computer in real time.   The face image of the player is photographed by the camera, and the face image is transmitted to the opponent computer in real time, whereby the face image together with the moving image of the player is displayed on the display means in each virtual reality tennis game device. The communication battle type virtual reality tennis game system according to any one of claims 2 to 5. A virtual reality tennis game device is installed at each of the two sites, and each virtual reality tennis game device is provided with a computer and display means for displaying a game screen in accordance with image processing by the computer, and the player displays on the display means. A virtual reality reality tennis game system in which a game is progressed by swinging a racket in real space so as to hit a ball figure, and the computers of the virtual reality tennis game devices are connected to each other via a communication line A control method in which
Each virtual reality tennis game device is provided with a camera, and the camera is used to capture a moving image of the player,
Before the game starts or at the beginning of the game, the moving image of the player is transmitted to the opponent computer, whereby the moving image of the player is stored in advance in the storage means in each virtual reality tennis game device,
In each of the virtual reality tennis game apparatuses, the computer reads out the moving images of the player stored in the storage means as necessary during the game and displays them on the display means.

Images