JP3451343B2 - Signal generation apparatus and signal generation method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a video game , for example.
In etc., generates a signal in response to a user operation corresponding to the screen to be displayed on a television or a liquid crystal display device
And a signal generation method .

[0002]

2. Description of the Related Art Conventionally, there is known a video game or the like in which an image on a display screen such as a television or a liquid crystal display is changed by operating a pointing device such as a mouse or a joystick. In such a conventional technique, it is possible to control the operation of the display image through an operation by a user's finger or hand.

On the other hand, there is also known a device that uses a so-called virtual reality technique to generate an effect as if the user were in the virtual space shown on the display screen based on the movement of a glove or the like worn by the user. Has been.

[0004]

However, in the prior art in which the operation of a display image is controlled by operating a pointing device such as a mouse or a joystick,
The actions that the user can perform are limited to operations with a finger on a mouse or a joystick.
For this reason, there is a problem that, for example, in a sports game or the like, only a game in which the player feels that he or she is actually playing is not expected.

Further, in the prior art for realizing virtual reality, it is possible to obtain the virtual reality that the user exists in the virtual space, but unless the user wears electronic equipment such as gloves or a three-dimensional stereoscope. In addition, there is a problem in that it is not possible to eliminate the discomfort caused by mounting them. In addition, these instruments have a problem that the cost of the device is increased.

An object of the present invention is to make it possible to control a display screen according to a user's operation without mounting or operating a special electronic device.

[0007]

Means for Solving the ProblemsSignal generator
Was first imagedBased on the color signal of the screen,
Extraction means for extracting the image area of interest (image recognition in FIG.
Knowledge circuit 508)Have. This imageArea andIs for example
Obtained by sequentially capturing the motion of the user with a bat etc.
It is something.

Next, the image extracted by the extraction means
First storage means (image recognition circuit 50 that stores a processing screen map) that stores the existence range of the image region on the imaging screen.
8 etc.).

Next, the image stored in the first storage means
A center of gravity calculating means for calculating the center of gravity position of the image area from the existence range of the area (step 1011 in FIG. 10 , and
The image recognition circuit 508 for executing the operation flowchart of the center of gravity calculation processing of FIG .

It also has a second storage means (application ROM 514, DRAM 507, etc. in FIG. 5 ) for storing object screen data to be output . Here, a plurality of object screen data can be stored.

[0011] Then, O stored in said second storage means
Display position specifying means (CPU 509 in FIG. 5) for specifying the position of the object screen data to be output on the display screen .
Etc.). This means also includes a plurality of predetermined object screen data stored in the second storage means.
It may be configured to specify at least one type of data.

Furthermore, there range stored in the specified location and the first storage means by the display position specifying means
Judge whether the positional relationship with
Disconnection means (step 1006 in FIG. 10 is executed)
Image recognition circuit 508). Where the prescribed article
The case is, for example, specified by the display position specifying means.
Position and existence range of image area stored in first storage means
It is whether or not and overlap.

In addition , the display position finger is determined by the determination means.
Stored in the first storage means and the position designated by the setting means
The positional relationship with the existing range that meets the
If it is determined that the
The center of gravity of the object is calculated, and the center of gravity calculation means
The calculated center of gravity of the image area and the center of gravity of the object
The screen to be displayed next based on the positional relationship with the
Signal generating means for generating a corresponding signal (CP in FIG. 5
U509 etc.) .

The signal generator of the present invention having the above configuration
Specified by the display position designating means in the computer
Position and existence range of image area stored in first storage means
Based on the
When a signal corresponding to the combined display screen is generated,
Generated signal generating means (video signal input / output circuit 5 in FIG. 5)
03, CPU 509, etc.)
It Also, the output procedure of the signal corresponding to the screen that can be displayed
Is further provided with a procedure storage means that stores
Means is the center of gravity of the image area calculated by the center of gravity calculating means
Positional relationship between the position and the calculated center of gravity of the object
Corresponding to the person in charge
Read the procedure of the signal corresponding to the screen
Is also possible. Furthermore, based on the color signal of the imaged screen
Signal output means for outputting a chroma key signal (see the video
Chroma key circuit 706 in signal conversion circuit 505
Etc.) is further provided so that the extraction means can output the signal.
To recognize the chromakey signal output by the stage
It is also possible to extract more specific image areas.
Noh.

In the above-mentioned configuration of the invention, the second storage means or the procedure storage means can be constructed in the form of, for example, an application cartridge which can be detached from the main body of the signal generating device. Further, the signal generation method of the present invention
Method is based on the color signal of the imaged screen,
Extraction step for extracting the image area (processing screen
Step 1 in the operation flowchart of the backup creation process
202 etc.) and the image area extracted in this extraction step.
The range of existence of the image area on the imaging screen in the memory
Storage step (operation of processing screen map creation processing of FIG. 13
Step 1219, etc. in the flowchart)
From the existence range of the image area stored in the storage step,
Center of gravity calculation step for calculating the center of gravity position of this image area
(See the overall operation flowchart of the screen recognition circuit in FIG.
Step 1011 of moving the center of gravity calculation process of FIG.
Flowchart) and objects previously stored in memory
Specify the position of the output screen data to be output to the display screen
Step for specifying display position (operation of CPU processing in FIG. 16
Step 1606, etc. in the flow chart)
The position specified in the display position specification step and the above memory switch
The positional relationship with the existing range stored in
Judgment step for judging whether or not the condition is satisfied (Fig. 1
0 in the overall operation flowchart of the screen recognition circuit
Step 1006) and the above display in this judgment step.
The position specified in the position specification step and the memory step
The positional relationship with the existing range stored in
When it is judged that the condition is satisfied, the position is displayed at the specified position above.
Calculate the position of the center of gravity of the indicated object and calculate the center of gravity
The center of gravity of the image area calculated in the step
Based on the positional relationship with the center of gravity of the object,
Signal generation that generates the signal corresponding to the screen to be displayed
Step (Operation flow chart of collision detection process in FIG. 14)
And the operation flow chart of the collision flag interrupt processing of FIG.
Etc.) and In the signal generating method of the present invention
The chroma key signal based on the captured color signal of the screen.
A signal output step for outputting (of the screen recognition circuit of FIG.
Step 1003) in the overall operation flowchart
In addition, the extraction step described above includes
To recognize the chroma key signal output at the step
Yes it is possible to extract more image areas of specific conditions
Noh.

[0016]

In the present invention, a specific condition on a screen obtained by capturing an actual scene of a user swinging a bat
Existence range of image area and display of object screen data
The positional relationship with the position on the screen does not meet the specified conditions.
The center of gravity of the image area and the object
Based on the positional relationship with the center of gravity position of
A signal corresponding to the screen is generated. Thus, the user can participate in the video game displayed by the CG image or the like by performing a motion of swinging the bat as if he / she actually played the game.

In this case, in particular, the bat is displayed from the imaged screen.
When extracting the specific image region such as, extracted as predetermined color image region due to chroma key, further, by calculating the center of gravity of that, it is possible to accurately grasp the behavior of the user.

[0018]

Embodiments of the present invention will now be described in detail with reference to the drawings. <External Appearance of User's Operation in Example and Operating Principle of Example> Before describing the configuration and specific operation of the example, the external appearance of the user's operation in the example and the operating principle of the example will be briefly described. To do.

In this embodiment, the present invention is applied to a baseball video game. As shown in FIG. 1, the user holds the bat 101 painted in a predetermined color, for example, blue, and the C thrown by the pitcher is displayed on the display device 103 as a CG (computer graphics) image 104.
The bat 101 is swung toward the ball 105 by G.

The camera 102 captures a scene including a user holding the bat 101. By extracting the chroma key signal relating to the above-described predetermined color from the captured image, a coordinate map 202 of the bat 101 as shown in FIG. 2 is created, and further, the coordinate map 202.
The barycentric coordinates of the bat 101 (marked with X in the figure) above are detected. Then, the positional relationship 203 between the coordinate map 202 and the barycentric coordinates of the bat 101 and the coordinates 201 of the ball 105 and the barycentric coordinates (X mark in the drawing) of the CG as shown in FIG. By doing so, the presence or absence of collision of the ball 105 with the bat 101 and the CG, and the positional relationship between the barycentric coordinates of the bat 101 and the barycentric coordinate of the ball 105 with the CG when the collision occurs are extracted.

As shown in FIG. 2, the bat 101 has a C
When it is determined that the ball 105 by G hits, the center of gravity of the bat 101 and the ball 105 by CG
If the deviation from the barycentric coordinates of is within a predetermined allowable range, for example, it is determined that the user has hit a home run and CG
The image 104 changes to an image corresponding to a home run.

Alternatively, as shown in FIG. 3, the bat 1
When it is determined that the ball 105 by CG hits 01, and the center of gravity coordinates of the ball 105 by CG deviate below the center of gravity coordinates of the bat 101, for example, it is determined that the user has hit the ground, and the CG image 104 Changes to an image that corresponds to Goro.

Further, as shown in FIG.
When it is determined that the CG ball 105 hits 1, the CG image 104 is determined to be hit by a user, for example, when the CG ball 105 gravity center coordinate is deviated above the bat 101 gravity center coordinate. Changes to the image corresponding to the fly.

As described above, in the embodiment, the user can participate in the video game displayed by the CG image 104 by swinging the bat 101 as if he / she actually played. <Structure of Embodiment> A specific structure of the embodiment for realizing the above-described operation will be described below.

FIG. 5 is an overall configuration diagram of an embodiment of the present invention. First, the clock driver 502 directly controls the operation of the CCD 501 provided in the camera 102 of FIG. 1 based on the clock input from the oscillator (OSC) 504 via the video signal input / output circuit 503, and at the same time, HSYNC. Signal and a sync signal 519 such as a VSYNC signal are generated, and are generated by the video signal input / output circuit 503,
VDP 506, image recognition circuit 508, and CPU 509
Supply to. Therefore, these circuits are
, They can operate in synchronization with each other.

The CCD output signal 518 output from the CCD 501 is input to the video signal input / output circuit 503. The video signal input / output circuit 503 outputs the CCD output signal 5
18 is converted into a digital composite video input signal 520, and a digital composite video output signal 521 from the video signal conversion circuit 505 is converted into an analog composite video output signal 522.

The configuration of the video signal input / output circuit 503 is shown in FIG.
Shown in. In FIG. 6, C from the CCD 501 (FIG. 5)
The CD output signal 518 is analogly sampled by the sampling circuit 601.

The amplitude level of the output of the sampling circuit 601 is adjusted by an automatic gain adjustment circuit (AGC) 602. The output of the AGC 602 is the γ correction circuit 60.
3, the signal level with respect to the power of light has a predetermined γ
Converted to be characteristic.

The output of the γ correction circuit 603 is an 8-bit A /
It is converted into an 8-bit digital composite video input signal 520 by the D converter 604. The digital composite video input signal 520 is output to the video signal conversion circuit 505 shown in FIG.

On the other hand, the 8-bit digital composite video output signal 521 output from the video signal conversion circuit 505 is converted into an analog composite video output signal 522 by the 8-bit D / A converter 605. This analog composite video output signal 522 is shown in FIG.
Output to the display device 103. As a result, the display device 1
03 displays the CG image 104 corresponding to the analog composite video output signal 522.

Returning to FIG. 5, the video signal conversion circuit 505.
Generates a chroma key signal 523 from the digital composite video input signal 520 and outputs it to the image recognition circuit 508.
Output to the video signal input / output circuit 5 based on the superimposing signal 525 from the image recognition circuit 508.
03 corresponding to the digital composite video output signal 521 or the digital RGB output from the video display processor (VDP) 506.
An operation of selecting something in the signal 524 and outputting the digital composite video output signal 521 corresponding to the selected color signal is performed.

The configuration of the video signal conversion circuit 505 is shown in FIG. The color separation filter 701 inputs the digital composite video input signal 520 input from the video signal input / output circuit 503 (FIG. 5) and G corresponding to the signal.
It outputs three types of color signals: a (green) signal, a Cy (cyan) signal, and a Ye (yellow) signal.

The contour correction circuit 702 inputs the above-mentioned three types of color signals, and performs processing for correcting the contour of the image determined by these color signals. Of the outputs of the contour correction circuit 702, signals other than the G (green) signal are input to the white balance circuit 703. The white balance circuit 703 adjusts the white balance of the input signal.

The output of the white balance circuit 703 is input to the selection filter 704. Selection filter 70
4 outputs an R (red) signal and a B (blue) signal corresponding to the input.

The matrix circuit 705 inputs the above-mentioned R (red) signal, B (blue) signal and the G (green) signal output from the contour correction circuit 702, and converts them into the following equations. The luminance signal Y and the color difference signals R-Y and B-Y are output by executing the calculation corresponding to. Y = (8/27) R + (16/27) G + (1 /
9) B R-Y = (15/27) R- (16/27) G- (
1/9) B B-Y =-(8/27) R- (16/27) G + (
In the 8/9) B chroma key circuit 706, as shown in FIG. 9, the color difference signal R-Y changes from the minimum value R-Ymin to the maximum value R-Ym.
When the value of the color difference signal BY is in the range of ax and the value of the color difference signal BY is in the range of minimum value B-Ymin to maximum value B-Ymax, the chroma key signal 52 having the logical value "1" is obtained.
3 is output. The chroma key signal 523 is, for example,
When a color signal showing bright blue corresponding to the bat 101 in FIG. 1 is input, it shows a logical value “1”. By using this chroma key signal 523, the image recognition circuit 508 (FIG. 5), which will be described later, causes the bat 101 shaken by the user.
Can be detected, and it can be determined whether or not the bat 101 and the CG ball 105 collide.

Next, the matrix circuit 708 outputs VDP5
The digital RGB signal 524 from 06 is input, and the luminance signal Y and the color difference signals RY and BY corresponding to them are output.

The selection circuit 707 is an image recognition circuit 508.
Based on the superimpose signal 525 from (FIG. 5), the output of either the matrix circuit 705 or 708 is selected and output to the modulation circuit 709. Modulation circuit 709
Generates and outputs a digital composite video output signal 521 corresponding to the input. In this embodiment, the superimpose signal 525 from the image recognition circuit 508 (FIG. 5) is used.
Is designed so that the selection circuit 707 always selects the output of the matrix circuit 708. As a result, the CG image 104 corresponding to the digital RGB signal 524 generated by the VDP 506 is always displayed on the display device 103 in FIG. However, by controlling the superimpose signal 525, a real image corresponding to the digital composite video input signal 520 generated based on the image pickup result of the camera 102 and a CG image generated by the VDP 506 are combined, and the combination is performed. The displayed image can be displayed on the display device 103.

Returning to FIG. 5, the video display processor (VDP) 506 stores the object screen data and the background screen data forming the CG image 104 in the DRAM 507 at any time based on the designation from the CPU 509, and the designation from the CPU 509. Based on the above, the data is arranged at the designated coordinate position, and the digital RGB signal 524 corresponding to the arrangement result is generated and output to the video signal conversion circuit 505. As a result, the video signal conversion circuit 505 generates the digital composite video output signal 521 corresponding to the digital RGB signal 524, as described above, and the video signal input / output circuit 50.
3 is the digital composite video output signal 521
Analog composite video output signal 522
To generate CG on the display device 103 (FIG. 1).
The image 104 will be displayed.

The structure of the VDP 506 is shown in FIG. CPU
Object screen data and background screen data transferred from the bus 509 on the bus 526 are stored in the DRAM 507 (FIG. 5) via the CPU interface circuit 801 and the DRAM control circuit 802.

The object table data transferred from the CPU 509 on the bus 526 is transferred to the object table RAM 80 via the CPU interface circuit 801 and the object control circuit 803.
4 is stored. This object table data is
9 is a table showing a correspondence table between each object screen data stored in a DRAM 507 and an ID number corresponding to each. When designating the display position of each object screen data on the CG image 104 (FIG. 1) over time, the CPU 509 designates the ID number corresponding to each object screen data and each display position. As a result, the object control circuit 803 reads the object screen data corresponding to the designated ID number from the DRAM 507 via the DRAM control circuit 802, and the line buffer RAM 8
After being expanded to 08, the contents are output to the RGB color difference data processing circuit 806. Further, the object control circuit 803 notifies the CPU 509 from the CPU interface circuit 801 via the bus 526 of the center-of-gravity position of the object showing the ball 105 (FIG. 1) currently displayed, based on the designation from the CPU 509.

On the other hand, the background control circuit 8
Reference numeral 05 denotes the background screen data designated by the CPU 509 via the RAM control circuit 802.
The data is read from the RAM 507, the display position is controlled or the scroll is controlled for the data, and then the data is output to the RGB color difference data processing circuit 806.

The RGB color difference data processing circuit 806 refers to the color lookup table in the color lookup table RAM 807 to refer to the object screen data designated by the object control circuit 803 and the background designated by the background control circuit 805. The digital RGB signals 524 corresponding to the ground screen data are generated and output to the video signal conversion circuit 505 (FIG. 5). In this case, the RGB color difference data processing circuit 806 also performs control for determining whether to display the object screen data or the background screen data for each display position. And RGB
The color difference data processing circuit 806 sets the logical value of the object screen data output signal YSOBJ (hereinafter referred to as YSOBJ signal) to “1” when outputting the object screen data, and when outputting the background screen data. , The logical value of the background screen data output signal YSBG (hereinafter referred to as YSBG signal) is set to "1". The YSOBJ signal and the YSBG signal are output to the image recognition circuit 508.

Returning to FIG. 5, the image recognition circuit 508 outputs the chroma key signal 523 for one frame, which is input from the video signal conversion circuit 505, in two internal frames for each frame period determined in synchronization with the synchronization signal 519. Alternately hold in the chroma key signal buffer and at the same time R in VDP506
A YSOBJ signal for one screen map (described later) output from the GB color difference data processing circuit 806 is converted into two Y
It is held alternately in the SOBJ signal buffer. At the same time, the image recognition circuit 508 causes the user to shake the bat 101 (FIG. 1) based on the one-frame chroma key signal 523 held in one of the two chroma key signal buffers in the previous frame cycle. Create a screen map showing the location. Then, the image recognition circuit 508
In the process of this process, if it is determined that the bat 101 is present at the block position (described later) currently being processed on the screen map, it is held in one of the two YSOBJ signal buffers in the previous frame cycle. Was done 1
In the YSOBJ signal for the screen map, whether the logical value of the YSOBJ signal corresponding to the block position is “1”, that is, the ball 105 (FIG. 1) at the current block position where it is determined that the bat 101 is present. It is determined whether or not the object screen data indicating is present. In this way, the image recognition circuit 508 detects whether or not there is a collision between the bat 101 and the ball 105. Further, the image recognition circuit 508 calculates the position of the center of gravity of the bat 101 on the screen map each time the screen map for one screen is completed, stores the calculation result in the DRAM 510 via the bus 526, and at the same time, the collision state. When the flag is detected, a collision flag is set as an interrupt flag to the CPU 509.

The CPU 509 operates according to the operating system stored in the OS ROM 511, and in particular, the application ROM 514 in the application cartridge 513 inserted in a slot (not shown).
A video game such as a baseball game described in this embodiment is executed by loading an application program from (# 1 or # 2) into the program area of the DRAM 510. In this case, the CPU 509
C, from the application ROM 514 in the application cartridge 513 with the passage of time while processing the key scan signal 527 input from a start switch and a stop switch (not shown).
The object screen data and the background screen data that form the G image 104 (FIG. 1) are sequentially read,
Within the vertical blanking period synchronized with the synchronization signal 519,
Those data are transferred from the bus 526 to the DRAM 507 and the like via the VDP 506.

At the same time, the CPU 509 changes the display position on the CG image 104 (FIG. 1) of each object screen data transferred to the DRAM 507 over time by VDP.
506 are sequentially designated.

Further, the CPU 509 is the image recognition circuit 50.
Collision flag interruption processing is executed at the timing when the collision flag indicating that the collision state 8 has been detected is set.
In this interrupt processing, the CPU 509 causes the DRAM 5
10, the center of gravity of the bat 101 on the screen map calculated by the image recognition circuit 508 stored in
The positional relationship with the center of gravity of the object showing the currently displayed ball 105 (FIG. 1) acquired from the object control circuit 803 therein is determined. Then, based on the determination result, the CPU 509 executes necessary image processing.

Further, the CPU 509 causes the tone generator circuit 515 to generate a tone signal by sequentially transferring necessary data to the tone generator circuit 515 in the application cartridge 513. The CPU 509 also causes the tone generator circuit 516 in the main body to generate a musical tone signal as necessary. The tone signal generated by the tone generator circuit 515 is sent to the amplifier 517 via the CPU 509 and the tone generator circuit 516, amplified here, and then output as a sound output signal 528 to a speaker (not shown).

In addition, the CPU 509 also performs an inspection process as to whether or not the application cartridge 513 has been imitated. <Specific Operation of Embodiment> The specific operation of the embodiment having the above-described configuration will be sequentially described below.

FIG. 10 is an overall operation flowchart realized by the image recognition circuit 508 as an operation for executing a program stored in a memory (not shown). The image recognition circuit 508 creates a screen map showing the position of the bat 101 (FIG. 1) swung by the user based on the chroma key signal 523 from the video signal conversion circuit 505, and based on this, supplies it as a YSOBJ signal from the VDP 506. The presence or absence of a collision between the bat 101 and the object screen data showing the ball 105 of FIG. 1 is detected.

Here, as shown in FIG. 26, a frame which is an image pickup unit for one screen in the CCD 501 of FIG. 5 has a structure of 576 columns in the horizontal direction × 192 rows in the vertical direction. Therefore, the chroma key signal 523 output from the video signal conversion circuit 505 is a signal indicating a logical value of 576 × 192 pixels. Then, in order to reduce the processing load, the image recognition circuit 508 causes the horizontal direction 6 in the chroma key signal 523 as shown in FIG.
A process of compressing a logical value of 12 pixels in total consisting of pixels × 2 pixels in the vertical direction into one logical value is performed. A unit in which this logical value is stored is called a block. As a result, as shown in FIG. 26, 576 in the chroma key signal 523.
The logical value of × 192 pixels is compressed into the logical value of a block of 96 columns in the horizontal direction and 96 rows in the vertical direction.
A screen map relating to the chroma key is defined by such a set of logical values for 96 × 96 blocks.

On the other hand, the object screen data and the background screen data specified by the YSOBJ signal and the YSBG signal are represented by a resolution of 96 columns in the horizontal direction and 96 rows in the vertical direction. Therefore, the VDP 506 converts them into these screen data. Based on the digital RGB signal 52
4 is generated, these screen data are expanded according to the display resolution of the display device 103 (FIG. 1).

In FIG. 10, first, step 1001
A RAM (not shown) in the image recognition circuit 508
The contents (see FIG. 27 described later) are initialized. Next, after the power is turned on, when it is determined in step 1002 that the initial screen map has not been created, the initial screen map creation process is executed in step 1003.

The initial screen map is the camera 102 of FIG.
It is a screen map regarding the chroma key about the scene where the user with the bat 101 does not exist captured by. The purpose of creating the initial screen map is that when a chroma key corresponding to the bat 101 is detected as a processing screen map to be described later, if an object of the same color as the bat 101 exists in the background, the background of the bat 101 is incorrect. In order to prevent it from being detected as
The position of the background object having the same color as the bat 101 is detected in advance. In the processing screen map creation processing of step 1005, which will be described later, a processing screen map is created for each frame cycle based on the result of the scene in which the camera 102 of FIG. In the case where the position recognized as the part related to the chroma key on the processing screen map is recognized at the same time as the position where the background object having the same color as the bat 101 exists on the initial screen map, Will be processed as if the bat 101 does not exist.

As a result of execution of the initial screen map creation processing in step 1003, an initial screen having a size of 96 × 96 blocks in the data format shown in FIG. 27 is stored in the RAM (not shown) in the image recognition circuit 508. A map is created. Since this RAM stores data with 16 bits as one word, the logical value for one row is stored as data of 6 words (= 96/16). Therefore, in RAM, the initial screen map is 6 × 9.
Occupies a data area for 6 words.

FIG. 11 shows a detailed operation flowchart of the initial screen map creation processing in step 1003. In the following process, in order to stably obtain the initial screen map,
Chroma key signal 523 for multiple frames (for multiple screens)
Is repeatedly executed, and the logical sum (OR) of the logical values obtained in each iterative process is calculated for each block position on the screen map to determine the final logical value for each block position. For that, step 1
In step 101, the number of times of sampling, which is the number of times the process is repeated, is set, and it is determined that the value of the register n has reached the set number of times, every time the processing for one frame ends with YES in step 1114 described later. Until step 11
At 16, the value of register n is incremented by +1
The processing for each frame shown in steps 1102 to 1114 is executed.

Here, although not specifically shown, as described above, the image recognition circuit 508 has two frames (two screens) of chroma key signal buffers each having the size shown in FIG. 26 (a). Therefore, for each frame period determined in synchronization with the sync signal 519 (FIG. 5), the chroma key signal 523 for one frame shown in FIG. The two chroma-key signal buffers are alternately held. And
The processing for each frame shown in steps 1102 to 1114 of FIG.
This is executed for one frame of chroma key signal 523 held in one of the one chroma key signal buffer. At the same time, the chroma key signal 523 corresponding to the current frame period is sequentially input to the other chroma key signal buffer. Then, step 111
Each time the determination of 4 is NO, the chroma key signal buffer to be processed is switched.

One chroma key signal buffer having the size shown in FIG. 26 (a) is a 1-bit buffer for storing the logical value of "0" or "1" of the chroma key signal 523 corresponding to each pixel. 1 if it can store data
Bit x 576 x 192 = 110592 Bit = 691
It only has to have a storage capacity of 2 words.

Next, steps 1102 to 111
The details of the processing for each frame shown by 4 will be described. First, in step 1102, the value of the register X indicating the horizontal column position and the value of the register Y indicating the vertical row position of the generated initial screen map are reset to 0, and then the value of the register X is determined in step 1113. Is cleared and the value of the register Y is incremented by +1 until the value of the register Y reaches 96 (exceeds 95) in step 1114.
3 to step 1112 → step 1113 → step 1
The process of 114 → step 1103 is repeatedly executed. In this way, the processing for 96 lines in the initial screen map is executed.

Further, while the value of the register X is incremented by +1 in step 1111, step 111
Until it is determined that the value of the register X reaches 96 (exceeds 95) in step 2, steps 1103 to 1112
→ The processing of step 1103 is repeatedly executed. In this way, the processing for 96 blocks in one line in the initial screen map is executed.

Then, steps 1103 to 1110 executed for each of the current values of the registers X and Y are the processing for one block in the initial screen map. In the following description, the current values of the registers X and Y are simply described as X and Y, and the block specified by the current values of X and Y is referred to as a processing target block.

In step 1103, the chroma key signals 523 at the column positions (6X + 1) to (6X + 6) in the horizontal direction and the row positions (2Y + 1) and (2Y + 2) in the vertical direction are output from the chroma key signal buffer to be processed. Each logical value is read. For example, X = 0, Y
If = 0, the data (block 1 in FIG. 26A) at column positions 1 to 6 in the horizontal direction and row positions 1 and 2 in the vertical direction are read. If X = 1 and Y = 0, the data (block 2 in FIG. 26A) at column positions 7 to 12 in the horizontal direction and row positions 1 and 2 in the vertical direction are read. Further, if X = 0 and Y = 1, column positions 1 to 6 in the horizontal direction,
Data at row positions 3 and 4 in the vertical direction (Fig. 26 (a)
Block 97) is read.

In step 1104, step 1103
It is determined whether or not the logical value "1" of the 12-pixel chroma key signal 523 in the block to be processed read in step 6 is 6 or more.

If the determination in step 1104 is YES, step 1105 sets "1" as a logical value corresponding to the block to be processed in a predetermined register, and step 1
If the determination in 104 is NO, in step 1106, "0" is set in the above register as a logical value.

When it is determined in step 1107 that the current process is the process within the first frame of the above-described processes for a plurality of frames for creating the initial screen map, in step 1110, The logical value of the processing target block set in the predetermined register in step 1105 or step 1106 is the RAM (see FIG.
In the initial screen map area in 7), it is written as it is to the address corresponding to the block to be processed in the X column in the horizontal direction and the Y row in the vertical direction.

On the other hand, if it is determined in step 1107 that the current process is the process within the second and subsequent frames among the processes for the plurality of frames, the initial screen in the RAM (FIG. 27) is displayed. The logical value of the address corresponding to the processing target block in the X-th column in the horizontal direction and the Y-th row in the vertical direction in the map area is read, and the logical value and the logical value are set in a predetermined register in step 1105 or step 1106. Logical sum (O
R) is calculated, and the contents of the predetermined register are rewritten with the calculation result. Then, in step 1110, the contents of the address corresponding to the processing target block in the X-th column in the horizontal direction and the Y-th row in the vertical direction in the initial screen map area in the RAM (FIG. 27) are set in the register. Rewritten with a value.

Steps 1103 to 111 above
As described above, the process of 0 corresponds to 9 of the initial screen map.
By repeating for 6 × 96 blocks, an initial screen map for one frame is created, further such initial screen maps are created for a plurality of frames, and the logical sum is calculated between the frames at each block position. Finally, the initial screen map for one screen is obtained in the RAM (FIG. 27), and the process of step 1003 of FIG. 10 is ended.

It should be noted that the initial screen map creation process of step 1003 is configured to be automatically executed when it is determined in step 1002 that the initial screen map has not been created. Camera 10
The initial screen map creation process of step 1003 may be executed at a timing when a predetermined set button (not shown) is pressed at a position where the image is not captured in 2.

Next, in synchronization with the frame period determined by the sync signal 519 (FIG. 5), step 1 of FIG.
The processing from 004 to step 1015 is repeatedly executed. Here, mainly the bat 101 that the user shakes (FIG. 1)
Screen map showing the position of the
Process screen map creating process for sequentially creating data required for the center of gravity calculation process executed in step 1 (step 1005), and the object screen showing the ball 105 (FIG. 1) at the block position where the presence of the bat 101 is detected by this process. Collision coordinate detection processing for determining whether or not data exists (step 1006), center of gravity calculation processing for calculating the center of gravity position of the bat 101 on the processing screen map (step 101)
1), and a process of setting a collision flag as an interrupt flag to the CPU 509 when it is determined that the bat 101 and the ball 105 collide (step 1013)
Is executed.

In this case, although not particularly shown, as described above, the image recognition circuit 508 has two chroma key signal buffers each having the size shown in FIG. 26 (a).
It has a frame (two screens) and a sync signal 519.
For each frame period determined in synchronism with (FIG. 5), one frame of chroma key signal 523 shown in FIG. 26 (a), which is input from the video signal conversion circuit 505, is alternated between two chroma key signal buffers. Hold on. Although not shown in particular, as described above, the image recognition circuit 508
Are YSs each having the size shown in FIG. 26 (b).
It has an OBJ signal buffer for two screen maps, and the RGB color difference data processing circuit 8 in the VDP 506 for each frame period determined in synchronization with the synchronization signal 519 (FIG. 5).
The YSOBJ signals for one screen map output from 06 are alternately held in the two YSOBJ signal buffers. Then, the processing for each frame shown in steps 1004 to 1015 of FIG. 10 is performed by the chroma key signal 523 for one frame held in one of the two chroma key signal buffers in the previous frame cycle, and the previous time. The YSOBJ signal for one screen map held in one of the two YSOBJ signal buffers in the frame period of. At the same time, the chroma key signal 523 and the YSOBJ signal corresponding to the current frame period are sequentially input to the other chroma key signal buffer and the YSOBJ signal buffer, respectively. Then, each time the processing of step 1015 is completed, the chroma key signal buffer and the YSOBJ signal buffer to be processed are switched.

In FIG. 10, the processing screen map creation processing of step 1005 and the collision coordinate detection processing of step 1006 are executed for each block position (see FIG. 26).
Therefore, as in the case of the initial screen map creation processing described in FIG. 11, first, at step 1004, the register X indicating the horizontal column position and the register Y indicating the vertical row position of the generated processing screen map. Is reset to 0, the value of the register X is cleared in step 1009, the value of the register Y is incremented by +1, and the value of the register Y is set to 96 in step 1010.
Until it is determined that the value has reached (exceeds 95), steps 1005 to 1008 → step 1009 → step 1010 → step 1005 are repeatedly executed. In this way, the processing for 96 lines in the processing screen map is executed.

Further, while the value of the register X is incremented by +1 in step 1007, step 100
Until it is determined that the value of the register X reaches 96 (exceeds 95) in step 8, steps 1005 to 1008.
→ The process of step 1005 is repeatedly executed. In this way, the processing for 96 blocks in one line in the processing screen map is executed.

The processing screen map creation processing of step 1005 and the collision coordinate detection processing of step 1006 executed for each current value of the registers X and Y are processing for one block in the processing screen map.

In step 1004, the values of the upper end coordinates and the left end coordinates stored in the RAM of FIG. 27 used in the processing screen map creation processing of step 1005 which will be described later are initialized to 97, and the lower end coordinates are also set. The value of the right end coordinate is initialized to 0, and the center-of-gravity calculation processing data S_M and S_M are also used in the processing screen map creation processing.
The values of xX and S_MyY are initialized to 0. In addition,
These data are secured in a work area (not shown) in the same RAM as the RAM in FIG.

In the following description, the current values of the registers X and Y are simply described as X and Y, and the block specified by the current X and Y values is called a block to be processed. 12 and 13 show detailed operation flowcharts of the processing screen map creation processing in step 1005. Here, the processing for creating the processing screen map stored in the RAM shown in FIG. 27 and the RAM shown in FIG.
Upper end coordinates indicating the existence range of the bat 101 stored in
A process of detecting the lower end coordinates, the left end coordinates, and the right end coordinates, and a process of calculating data for obtaining the barycentric coordinates of the bat 101 are executed.

First, at step 1201, the chroma key signals at the column positions (6X + 1) to (6X + 6) in the horizontal direction and the row position (2Y + 1) and (2Y + 2) in the vertical direction are output from the chroma key signal buffer to be processed. Each logical value of 523 is read. Similar to the case of the initial screen map creation processing, for example, if X = 0 and Y = 1, the data at the column positions 1 to 6 in the horizontal direction and the row positions 3 and 4 in the vertical direction (block 97 in FIG. ) Is read.

In step 1202, step 1201
It is determined whether or not the logical value "1" of the 12-pixel chroma key signal 523 in the block to be processed read in step 6 is 6 or more.

If the judgment in step 1202 is YES, and if a reject switch (not shown) is turned off in step 1203, "1" is set as a logical value corresponding to the block to be processed in a predetermined register in step 1206, and step 1202 If the determination is NO, step 1
At 207, "0" is set as a logical value in the above register.

Here, the reject switch is operated by the user. When the position recognized as the part related to the chroma key on the processing screen map is recognized at the same time as the position where the background object having the same color as the bat 101 exists on the initial screen map, the bat 101 is located at that position. The user turns on this switch when he or she wishes to treat it as nonexistent.

Then, the determination at step 1202 is YE.
S, that is, when the block to be processed is recognized as a portion related to the chroma key, and if it is determined in step 1203 that a reject switch (not shown) is turned on, in step 1204, the RAM (FIG.
From the initial screen map area in 7), the Xth column in the horizontal direction,
The logical value at the same block position as the block to be processed on the Y-th row is read out in the vertical direction.

Then, at step 1205, it is judged if the logical value is "1". Step 120
5 is YES, that is, the position of the processing target block recognized as a portion related to the chroma key on the processing screen map is simultaneously recognized as the position on the initial screen map where the background object having the same color as the bat 101 exists. If so, the logical value "0" is set in the above-mentioned predetermined register in step 1207, so that the bat 101 does not exist at the block position.

On the other hand, if the determination in step 1205 is NO, that is, the processing target block position recognized as a portion related to the chroma key on the processing screen map has a background object having the same color as the bat 101 on the initial screen map. If the position is not recognized, step 12
By setting "1" as a logical value in the above-mentioned predetermined register at 06, it is processed that the bat 101 exists at the block position. Step 12
After the processing of 07, that is, the bat 10 on the processing screen map
For the processing target block for which 1 is not recognized, step 1219 of FIG. 13 described later is executed.

On the other hand, after the processing of step 1206, that is, for the processing target block in which it is recognized that the bat 101 exists on the processing screen map, steps 1208 to 1218 of FIG. 13 are further executed. Here, the process of updating the upper end coordinates, the lower end coordinates, the left end coordinates, and the right end coordinates stored in the RAM of FIG. 27, which shows the existence range of the bat 101 on the processing screen map, and the barycentric coordinates of the bat 101 are obtained. A process of calculating data for is executed.

That is, first, at step 1208, the value (Y
+1) is smaller than the value of the upper end coordinates stored in the RAM (FIG. 27), and this determination is YES.
Then, in step 1209, the value of the upper end coordinate is the value (Y +
It is rewritten as 1).

After step 1209 or step 120
If the determination in step 8 is NO, in step 1210, it is determined whether or not the value (Y + 1) is larger than the value of the lower end coordinate stored in the RAM (FIG. 27).
If it is ES, in step 1211, the value of the lower end coordinate is rewritten to the value (Y + 1).

After step 1211, or step 121
When the determination of 0 is NO, in step 1212, it is determined whether or not the value (X + 1) is smaller than the value of the left end coordinate stored in the RAM (FIG. 27), and this determination is Y.
If it is ES, the value of the leftmost coordinate is rewritten to the value (X + 1) in step 1213.

After step 1213 or step 121
When the determination result in No. 2 is NO, it is determined in step 1214 whether the value (X + 1) is larger than the value of the right end coordinate stored in the RAM (FIG. 27), and this determination is Y.
If it is ES, the value of the right end coordinate is rewritten to the value (X + 1) in step 1215.

As a result of the above processing of steps 1208 to 1215, for example, as shown in FIG. 28, in the range of the block having the logical value "1" on the processing screen map,
The upper end coordinate, the lower end coordinate, the left end coordinate, and the right end coordinate indicating the range are obtained in the RAM of FIG. This value is transferred to the DRAM 510 of FIG. 5 at the same time.

After step 1215 or step 121
If the determination result in 4 is NO, in step 1216, the value of the data S_M is incremented by +1 and
In step 1217, the value of the data S_MxX is incremented by X, and similarly, in step 1218, the value of the data S_MyY is incremented by Y. The principle of obtaining the barycentric coordinates from these data will be described later.

After the processing of step 1218 or the processing of step 1207 (FIG. 12) described above, step 1
In 219, the contents of the address corresponding to the processing target block in the X-th column in the horizontal direction and the Y-th row in the vertical direction in the processing screen map area in the RAM shown in FIG. 27 are predetermined in step 1206 or step 1207 described above. It is rewritten to the logical value set in the register.

The processing screen map creation processing of step 1005 of FIG. 10 shown in the operation flowcharts of FIGS. 12 and 13 is performed by steps 1005 to 1010.
27 is executed as a part of the iterative processing, a processing screen map for one screen is created in the RAM shown in FIG. 27 for each frame cycle, and the RAM shown in FIG.
The upper end coordinate, the lower end coordinate, indicating the existence range of the bat 101,
The left end coordinates and the right end coordinates are obtained, and further, data S_M,
S_MxX and S_MyY are obtained.

Next, FIG. 14 shows a detailed operation flowchart of the collision detection processing in step 1006 of FIG. First, in step 1401, the logical value of the processing target block in the X-th column in the horizontal direction and the Y-th row in the vertical direction is read from the processing screen map area in the RAM (FIG. 27).

In step 1402, it is judged whether or not the logical value is "1", that is, whether or not the bat 101 exists at the position of the block to be processed. When the determination in step 1402 is NO, the process in FIG.
The collision detection process is ended and the process returns to the operation flowchart of FIG. 10 to continue the process for the next block position.

If the determination in step 1402 is yes, in step 1403, the YSO to be processed is processed.
From the BJ signal buffer, the logical value of the YSOBJ signal (CG data) corresponding to the block to be processed in the X column in the horizontal direction and the Y row in the vertical direction is read.

In step 1404, the read YS
It is determined whether or not the logical value of the OBJ signal is "1", that is, whether or not the object screen data indicating the ball 105 (FIG. 1) is present in the processing target block.

If the determination in step 1404 is NO, the collision detection process of FIG. 14 is terminated, and the process returns to the operation flowchart of FIG. 10 to continue the process for the next block position.

If the determination in step 1404 is YES, in step 1405, it is determined whether or not the value of the predetermined collision register is "0", that is, the collision has not been detected so far, and the collision detection of this time is the first time. It is determined whether or not.

If the determination in step 1405 is YES, in steps 1406 and 1407, the X and Y coordinates of the first collision coordinate held in the RAM of FIG. 27 are updated to the respective values of the registers X and Y. To be done. Then, in step 1408, the value of the collision register is set to the value "1" indicating that a collision has occurred. After that, the collision detection process of FIG. 14 is ended, the process returns to the operation flowchart of FIG. 10, and the process for the next block position is continued.

On the other hand, if the determination in step 1405 is NO, then in steps 1409 and 1410, as shown in FIG.
X and Y of the last collision coordinates held in RAM of 7
The coordinates are updated to the values in registers X and Y. afterwards,
The collision detection process of FIG. 14 is terminated, the process returns to the operation flowchart of FIG. 10, and the process for the next block position is continued.

By executing the collision detection processing of step 1006 of FIG. 10 shown in the operation flow chart of FIG. 14 over the entire range of the processing screen map based on the repetition of the processing of steps 1005 to 1010, The presence or absence of a collision and the block range on the processing screen map where the collision has occurred are detected.

Next, the center-of-gravity calculation process of step 1011 of FIG. 10 is executed once in one frame cycle, that is, every time when the creation of the process screen map for one screen is completed and the determination of step 1010 becomes YES. It In this process, RAM
Bat 10 on the processing screen map stored in (FIG. 27)
The centroid position of 1 is calculated.

Generally, the x-coordinate XG and the y-coordinate YG of the center of gravity of the region existing on the two-dimensional coordinates are expressed by the following equation 1 and equation 2
It can be calculated by a formula.

[0102]

XG = Σ (m _xi · x _i ) / Σm _xi = Σ (m _xi ·
x _i ) / M

[0103]

[ _Formula 2] YG = Σ (m _yj · y _j ) / Σm _yj = Σ (m _yj ·
y _j ) / M where the suffixes i and j are the respective positions of the centroid calculation region in the column direction (x direction) and the row direction (y direction), and x _i and y _j are the column direction position i and the row, respectively. Direction position j
X coordinate value and y coordinate value at, and m _xi and m _yj are sums of masses at column-direction position i and row-direction position j, respectively. Further, M is the total sum of the masses of the regions existing on the two-dimensional coordinates.

Here, when the x-coordinate value and the y-coordinate value are values in units of the blocks described above, m _xi and m _yj
Are respectively equivalent to the total number of blocks at the column direction block position i and the row direction block position j. Then, in the processing screen map creation processing of FIGS. 12 and 13 corresponding to step 1005 of FIG. 10 described above, the mass M that is the denominator in the equations 1 and 2 is set as the area S_M in step 1216 of FIG. The term Σ (m _xi · x _i ) calculated in Equation 1 is calculated in step 12 of FIG.
Calculated as data S_MxX by 17 repetitions of the term in equation 2 expression Σ (m _yj · y _j) are calculated as data S_MyY by repetition of step 1218 of FIG. 13.

The above-mentioned data S_M, S_MxX, S_M
FIG. 15 shows a more detailed operation flowchart of the centroid calculation process of step 1011 of FIG. 10 based on yY and the equations of the equations 1 and 2.

That is, in step 1501, the barycentric coordinates (X) of the bat 101 existing on the processing screen map are expressed by the following formulas 3 and 4 corresponding to the formulas 1 and 2.
G, YG) is obtained.

[0107]

## EQU3 ## XG = S_MxX / S_M

[0108]

## EQU00004 ## YG = S_MyY / S_M For example, in the example shown in FIG. 28, the barycentric coordinates (XG, YG) = (32.
6, 16.1) is obtained. S_MxX = 2 × 30 + 3 × 31 + 3 × 32 + 3 × 33
+ 3 × 34 + 1 × 35 + 1 × 36 = 521 S_Mx = 2 + 3 + 3 + 3 + 3 + 1 + 1 = 16 Therefore, XG = 521/16 = 32.6 S_MyY = 4 × 15 + 7 × 16 + 5 × 17 = 257 S_My = 4 + 7 + 5 = 16 Therefore YG = 257/16 = 16 .1 After the center of gravity calculation processing in step 1011 in FIG. 10 shown in the operation flowchart in FIG. 15 above, in step 1012 in FIG. 10, it is determined whether or not the value of the collision register is “1”. The value of this register is Y indicating the presence of the ball 105 even if there is one block having a logical value “1” indicating the presence of the bat 101 on the processing screen map in the collision coordinate detection processing of step 1006 described above.
When it overlaps with the SOBJ signal, it is set to "1" (see step 1408 in FIG. 14).

Then, the determination in step 1012 is YES.
In this case, in step 1013, the collision flag which is an interrupt flag to the CPU 509 of FIG. 5 is set to "1". As a result, a collision flag interrupt process (FIG. 20) described later occurs in the CPU 509, and in this interrupt process, the CG image 104 displayed on the display device 103 in response to the bat 101 hitting the ball 105 in FIG.
A process for controlling is executed. The image recognition circuit 50
8 transfers the barycentric coordinates (XG, YG) calculated in step 1011 to the DRAM 507 of FIG. 5 together with the above-mentioned collision flag setting processing. In addition, the image recognition circuit 508
Is RA by the collision coordinate detection processing of step 1006.
The first collision coordinate and the last collision coordinate stored in M (FIG. 26) and the upper end coordinate, the lower end coordinate, the left end coordinate, and the right end coordinate stored in the above-mentioned RAM by the processing screen map creation processing of step 1005 are required. Depending on the DRA
Transfer to M507.

After the processing of step 1013, it is determined in step 1014 whether the collision flag has returned to "0". The value of the collision flag is returned to "0" at the time when the later-described collision flag interrupt process executed by the CPU 509 ends. As a result, for example, the image recognition circuit 508 of FIG. 5 cannot advance the process of the operation flowchart of FIG. Note that even while the collision flag interrupt process is being executed, step 1
The processing from 004 to step 1015 is configured to proceed in synchronization with the frame cycle, and CP is set according to the value of the collision register.
Further interruptions may be made to U509.

The value of the collision flag returns to "0" and step 1
When the determination of 014 is YES, the value of the collision register is also returned to “0” in step 1015. Step 1015
After the processing of 1) or when the determination in step 1012 is NO, the processing returns to the processing of step 1004 in synchronization with the start time of the next frame period. In this case, as described above, the chroma key signal buffer and YS to be processed are
The OBJ signal buffer is changed.

Next, FIGS. 16 and 17 show the CPU of FIG.
509 is an operation flowchart of CPU processing executed by 509. The CPU 509 basically processes a key scan signal 527 input from a start switch, a stop switch, or the like (not shown), and with the passage of time, from the application ROM 514 in the application cartridge 513 to the CG image 104 (FIG. 1). The object screen data, the object table data, and the background screen data constituting the above are sequentially read, and those data are transferred from the bus 526 to the VDP 506 within the vertical blanking period synchronized with the synchronization signal 519. The display position on the CG image 104 (FIG. 1) of each object screen data transferred to the DRAM 507 is sequentially designated to the VDP 506.

In FIG. 16, first, step 1601.
Then, the contents of the DRAM 507 in FIG. 5 are initialized. In step 1602, various interrupt processes described later are permitted.

In step 1603, the application program is loaded from the application ROM 514 (either # 1 or # 2) in the application cartridge 513 of FIG. 5 inserted into a slot (not shown) into the program area of the DRAM 510 of FIG. Loaded. After that, the CPU 509 executes the operations of step 1604 and thereafter according to this application program.

At step 1604, a DMAC (Direct Memory Access Controller) in the CPU 509.
The read start address and the number of read data of the background screen data forming the CG image 104 (FIG. 1) on the application ROM 514 in the application cartridge 513 are set.

At step 1605, the DMAC is
The read start address and the number of read data on the application ROM 514 of the first object screen data forming the CG image 104 (FIG. 1) are set.

At step 1606, the DMAC is
Application R of object table data
The read start address and the number of read data on the OM 514 are set. This object table data is
As described above, it is a table showing a correspondence table of each object screen data stored in the DRAM 507 and the ID number corresponding to each.

Steps 1604 to 1606 described above
After the above processing, in step 1607, "1" is set in the register ACKF. Then, in step 1608,
It is in a standby state until the value of the register ACKF returns to "0".

Here, the CPU 509 uses the synchronization signal 519.
The vertical blanking interrupt process shown in the operation flowchart of FIG. 18 is executed by interrupting the process of step 1608 at each time when the vertical blanking period in each frame period determined based on the above is started.

When "1" is set in the register ACKF by the above-mentioned CPU processing, the determination at step 1801 becomes YES at the timing when this interrupt processing is executed, so that the step shown in FIG. CPU 5 by the processing from 1604 to step 1606
Each data stored in the application ROM 514 in the application cartridge 513 is converted to the VDP50 based on each data set in the DMAC in the 09.
6 is transferred.

That is, since the processes of steps 1604 to 1606 of FIG. 16 are executed immediately after the power is turned on, the determination of step 1802 of FIG. 18 becomes YES. As a result, first, in step 1803, the DMAC transfers the background screen data in the application ROM 514 via the bus 526 based on the read start address and the number of read data of the background screen data set by the process of step 1604. DM to VDP506
A transfer. The VDP 506 transfers the background screen data from the CPU interface circuit 801 of FIG. 8 to the DRAM control circuit 802 of FIG.
Transfer to M507.

Next, in step 1804, the DMAC
The object screen data in the application ROM 514 is transferred to the VDP 506 via the bus 526 based on the read start address and the number of read data of the object screen data set by the process of step 1605.
MA transfer. The VDP 506 also transfers this object screen data from the CPU interface circuit 801 of FIG. 8 through the DRAM control circuit 802 to the DRA of FIG.
Transfer to M507.

Further, in step 1805, the DMAC
The object table data in the application ROM 514 is transferred to the VDP 50 via the bus 526 based on the read start address and the read data number of the object table data set by the process of step 1606.
DMA transfer to 6. The VDP 506 transfers this object table data to the CPU interface circuit 801.
To the object table RAM 804 via the object control circuit 803.

Thereafter, in step 1806, register A
The value of CKF is returned to "0", and the vertical blanking interrupt processing shown in FIG. 18 ends. As a result, the process of step 1608 of FIG. 16 is restarted, and the determination in this process is YES, so that the user first presses a start switch (not shown) and inputs the corresponding key scan signal 527 (FIG. 5). Until the determination in step 1609 is NO → the determination in step 1612 is N
The process loop in which the determination of O → step 1609 is NO ... is repeated, and the process enters the standby state. The register S
The contents of TF are initialized to "0" in step 1601 of FIG.

When the start switch is pressed and the corresponding key scan signal 527 is input, step 16
The determination of 09 is YES. As a result, first, in step 1610, the time 0 is set in the register t. The register t is a register indicating the elapsed time from the time when the user starts the game by pressing the start switch, and the value of the register t is the value of the clock circuit 512 connected to the bus 526 (see FIG. 5). ) Is incremented by +1 by the timer interrupt processing shown as step 1901 in FIG. 19, which is executed based on the interrupt request input at regular intervals.

Next, at step 1611, the register ST
“1” is set in F. As a result, step 161
The determination of 2 is YES, and thereafter, normally, step 1613 to step 1617 of FIG. 17 → step 161.
9 → step 1609 (FIG. 16) → step 1612 →
The processing loop of step 1613 is repeated.

In the above processing loop, step 16
At 14, the ID number designated for the object table data corresponding to the object screen data to be displayed is calculated based on the current value of the register t indicating the elapsed time, and at step 1615, the current register t is calculated.
The display position of the object screen data is calculated based on the value of. These data are set to a predetermined address on the DRAM 507, and the read start address and the number of read data are set to the DMAC in the CPU 509. In step 1614 or step 1615, the tone generator circuit 515 in the application cartridge 513 is used in the tone generator interrupt process (see FIG. 22) described later.
Pronunciation instruction data to be transferred to the
Set to 7.

If it is necessary to change the object screen data, in step 1614, the CPU 50
In the DMAC in 9, the read start address and the read data number of the object table data corresponding to the new object screen data on the application ROM 514 are set. Also, in step 1615, DMA
In C, the read start address of the new object screen data on the application ROM 514 and the number of read data are set.

In step 1616, the register ACKF
Is set to "1". Thereafter, until the above-described vertical blanking interrupt process is executed and the value of the register ACKF is returned to "0", normally, the determination in step 1617 is NO → the determination in step 1619 is NO → step 1
NO in step 609 (FIG. 16) → YES in step 1612 → NO in step 1613 (FIG. 17)
→ The processing loop in which the determination in step 1617 is NO ... Is repeated, and a standby state is entered.

As described above, the CPU 509 interrupts the processing of the processing loop described above at each time when the vertical blanking period is started, and executes the vertical blanking interrupt processing shown in the operation flowchart of FIG. .

As a result, the determination in step 1801 is YE.
S → because the determination in step 1802 is NO,
Step 1804 is executed. In step 1804, the DMAC in the CPU 509 sends data indicating the display position of the object screen data to the VDP 5 via the bus 526 based on the setting made in the process of step 1615.
DMA transfer to 06. Also, in step 1805,
The DMAC DMA-transfers the ID number of the object screen data to be displayed to the VDP 506 via the bus 526 based on the setting made by the process of step 1614. As a result, the object control circuit 803 in the VDP 506 refers to the object table RAM 804 by the ID number designated by the CPU 509, and the object screen data corresponding to the ID number is transferred via the DRAM control circuit 802 to the D screen.
The line buffer RAM 80 is read out from the RAM 507 and based on the display position information designated by the CPU 509.
After being expanded to 8, the contents are output to the RGB color difference data processing circuit 806.

Step 18 when new object screen data and object table data are transferred
The processing of 04 and 1805 is the same as step 1 of FIG. 16 described above.
This is similar to the case immediately after the execution of Steps 604 to 1606.

Thereafter, in step 1806, register A
The value of CKF is returned to "0", and the vertical blanking interrupt processing shown in FIG. 18 ends. As a result, the above-mentioned processing loop in the CPU processing is restarted, and step 1
When the determination in 613 is YES, the next screen control process is executed.

When the user presses the stop switch (not shown) and inputs the corresponding key scan signal 527 (FIG. 5) in the execution process of the above processing loop, the determination in step 1617 becomes YES. As a result, in step 1618, "0" is set in the register STF. Therefore, after this point, step 1609 is repeated until the user presses a start switch (not shown) again and the corresponding key scan signal 527 is input.
The determination in (FIG. 16) is NO, and the determination in step 1612 is N.
The process loop in which the determination of O → step 1609 is NO ... is repeated, and the process enters the standby state. Since the value of the register t is not cleared, the next time the start switch is pressed, the progress of the game is resumed from the time when the stop switch is pressed.

When the value of the register t reaches the preset end time T in the execution process of the above processing loop, the determination in step 1619 becomes YES. As a result, in step 1620, the value of the register t is reset to 0. Therefore, after this point, the game is repeated from the beginning.

Further, when the user presses the start switch (not shown) again and inputs the corresponding key scan signal 527 in the execution process of the above-mentioned processing loop,
The determination in step 1609 becomes YES. As a result, the value of the register t is forcibly reset to 0 in step 1610. Therefore, even after this point, the game is repeated again from the beginning.

On the other hand, when the image recognition circuit 508 sets the collision flag to 1 as described above (see step 1013 in FIG. 10) in the execution process of the above processing loop, the CPU 509 executes the above processing loop. The processing is interrupted and the collision flag interrupt processing shown in the operation flowchart of FIG. 20 is executed.

First, in step 2001, the register t
It is determined whether or not the current time t indicated by is within the range of a predetermined collision determination period T _{1 to} T ₂ set in advance. The reason for providing such a collision determination period is that the chroma key corresponding to the bat 101 overlaps with the object screen data corresponding to the ball 105 at a time other than the time when the bat 101 of FIG. This is to prevent erroneous processing from being executed in the case of failure.

In step 2002, the bat 1 transferred from the image recognition circuit 508 to the DRAM 510.
The barycentric coordinate of the chroma key indicating 01 is loaded into the CPU 509.

At step 2003, the chroma key barycentric coordinates indicating the bat 101 and the VDP 506 are set.
The positional relationship with the barycentric coordinates of the object screen data indicating the currently displayed ball 105 acquired from the object control circuit 803 therein is determined.

As a result, as shown in FIG. 2, when the deviation between the two barycentric coordinates is within a predetermined allowable range, the CG image 104 on the display device 103 in FIG. 1 is converted into an image corresponding to a home run, for example. Step 200 to change to
At 4, the first image processing is executed. It should be noted that the barycentric coordinates of the chroma key indicating the bat 101 on the processing screen map and the barycentric coordinates of the object screen data indicating the ball 105 should match when the user swings the bat 101 right down to a position corresponding to the strike zone. First, it is assumed that the direction of the camera 102 in FIG. 1 is adjusted in advance.

Also, as shown in FIG.
The barycentric coordinates of the object screen data indicating 5 are bat 1
If the displacement is below the barycentric coordinate of the chroma key indicating 01, the CG image 104 on the display device 103 in FIG.
At 005, the second image processing is executed.

Further, as shown in FIG.
The barycentric coordinates of the object screen data indicating 5 are bat 1
If the displacement is above the center of gravity of the chroma key indicating 01, the third image processing is executed in step 2006 in order to change the CG image 104 on the display device 103 in FIG. 1 into an image corresponding to a fly, for example. To be done.

FIG. 21 is an operational flowchart of the nth image processing generally showing the first to third image processings shown in steps 2004 to 2006. In step 2101, depending on whether the current processing is the first, second, or third image processing, and the current register t indicating the passage of time.
The display position of the object screen data is calculated based on the value of. In step 2102, depending on whether the current processing is the first, second, or third image processing,
Further, the ID number designated for the object table data corresponding to the object screen data to be displayed is calculated based on the current value of the register t. In step 2101 or step 2102, tone generation instruction data transferred to the tone generator circuit 515 in the application cartridge 513 in tone generator interrupt processing (see FIG. 22) described later is also generated and set in the DRAM 507 or the like.

In step 2103, the register ACKF
Is set to "1". After that, until the above-described vertical blanking interrupt process is executed and the value of the register ACKF is returned to “0”, normally, the determination in step 2104 is NO → the determination in step 2105 is NO → the determination in step 2106 is NO. → NO in step 2104
The processing loop of ... Is repeated, and the standby state is entered.

As described above, the CPU 509 interrupts the processing of the above-described processing loop and executes the vertical blanking interrupt processing shown in the operation flowchart of FIG. 18 each time the vertical blanking period is started. .

As a result, the determination in step 1801 is YE.
S → because the determination in step 1802 is NO,
Step 1804 is executed. In step 1804, the DMAC in the CPU 509 sends the data indicating the display position of the object screen data to the VDP 5 via the bus 526 based on the setting made in the process of step 2101.
DMA transfer to 06. Also, in step 1805,
The DMAC DMA-transfers the ID number of the object screen data to be displayed to the VDP 506 via the bus 526 based on the setting made by the process of step 2102. As a result, the object control circuit 803 in the VDP 506 refers to the object table RAM 804 by the ID number designated by the CPU 509, and the object screen data corresponding to the ID number is transferred via the DRAM control circuit 802 to the D screen.
The line buffer RAM 80 is read out from the RAM 507 and based on the display position information designated by the CPU 509.
After being expanded to 8, the contents are output to the RGB color difference data processing circuit 806.

Thereafter, in step 1806, register A
The value of CKF is returned to "0", and the vertical blanking interrupt processing shown in FIG. 18 ends. As a result, the above-described processing loop in the nth image processing is restarted, and the determination in step 2106 becomes YES, so that the screen control processing corresponding to the value of the register t next to the nth image processing is executed.

When the user presses the stop switch (not shown) and inputs the corresponding key scan signal 527 (FIG. 5) in the execution process of the above processing loop, the determination in step 2105 becomes YES. As a result, in step 2108, "0" is set in the register STF, and the n-th image processing in FIG. 21 ends. As a result, the collision flag interrupt process is also terminated after executing step 2007 in FIG. 20 described later, and the process returns to the CPU 509 process loop. Therefore, after this point, step 1609 is repeated until the user presses a start switch (not shown) again and the corresponding key scan signal 527 is input.
The determination in (FIG. 16) is NO, and the determination in step 1612 is N.
The process loop in which the determination of O → step 1609 is NO ... is repeated, and the process enters the standby state.

Further, when the value of the register t reaches the preset end time END of the nth image processing in the execution process of the nth image processing of FIG. 21, the determination in step 2104 is YE.
It becomes S. As a result, in step 2107, the register t
21 is reset to 0, the nth image processing in FIG. 21 is terminated, and further, the collision flag interrupt processing is terminated after executing step 2007 in FIG. 20 described later, and the process returns to the CPU 509 processing loop. Therefore, after this point, the game is repeated from the beginning.

In step 2107, the value of the register t is not reset to 0, but is forcibly set to a predetermined value, so that the CG image 104 displayed on the display device 103 is displayed as follows. It is also possible to shift to a scene.

At step 2007 in the collision flag interruption processing of FIG. 20 executed after the nth image processing of FIG. 21 is completed, the collision flag is returned to “0”.
This process is performed by the image recognition circuit 508 in the operation flowchart shown in FIG.
This is a process for advancing the process before step 1014.

Next, FIG. 22 is an operation flowchart of the sound source interruption processing executed by the CPU 509. This interrupt processing is performed by the application cartridge 513 of FIG.
The tone generator circuit 515 therein executes the tone generator processing (FIG. 25) described later when the tone generator interrupt request flag is set as an interrupt flag to the CPU 509.

First, in step 2201, the monitoring timer is reset. The watch timer is used to prevent the use of the imitated application cartridge 513.

The value of the above-mentioned monitoring timer is executed based on the interrupt request input from the clock circuit 512 (FIG. 5) connected to the bus 526 at regular intervals.
In step 2301 of the monitoring timer count process shown in the operation flow chart of No. 3, it is sequentially incremented. Then, in step 2302, it is determined whether or not the value of this monitoring timer has overflowed.

Here, the application cartridge 5
If 13 is manufactured by a legitimate maker, the application cartridge 513 uses the CPU in the process of executing a sound source process (FIG. 25) described later.
The operation of setting and resetting the sound source interrupt request flag which is an interrupt flag to 509 is sequentially executed. On the other hand, the CPU 509 executes step 220 of the sound source interrupt processing of FIG. 22 every time the sound source interrupt request flag is set.
Performing a 1 will cause the watchdog timer to be reset at the appropriate time intervals.

Therefore, the application cartridge 5
If 13 is manufactured by a legitimate manufacturer, step 2 of the monitoring timer count process of FIG. 23.
The determination at 302 is always NO.

On the other hand, the application cartridge 51
In the case where 3 is imitated, it is almost the case that the operation for setting and resetting the sound source interrupt request flag which is the interrupt flag to the CPU 509 is not provided due to cost reasons or the like. CP
A sound source interrupt request is not generated to U509 at an appropriately short time interval, and as a result, the monitoring timer is not reset or reset at an appropriately short time interval, and eventually the monitoring timer overflows. By doing so, the determination in step 2302 of the monitoring timer count processing in FIG. 23 becomes YES.

In this case, in step 2303, CP
The overflow flag OVF is set to 1 as the interrupt flag to U509 again. As a result, the CPU
As the interrupt processing again, the CPU 509 executes the overflow interrupt processing shown in the operation flowchart of FIG. 24, and stops (halts) the operation of the CPU 509 itself in step 2401.

By the control using the above monitoring timer,
The use of the imitated application cartridge 513 is prevented. Next, in the sound source interruption process of FIG. 22, in step 2202, the pronunciation instruction data created in advance in the DRAM 507 is transferred via the bus 526.
Sound source circuit 51 in the application cartridge 513
5 is transferred. This pronunciation instruction data is stored in the CPU 509.
Is the above-mentioned step 1614 or step 16 of FIG.
15, or generated in accordance with the control processing of the CG image 104 (FIG. 1) such as step 2101 or step 2102 of FIG. 21 and stored in the DRAM 507.

The application cartridge 51
The same control may be executed not only for the tone generator circuit 515 in the No. 3 but also for the tone generator circuit 516 mounted on the main body side.

Each process shown in the operation flowcharts of FIGS. 16 to 24 described above is a process executed by the CPU 509. Finally, FIG. 25 is an operation flowchart of the sound source processing executed by the sound source circuit 515 in the application cartridge 513.

When the application cartridge 513 is attached to a holder (not shown), execution of the operation flowchart of FIG. 25 is started, and first, step 25
At 01, the contents of the registers and work RAM inside the tone generator circuit 515 are initialized.

Then, steps 2502 to 25
Each processing of 07 is repeatedly executed. First, the tone generator circuit 515 generates a tone signal for one tone generation unit (for example, every time reading of tone waveform data for several seconds is completed).
In step 2502, a sound source interrupt request flag which is an interrupt flag to the CPU 509 of FIG. 5 is set. On the other hand, the CPU 509 outputs the tone generation instruction data for the next tone generation unit to the tone generator circuit 515 by executing the tone generator interrupt process of FIG.

In step 2503, the tone generator circuit 515 waits until the input data changes. When the input data changes, in step 2504, a standby state is entered until the input data becomes stable.

When the input data becomes stable, the tone generator interrupt request flag is reset to 0 in step 2505, and then the input sound generation instruction data is interpreted in step 2506.

Then, in step 2507, a tone signal corresponding to the data is generated. When the tone signal generation for one tone generation unit is completed, the tone generator circuit 515 again executes step 2502 and sets a tone generator interrupt request flag which is an interrupt flag to the CPU 509 in FIG.

The musical tone signal generated by the tone generator circuit 515 as described above is sent to the amplifier 517 via the CPU 509 and the tone generator circuit 516, where it is amplified and then output to a speaker (not shown). It is output as the signal 528. <Other Embodiments> In the embodiment described above, the CPU 509
Is the bat 101 on the processing screen map (see FIG. 26) transferred from the image recognition circuit 508 to the DRAM 510.
1 is displayed on the display device 103 of FIG. 1 based only on the positional relationship between the center of gravity of the object and the center of gravity of the object showing the currently displayed ball 105 (FIG. 1) acquired from the object control circuit 803 in the VDP 506. CG image 104 is controlled. However, the present invention is not limited to this, and the image recognition circuit 508 performs the R by the collision coordinate detection processing of step 1006 of FIG.
First collision coordinates and last collision coordinates stored in AM (FIG. 26), and upper end coordinates, lower end coordinates, left end coordinates indicating the existence range of the bat 101 stored in the above-mentioned RAM by the processing screen map creation processing of step 1005. , And the right edge coordinates are transferred to the DRAM 507, and based on these various position data, the display device 103 in FIG.
May be configured to control the CG image 104 displayed in.

[0169]

According to the present invention, the user can participate in the video game displayed by the CG image by swinging the bat as if he / she actually played the game.

[0170] Particularly, in this case, from the captured screen, bat, etc., extracted, etc. Te by a predetermined color image area <br/> chromakey, further, by calculating the center of gravity of its accurate operation of the user It becomes possible to grasp.

According to the present invention, not only a video game in which image control is performed in response to a user's action, but also a device for performing various image controls based on the results of capturing various real scenes are realized. It becomes possible.

[Brief description of drawings]

FIG. 1 is an external view of an operation of an embodiment.

FIG. 2 is a diagram (No. 1) of the operation principle of the embodiment.

FIG. 3 is a second operational principle diagram of the embodiment.

FIG. 4 is a diagram (part 3) of the operation principle of the embodiment.

FIG. 5 is an overall configuration diagram of an embodiment of the present invention.

FIG. 6 is a configuration diagram of a video signal input / output circuit.

FIG. 7 is a configuration diagram of a video signal conversion circuit.

FIG. 8 is a block diagram of a VDP.

FIG. 9 is an explanatory diagram of a clock circuit.

FIG. 10 is an overall operation flowchart of the image recognition circuit.

FIG. 11 is an operation flowchart of an initial screen map creation process.

FIG. 12 is an operation flowchart (No. 1) of processing screen map creation processing.

FIG. 13 is an operation flowchart (No. 2) of processing screen map creation processing.

FIG. 14 is an operation flowchart of a collision detection process.

FIG. 15 is an operation flowchart of a center of gravity calculation process.

FIG. 16 is an operation flowchart of CPU processing (No. 1)
Is.

FIG. 17 is an operation flowchart of CPU processing (No. 2)
Is.

FIG. 18 is an operation flowchart of vertical blanking interrupt processing.

FIG. 19 is an operation flowchart of timer interrupt processing.

FIG. 20 is an operation flowchart of collision flag interrupt processing.

FIG. 21 is an operation flowchart of n-th image processing.

FIG. 22 is an operation flowchart of a sound source interrupt process.

FIG. 23 is an operation flowchart of monitoring timer count processing.

FIG. 24 is an operation flowchart of overflow interrupt processing.

FIG. 25 is an operation flowchart of sound source processing.

FIG. 26 is a relationship diagram between a pixel configuration of a frame and a screen map.

FIG. 27 is a diagram showing a RAM map of the image recognition circuit.

FIG. 28 is an explanatory diagram of detection processing of upper / lower / left / right edge coordinates.

[Explanation of symbols]

101 Bat 102 camera 103 display device 104 CG image 105 balls 501 CCD 502 clock driver 503 video signal input / output circuit 504 Oscillator (OSC) 505 Video signal conversion circuit 506 VDP 507 DRAM 508 image recognition circuit 509 CPU 510 DRAM 511 OS ROM 512 clock circuit 513 application cartridge 514 Application ROM 515, 516 sound source circuit 517 amplifier 518 CCD output signal 519 Sync signal 520 digital composite video input signal 521 Digital composite video output signal 522 analog composite video output signal 523 Chromakey signal 524 digital RGB signals 525 Superimpose signal 526 bus 527 key scan signal 528 audio output signal 528

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI G09G 5/00 510 G09G 5/00 510A 510H 550 550C 5/34 5/34 Z H04N 7/18 H04N 7/18 K P (58 ) Fields surveyed (Int.Cl. ⁷ , DB name) G09G 5/00-5/42 A63F 13/00 G06F 3/00 G06F 3/033 H04N 7/18

Claims (9)

(57) [Claims] 1. A feature based on a color signal of an imaged screen.
Extraction means for extracting an image region of constant conditions, the extraction of the image area extracted by the means and the imaging screen
A first storage unit for storing the existence range of the image area, and an existence range of the image area stored in the first storage unit .
Et al., A center-of-gravity calculating means for calculating the centroid position of the image area, the second storage means and the object picture data stored in the storage means of said second storing an object screen data to be output
A display position designating means for designating a position to be output on the display screen, the position designated by the display position designating means, and the first position.
The positional relationship with the existing range stored in the storage means of
Determination means for determining whether or not the condition is satisfied, and the display position designating means designated by the determination means
Location and existence range stored in the first storage means
It is determined that the positional relationship with
And the weight of the object displayed at the specified position.
Calculated by the center of gravity calculation means by calculating the center position
Center of gravity of the image area and center of gravity of the object
Corresponds to the screen to be displayed next based on the positional relationship with
And signal generating means for generating a signal, a signal generation device and having a. 2. Designated by the display position designating means
Position and the existence of the image area stored in the first storage means.
The imaged screen and the object based on the presence range.
Image corresponding to the display screen
The signal generation apparatus according to claim 1 , further comprising a synthesized signal generation unit that generates a signal . 3. Output of a signal corresponding to a screen that can be displayed
The procedure further includes procedure storing means for storing procedures, wherein the signal generating means is calculated by the center of gravity calculating means.
Center of gravity of the image area and the weight of the calculated object
Corresponding to the positional relationship with the cardiac position, from the procedure storage means
Read out the procedure of the signal corresponding to the screen to be displayed next
To, signal generating instrumentation <br/> location according to claim 1 or 2, characterized in that. 4. A positional relationship determined by the determination means.
The predetermined condition of is specified by the display position specifying means.
Position and the image area stored in the first storage means
4. It is whether or not the existence range of the object overlaps with the existing area of the object.
Signal generator . 5. A color signal based on a color signal of an imaged screen is used.
Further comprising a signal output means for outputting a mackey signal,
The output means is the chroma key output by the signal output means.
Extraction of image area of specific condition by recognizing signal
According to any one of claims 1 to 4, characterized in that
Signal generator . 6. The procedure storage means is a book of a signal generator.
Configured to be removable from the body, A contract characterized by
The signal generation device according to any one of claim 3 to claim 5. 7. The second storage means is a signal generating device.
It is configured to be removable from the main body, 7. The method according to any one of claims 1 to 6, characterized in that
Signal generator. 8. Based on the captured color signal of the screen,
An extraction step of extracting an image region of a constant condition, On the imaging screen of the image area extracted in the extraction step
A storage step for storing the existence range in the memory, Existence range of the image area stored in the storing step
, The center of gravity calculation step for calculating the center of gravity position of the image area
When, A table of object screen data stored in memory in advance.
Display position specification step to specify the position to be output on the display screen.
And The position specified in the display position specifying step and the memory
The positional relationship with the existing range stored in step is
A determination step of determining whether or not the condition is satisfied, In the judging step, the finger is pressed in the display position specifying step.
The determined position and the existence range stored in the storage step.
If the positional relationship with the enclosure meets the specified conditions Judged
Then, of the object displayed at the specified position
The position of the center of gravity is calculated and calculated in the step of calculating the center of gravity.
Center of gravity of the image area and center of gravity of the object
The screen to be displayed next based on the positional relationship with the
A signal generating step for generating a corresponding signal, A signal generation method comprising: 9. Based on the color signal of the imaged screen,
Further comprising a signal output step of outputting a mackey signal,
The extraction step includes the clock output in the signal output step.
Image area of specific condition by recognizing Romakey signal
To extract, The signal generating method according to claim 8, wherein