JP2011008830A - Input coordinate processing program, device, system, and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an input coordinates processing program and device, which improves operability of coordinate input by making processing to an unsuitable locus simple while keeping the number of variations of processing which can be specified by coordinates input operation.SOLUTION: The input coordinate processing program detects a series of coordinate information in accordance with input coordinates outputted from a pointing device 15 (S43 etc.) and sets up a standard coordinates DC2 (S44). When the final coordinates information indicates the inside of domains AT, AB, AL, and AR except a standard domain AM, the input coordinates processing program determines processing, based on a direction from the standard coordinate DC2 to the domain indicated by the final coordinate information (S115 etc.). When the final coordinate information indicate the inside of the standard domain AM after a part of coordinate information indicates the inside of the domains except the standard domain AM, the program determines processing, based on combination of directions for reciprocating between the standard coordinate DC2 and the domain indicated by a part of coordinate information (S121).

Description

  The present invention relates to an input coordinate processing program, an input coordinate processing device, an input coordinate processing system, and an input coordinate processing method, and more specifically, is output from a pointing device such as a touch panel that outputs coordinate information of a predetermined coordinate system. The present invention relates to an input coordinate processing program, an input coordinate processing device, an input coordinate processing system, and an input coordinate processing method for determining processing using input coordinates.

  2. Description of the Related Art Conventionally, a technique for determining a process to be performed next (hereinafter referred to as a next process) based on coordinates input by operating a mouse is known. For example, a web browser for browsing web pages on the Internet has been proposed that enables an operation called “mouse gesture” (see, for example, Non-Patent Document 1).

  In the browser described in Non-Patent Document 1, when the mouse is moved to the left in the state where the mouse is right-clicked in the background area of the web page and the right click is released, the web page is advanced to the next page. On the other hand, after moving to the right with the mouse right-clicked in the background area of the web page, when the right-click is released, the previous page is restored. Also, after moving the mouse down with the right mouse click in the background area of the web page and releasing the right click, a new window is opened. Furthermore, after the mouse is moved down with the right mouse click in the background area of the web page, the window is duplicated when the mouse is moved up and the right click is released.

  Also, after right-clicking the mouse on the link of the web page, moving the mouse up in the state of right-clicking and then releasing the right-click, the link destination is opened in a new window. Also, after right-clicking the mouse on a link on a web page, moving the mouse down with the right-click, moving up and releasing the right-click opens the link destination in the background. These are different processes when the left mouse button is clicked on a link on a web page compared to opening the link destination in the original window.

  As described above, in the mouse gesture disclosed in Non-Patent Document 1, the next process is determined based on the direction in which the locus of the input coordinates by the mouse is accumulated or a combination of these directions. As a result, the mouse gesture can save the trouble of clicking on the buttons arranged around the web page or operating the toolbar on the display screen.

  In a browser or the like called “Slipnipir”, a recognized gesture is displayed at the bottom on the left side of the screen. For example, the direction in which the locus of the input coordinates is accumulated is displayed sequentially in the order of recognition such as “↑”, “↓”, “↑”, and “→”. The user can predict the next processing by viewing this display.

“Opera FAQ> Mouse related”, [online], [March 15, 2005 search], Internet <URL: http: // dai. pekori. to / opera / faq / mice>

  However, in the browser described in Non-Patent Document 1, there is no problem when the user can draw an appropriate trajectory at a time, but a problem occurs when an inappropriate trajectory is drawn. That is, as the result of the accumulation of the input coordinate trajectory by the mouse, the unintended trajectory accumulates as the movement is continued to correct the inappropriate trajectory, and as a result, the next processing desired by the user is performed. Can not be made. In addition, in order to delete this inappropriate locus, the user had to perform a complicated operation of releasing the right mouse button, right-clicking again, and redrawing the desired locus again. .

  On the other hand, it is conceivable that the next process is determined based on only one direction obtained by accumulating the locus of input coordinates by the mouse without combining a plurality of directions. However, the number of variations for determining the next process becomes a number that can be distinguished in one direction, and the operation gesture for instructing the next process becomes poor. Although there is a method of increasing the number of distinctions in one direction, since the angle for determination for each direction becomes small, the direction that the user tried to draw is often different from the direction recognized on the computer side. Thus, the user's operation becomes difficult.

  In addition, as described above, there is a browser in which the recognized operation gesture is displayed at the bottom on the left side of the display screen. However, since the operation gesture is displayed at a less noticeable position on the left bottom of the display screen, it is difficult for the user to recognize. In addition, even if the user is aware, in order to view the operation gesture display, it is necessary to switch the line of sight from the browsing position of the web page to the bottom part on the left side of the screen, which is troublesome for the user. It was.

  SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an input coordinate processing program and input coordinate processing that improve the operability of coordinate input by simplifying processing for an inappropriate locus while maintaining the number of processing variations that can be specified by coordinate input operation. An apparatus, an input coordinate processing system, and an input coordinate processing method are provided. Another object of the present invention is to provide an input coordinate processing program, an input coordinate processing device, an input coordinate processing system, and an input coordinate processing method that allow a user to easily recognize a recognized operation gesture.

  In order to achieve the above object, the present invention employs the following configuration. Reference numerals in parentheses, step numbers (steps are abbreviated as S, and only step numbers are described), and the like indicate correspondence with embodiments described later in order to help understanding of the present invention. The scope of the present invention is not limited at all.

  The first invention is an input coordinate processing program executed by a computer (21) of an apparatus operated by input coordinates (coordinate data corresponding to a contact position) output from a pointing device (15) in accordance with a user operation. It is. The input coordinate processing program includes an input coordinate detection step (S43, S63, S68, S73, S84, S87, S92, S104, S107, S114, S120, S134, S139, S144), a reference coordinate storage step (S44), and a plurality of areas. The computer is caused to execute the setting step (FIG. 11) and the process determining step (S50 to S52). The input coordinate detection step detects coordinate information (DC1) based on a predetermined coordinate system (screen coordinate system) in accordance with the input coordinates output from the pointing device. The reference coordinate storage step sets the reference coordinate (DC2) based on the first coordinate information out of the series of coordinate information detected in the input coordinate detection step, and stores it in the memory (24). In the multiple area setting step, a plurality of areas (AM, AT, AB, AL, AR) are formed with reference coordinates as a reference, and an area including the reference coordinates is set as a reference area (AM). In the process determination step, the process is determined based on the last coordinate information (coordinate at the time of touch-off) and the reference coordinates among the series of coordinate information detected in the input coordinate detection step. The process determination step includes a first process determination step (S115, S135, S140, S145) and a second process determination step (S121). In the first process determination step, when the last coordinate information indicates an area other than the reference area (AT, AB, AL, AR), based on the direction from the reference coordinate to the area indicated by the last coordinate information. Determine processing. In the second process determination step, the last coordinate information is the reference region after a part of the coordinate information in the series of coordinate information detected in the input coordinate detection step indicates an area other than the reference region (DF8 is on). When the inside is indicated, the processing is determined based on the combination of the direction from the reference coordinate to the region indicated by the partial coordinate information and the direction from the region indicated by the partial coordinate information to the reference coordinate. The pointing device is an input device that specifies an input position and coordinates on the screen, and is realized by, for example, a touch panel, a mouse, a track pad, a track ball, and the like. A coordinate system used in each input device is a touch panel coordinate system or a screen coordinate system.

  In a second aspect based on the first aspect, the process determining step includes a flag setting step (S112, S132, S137, S142), a first flag updating step (S112, S132, S137, S142), and a second flag. An update step (S118) is further included. In the flag setting step, when the position indicated by the last coordinate information moves from the reference area to another area, flags (DF8, DF10 to DF12) corresponding to the other areas are set and stored in the memory. The first flag update step is currently set when the position indicated by the last coordinate information further moves from the region where the flag is set to another region different from the reference region after the flag is set in the flag setting step. Are updated to the flags (DF8, DF10 to DF12) corresponding to the area indicated by the last coordinate information and stored in the memory. The second flag update step is currently set when the position indicated by the last coordinate information further moves from the region where the flag is set to the reference region after setting the flag in the flag setting step or the first flag update step. Is updated to a flag (DF9) indicating a round trip between the reference area and the area where the flag is set, and stored in the memory. The first process determination step executed by the computer determines a process based on the flag set by the flag setting step or the first flag update step. The second process determination step executed by the computer determines the process based on the flag set by the second flag update step.

  In a third aspect based on the first aspect, the computer is further caused to execute display control steps (S113, S119, S133, S138, S143). In the display control step, the direction (upward, downward, leftward, rightward) from the reference coordinate determined in the first processing determination step to another region indicated by the last coordinate information and the second processing determination Indices (M8 to M12) indicating the direction (up and down direction) of the combination determined to be processed in the step are respectively displayed on the display device (12).

  In a fourth aspect based on the first aspect, the computer further executes a display control step. In the display control step, an image (M7) indicating the reference region set in the multiple region setting step is displayed on the display device.

  In a fifth aspect based on the first aspect, the computer further executes a display control step. The display control step displays an image based on the coordinate system on the display device. The multiple region setting step executed by the computer forms a plurality of regions by dividing a peripheral image region centered on the reference region into a plurality of regions.

  A sixth invention is an input coordinate processing program executed by a computer of an apparatus operated by input coordinates output from a pointing device in response to a user operation. The input coordinate processing program causes the computer to execute an input coordinate detection step, a reference coordinate storage step, a first process determination step, a second process determination step, and a display control step. The input coordinate detection step detects coordinate information based on a predetermined coordinate system according to the input coordinates output from the pointing device. In the reference coordinate storage step, reference coordinates are set based on the first coordinate information in the series of coordinate information detected in the input coordinate detection step, and stored in the memory. In the first process determination step, the process is determined based on the direction from the position indicated by the reference coordinates to the position indicated by the last coordinate information in the series of coordinate information detected by the input coordinate detection step. In the second process determination step, the reference coordinates indicate the direction from the position indicated by the reference coordinates to the position indicated by the intermediate coordinate information in the series of coordinate information detected by the input coordinate detection step and the position indicated by the intermediate coordinate information. Processing is determined based on the combination of directions to positions. In the display control step, an image based on the coordinate system is displayed on the display device, and the index indicating the direction of the process determined in the first process determination step and the direction of the combination determined in the second process determination step is the last. The image is displayed in the vicinity of the position indicated by the coordinate information. Here, the position “vicinity” indicated by the last coordinate information includes at least the position itself indicated by the last coordinate information and a region adjacent to the position.

  A seventh aspect of the present invention is an input coordinate processing apparatus operated by input coordinates output from a pointing device in accordance with a user operation. The input coordinate processing apparatus includes storage means (24), input coordinate detection means, reference coordinate storage processing means, multiple area setting means, and process determination means. The input coordinate detection means detects coordinate information based on a predetermined coordinate system according to the input coordinates output from the pointing device. The reference coordinate storage processing means sets reference coordinates based on the first coordinate information among a series of coordinate information detected by the input coordinate detection means, and stores the reference coordinates in the storage means. The multi-area setting unit forms a plurality of areas with reference to the reference coordinates, and sets the area including the reference coordinates as the reference area. The process determining means determines the process based on the last coordinate information and the reference coordinates in the series of coordinate information detected by the input coordinate detecting means. The process determining unit includes a first process determining unit and a second process determining unit. When the last coordinate information indicates an area other than the reference area, the first process determining means determines the process based on the direction from the reference coordinate to the area indicated by the last coordinate information. The second processing determining unit is configured to perform reference processing when the last coordinate information indicates the inside of the reference region after the partial coordinate information indicates the region other than the reference region in the series of coordinate information detected by the input coordinate detecting unit. The processing is determined based on the combination of the direction from the coordinate to the area indicated by the partial coordinate information and the direction from the area indicated by the partial coordinate information to the reference coordinate.

  An eighth invention is an input coordinate processing apparatus operated by input coordinates output from a pointing device in accordance with a user operation. The input coordinate processing device includes storage means, input coordinate detection means, reference coordinate storage processing means, first process determination means, second process determination means, and display control means. The input coordinate detection means detects coordinate information based on a predetermined coordinate system according to the input coordinates output from the pointing device. The reference coordinate storage processing means sets reference coordinates based on the first coordinate information among a series of coordinate information detected by the input coordinate detection means, and stores the reference coordinates in the storage means. The first process determining means determines the process based on the direction from the position indicated by the reference coordinates to the position indicated by the last coordinate information in the series of coordinate information detected by the input coordinate detecting means. The second processing determining means indicates the direction from the position indicated by the reference coordinates to the position indicated by the intermediate coordinate information in the series of coordinate information detected by the input coordinate detection means and the reference coordinates indicated by the position indicated by the intermediate coordinate information. Processing is determined based on the combination of directions to positions. The display control means displays an image based on the coordinate system on the display device, and the index indicating the direction in which the first process determining means determines the process and the direction of the combination in which the second process determining means determines the process is the last. The image is displayed in the vicinity of the position indicated by the coordinate information.

  According to the first aspect of the invention, in the second process determination step, a process for recognizing and executing a user gesture with a combination of a plurality of operation directions in accordance with an input from the pointing device is determined. Therefore, there are more process variations that can be determined than distinguishing user operation gestures by a single operation direction. On the other hand, in the first process determination step, it is possible to easily cancel the gesture based on the operation direction, and only the operation direction from the reference coordinate to the area is effective only by operating to another area other than the reference area. It can be recognized as a gesture. Therefore, when an operation performed by the user is inappropriate, the user can easily cancel the operation, and can give a new operation instruction by an operation continuous with the operation. That is, it is possible to improve the operability of coordinate input by reducing the inconvenience caused when an inappropriate trajectory is drawn while increasing the variations of processing that can be determined by the coordinate input operation.

  According to the second aspect of the invention, since entering and exiting a plurality of areas are managed by flags, it is possible to distinguish user gestures more easily than processing based on all input coordinates.

  According to the third aspect, since the operation gesture recognized by the coordinate input operation input by the user is displayed as an index, the user can surely recognize the input operation gesture and the determined process.

  According to the fourth aspect, by displaying the reference area that serves as a reference for distinguishing the operation gesture, the target image for the user to input an effective operation using the pointing device can be obtained.

  According to the fifth aspect of the present invention, the operation direction centered on the reference coordinate is easily distinguished by dividing the peripheral region centered on the reference region to form a plurality of regions for distinguishing the operation direction. be able to.

  According to the sixth aspect, since the operation gesture recognized by the coordinate input operation input by the user is displayed as an index near the position where the user is currently performing the input operation on the displayed image, the user can It is possible to reliably recognize the input operation gesture and the determined process.

  Further, according to the input coordinate processing apparatus of the present invention, the same effect as the above-described input coordinate processing program can be obtained.

1 is an external view of a game apparatus 1 that executes an input coordinate processing program of the present invention. The block diagram which shows the internal structure of the game device 1 of FIG. The figure which shows the example of a screen display of the initial stage displayed on 1st LCD11 and 2nd LCD12 The figure which shows the example of a screen display displayed on the 1st LCD11 and the 2nd LCD12 when tapping the touch panel 15 on the link destination displayed on the 2nd LCD12. The figure which shows the example of a screen display displayed on the 1st LCD11 and the 2nd LCD12 when the touch panel 15 is slid upward from the link destination displayed on the 2nd LCD12. Screen display examples displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the link destination displayed on the second LCD 12 and then reciprocated downward. Illustration The figure which shows the example of a screen display displayed on the 1st LCD11 and the 2nd LCD12 when the touch panel 15 is slid downward from the tab displayed on the 2nd LCD12. A screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid downward from the tab displayed on the second LCD 12 and then reciprocated upward is shown. Figure The figure which shows the example of a screen display displayed on the 1st LCD11 and the 2nd LCD12 when the touch panel 15 is slid so that the one part information displayed on the 2nd LCD12 may be enclosed. The figure which shows the example of a screen display displayed on the 1st LCD11 and the 2nd LCD12 when the touch panel 15 is slid upward from the background image displayed on the 2nd LCD12. The figure for demonstrating each area | region set when the touch panel 15 on the background image displayed on the 2nd LCD 12 is touch-operated. Screen display examples displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the background image displayed on the second LCD 12 and then reciprocated downward. Illustration Flowchart showing an operation in which the game apparatus 1 performs an input coordinate process by executing an input coordinate process program executed by the game apparatus 1 Subroutine showing detailed operation of the link designation process in step 50 in FIG. Subroutine showing detailed operations for the tab designation process in step 51 in FIG. Subroutine showing the detailed operation of the background designating process in step 52 in FIG. Subroutine showing the detailed operation of the background designating process in step 52 in FIG. Subroutine showing the detailed operation of the background designating process in step 52 in FIG. The figure which shows an example of the various data memorize | stored in RAM24 by the processing operation based on FIG.

  An input coordinate processing apparatus that executes an input coordinate processing program according to an embodiment of the present invention will be described with reference to the drawings. The input coordinate processing program of the present invention can be applied by being executed by an arbitrary computer system that can be displayed on the display device, but is executed by the game device 1 as an example of an information processing device (input coordinate processing device). This will be described using an input coordinate processing program. FIG. 1 is an external view of a game apparatus 1 that executes the input coordinate processing program of the present invention. Here, a portable game device is shown as an example of the game device 1.

  In FIG. 1, the game apparatus 1 includes a first LCD (Liquid Crystal Display) 11 and a second LCD 12. The housing 13 includes an upper housing 13a and a lower housing 13b. The first LCD 11 is accommodated in the upper housing 13a, and the second LCD 12 is accommodated in the lower housing 13b. The resolutions of the first LCD 11 and the second LCD 12 are both 256 dots × 192 dots. In this embodiment, an LCD is used as the display device. However, any other display device such as a display device using EL (Electro Luminescence) can be used. The first LCD 11 and the second LCD 12 can use any resolution.

  The upper housing 13a is formed with sound release holes 18a and 18b for releasing sound from a pair of speakers (30a and 30b in FIG. 2) to be described later.

  The lower housing 13b has a cross switch 14a, a start switch 14b, a select switch 14c, an A button 14d, a B button 14e, an X button 14f, a Y button 14g, a power switch 14h, an L button 14L, and an R button as input devices. 14R is provided. As a further input device, a touch panel 15 is mounted on the screen of the second LCD 12. The lower housing 13b is also provided with an insertion port for storing the memory card 17 and the stick 16.

  As the touch panel 15, an arbitrary type such as a resistive film type, an optical type (infrared type), and a capacitive coupling type can be used. The touch panel 15 is an example of a pointing device having a function of outputting coordinate data corresponding to the contact position when the surface of the touch panel 15 is touched with the stick 16. In the following description, it is assumed that the player operates the touch panel 15 with the stick 16, but it is of course possible to operate the touch panel 15 with a pen (stylus pen) or a finger instead of the stick 16. In the present embodiment, the touch panel 15 having a resolution (detection accuracy) of 256 dots × 192 dots is used as in the resolution of the second LCD 12. However, the resolution of the touch panel 15 and the resolution of the second LCD 12 are not necessarily the same.

  The memory card 17 is a recording medium that records an input coordinate processing program and the like, and is detachably attached to an insertion port provided in the lower housing 13b.

  Next, the internal configuration of the game apparatus 1 will be described with reference to FIG. FIG. 2 is a block diagram showing the internal configuration of the game apparatus 1.

  In FIG. 2, a CPU core 21 is mounted on the electronic circuit board 20 accommodated in the housing 13. A connector 23 is connected to the CPU core 21 via a bus 22, an input / output interface circuit (referred to as I / F circuit in the drawing) 25, a first GPU (Graphics Processing Unit) 26, a second GPU 27, a RAM 24, The LCD controller 31 and the wireless communication unit 33 are connected. The memory card 17 is detachably connected to the connector 23. The memory card 17 includes a ROM 17a that stores an input coordinate processing program and a RAM 17b that stores backup data in a rewritable manner. The input coordinate processing program stored in the ROM 17a of the memory card 17 is loaded into the RAM 24, and the input coordinate processing program loaded into the RAM 24 is executed by the CPU core 21. In addition to the input coordinate processing program, the RAM 24 appropriately stores data for generating temporary data obtained by the CPU core 21 executing the program. The I / F circuit 25 is connected to the touch panel 15, the right speaker 30a, the left speaker 30b, and the operation switch unit 14 including the cross switch 14a and the A button 14d shown in FIG. The right speaker 30a and the left speaker 30b are disposed inside the sound release holes 18a and 18b, respectively.

  A first VRAM (Video RAM) 28 is connected to the first GPU 26, and a second VRAM 29 is connected to the second GPU 27. In response to an instruction from the CPU core 21, the first GPU 26 generates a first display image based on data for generating a display image stored in the RAM 24, and draws the first display image in the first VRAM 28. Similarly, the second GPU 27 generates a second display image in accordance with an instruction from the CPU core 21 and draws it in the second VRAM 29. The first VRAM 28 and the second VRAM 29 are connected to the LCD controller 31.

  The LCD controller 31 includes a register 32. The register 32 stores a value of 0 or 1 according to an instruction from the CPU core 21. When the value of the register 32 is 0, the LCD controller 31 outputs the first display image drawn in the first VRAM 28 to the first LCD 11 and the second display image drawn in the second VRAM 29 as the second display image. Output to the LCD 12. When the value of the register 32 is 1, the first display image drawn in the first VRAM 28 is output to the second LCD 12, and the second display image drawn in the second VRAM 29 is output to the first LCD 11. To do.

  The wireless communication unit 33 has a function of exchanging data used for game processing and other data with the wireless communication unit 33 of other game devices. As an example, the wireless LAN standard of IEEE802.11. Provides wireless communication functions that comply with Then, the wireless communication unit 33 outputs the received data to the CPU core 21. Further, the wireless communication unit 33 transmits data instructed by the CPU core 21 to another game device. Note that the game device 1 is connected via the wireless communication unit 33 by mounting a protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol) or a predetermined browser in the wireless communication unit 33 or the storage unit in the game device 1. This makes it possible to connect to a network such as the Internet. The game apparatus 1 can browse data such as documents and images published on the network on the first LCD 11 and the second LCD 12.

  The input coordinate processing program of the present invention may be supplied not only to the computer system through an external storage medium such as the memory card 17 but also to the computer system through a wired or wireless communication line. The input coordinate processing program may be recorded in advance in a nonvolatile storage device inside the computer system. The information storage medium for storing the input coordinate processing program is not limited to the nonvolatile semiconductor memory, but may be a CD-ROM, a DVD, or an optical disk storage medium similar to them.

  Next, with reference to FIG. 3 to FIG. 12, before describing specific processing operations by the input coordinate processing program executed in the game apparatus 1, the processing operations are performed on the first LCD 11 and the second LCD 12. A display form example to be displayed will be described. In order to make the description concrete, the first LCD 11 and the second LCD 12 are used to connect to a network such as the Internet via the wireless communication unit 33 and to display data such as documents and images disclosed on the network. An example will be described. FIG. 3 is a diagram showing an example of an initial screen display displayed on the first LCD 11 and the second LCD 12. FIG. 4 is a diagram illustrating a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 on the link destination displayed on the second LCD 12 is tapped. FIG. 5 is a diagram showing a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the link destination displayed on the second LCD 12. FIG. 6 is displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the link destination displayed on the second LCD 12 and then reciprocated downward. It is a figure which shows the example of a screen display. FIG. 7 is a diagram illustrating a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid downward from the tab displayed on the second LCD 12. FIG. 8 shows screens displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid downward from the tab displayed on the second LCD 12 and then reciprocated upward. It is a figure which shows the example of a display. FIG. 9 is a diagram illustrating a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid so as to surround a part of the information displayed on the second LCD 12. . FIG. 10 is a diagram illustrating a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the background image displayed on the second LCD 12. FIG. 11 is a diagram for explaining each area set when the touch panel 15 on the background image displayed on the second LCD 12 is touch-operated. FIG. 12 is displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the background image displayed on the second LCD 12 and then reciprocated downward. It is a figure which shows the example of a screen display.

  In FIG. 3, when browsing information such as documents and images published on the network via the wireless communication unit 33 on the game apparatus 1, the current browsing is selected only on the second LCD 12 as an example of the initial stage. Information is displayed. For example, homepage information selected by the user is displayed on the second LCD 12. In the example shown in FIG. 3, a text link and an image link indicating a link destination, and a tab for changing browsing to other information in the home page information are displayed as response areas in the home page information. An icon set for browsing the network in the game apparatus 1 is displayed side by side at the left end of the display screen of the second LCD 12. As one of these icons, an enclosure mode icon is set.

  In FIG. 4, when the user touches the touch panel 15 on the link destination displayed on the second LCD 12 (on the image link in FIG. 4) using the stick 16, a circle centered on the touched point. M1 is displayed (state shown in the left diagram of FIG. 4). Then, when the user releases the stick 16 from the touch panel 15, the home page information of the link destination that has been touched is displayed on the second LCD 12 (state shown in the right diagram of FIG. 4). Although the home page information displayed on the second LCD 12 is changed to the linked home page information, the display mode of the first LCD 11 does not change. Hereinafter, as described above, an operation of touching and releasing the touch panel 15 for a moment is referred to as a tap operation.

  In FIG. 5, the user touches the touch panel 15 on the link destination displayed on the second LCD 12 (on the text link in FIG. 5), and slides the stick 16 upward without releasing from the touch panel 15. When the operation is performed, an upward arrow M2 is displayed in the vicinity of the point where the touch operation is currently performed (the state in the left diagram of FIG. 5). Hereinafter, an operation in which the touch panel 15 is touched and slid without leaving the touch panel 15 is referred to as a slide operation. Specifically, the screen coordinate corresponding to the touch input coordinate where the user first touched the touch panel 15 is set as the start point coordinate (x1, y1), and the determination line y = y1 + c1 is set in the same screen coordinate system. Here, c1 is a predetermined constant. Then, when a sliding operation is performed on the touch panel 15 upward to an area above the determination line (touch input coordinates (xt, yt) in FIG. 5), an upward arrow M2 is displayed.

  When the user releases the stick 16 from the touch panel 15 in an area above the determination line after the slide operation, the link destination corresponding to the first touch destination (that is, the link destination corresponding to the start point coordinates (x1, y1)). ) Home page information is displayed on the first LCD 11. The second LCD 12 continuously displays the homepage information as it is without changing the display (the state shown in the right diagram of FIG. 5). That is, the information specified by the first touch operation is displayed on the display screen (first LCD 11) mounted in the user's slide operation direction (upward) while leaving the display information on the second LCD 12.

  In FIG. 6, the user touches the link destination displayed on the second LCD 12 (on the text link in FIG. 6) and slides the touch panel 15 upward without releasing the touch panel 15. When a slide operation reciprocating in the direction is performed, an up and down arrow M3 is displayed in the vicinity of the point where the current touch operation is performed (state in the left diagram of FIG. 6). Specifically, the start point coordinates (x1, y1) and the determination line y = y1 + c1 are set as in FIG. Then, after sliding on the touch panel 15 upward to the area above the determination line, when sliding to the area below the determination line in the reverse direction (touch input coordinates (xt, yt in FIG. 6) )), An up and down arrow M3 is displayed.

  When the user releases the stick 16 from the touch panel 15 in an area below the determination line after the above-and-down sliding operation, the link destination (that is, the start point coordinates (x1, y1) that is touched first) corresponds. Linked home page information is displayed on the second LCD 12. Then, the first LCD 11 displays the home page information that has been displayed on the second LCD 12 until then (the state shown in the right diagram of FIG. 6). That is, the display information on the second LCD 12 is displayed on the first LCD 11, the information specified by the first touch operation is displayed on the second LCD 12, and according to the user's slide operation direction (vertical direction), The information displayed on the upper and lower display screens (first LCD 11 and second LCD 12) is displayed so as to be switched.

  In FIG. 7, when the user touches the touch panel 15 on the tab displayed on the second LCD 12 (on the tab “xxx” in FIG. 7), the tab (that is, the tab corresponding to the start point coordinates (x1, y1)). Other information in the homepage information being browsed corresponding to “xxx”) is immediately displayed on the first LCD 11. When an operation of sliding the stick 16 downward without releasing from the touch panel 15 is performed, a downward arrow M4 is displayed in the vicinity of the currently touched point (state in the left diagram of FIG. 7). Specifically, the screen line corresponding to the touch input coordinate where the user first touched the touch panel 15 is set as the start point coordinate (x1, y1), and the determination line y = y1-c2 is set in the same screen coordinate system. Here, c2 is a predetermined constant. Then, when a sliding operation is performed on the touch panel 15 downward to an area below the determination line (touch input coordinates (xt, yt) in FIG. 7), a downward arrow M4 is displayed.

  When the user releases the stick 16 from the touch panel 15 in an area below the determination line after the slide operation, the information displayed on the first LCD 11 corresponding to the touched tab is displayed on the second LCD 12. The state is displayed (the state shown in the right diagram of FIG. 7). Although the home page information displayed on the second LCD 12 is changed to other information corresponding to the tab, the first LCD 11 continues to display other information corresponding to the tab. In other words, the information specified by the first touch operation is displayed on the display screen (second LCD 12) mounted in the user's slide operation direction (downward) while leaving the display information on the first LCD 11.

  In FIG. 8, when the user performs a touch operation on a tab displayed on the second LCD 12 (on the tab “xxx” in FIG. 8), the tab “xxx” corresponding to the tab (that is, the start point coordinates (x1, y1)). Other information in the homepage information being browsed corresponding to) is immediately displayed on the first LCD 11. Then, when a slide operation is performed to reciprocate upward after sliding the touch panel 15 downward without being separated from the top of the touch panel 15, a vertical arrow M5 is displayed in the vicinity of the currently touched point (see FIG. 8 (state in the left figure). Specifically, the start point coordinates (x1, y1) and the determination line y = y1-c2 are set as in FIG. Then, after sliding on the touch panel 15 downward to the area below the determination line, when sliding to the area above the determination line in the reverse direction (touch input coordinates (xt, yt in FIG. 8) )), An up and down arrow M5 is displayed.

  When the user releases the stick 16 from the touch panel 15 in an area above the determination line after the slide operation that reciprocates up and down, information corresponding to the touched tab is displayed on the second LCD 12. Become. Then, the first LCD 11 displays the original homepage information that has been displayed on the second LCD 12 until then (the state in the right diagram of FIG. 8). That is, the original display information on the second LCD 12 is displayed on the first LCD 11 and the information specified by the first touch operation is displayed on the second LCD 12 according to the user's slide operation direction (vertical direction). Thus, the information displayed on the upper and lower display screens (first LCD 11 and second LCD 12) is displayed so as to be switched.

  In FIG. 9, when the user taps the enclosure mode icon Ia, the game apparatus 1 moves to the enclosure mode and operates. When the user performs a touch operation on the touch panel 15 using the stick 16 so as to surround some information (character information described in the background image in FIG. 9) displayed on the second LCD 12 in the enclosing mode, A trajectory M6 is displayed in accordance with the touched trajectory (the state shown in the left diagram of FIG. 9). Specifically, the screen coordinates corresponding to the touch input coordinates with which the user first touched the touch panel 15 are set as the start point coordinates (x1, y1), and the start point centered on the start point coordinates (x1, y1) in the same screen coordinate system. Set the neighborhood area. Then, when a slide operation is performed to the vicinity of the start point so as to surround the touch panel 15 (touch input coordinates (xt, yt) in FIG. 9), a locus M6 is displayed. When the user releases the stick 16 from the touch panel 15 in the vicinity of the start point, the information surrounded by the locus M6 is enlarged and displayed on the first LCD 11. The second LCD 12 continuously displays the homepage information as it is without changing the display (the state shown in the right diagram of FIG. 9). That is, the information surrounded by the touch operation is displayed on the other display screen (first LCD 11) while leaving the display information on the second LCD 12.

  In FIG. 10, when the user touches the touch panel 15 on the background image displayed on the second LCD 12 and slides upward without releasing from the touch panel 15, the guide image M <b> 7 is located near the start point coordinates. In addition, an upward arrow M8 is displayed in the vicinity of the currently touched point (the state shown in the left diagram of FIG. 10).

  Specifically, as shown in FIG. 11, the screen coordinates corresponding to the touch input coordinates with which the user first touched the touch panel 15 is used as a reference with the start point coordinates (x1, y1) as a plurality of regions in the same screen coordinate system. Set. A predetermined area AM is set around the start point coordinates (x1, y1). In the screen coordinate system (x, y), the predetermined area AM is an area where x1−k1 ≦ x ≦ x1 + k1 and y1−k2 <y <y1 + k2. Here, k1 and k2 are predetermined constants, respectively. Then, a guide image M7 is displayed at the boundary of the predetermined area AM.

  An upper area AT is set above the predetermined area AM. In the screen coordinate system (x, y), the upper area AT is an area where x1−k1 ≦ x ≦ x1 + k1 and y ≧ y1 + k2. A lower area AB is set below the predetermined area AM. In the screen coordinate system (x, y), the lower area AB is an area where x1−k1 ≦ x ≦ x1 + k1 and y ≦ y1−k2. A left area AL is set on the left side of the predetermined area AM. In the screen coordinate system (x, y), the left side area AL is an area where x <x1-k1. A right area AR is set on the right side of the predetermined area AM. In the screen coordinate system (x, y), the right area AR is an area where x> x1 + k1. Then, as shown in FIG. 10, when a slide operation is performed upward on the touch panel 15 to an upper area AT above a predetermined area AM (that is, an area surrounded by the guide image M7) (touch input coordinates (FIG. 10)). xt, yt)), and an upward arrow M8 is displayed.

  After the slide operation, when the user releases the stick 16 from the touch panel 15 in the upper area AT, the home page information being browsed on the second LCD 12 (that is, the home page information designated by the start point coordinates (x1, y1)) ) Is displayed on the first LCD 11. The second LCD 12 continues to display the homepage information as it is without changing the display (the state shown in the right figure of FIG. 10). That is, information displayed at the starting point of the user's slide operation direction (upward) while leaving the display information on the second LCD 12 (information displayed on the second LCD 12 specified by the first touch operation) Is displayed on the display screen (first LCD 11) mounted in the slide operation direction.

  In FIG. 12, the user touches the touch panel 15 on the background image displayed on the second LCD 12 and slides upward to the upper area AT without releasing from the touch panel 15, and then into the predetermined area AM. When a slide operation that reciprocates up and down (touch input coordinates (xt, yt) in FIG. 12) is performed, a guide image M7 is displayed in the vicinity of the start point coordinates, and an up and down arrow M9 is displayed in the vicinity of the currently touched point (The state shown in the left diagram of FIG. 12). Each area set at this time is the same as the area described in FIG.

  When the user releases the stick 16 from the touch panel 15 within the predetermined area AM after the above-described slide operation that reciprocates up and down, the home page information being browsed on the second LCD 12 (that is, designated by the start point coordinates (x1, y1)). Homepage information) is displayed on the first LCD 11. Then, the home page information being browsed on the first LCD 11 is displayed on the second LCD 12 (state shown in the right diagram of FIG. 12). That is, the display information on the second LCD 12 is displayed on the first LCD 11, the display information on the first LCD 11 is displayed on the second LCD 12, and the up and down directions are changed according to the user's slide operation direction (vertical direction). Information displayed on the display screens (first LCD 11 and second LCD 12) is displayed so as to be switched.

  Next, with reference to FIGS. 13 to 19, a specific processing operation by the input coordinate processing program executed by the game apparatus 1 will be described. FIG. 13 is a flowchart showing an operation in which the game apparatus 1 performs input coordinate processing by executing the input coordinate processing program. FIG. 14 is a subroutine showing the detailed operation of the link designation process in step 50 in FIG. FIG. 15 is a subroutine showing the detailed operation of the tab designation process in step 51 in FIG. FIGS. 16 to 18 are subroutines showing detailed operations for the background designation processing in step 52 in FIG. FIG. 19 is a diagram showing an example of various data stored in the RAM 24 by the processing operation based on FIG. The program for executing these processes is included in the input coordinate processing program stored in the ROM 17a, and is read from the ROM 17a to the RAM 24 when the game apparatus 1 is turned on. It is executed by the CPU core 21. In order to make the description more specific, the first LCD 11 and the second LCD 12 are connected to a network such as the Internet via the wireless communication unit 33, and data such as documents and images disclosed on the network are displayed. The operation will be described using an example of browsing in the above.

  First, when a power source (not shown) of the game apparatus 1 is turned on, a boot program (not shown) is executed by the CPU core 21, whereby the input coordinate processing program stored in the memory card 17 is stored in the RAM 24. Loaded. When the loaded input coordinate processing program is executed by the CPU core 21, the steps shown in FIG. 13 (abbreviated as “S” in FIGS. 13 to 18) are executed.

  In FIG. 13, the CPU core 21 displays information such as documents and images published on the designated web page on at least the second LCD 12 in accordance with a user operation (step 41; see FIG. 3). Then, the CPU core 21 sets the touch input flag DFt stored in the RAM 24 to OFF (step 42), and advances the processing to the next step. In step 42, the CPU core 21 sets all the first to twelfth mode flags DF1 to DF12 stored in the RAM 24 to off.

  Here, as shown in FIG. 19, the coordinate data input from the touch panel 15 is converted into coordinates on the image displayed on the second LCD 12 corresponding to the touch position where the touch panel 15 is touched, and the RAM 24. Are stored as touch input coordinates DC1. In addition, in the RAM 24, start point coordinates DC2, trajectory coordinates DC3, and the like are appropriately stored as position data DC for generating an image. In addition to the touch input flag DFt, the RAM 24 includes first to twelfth mode flags DF1 to DF12 as flag data DF for determining a process to be performed next (hereinafter referred to as the next process). Remembered. Further, an index image DI1, a trajectory image DI2, a guide image DI3, and the like are appropriately stored as image data DI for generating an image indicating the recognized operation gesture.

  Returning to FIG. 13, the CPU core 21 determines whether or not there is a touch input from the touch panel 15 according to a user operation (step 43). If there is a touch input, the process proceeds to the next step 44. On the other hand, if there is no touch input, it is determined whether or not browsing of the currently displayed information is to be terminated (step 53). Next, when continuing browsing, the CPU core 21 returns to step 43 to repeat the process, and when ending browsing, ends the process according to the flowchart.

  In step 44, the CPU core 21 sets the coordinates on the image displayed on the second LCD 12 corresponding to the touch position where the touch panel 15 is currently touched (that is, the current touch input coordinates DC1) as the start point coordinates DC2 in the RAM 24. To remember. Next, the CPU core 21 sets the touch input flag DFt stored in the RAM 24 to ON (step 45), and advances the processing to the next step.

  Next, the CPU core 21 determines whether the image corresponding to the position indicated by the starting point coordinate DC2 is a link (step 46), a tab (step 47), and a background image (step 48). to decide. When the image corresponding to the position indicated by the start point coordinate DC2 is a link such as an image link or a text link (Yes in Step 46), the CPU core 21 performs a link designation process (Step 50) and performs the above Step 42. Return to and repeat the process. If the image corresponding to the position indicated by the start point coordinate DC2 is a tab (Yes in step 47), the CPU core 21 performs tab designation processing (step 51), returns to step 42, and repeats the processing. When the image corresponding to the position indicated by the start point coordinate DC2 is a background image (Yes in step 48), the CPU core 21 performs background designation processing (step 52), returns to step 42, and repeats the processing. . Further, when the image corresponding to the position indicated by the start point coordinate DC2 is neither a link, a tab nor a background image (No in Steps 46 to 48), the CPU core 21 responds to the position indicated by the start point coordinate DC2. The process is repeated (step 49), and the process returns to step 42 to repeat the process. Hereinafter, detailed operations of the link designation process, the tab designation process, and the background designation process will be described.

  In FIG. 14, in the link designation process in step 50, first, the CPU core 21 sets the first mode flag DF1 stored in the RAM 24 to ON (step 61). Then, the CPU core 21 displays the circle M1 centered on the current touch input coordinate DC1 on the information displayed on the second LCD 12 using the index image DI1 (step 62; see FIG. 4), and the user It is determined whether or not has been touched off (step 63). The CPU core 21 proceeds to the next step 64 when the user touches off, and proceeds to step 65 when the touch operation is continued.

  In step 64, the CPU core 21 executes the first mode based on the first mode flag DF1 that is currently turned on, and ends the processing by the subroutine. Here, the first mode is a process for performing the display described with reference to the right side of FIG. 4. The display mode of the first LCD 11 is not changed, and the home page information of the link destination that has been tapped is changed to the second mode. This is a process of displaying on the LCD 12.

  In step 65, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in an image area above the determination line including the determination line (hereinafter referred to as an area equal to or higher than the determination line). Then, the CPU core 21 proceeds to the next step 66 when the touch input coordinate DC1 is in an area equal to or higher than the determination line, and returns to step 62 and repeats the process when it is in an area below the determination line. Here, the CPU core 21 sets the determination line used in step 65 based on the starting point coordinates DC2 (x1, y1) as y = y1 + c1 (c1: constant). Then, if the current touch input coordinate DC1 (xt, yt) is yt ≧ y1 + c1, the CPU core 21 determines that the current touch input coordinate DC1 is in an area greater than the determination line.

  In step 66, the CPU core 21 sets the second mode flag DF2 stored in the RAM 24 to ON, and sets the first mode flag DF1 and the third mode flag DF3 to OFF. Then, the CPU core 21 displays an upward arrow M2 on the information displayed on the second LCD 12 in the vicinity of the current touch input coordinate DC1 using the index image DI1 (step 67; see FIG. 5), and the user It is determined whether or not has been touched off (step 68). Then, the CPU core 21 proceeds to the next step 69 when the user touches off, and proceeds to step 70 when the touch operation is continued.

  In step 69, the CPU core 21 executes the second mode based on the second mode flag DF2 that is currently turned on, and ends the processing by the subroutine. Here, the second mode is a process for performing the display described with reference to the right side of FIG. 5, and the link destination home page information is displayed on the first LCD 11 without changing the display mode of the second LCD 12. It is processing to do.

  In step 70, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in an area below the determination line used in step 65. If the touch input coordinate DC1 is in the area below the determination line, the CPU core 21 proceeds to the next step 71. On the other hand, when the second mode flag DF2 is set to ON when the CPU core 21 is in an area equal to or greater than the determination line, the process returns to step 67 to repeat the process, and the second mode flag DF2 is set to OFF. Then, the process returns to step 66 and the process is repeated (step 75).

  In step 71, the CPU core 21 sets the third mode flag DF3 stored in the RAM 24 to ON and sets the second mode flag DF2 to OFF. Then, the CPU core 21 displays an up and down arrow M3 on the information displayed on the second LCD 12 in the vicinity of the current touch input coordinate DC1 using the index image DI1 (step 72; see FIG. 6). It is determined whether or not the user has touched off (step 73). Then, the CPU core 21 proceeds to the next step 74 when the user touches off, and returns to step 70 to repeat the process when the touch operation is continued.

  In step 74, the CPU core 21 executes the third mode based on the third mode flag DF3 that is currently set to ON, and ends the processing by the subroutine. Here, the third mode is a process for performing the display described with reference to the right diagram of FIG. It is a process displayed on LCD12.

  Here, after the third mode flag DF3 is set to ON in step 71, when the slide operation is performed to an area equal to or more than the determination line (that is, No in step 70), step 66 is executed. Therefore, the third mode flag DF3 is set to off and the second mode flag DF2 is set to on again. That is, the downward slide operation gesture using the touch panel 15 from the region above the determination line to the region below the determination line is canceled, and the upward slide operation from the start point DC2 to the region above the determination line. Only is recognized as a valid operation gesture.

  In FIG. 15, in the tab designating process in step 51, the CPU core 21 first turns on the fourth mode flag DF4 stored in the RAM 24 (step 81). Next, the CPU core 21 executes the fourth mode based on the fourth mode flag DF4 that is currently turned on (step 82), and advances the processing to the next step. Here, the fourth mode is a process of immediately displaying other information in the browsed homepage information corresponding to the touch-operated tab (that is, the tab corresponding to the start point coordinate DC2) on the first LCD 11. (For example, see the left diagram in FIG. 7 and the left diagram in FIG. 8).

  Next, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in an image area below the determination line including the determination line (hereinafter referred to as an area below the determination line) (step 83). ). Then, the CPU core 21 advances the process to the next step 85 when the touch input coordinate DC1 is in the area below the determination line. On the other hand, when the touch input coordinate DC1 is in an area above the determination line, the CPU core 21 determines whether or not the user has touched off (step 84). Then, when the touch operation is continued, the CPU core 21 returns to step 83 and repeats the process. When the user touches off, the CPU core 21 ends the subroutine. Here, the CPU core 21 sets y = y1−c2 as a determination line used in step 83 based on the start point coordinate DC2 (x1, y1) (c2: constant). Then, if the current touch input coordinate DC1 (xt, yt) is yt ≦ y1-c2, the CPU core 21 determines that the current touch input coordinate DC1 is in the area below the determination line.

  In step 85, the CPU core 21 sets the fifth mode flag DF5 stored in the RAM 24 to ON, and sets the fourth mode flag DF4 and the sixth mode flag DF6 to OFF. Then, the CPU core 21 displays a downward arrow M4 on the information displayed on the second LCD 12 in the vicinity of the current touch input coordinate DC1 using the index image DI1 (step 86; see FIG. 7), and the user It is determined whether or not has been touched off (step 87). Then, the CPU core 21 proceeds to the next step 88 when the user touches off, and proceeds to step 89 when the touch operation is continued.

  In step 88, the CPU core 21 executes the fifth mode based on the fifth mode flag DF5 that is currently turned on, and ends the processing by the subroutine. Here, the fifth mode is a process for performing the display described with reference to the right diagram of FIG. 7, and other information displayed on the first LCD 11 corresponding to the touch-operated tab is displayed as the second mode. This is a process of displaying on the LCD 12.

  In step 89, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in an area above the determination line used in step 83. Then, the CPU core 21 advances the process to the next step 90 when the touch input coordinate DC1 is in an area above the determination line. On the other hand, if the CPU core 21 is in the area below the determination line, if the fifth mode flag DF5 is set to ON, the CPU core 21 returns to step 86 and repeats the process, and the fifth mode flag DF5 is set to OFF. If so, the process returns to step 85 to repeat the process (step 94).

  In step 90, the CPU core 21 sets the sixth mode flag DF6 stored in the RAM 24 to ON and sets the fifth mode flag DF5 to OFF. Then, the CPU core 21 displays an up and down arrow M5 on the information displayed on the second LCD 12 in the vicinity of the current touch input coordinate DC1 using the index image DI1 (step 91; see FIG. 8). It is determined whether or not the user has touched off (step 92). Then, the CPU core 21 proceeds to the next step 93 when the user touches off, and returns to step 89 and repeats the process when the touch operation is continued.

  In step 93, the CPU core 21 executes the sixth mode based on the sixth mode flag DF6 that is currently turned on, and ends the processing by the subroutine. Here, the sixth mode is a process for performing the display described with reference to the right diagram of FIG. 8. The original display information on the second LCD 12 is displayed on the first LCD 11 and the touch-operated tab is displayed. Correspondingly, this is a process of displaying other information displayed on the first LCD 11 on the second LCD 12.

  Here, after the sixth mode flag DF6 is set to ON in Step 90, when the slide operation is performed to the area below the determination line (that is, No in Step 89), Step 85 is executed. Accordingly, the sixth mode flag DF6 is set off and the fifth mode flag DF5 is set on again. That is, the upward slide operation gesture using the touch panel 15 from the area below the determination line to the area above the determination line is canceled, and the downward slide operation from the start point DC2 to the area below the determination line. Only is recognized as a valid operation gesture.

  In FIG. 16, in the background designating process in step 52, first, the CPU core 21 determines whether or not the enclosing mode is currently set (step 101). If the CPU core 21 is not in the enclosing mode, the CPU core 21 proceeds to the next step 111 (FIG. 17). On the other hand, the CPU core 21 proceeds to the next step 102 in the case of the enclosure mode. Here, as described with reference to FIG. 9, the enclosure mode is an operation mode that is shifted when the user taps the enclosure mode icon Ia.

  In step 102, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in the start point vicinity region. Next, the CPU core 21 determines whether or not to form an enclosing curve when the locus coordinates DC3 are connected in time series order (step 103). When the touch input coordinate DC1 is in the vicinity of the start point and the locus coordinate DC3 group forms an encircling curve (Yes in steps 102 and 103; see the left diagram in FIG. 9), the CPU core 21 The process proceeds to the next step 105. On the other hand, when the touch input coordinate DC1 is not in the vicinity of the start point or the locus coordinate DC3 group does not form an enclosing curve (No in Step 102 and Step 103), the CPU core 21 is touched off by the user. Whether or not (step 104). Then, when the user touches off, the CPU core 21 ends the processing by the subroutine. On the other hand, when the touch operation is continued, the CPU core 21 adds and stores the current touch input coordinate DC1 as the locus coordinate DC3 in the RAM 24 (step 109), and returns to step 101 to continue the process.

  In step 105, the CPU core 21 sets the seventh mode flag DF7 stored in the RAM 24 to ON. Then, the CPU core 21 displays the trajectory M6 along the current trajectory coordinate DC3 on the information displayed on the second LCD 12 using the trajectory image DI2 (step 106; see FIG. 9), and the user It is determined whether or not touch-off has been performed (step 107). Then, the CPU core 21 proceeds to the next step 108 when the user touches off, and returns to step 102 to continue the process when the touch operation is continued.

  In step 108, the CPU core 21 executes the seventh mode based on the seventh mode flag DF7 that is currently turned on, and ends the processing by the subroutine. The CPU core 21 clears all the trajectory coordinates DC3 stored in the RAM 24 after executing the seventh mode. Here, the seventh mode is a process for performing the display described with reference to the right side of FIG. 9, and the information surrounded by the locus M6 is enlarged without changing the display mode of the second LCD 12. This is a process of displaying on one LCD 11.

  In FIG. 17, when the background designation process in step 52 is not the enclosed mode (No in step 101), the CPU core 21 executes step 111. In step 111, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in the upper area AT (see FIG. 11). Then, when the touch input coordinate DC1 is in the upper area AT, the CPU core 21 advances the process to the next step 112. On the other hand, if the touch input coordinate DC1 is not in the upper area AT, the CPU core 21 proceeds to the next step 116. Here, the CPU core 21 sets the upper area AT used in step 111 with x1−k1 ≦ x ≦ x1 + k1 and y ≧ y1 + k2 based on the starting point coordinate DC2 (x1, y1) (k1, k2: constant). . Then, the CPU core 21 determines that the current touch input coordinate DC1 is in the upper area AT if the current touch input coordinate DC1 (xt, yt) is x1−k1 ≦ xt ≦ x1 + k1 and yt ≧ y1 + k2. To do.

  In step 112, the CPU core 21 sets the eighth mode flag DF8 stored in the RAM 24 to ON, and sets the ninth to twelfth mode flags DF9 to DF12 to OFF. Then, the CPU core 21 displays the guide image M7 in the vicinity of the current touch input coordinate DC1 and the guide image M7 in the vicinity of the start point coordinate DC2 on the information displayed on the second LCD 12 using the index image DI1. (Step 113; see FIG. 10), it is determined whether or not the user has touched off (Step 114). Then, the CPU core 21 proceeds to the next step 115 when the user touches off, and proceeds to step 116 when the touch operation is continued.

  In step 115, the CPU core 21 executes the eighth mode on the basis of the eighth mode flag DF8 that is currently turned on, and ends the processing by the subroutine. Here, the eighth mode is a process for performing the display described with reference to the right diagram of FIG. 10, and the information displayed on the second LCD 12 is changed to the first information while the display information on the second LCD 12 is left. This process is also displayed on the LCD 11.

  In step 116, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in the predetermined area AM (see FIG. 11). Then, if the touch input coordinate DC1 is not in the predetermined area AM, the CPU core 21 proceeds to the next step 131 (FIG. 18). On the other hand, when the touch input coordinate DC1 is in the predetermined area AM, the CPU core 21 proceeds to the next step 118 if the eighth mode flag DF8 is set to ON, and sets the eighth mode flag DF8 to OFF. If so, the process proceeds to the next step 131 (step 117). Here, the CPU core 21 sets the predetermined area AM used in step 116 based on the start point coordinate DC2 (x1, y1) as x1−k1 ≦ x ≦ x1 + k1 and y1−k2 <y <y1 + k2 (k1, k2: constant). If the current touch input coordinate DC1 (xt, yt) is x1−k1 ≦ xt ≦ x1 + k1 and y1−k2 <yt <y1 + k2, the CPU core 21 determines that the current touch input coordinate DC1 is the predetermined area AM. It is determined that

  In step 118, the CPU core 21 sets the ninth mode flag DF9 stored in the RAM 24 to ON, and sets the eighth mode flag DF8 to OFF. Then, the CPU core 21 uses the index image DI1 to display a guide image M7 in the vicinity of the current touch input coordinate DC1 and a guide image M7 around the start point coordinate DC2 on the information displayed on the second LCD 12. (Step 119; see FIG. 12), it is determined whether or not the user has touched off (Step 120). Then, the CPU core 21 proceeds to the next step 121 when the user touches off, and proceeds to step 131 (FIG. 18) when the touch operation is continued.

  In step 121, the CPU core 21 executes the ninth mode based on the ninth mode flag DF9 that is currently turned on, and ends the processing by the subroutine. Here, the ninth mode is a process for performing the display described with reference to the right diagram of FIG. 12, and the information displayed on the first LCD 11 and the information displayed on the second LCD 12 are switched. This is a process of displaying on the first LCD 11 and the second LCD 12.

  In step 131 of FIG. 18, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in the lower area AB (see FIG. 11). Then, if the touch input coordinate DC1 is not in the lower area AB, the CPU core 21 advances the process to the next step 136. On the other hand, if the touch input coordinate DC1 is in the lower area AB, the CPU core 21 advances the process to the next step 132. Here, the CPU core 21 sets the lower region AB used in step 131 with reference to the starting point coordinate DC2 (x1, y1) by x1−k1 ≦ x ≦ x1 + k1 and y ≦ y1−k2 (k1, k2: constant). Then, the CPU core 21 determines that the current touch input coordinate DC1 is in the lower area AB if the current touch input coordinate DC1 (xt, yt) is x1−k1 ≦ xt ≦ x1 + k1 and yt ≦ y1−k2. Judge.

  In step 132, the CPU core 21 turns on the tenth mode flag DF10 stored in the RAM 24, and turns off the eighth, ninth, eleventh, and twelfth mode flags DF8, DF9, DF11, and DF12. Set to. Then, the CPU core 21 uses the index image DI1 to display a down arrow M10 (not shown) near the current touch input coordinate DC1 and a guide image M7 (see FIG. 10) around the start point coordinate DC2 on the second LCD 12. (Step 133), and it is determined whether or not the user has touched off (step 134). Then, the CPU core 21 proceeds to the next step 135 when the user touches off, and proceeds to step 136 when the touch operation is continued.

  In step 135, the CPU core 21 executes the tenth mode based on the tenth mode flag DF10 that is currently turned on, and ends the processing by the subroutine. Here, the tenth mode is a process of displaying the information displayed on the first LCD 11 on the second LCD 12 while leaving the display information on the first LCD 11. That is, in the tenth mode, the information displayed on the reverse direction of the user's slide operation direction (downward) (information displayed on the first LCD 11) remains while the display information on the first LCD 11 remains. It is displayed on the display screen (second LCD 12) mounted in the slide operation direction.

  In step 136, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in the left area AL (see FIG. 11). Then, when the touch input coordinate DC1 is not in the left area AL, the CPU core 21 advances the processing to the next step 141. On the other hand, if the touch input coordinate DC1 is in the left area AL, the CPU core 21 proceeds to the next step 137. Here, the CPU core 21 sets the left area AL used in step 136 with x <x1-k1 based on the starting point coordinate DC2 (x1, y1) (k1: constant). Then, if the current touch input coordinate DC1 (xt, yt) is xt <x1-k1, the CPU core 21 determines that the current touch input coordinate DC1 is in the left area AL.

  In step 137, the CPU core 21 sets the eleventh mode flag DF11 stored in the RAM 24 to ON, and sets the eighth to tenth mode flags DF8 to DF10 and the twelfth mode flag DF12 to OFF. Then, the CPU core 21 uses the index image DI1 to display the guide image M7 (see FIG. 10) around the start point coordinate DC2 and the left arrow M11 (not shown) near the current touch input coordinate DC1 and the second LCD 12. (Step 138), and it is determined whether or not the user has touched off (step 139). Then, the CPU core 21 proceeds to the next step 140 when the user touches off, and proceeds to step 141 when the touch operation is continued.

  In step 140, the CPU core 21 executes the eleventh mode based on the eleventh mode flag DF11 that is currently turned on, and ends the processing by the subroutine. Here, in the eleventh mode, the information displayed on the second LCD 12 in the previous process or the current second display is left while the display information on the first LCD 11 is left in response to the user's leftward sliding operation. This is a process of displaying information corresponding to the previous page of the information displayed on the LCD 12 on the second LCD 12. For example, in the eleventh mode, a so-called “return” operation of a browser or an operation of displaying a previous page in an electronic book or the like can be performed on the information displayed on the second LCD 12. When the electronic book is displayed on the second LCD 12, the left direction is generally an instruction to display the previous page in the horizontally written electronic book, but the document configuration of the electronic book to be displayed (for example, vertical writing, comics, Depending on the (foreign language document), the instruction to display the next page may be set to the left slide operation.

  In step 141, the CPU core 21 determines whether or not the current touch input coordinate DC1 is in the right area AR (see FIG. 11). If the touch input coordinate DC1 is not in the right area AR, the CPU core 21 returns to step 111 (FIG. 17) and continues the process. On the other hand, when the touch input coordinate DC1 is in the right area AR, the CPU core 21 proceeds to the next step 142. Here, the CPU core 21 sets the right area AR used in step 141 with x> x1 + k1 based on the start point coordinate DC2 (x1, y1) (k1: constant). Then, if the current touch input coordinate DC1 (xt, yt) is xt> x1 + k1, the CPU core 21 determines that the current touch input coordinate DC1 is in the right area AR.

  In step 142, the CPU core 21 sets the twelfth mode flag DF12 stored in the RAM 24 to ON, and sets the eighth to eleventh mode flags DF8 to DF11 to OFF. Then, the CPU core 21 uses the index image DI1 to display a guide image M7 (see FIG. 10) in the vicinity of the current touch input coordinate DC1 and a guide image M7 (see FIG. 10) around the start point coordinate DC2 on the second LCD 12. Is displayed on the information displayed in (Step 143), and it is determined whether or not the user has touched off (Step 144). Then, the CPU core 21 proceeds to the next step 145 when the user touches off, and proceeds to step 111 when the touch operation is continued.

  In step 145, the CPU core 21 executes the twelfth mode based on the twelfth mode flag DF12 that is currently set to ON, and ends the processing by the subroutine. Here, in the twelfth mode, the information displayed on the second LCD 12 as the next process and the current second display are left while the display information on the first LCD 11 is left in response to the user's rightward sliding operation. This is processing for displaying information corresponding to the next page of information displayed on the LCD 12 on the second LCD 12. For example, in the twelfth mode, a so-called “forward” operation of the browser or an operation of displaying the next page in an electronic book or the like can be performed on the information displayed on the second LCD 12. When an electronic book is displayed on the second LCD 12, the right direction is generally an instruction to display the next page in a horizontally written electronic book, but the previous page is displayed in accordance with the document configuration of the electronic book to be displayed. You may set the instruction to slide to the right.

  Here, in the background designating process other than the enclosing mode, after any of the eighth to twelfth mode flags DF8 to DF12 is set to ON, when the slide operation is performed from the currently touched area to another area, the ON is performed. The mode flag that was set to is set to OFF, and the other mode flags are set to ON. For example, when the eighth mode flag DF8 is set to ON in step 112 and a slide operation is performed to the left area AL (that is, Yes in step 136), the eighth mode flag DF8 is set to OFF and the eleventh mode is set. The flag DF11 is set on. In this case, the upward slide operation gesture using the touch panel 15 from the start point coordinate DC2 to the upper region AT is canceled, and only the left slide operation from the start point coordinate DC2 to the left region AL is recognized as an effective operation gesture. Is done. That is, it is possible to easily cancel a slide operation gesture for performing a predetermined operation, and only a slide operation from the start point DC2 to the corresponding region is recognized as an effective operation gesture by simply sliding to another region. can do.

  In the above description, the gesture in the four directions using the touch panel 15 is configured to be recognized. However, the operation gesture in five or more directions may be recognized. By further subdividing the other areas around the predetermined area AM, gestures in five directions or more using the touch panel 15 can be recognized. In this case, the guide image M7 is displayed as a rectangular pattern as a boundary of the predetermined area AM (see FIG. 10), and is a target image for the user to recognize an effective operation gesture using the touch panel 15. It may be displayed in the shape of the shape. For example, when making it possible to recognize not only four-way gestures using the touch panel 15 as described above but also multi-directional operation gestures, polygonal symbols corresponding to the number of direction segments may be displayed.

  As described above, in the game apparatus 1, in the third mode in step 74, the sixth mode in step 93, the seventh mode in step 108, the ninth mode in step 121, and the like, a plurality of responses corresponding to the input from the touch panel 15 are provided. A process to be executed by recognizing the user's gesture by a combination of slide operation directions (for example, the vertical direction) is determined. Therefore, there are more variations in the next process than distinguishing user operation gestures by a single slide operation direction. On the other hand, it is possible to easily cancel the slide operation gesture, and only the slide operation from the start point coordinates to the region can be recognized as an effective operation gesture by simply sliding to another region. Therefore, when the slide operation performed by the user is inappropriate, the user can easily cancel the operation, and can give a new operation instruction by an operation continuous with the operation. That is, according to the present invention, while increasing the variation of the next process that can be specified by the coordinate input operation, the inconvenience caused when an inappropriate locus is drawn is reduced, and the operability of the coordinate input is improved. be able to. In addition, since the operation gesture recognized by the coordinate input operation input by the user is displayed as an index near the touch position, the user can surely recognize the input operation gesture and the next process.

  In the above description, each input coordinate process has been described using a specific operation procedure for specific description. However, these are only examples, and the present invention is limited to these operation procedures. It goes without saying that there is nothing. For example, the number of slide operations that can be recognized as operation gestures on the touch panel 15 may be further increased.

  Further, in the above description, in order to make the description more specific, description has been made using an example in which data such as documents and images published on the network is browsed on the first LCD 11 and the second LCD 12, but other information is also available. Alternatively, the image may be displayed on the first LCD 11 and the second LCD 12. For example, a digital book that is converted into digital data and recorded on an electronic medium, such as a dictionary, may be displayed on the first LCD 11 and the second LCD 12 as viewed on the game apparatus 1.

  In the above-described embodiment, as an example of the liquid crystal display unit for two screens, a case where the first LCD 11 and the second LCD 12 that are physically separated from each other are arranged one above the other (in the case of two upper and lower screens) will be described. did. However, the configuration of the display screen for two screens may be other configurations. For example, the first LCD 11 and the second LCD 12 may be arranged on the left and right on one main surface of the lower housing 13b. In addition, a vertically long LCD having the same horizontal width as the second LCD 12 and having a vertical length twice as long (that is, a single LCD having a vertical display size of two screens) is placed below. It may be arranged on one main surface of the side housing 13b so that the first and second display images are displayed vertically (that is, displayed adjacently without the upper and lower boundary portions). In addition, a horizontally long LCD having the same vertical width as the second LCD 12 and having a width twice as long as the second LCD 12 is disposed on one main surface of the lower housing 13b, and the first and second in the horizontal direction. These display images may be displayed on the left and right (that is, displayed adjacently without the left and right boundary portions). That is, the first and second display images may be displayed by physically dividing one screen into two. Regardless of the form of the game image, the present invention can be similarly realized by disposing the touch panel 15 on the screen on which the second display image is displayed. When the first and second display images are displayed by physically dividing one screen into two, the touch panel 15 may be disposed on the entire screen.

  In the above-described embodiment, the touch panel 15 is integrally provided on the game apparatus 1. However, it goes without saying that the present invention can be realized even if the game apparatus and the touch panel are configured separately. Further, the touch panel 15 may be provided on the upper surface of the first LCD 11. Furthermore, in the above embodiment, two display screens (first LCD 11 and second LCD 12) are provided, but one display screen may be used. That is, in the above-described embodiment, the touch panel 15 may be provided by using only the second LCD 12 as a display screen without providing the first LCD 11. Further, in the above embodiment, the touch panel 15 may be provided on the upper surface of the first LCD 11 without providing the second LCD 12.

  Moreover, in the said Example, although the touch panel was used as an input means of the game device 1, other pointing devices may be used. Here, the pointing device is an input device that specifies an input position and coordinates on the screen. For example, a mouse, a trackpad, a trackball, or the like is used as an input means, and is calculated from an output value output from the input means. The present invention can be similarly realized by using the information on the screen coordinate system.

  In the above embodiment, the touch panel 15 is integrally provided on the game apparatus 1, but an information processing apparatus (input coordinate processing apparatus) such as a general personal computer using the touch panel as one of input means may be used. .

  The input coordinate processing program, the input coordinate processing device, the input coordinate processing system, and the input coordinate processing method of the present invention facilitate cancellation of an input operation gesture while increasing the number of user operation gestures that can be recognized using a pointing device. Information processing apparatus such as a game apparatus that can browse information such as documents and images published on a network and display display information such as an electronic book on a display apparatus, and the information processing apparatus It is useful as a program executed in

1 game device 11 first LCD
12 Second LCD
13 Housing 13a Upper housing 13b Lower housing 14 Operation switch 14a Cross switch 14b Start switch 14c Select switch 14d A button 14e B button 14f X button 14g Y button 14h Power switch 14L L button 14R R button 15 Touch panel 16 Stick 17 Memory card 17a ROM
17b RAM
18a, 18b Sound release hole 20 Electronic circuit board 21 CPU core 22 Bus 23 Connector 24 RAM
25 I / F circuit 26 1st GPU
27 Second GPU
28 First VRAM
29 Second VRAM
30a Right speaker 30b Left speaker 31 LCD controller 32 Register 33 Wireless communication unit

Claims (4)

An input coordinate processing program executed by a computer of an apparatus operated by input coordinates output from a pointing device according to a user operation,
In the computer,
An input coordinate detection step for detecting coordinate information based on a predetermined coordinate system in accordance with input coordinates output from the pointing device;
Of the series of coordinate information detected in the input coordinate detection step, a reference coordinate storage step of setting a reference coordinate based on the first coordinate information and storing it in a memory;
A first process determination step for determining a process based on a direction from a position indicated by the reference coordinates to a position indicated by the last coordinate information in a series of coordinate information detected in the input coordinate detection step;
The direction from the position indicated by the reference coordinates to the position indicated by intermediate coordinate information in the series of coordinate information detected in the input coordinate detection step, and the direction from the position indicated by the intermediate coordinate information to the position indicated by the reference coordinates A second process determining step for determining a process based on a combination of the above, and an image based on the coordinate system is displayed on a display device, and a position indicated by the coordinate information detected at present among the series of coordinate information is In the vicinity of the position indicated by the current coordinate information, an index indicating the direction from the position indicated by the reference coordinate to the position indicated by the current coordinate information when the distance is greater than a predetermined distance from the position indicated by the reference coordinate The position indicated by the coordinate information in the middle of the series of coordinate information is indicated by the reference coordinates. From a position indicated by the reference coordinates to a position indicated by the intermediate coordinate information when the position indicated by the current coordinate information is approaching a position less than the predetermined distance after being separated by a predetermined distance or more A display control step of displaying an index combining a direction from the position indicated by the coordinate information and the position indicated by the reference coordinate information to the position indicated by the reference coordinates on the image in the vicinity of the position indicated by the current coordinate information. Input coordinate processing program. An input coordinate processing apparatus operated by input coordinates output from a pointing device in accordance with a user operation,
Storage means;
Input coordinate detection means for detecting coordinate information based on a predetermined coordinate system according to input coordinates output from the pointing device;
Among the series of coordinate information detected by the input coordinate detection means, reference coordinate storage processing means for setting reference coordinates based on the first coordinate information and storing them in the storage means;
First processing determining means for determining processing based on a direction from a position indicated by the reference coordinates to a position indicated by the last coordinate information in a series of coordinate information detected by the input coordinate detecting means;
The direction from the position indicated by the reference coordinates to the position indicated by intermediate coordinate information in the series of coordinate information detected by the input coordinate detection means, and the direction from the position indicated by the intermediate coordinate information to the position indicated by the reference coordinates Second process determining means for determining a process based on the combination of:
When an image based on the coordinate system is displayed on a display device, and the position indicated by the coordinate information detected at present in the series of coordinate information is more than a predetermined distance from the position indicated by the reference coordinates An index indicating the direction from the position indicated by the reference coordinates to the position indicated by the current coordinate information is displayed on the image in the vicinity of the position indicated by the current coordinate information. The position indicated by the reference coordinates when the position indicated by the current coordinate information approaches a position less than the predetermined distance after the position indicated by the coordinate information is separated from the position indicated by the reference coordinates by a predetermined distance or more. The direction from the position to the position indicated by the intermediate coordinate information and the intermediate coordinate information An index that combines direction from the position to the position indicated by the reference coordinate and a display control means for displaying the image near a position indicated by the coordinate information of the current input coordinate processor. An input coordinate processing system operated by input coordinates output from a pointing device in accordance with a user operation,
Storage means;
Input coordinate detection means for detecting coordinate information based on a predetermined coordinate system according to input coordinates output from the pointing device;
Among the series of coordinate information detected by the input coordinate detection means, reference coordinate storage processing means for setting reference coordinates based on the first coordinate information and storing them in the storage means;
First processing determining means for determining processing based on a direction from a position indicated by the reference coordinates to a position indicated by the last coordinate information in a series of coordinate information detected by the input coordinate detecting means;
The direction from the position indicated by the reference coordinates to the position indicated by intermediate coordinate information in the series of coordinate information detected by the input coordinate detection means, and the direction from the position indicated by the intermediate coordinate information to the position indicated by the reference coordinates Second process determining means for determining a process based on the combination of:
When an image based on the coordinate system is displayed on a display device, and the position indicated by the coordinate information detected at present in the series of coordinate information is more than a predetermined distance from the position indicated by the reference coordinates An index indicating the direction from the position indicated by the reference coordinates to the position indicated by the current coordinate information is displayed on the image in the vicinity of the position indicated by the current coordinate information. The position indicated by the reference coordinates when the position indicated by the current coordinate information approaches a position less than the predetermined distance after the position indicated by the coordinate information is separated from the position indicated by the reference coordinates by a predetermined distance or more. The direction from the position to the position indicated by the intermediate coordinate information and the intermediate coordinate information An index that combines direction from the position to the position indicated by the reference coordinate and a display control means for displaying the image near a position indicated by the coordinate information of the current input coordinate processing system. An input coordinate processing method operated by input coordinates output from a pointing device in accordance with a user operation,
An input coordinate detection step for detecting coordinate information based on a predetermined coordinate system in accordance with input coordinates output from the pointing device;
Of the series of coordinate information detected in the input coordinate detection step, a reference coordinate storage step of setting a reference coordinate based on the first coordinate information and storing it in a memory;
A first process determination step for determining a process based on a direction from a position indicated by the reference coordinates to a position indicated by the last coordinate information in a series of coordinate information detected in the input coordinate detection step;
The direction from the position indicated by the reference coordinates to the position indicated by intermediate coordinate information in the series of coordinate information detected in the input coordinate detection step, and the direction from the position indicated by the intermediate coordinate information to the position indicated by the reference coordinates A second process determining step for determining a process based on a combination of the above, and an image based on the coordinate system is displayed on a display device, and a position indicated by the coordinate information detected at present among the series of coordinate information is In the vicinity of the position indicated by the current coordinate information, an index indicating the direction from the position indicated by the reference coordinate to the position indicated by the current coordinate information when the distance is greater than a predetermined distance from the position indicated by the reference coordinate The position indicated by the coordinate information in the middle of the series of coordinate information is indicated by the reference coordinates. From a position indicated by the reference coordinates to a position indicated by the intermediate coordinate information when the position indicated by the current coordinate information is approaching a position less than the predetermined distance after being separated by a predetermined distance or more A display control step of displaying on the image in the vicinity of the position indicated by the current coordinate information an indicator that combines the direction from the position indicated by the coordinate information and the position indicated by the reference coordinate information to the position indicated by the reference coordinates, Input coordinate processing method.

Images