JP7273098B2 - Method of operation - Google Patents

Description

TECHNICAL FIELD The present invention relates to a small input terminal device and an operation input method to be worn on a wrist or the like.

2. Description of the Related Art In recent years, mobile information terminals such as smartphones that perform not only phone calls but also processing such as searching for various information, sending and receiving e-mails, managing schedules, etc., have rapidly spread. However, these portable information terminals are often stored in a bag or pocket when moving, and it is necessary to take out the portable information terminal from the bag or pocket each time in order to check a received e-mail or the like. For this reason, smaller input terminal devices such as wristwatches that are connected to these portable information terminals by wireless communication have been developed.

In portable information terminals such as smartphones, information can be displayed on a touch panel and input can be performed with a finger. However, in a small input terminal device, the touch panel is also small, making it difficult to perform detailed operation input by finger touch. . Therefore, a non-contact input method using the back of the user's hand as an input surface has been proposed.

For example, in the wristband type input device described in Patent Document 1, a light-emitting part and a light-receiving part are provided in the main body, and characters can be input by detecting the positions of the fingertips on the back of the hand and the palm without contact. It is configured.

Japanese Patent Application Laid-Open No. 2014-106765

In a conventional character input method using a touch panel or the like, characters are drawn according to the movement of the user's finger when the user's finger is in contact with the panel surface, and characters are drawn when the user's finger is away from the panel surface. You can enter the valid period and invalid period of the input separately so that the input is not valid. As a result, when entering a character composed of multiple "images" such as line segments and dots, there are valid "images" and invalid parts that move between "images". By distinguishing between , a desired character can be entered correctly.

On the other hand, in the non-contact input method disclosed in Patent Document 1 and the like, a plurality of grid points on the back of the hand passed by the fingertips are detected, and input characters are recognized from the dot matrix information. However, regarding the position where the fingertip passes, it does not distinguish between the effective part for the "stroke" that constitutes the character and the ineffective part that moves between the "strokes". In other words, there is a problem that input characters are likely to be erroneously recognized by detecting even invalid parts unrelated to "strokes" as part of characters. In other words, the start and end points of character rendering are ambiguous. Even if this invalid part is predicted and excluded in advance, if the stroke order is different at the time of input, or if a character containing voiced sound or a character with a large number of strokes is input, the trajectory of the invalid part will not be constant. Mistakes seem inevitable.

SUMMARY OF THE INVENTION An object of the present invention is to provide an input terminal device that, when performing an input operation by detecting the position of a finger in a non-contact manner, does not respond to invalid finger movements and correctly executes processing according to the intended input operation. is to provide

The present invention provides an input terminal device for inputting a user's operation via a position input object, wherein a position detection unit detects the position of the position input object operated by the user without contact, and the position detection unit a display unit for displaying a cursor based on the detected position; and an operation processing control unit for executing corresponding operation processing based on the position detected by the position detection unit, wherein the operation processing control unit performs the operation processing. having a plurality of operation modes relating to execution, one of the operation modes including a state in which no operation processing other than movement of the cursor according to the position of the position input object is executed; The unit switches between the plurality of operation modes when the user performs a specific operation via the position input object.

Further, the present invention provides an operation input method for inputting a user's operation via a position input object, comprising: a position detection step of detecting a position of the position input object operated by the user without contact; and an operation processing step of executing corresponding operation processing based on the position detected in the position detection step, wherein the operation processing step includes the operation a plurality of operation modes relating to execution of processing, one of the operation modes including a state in which no operation processing other than movement of the cursor according to the position of the position input object is executed; The processing step switches between the plurality of operation modes when the user performs a specific operation via the position input object.

According to the present invention, when an input operation is performed by detecting the position of a finger in a non-contact manner, the input terminal device does not respond to invalid motions of the finger and correctly executes processing according to the intended input operation. can provide

1 is a block diagram showing the configuration of an input terminal device (first embodiment); FIG. The figure which shows an example of the software configuration of an input terminal device. The figure which shows the external appearance of an input terminal device. FIG. 4 is a diagram showing the detection principle of the finger operation mode; FIG. 10 is a diagram showing a display example of a display unit in handwritten character input processing; 4 is a flowchart showing operation switching of the entire input terminal device. 4A and 4B are diagrams showing examples of operation modes and cursor displays corresponding thereto; FIG. The figure which shows the example of a screen display in a handwritten character input process. FIG. 8 is a diagram showing temporal changes in operation modes and drawing positions in FIG. 7 ; 4 is a flowchart showing handwritten character input processing in finger operation mode; FIG. 10 is a flowchart showing an operation mode determination process S605 in FIG. 9; FIG. The figure which shows the structure of the display part for a key character input process (Example 2). FIG. 10 is a diagram showing an example of key character input processing in a finger operation mode; 4 is a flowchart showing key character input processing in finger operation mode. FIG. 14 is a flowchart showing operation mode determination processing S1105 in FIG. 13 ; FIG. FIG. 11 is a diagram showing an example of operation modes and cursor display used in mouse operation input processing (Embodiment 3); FIG. 7 is a diagram showing an example of screen display in mouse operation input processing; 4 is a flowchart showing mouse operation input processing in finger operation mode. FIG. 18 is a flowchart showing operation mode determination processing S1504 in FIG. 17; FIG. The figure which shows the detection principle of 3D finger operation mode (Example 4). FIG. 4 is a diagram showing an example of screen display in the 3D finger operation mode; 6 is a flowchart showing mouse operation input processing in a 3D finger operation mode; FIG. 4 is a diagram showing an example of screen display in key character input processing; FIG. 11 is a diagram showing an example of screen display when a transition area is provided between layers (Embodiment 5); 6 is a flowchart showing mouse operation input processing when a transition region is provided between layers.

An embodiment of the present invention will be described below with reference to the drawings.

In the first embodiment, a configuration of an input terminal device and a case of detecting a finger position and inputting handwritten characters will be described.

FIG. 1A is a block diagram showing the configuration of the input terminal device 1 according to the first embodiment, and the internal configuration (hardware configuration) will be described. The input terminal device 1 is used by being connected to a mobile information terminal 2 such as a smart phone. The input terminal device 1 of this embodiment has, as operation input functions, a finger operation mode (non-contact detection method) using light reflection and a touch operation mode (contact detection method).

The main control unit 101 is composed of a CPU (Central Processing Unit) or the like, and operates the entire input terminal device 1 according to various operating programs and data stored in a ROM (Read Only Memory) 102 or a RAM (Random Access Memory) 103. Control. A system bus 100 is a data communication path for transmitting and receiving data between the control section 101 and each section in the input terminal device 1 .

The ROM 102 is a memory storing various programs for controlling the input terminal device 1, and uses a rewritable ROM such as an EEPROM (Electrically Erasable Programmable ROM) or a flash ROM. The RAM 103 is used as a temporary storage area during execution of the program stored in the ROM 102 or as a temporary storage of finger position information detected by the finger position detection sensor 106 . The storage unit 104 stores information such as operation setting values of the input terminal device 1, and uses a non-volatile rewritable device such as a flash ROM or SSD (Solid State Drive). The ROM 102 and RAM 103 may be integrated with the main controller 101 . Also, the ROM 102 may be configured to use a partial storage area in the storage unit 104 instead of having an independent configuration.

The acceleration sensor 105 is a sensor that detects acceleration of the input terminal device 1 . The finger position detection sensor 106 is a sensor that detects the position of a position input object such as a user's finger or pen using reflected light in a non-contact manner. The short-range wireless communication unit 107 connects to the mobile information terminal 2 by wireless communication, and transmits information input to the input terminal device 1 . As a communication method, for example, Bluetooth (registered trademark), WiFi (registered trademark), or the like is used.

The display control unit 108 controls display of input characters, various setting images of the input terminal device 1, and the like on the display unit 109a. The touch panel 109 includes a display section 109a and a touch operation input section 109b. The display unit 109 a is a display device such as a liquid crystal panel, and displays characters, images, etc. under the control of the display control unit 108 . The touch operation input unit 109b is arranged on the display surface of the display unit 109a.

The touch operation input unit 109b is, for example, a touch pad type input device such as a capacitive type, and detects a contact operation by a finger, a touch pen, or the like as an operation input. Specifically, for example, a gesture called swipe is performed by touching a touch panel with a finger and then moving the finger in a specific direction while touching the touch panel, moving the finger at a speed equal to or greater than a predetermined value after touching the touch panel with the finger, and then A gesture called a flick to release a touch panel, a gesture called a tap to quickly release a touch panel after touching it with a finger, etc. are detected. This enables operation input to the input terminal device 1 by a touch method.

The timer unit 110 measures the elapsed time from the date/time set by the user, for example, using an RTC (Real Time Clock) circuit, and outputs the date/time information. The main control unit 101 reads the date/time information output from the clock unit 110 to perform processing such as a timer. The operation input unit 111 is an instruction input unit for inputting an operation instruction to the input terminal device 1, and has buttons for turning on/off the power and setting an operation mode.

As described above, the input terminal device 1 of the present embodiment includes the finger position detection sensor 106 that detects the position of the user's finger in a non-contact manner to input characters and the like, and the touch panel 109 that touches the touch panel 109 with the finger to perform operation input. It is configured to include an operation input section 109b and a display section 109a for displaying characters, images, and the like. This enables two operation input functions, a finger operation mode and a touch operation mode.

FIG. 1B is a diagram showing an example of the software configuration of the input terminal device 1, and shows the software configuration in the ROM 102, RAM 103 and storage unit 104. As shown in FIG.

The main control unit 101 configures a basic operation processing control unit 1021 , a finger position information acquisition unit 1022 and a touch position information acquisition unit 1023 by executing programs stored in the ROM 102 . The RAM 103 also includes a temporary storage area 1031 that temporarily holds data as necessary when executing the program stored in the ROM 102 . The storage unit 104 stores setting information 1041 for executing the program stored in the ROM 102 . In order to simplify the explanation below, it is assumed that each operation block is controlled by the main control unit 101 executing a program stored in the ROM 102 .

The basic operation processing control unit 1021 controls various settings and overall operations of the input terminal device 1 . A finger position information acquisition unit 1022 acquires position information of a position input object such as a user's finger or pen based on the output of the finger position detection sensor 106 . The touch position information acquisition unit 1023 acquires position information of the touch panel 109 touched by a finger or a touch pen based on the output of the touch operation input unit 109b.

The finger operation processing control unit 1024 determines what kind of operation is input based on the position information of the position input object acquired from the finger position information acquisition unit 1022, and controls processing corresponding to the determination result. conduct. The touch operation processing control unit 1024 determines what kind of operation is input based on the information of the touch position on the touch panel 109 acquired from the touch position information acquisition unit 1023, and controls processing corresponding to the determination result. conduct.

FIG. 2 is a diagram showing the appearance of the input terminal device 1. As shown in FIG. It is a wristwatch-type input terminal device that is worn on the user's wrist. A touch panel 109 (display unit 109a) is arranged on the front side of the device. Two sensors 106a and 106b are arranged as a finger position detection sensor 106 on the side of the device on the back side of the user's hand. These sensors consist, for example, of light-emitting/light-receiving elements. Further, operation buttons are arranged as an operation input unit 111 on the user's arm side of the apparatus.

FIG. 3 is a diagram showing the detection principle of the finger operation mode, showing the relationship between the position of the finger and the display position of the display unit. A user's finger (position input object) PS moves, for example, within positions ABCD on the back side of the user's hand. The finger position detection sensors 106a and 106b are composed of an infrared light emitting element and a light receiving element. Infrared light emitted from the light emitting element is reflected by the finger PS, and the light receiving element detects the reflected light. The level of the detection signal in the light receiving element corresponds to the distances La and Lb from the sensors 106a and 106b to the finger PS.

The output signal of the finger position detection sensor 106 is sent to the finger position information acquisition unit 1022 to acquire the position F of the finger PS in the position input detection range Pa represented by the position ABCD. Further, positions ABCD of the position input detection range Pa correspond to positions abcd of the display range Da of the display section 109a of the touch panel 109, respectively. Accordingly, the display position f of the cursor or the like in the display range Da is determined corresponding to the position F of the finger PS in the position input detection range Pa.

FIG. 4 is a diagram showing a display example of the display unit 109a in the handwritten character input process. An area 501 displays the result of recognition of input handwritten characters, and an area 502 displays handwritten characters input with a finger. Buttons 503 selected by the user for character input processing include a recognition processing instruction button 503a for selecting processing for recognizing handwritten characters, a character type switching processing instruction button 503b for switching the type of input characters such as hiragana and alphabet, and an input handwriting. A deletion processing instruction button 503c for erasing the display of characters, a character conversion processing instruction button 503d for selecting processing for converting the character string of the recognition result displayed in the area 501 into kanji characters, etc., and an end processing instruction for terminating handwritten character input processing. A button 503e and the like are arranged.

FIG. 5 is a flow chart showing operation switching of the input terminal device 1 as a whole. Here, power on/off and operation input mode (finger operation mode/touch operation mode) are switched by pressing a button on the operation input unit 111 .

In S301, the output of the operation input unit 111 is captured, and in S302, it is determined whether or not the operation input unit 111 is pressed. If the operation input unit 111 is pressed (Yes), the process branches to S306. In S306, the time during which the operation input unit 111 is pressed is determined, and the process proceeds to power on/off switching or operation input mode switching processing.

If the operation input unit 111 has been pressed for the predetermined time Tp or longer (Yes) in determination processing S306, the process branches to S313, and ON/OFF switching processing is performed depending on whether the power is ON. If the power is on in S313 (Yes), the process branches to S314, turns off the power, and returns to S301. When the power is turned off, in order to reduce power consumption, the power supply to the parts other than the main control unit 101, the timer unit 110, the operation input unit 111, and the like is stopped. If the power is off in S313 (No), the process branches to S315 to turn on the power and returns to S301. When the power is turned on, the power is supplied to each part constituting the input terminal device 1 .

If the operation input unit 111 has not been pressed for the predetermined time Tp or longer (No) in determination processing S306, the process branches to S307 to determine whether or not the power is on. If the power is off (No), the process returns to S301. If the power is on (Yes), the process branches to S308.

In S308, operation input mode switching processing is performed. That is, if the current operation input mode is set to the finger operation mode (Yes), the switch is set to the touch operation mode in S309, and the cursor is not displayed on the display unit 109a of the touch panel 109 in S310. set to If the current operation input mode is not set to the finger operation mode (No) (that is, if the touch operation mode is set), the switch is set to the finger operation mode in S311, and the touch panel 109 is set in S312. is set to display a cursor on the display unit 109a.

If the operation input unit 111 is not pressed (No) in determination processing S302, the process proceeds to S303, and processing is performed according to the set operation input mode. If it is the finger operation mode (Yes), the process advances to S304 to determine what operation has been input based on the finger position information acquired from the finger position information acquisition unit 1022, and perform processing corresponding to the determination result. Execute. If it is not the finger operation mode (No) (that is, if it is the touch operation mode), the process proceeds to S305, and what kind of operation was input based on the information of the touched position acquired from the touch position information acquisition unit 1023. is determined, and processing corresponding to the determination result is executed. After executing the finger operation execution processing S304 or the touch operation execution processing S305, the process returns to S301.

The handwritten character input process in the finger operation mode will be specifically described below. In this embodiment, a plurality of operation states (operation modes) are provided for finger operation, so that handwritten character input can be performed correctly and easily.

FIG. 6 is a diagram showing examples of operation modes and cursor displays corresponding thereto. The operation mode M0 is a state (drawing stop) in which drawing for character input other than cursor movement is not performed even if a finger is operated. The operation mode M1 is a state in which, when a finger is operated, drawing for character input is executed according to the position of the finger. In addition, cursor display (for example, white/black display) is switched according to the operation mode so that the user can easily identify the current operation mode. As will be described later, the operation mode is switched from M0 to M1 or vice versa by performing a specific operation of holding the operating finger at a certain position within the operation surface for a predetermined time t0 or longer. can be switched from M1 to M0 at any time.

As a result, drawing is possible only when the operation mode is M1. Therefore, when inputting a character composed of multiple "images" such as line segments and dots, set the operation mode to M1 for valid "images" and Setting the operating mode to M0 for the invalid portion that moves between makes it easier to correctly enter the desired character.

FIG. 7 is a diagram showing an example of screen display in the handwritten character input process. Here, the case of inputting "a" in hiragana is taken as an example. The character input procedure is indicated by (S1) to (S9), and the operation mode (M0/M1) at that time is added.

In the following explanation, the cursor is expressed as "holding" at a certain position, but even if the cursor moves within a predetermined distance r0, it is allowed to move (shake) and is regarded as a "holding" state. and This r0 is due to an unintended movement of the user (hand shake, etc.), and is set to a value of several millimeters, for example. Also, the time t0 for determining the "holding" state is, for example, a value of about several seconds. These thresholds may be set according to the user and usage environment.

(S1): In the state where the operation mode is M0 (stop drawing), the cursor is moved to the position 7a in the character input display area 502 by moving the finger PS within the position input detection range Pa.
(S2): Hold the cursor at the position 7a for a predetermined time t0 or longer. As a result, the operation mode is switched to M1 (drawing execution), and the display of the cursor is changed.

(S3): When the display of the cursor changes, the finger is moved to move the cursor to position 7b in the character input display area 502. FIG. This draws a line from position 7a to position 7b.
(S4): Hold the cursor at position 7b for a predetermined time t0 or more. As a result, the operation mode is switched to M0 (stop drawing), and the display of the cursor is changed.

(S5): When the display of the cursor changes, move the finger to move the cursor to position 7c in the character input display area 502. FIG.
(S6): Hold at position 7c for a predetermined time t0 or more. As a result, the operation mode is switched to M1 and the display of the cursor is changed.

Thereafter, drawing operations are continued in the same manner. That is, when drawing is performed, the cursor is moved with the operation mode set to M1, and when drawing is not performed, the cursor is moved with the operation mode set to M0. Further, by holding the cursor at the moved position for a predetermined time t0 or more, the operation mode is switched to input handwritten characters.

(S7): When the drawing of the hiragana letter "a" is completed, it is held at the position 7d for a predetermined time t0 or more. As a result, the operation mode is switched to M0 and the display of the cursor is changed.
(S8): Move the cursor to the recognition process instruction button 503a.
(S9): Recognition processing is executed by holding the cursor on the recognition processing instruction button 503a for a predetermined time t0 or longer, and the recognition result is displayed in the display area 501. FIG.

FIG. 8 is a diagram showing temporal changes in operation modes and drawing positions in FIG. (a) shows the position of the position input object (user's finger) PS, (b) shows the operation mode, and (c) shows the display drawing position. (S1) to (S6) and symbols (7a) to (7c) on the horizontal axis correspond to the symbols in FIG. In this way, drawing is performed only during the period when the operation mode is M1, and the operation mode is switched by holding the position input object PS at the same position for time t0 or longer.

FIG. 9 is a flowchart showing handwritten character input processing in the finger operation mode. The contents of each process will be described below in order.

In S601, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S602, the acquired position F of the finger PS is converted into the position f in the display range Da as described with reference to FIG. Next, in S603, it is determined whether or not the distance r between the position f and the previously acquired position fd stored in the temporary storage area of the RAM 103 is less than or equal to a predetermined distance r0.

In determination processing S603, if the distance r is equal to or greater than the predetermined distance r0 (No), the process branches to S617. In S617, it is determined whether the position f is within the display range of the character input display area 502 or not. If the position f is within the range of the character input display area 502 (Yes), the process branches to S618. If the position f is outside the range of the character input display area 502 (No), the process branches to S620. At S620, the operation mode is set to M0, and the process proceeds to S615.

In S618, the set operation mode is determined. If the operation mode is M0, the process advances to S615 without performing drawing processing. If the operation mode is M1, the process branches to S619, performs drawing processing from the position fd to the position f in the display range Da, and proceeds to S615.

On the other hand, in determination processing S603, if the distance r is equal to or less than the predetermined distance r0 (Yes), the process branches to S604. In S604, it is determined whether or not the position f is within the display range of the predetermined area 502 for character input in the display range Da or within the display range of the processing instruction button 503. FIG. If the position f is within the character input display area 502 or the display range of the processing instruction button 503 (Yes), the process proceeds to S605. If the position f is outside the character input display area 502 or the display range of the processing instruction button 503 (No), the process proceeds to S615.

At S605, the operation mode is determined by a timer. That is, the operation mode is switched by comparing the holding time with the predetermined time t0. Details of the determination processing in S605 will be described with reference to FIG. In S606, branch processing is performed according to the determined operation mode. If the operation mode is M0, the process proceeds to S615, and if the operation mode is M1, the process proceeds to S607. In S607, it is determined whether the position f is within the display range of the processing instruction button 503 or not. If the position f is within the display range of the process instruction button 503 (Yes), the process proceeds to S608, and if the position f is outside the display range of the process instruction button 503 (No), the process proceeds to S615.

In S608, branch processing is performed depending on which of the processing instruction buttons 503a to 503e the position f is in the display range. If the position f is within the display range of the recognition process instruction button 503a, the process advances to S609 to execute the recognition process. If the position f is within the display range of the character type switching process instruction button 503b, the process advances to S610 to execute the character type switching process. If the position f is within the display range of the erasing process instruction button 503c, the process proceeds to S611 to execute the erasing process. If the position f is within the display range of the conversion processing instruction button 503d, the process advances to S612 to execute the conversion processing.
After each process is executed, the process advances to S613 to reset and stop the timer, and the operation mode is set to M0 in S614. If the position f is within the display range of the end process instruction button 503e, the handwritten character input process is ended.

In S615, a cursor as shown in FIG. 6, for example, is displayed at position f in display range Da according to the set operation mode. Next, in S616, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and the process returns to S601.

FIG. 10 is a flowchart showing operation mode determination processing S605 in FIG.

In S701, it is determined whether or not the timer is activated. If the timer has not been started (No), the process advances to S702 to start the timer, and the process ends. If the timer has been started (Yes), the process proceeds to S703.

In S703, it is determined whether or not the timer has exceeded a predetermined time t0. If the predetermined time t0 is not exceeded (No), the process is terminated. If the predetermined time t0 is exceeded (Yes), the process proceeds to S704.

In S704, branch processing is performed depending on the operation mode. If the operation mode is M0, the process advances to S705 to switch and set the operation mode to M1. If the operation mode is M1, the process advances to S706 to switch and set the operation mode to M0. After that, the process advances to S707 to reset/stop the timer and terminate the process.

As described above, in the first embodiment, the position of the position input object such as the user's finger on the display screen is held within the predetermined distance r0 for the predetermined time t0 or more, thereby switching the operation mode and performing drawing. When drawing is not performed, the operation mode is set to M0 and the cursor is moved. As a result, even when characters with a plurality of strokes are input, movement between strokes can be distinguished as a non-drawing section, and intended characters can be easily and correctly input even if the stroke order is different. becomes possible. In addition, it becomes possible to input more complicated characters such as Chinese characters.

Further, by moving the cursor to the area where the processing instruction button is displayed and holding the position within a predetermined distance r0 from the position where the cursor is moved for a predetermined time t0 or longer, the processing corresponding to the processing instruction button can be performed. It is possible to perform the same operation as moving the cursor to the position where the processing instruction button is displayed using an input means such as a mouse and clicking it.

In this embodiment, an example is shown in which the recognition processing instruction button 503a, the character type switching processing instruction button 503b, the deletion processing instruction button 503c, the character conversion processing instruction button 503d, and the end processing instruction button 503e are used as the processing instruction buttons. , without being limited to this, other processing instruction buttons may be provided.

Further, in this embodiment, the operation input unit 111 switches between the operation input mode (touch operation mode) by the touch panel 109 and the operation input mode (finger operation mode) by the finger position detection sensor 106, but other methods may be used. You may make it switch. For example, when the back of the hand is tapped with a finger, the vibration is detected by the acceleration sensor 105, but the touch panel 109 does not detect the touch. However, when the back of the hand is tapped with a finger, the operation input mode may be switched.

In addition, in this embodiment, as a specific operation for switching the operation mode, an operation of holding the position of the position input target such as the user's finger on the display screen within a predetermined distance r0 for a predetermined time t0 or more is adopted. However, it is not limited to this. As another specific operation, for example, the cursor may be moved by moving the position input object, and the operation mode may be switched by tapping the back of the hand at the moved position.

In the second embodiment, a case will be described where input is performed by selecting keys displayed on the screen like a keyboard. In a small portable information terminal, the keys displayed on the display unit 109a are also small, making it difficult to input by touching the touch panel 109 with a finger. Therefore, key character input is performed in the finger operation mode.

FIG. 11 is a diagram showing the configuration of the display unit 109a for key character input processing. An area 1001 is a portion for displaying characters input by key characters, and a character input key 1002 is a key for selecting characters to be input. Keys 1003 are used to instruct various types of processing. 1003b, and an end processing instruction key 1003c for ending key character input processing.

FIG. 12 is a diagram showing an example of key character input processing in the finger operation mode. Here, the case of inputting the alphabet "Y" is taken as an example. The input procedure will be explained in (S1) to (S4).

(S1): In the state where the operation mode is M0, by moving the finger PS within the position input detection range Pa, the cursor is moved to the position 12a where the character input key "WXYZ" is displayed.
(S2): The cursor is held at the position 12a for a predetermined time t0 or more (in this case also, movement within a predetermined distance r0 is permitted. The same applies hereinafter). As a result, the operation mode is switched to M1, and a display for selecting characters "W", "X", "Y", and "Z" is displayed.

(S3): When the character selection display is displayed, the finger is moved to move the cursor to the position 12b where the character "Y" is displayed.
(S4): Hold the cursor at the position 12b for a predetermined time t0 or longer. As a result, the character “Y” is determined as the input character, and the character “Y” is displayed in the input character display area 1001 . When character input is completed, the operation mode switches to M0.

FIG. 13 is a flow chart showing key character input processing in the finger operation mode.

In S1101, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S1102, the acquired position F of the finger PS is converted to the position f in the display range Da. Next, in S1103, it is determined whether or not the distance r between the position f and the previously obtained position fd is less than or equal to a predetermined distance r0.

In determination processing S1103, if the distance r is equal to or greater than the predetermined distance r0 (No), the process branches to S1114. If the distance r is less than or equal to the predetermined distance r0 (Yes), the process branches to S1104. In S1104, it is determined whether the position f is within the display area of the character input key 1002 or the processing instruction key 1003 or not. In determination processing S1104, if the position f is within the display area of the character input key 1002 or the processing instruction key 1003 (Yes), the process branches to S1105. ), the process branches to S1114.

At S1105, the operation mode is determined by a timer. That is, the operation mode is switched by comparing the holding time with the predetermined time t0. Details of the determination processing in S1105 will be described with reference to FIG. In S1106, branch processing is performed according to the determined operation mode. If the operation mode is M0, the process proceeds to S1114, and if the operation mode is M1, the process proceeds to S1107. In S1107, it is determined whether or not the character selection display is on.

In determination processing S1107, if the character selection display is on (Yes), the process proceeds to S1116, and if the character selection display is off (No), the process proceeds to S1108. In S1116, it is determined whether or not the position f is within the character selection display area. If the position f is within the character selection area (Yes), the process proceeds to S1117, and if the position f is outside the character selection display area (No), the process proceeds to S1114. In S1117, the character displayed at the position f is determined as an input character and displayed in the input character display area 1001. FIG. In S1118, the character selection display is turned off, and the process proceeds to S1112.

In S1108, branch processing is performed depending on whether the position f is within the display area of either the character input key 1002 or the processing instruction key 1003. FIG. If the position f is within the display area of the character input key 1002, the process advances to S1109 to turn on the character selection display and select and display the character corresponding to the key displayed at the position f. If the position f is within the display area of the character type switching processing instruction key 1003a, the process advances to S1110 to execute the character type switching processing. If the position f is within the display area of the deletion processing instruction key 1003b, the process advances to S1111 to execute deletion processing. After executing each process, the process advances to S1112 to reset and stop the timer, and the operation mode is set to M0 in S1113. If the position f is within the display area of the termination processing instruction key 1003c, the key character input processing is terminated.

In S1114, a cursor as shown in FIG. 6, for example, is displayed at position f in display range Da according to the set operation mode. Next, in S1115, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and the process returns to S1101.

FIG. 14 is a flowchart showing operation mode determination processing S1105 in FIG.

In S1201, it is determined whether or not the timer is activated. If the timer has not been started (No), the process advances to S1202 to start the timer, and the process ends. If the timer is activated (Yes), the process proceeds to S1203.

In S1203, it is determined whether or not the timer has exceeded a predetermined time t0. If the predetermined time t0 is not exceeded (No), the process is terminated. If the predetermined time t0 is exceeded (Yes), the process proceeds to S1204.

In S1204, the operation mode is switched to M1. In this case, in S1113 of FIG. 13, the operation mode after executing the key character input is switched to M0, so switching from M1 to M0 is unnecessary. After that, the process advances to S1205 to reset/stop the timer and terminate the process.

As described above, in the second embodiment, the cursor is moved to the position where the character input key or the processing instruction key is displayed, and the cursor is held within a predetermined distance r0 from the moved cursor position for a predetermined time t0 or longer. It is possible to easily select a character to be displayed or instruct a process to be executed. As a result, it is possible to perform an operation similar to moving the cursor to the position where the character input key or the processing instruction key is displayed using an input means such as a mouse and clicking the cursor.

In this embodiment, the processing instruction keys are composed of the character type switching processing instruction key 1003a, the deletion processing instruction key 1003b, and the end processing instruction key 1003c. Corresponding keys may be provided. As for the character input keys, a keyboard in which one alphabetic character or one hiragana character corresponds to one key, or a keyboard corresponding to the input of numbers may be displayed and input.

In the third embodiment, operations such as clicking, double-clicking, and dragging and dropping with a mouse (hereinafter referred to as mouse operation input processing) are implemented in finger operation mode. In order to cope with this, the types of operation modes are added in this embodiment.

FIG. 15 is a diagram showing examples of operation modes and cursor display used in mouse operation input processing. Six types of operation modes M0 to M5 are used here. The operation mode M0 is a state in which input operations other than cursor movement are not performed even if a finger is operated. Operation modes M1 to M5 are states in which mouse operations are performed for click processing (M1), double-click processing (M2), start of drag processing (M3), end of drag processing (M4), and drop processing (M5), respectively. be. In addition, cursor display (shape and/or color display) is switched according to each operation mode. As will be described later, the operation mode is switched by comparing the length of time during which the finger during operation is held at a certain position on the operation surface with three thresholds t0, t1, and t2.

FIG. 16 is a diagram showing an example of screen display in the mouse operation input process. The display unit 109a displays an icon group 1401 corresponding to the applications A to D as a window screen (home screen). The user can select a desired application by clicking the icon, execute the application by double-clicking, and move the display position of the icon by dragging and dropping. The flow of a series of processes will be explained in (S1) to (S7).

(S1): In the state where the operation mode is M0, by moving the finger within the position input detection range, the cursor is moved to the position 14a where the icon [D] is displayed.
(S2): Hold the cursor at the position 14a for a predetermined time (range of t0 to t1). As a result, the operation mode is switched to M1 (click processing), and the icon "D" is selected as the mouse operation target. Also, the display of the cursor changes to the display corresponding to M1 in FIG. After that, by arbitrarily moving the finger (in this case, moving the cursor to a position separated by a predetermined distance r0 or more from the position 14a) before a predetermined time t1 elapses, the selected icon "D" is Click processing is executed.

(S3): When the cursor is held at the position 14a for a predetermined time t1 or more, the operation mode is switched to M2 (double-click processing), and the cursor changes to display corresponding to M2 shown in FIG. Thereafter, by arbitrarily moving the finger (moving the cursor by a predetermined distance r0 or more from the position 14a) before the predetermined time t2 elapses, the selected icon "D" is double-clicked. .

(S4): When the cursor is held at the position 14a for a predetermined time t2 or longer, the operation mode is switched to M3 (drag processing start), and the cursor changes to display corresponding to M3 shown in FIG.
(S5): After that, by moving the finger to move the cursor to a position 14b that is away from the position 14a by a predetermined distance r0 or more, the operation mode is switched to M4 (end of drag processing). As a result, the cursor changes to display corresponding to M4 shown in FIG. Also, the icon "D" selected as the mouse operation target is moved and displayed at the position 14b to which the cursor has been moved, and drag processing is executed to move the position of the selected icon.

(S6): Move the finger further to move the cursor and the selected icon [D] to the position 14c, and hold the cursor at the position 14c for a predetermined time t0 or more. As a result, the operation mode is switched to M5 (drop processing), and the drop processing is executed at position 14c.
(S7): The display position of the selected icon "D" is set to 14c by the drop process. After that, the operation mode is switched to M0, and the cursor is displayed corresponding to M0.

FIG. 17 is a flow chart showing mouse operation input processing in the finger operation mode.

In S1501, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S1502, the acquired position F of the finger PS is converted to the position f in the display range Da. Next, in S1503, it is determined whether or not the distance r between the position f and the previously obtained position fd is less than or equal to a predetermined distance r0.

In determination processing S1503, when the distance r is equal to or less than the predetermined distance r0 (Yes), the process branches to S1504, and in S1504, the operation mode is determined by a timer. That is, the holding time is compared with predetermined times t0, t1, and t2 to switch the operation mode. Details of the determination processing in S1504 will be described with reference to FIG. After that, in S1505, it is determined whether or not the operation mode determined in S1504 is M5. If the operation mode is M5 (Yes), the process branches to S1508, and if the operation mode is not M5 (No), the process branches to S1506. In S1508, drop processing is executed to set the display position of the icon selected by the drag processing to position f, and the process advances to S1509.

In S1506, it is determined whether or not the position f is within the icon display area. If the position f is within the icon display area (Yes), the process branches to S1507, and if it is outside the icon display area (No), the process branches to S1509. In S1507, an icon whose position f is within the display area is selected as a mouse operation target, and the process proceeds to S1518. In S1509, the operation mode is set to M0, and the process proceeds to S1510. In S1510, the timer is reset/stopped, and the process proceeds to S1518.

On the other hand, in determination processing S1503, if the distance r is equal to or greater than the predetermined distance r0 (No), the process branches to S1511. In S1511, the following branch processing is performed according to the operation mode.
If the operation mode is M0, the process proceeds to S1518 without performing any processing.
If the operation mode is M1, the process advances to S1515, click processing is performed on the icon selected as the mouse operation target in S1507, and the process advances to S1516.
If the operation mode is M2, the process advances to S1514 to double-click the icon selected as the mouse operation target in S1507, and the process advances to S1516.
If the operation mode is M3, the process advances to S1513 to set the operation mode to M4, and the process advances to S1517.
If the operation mode is M4, the process advances to S1512 to display the icon selected as the mouse operation target in S1507 at the position f. As a result, drag processing for moving the position of the selected icon is performed, and the process proceeds to S1518.

In S1516, the operation mode is set to M0, in S1517 the timer is reset/stopped, and the process proceeds to S1518. In S1518, a cursor as shown in FIG. 15 is displayed at position f in display range Da according to the current operation mode. Next, in S1519, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and the process returns to S1501.

FIG. 18 is a flowchart showing operation mode determination processing S1504 in FIG.

In S1601, it is determined whether or not the timer is activated. If the timer has not been started (No), the process advances to S1602 to start the timer and terminates the process. If the timer has been started (Yes), the process proceeds to S1603.

In S1603, it is determined whether or not the operation mode is M4. If the operation mode is M4 (Yes), the process proceeds to S1604. In S1604, it is determined whether or not the timer has exceeded a predetermined time t0, and if the predetermined time t0 has not been exceeded (No), the process ends. If the predetermined time t0 is exceeded (Yes), the process advances to S1605 to set the operation mode to M5, and the process ends. In determination processing S1603, if the operation mode is not M4 (No), the process proceeds to S1606.

In S1606, the following branch processing is performed according to the elapsed time of the timer. Note that the following time thresholds have a relationship of t0<t1<t2.
If the timer has not exceeded the predetermined time t0 (timer≤t0), the process ends.
If the timer is between predetermined times t0 and t1 (t0<timer≤t1), the process proceeds to S1609, sets the operation mode to M1, and ends the process.
If the timer is between predetermined times t1 and t2 (t1<timer≤t2), the process proceeds to S1608, sets the operation mode to M2, and terminates the process.
If the timer exceeds the predetermined time t2 (timer>t2), the process advances to S1607 to set the operation mode to M3, and the process ends.

As described above, in the third embodiment, the operation mode is switched between M0 to M5 according to the length of time the cursor is held within the predetermined distance r0 from the moved position. After the operation mode is switched, by moving the cursor to a position separated by a predetermined distance r0 or more like a flick operation in touch operation of a smartphone, clicks, double clicks, drag-and-drops, etc. with a mouse can be performed. It becomes possible to realize the operation in the finger operation input mode. In addition, the fact that the operation mode has been switched can be confirmed by a change in the display of the cursor, and the mouse can be easily operated as intended by the user.

In the fourth embodiment, a case where an input operation is performed by three-dimensionally detecting the position of a user's finger (hereinafter referred to as a 3D finger operation mode) will be described.

FIG. 19 is a diagram showing the detection principle of the 3D finger operation mode, showing the relationship between the position of the finger and the display position of the display unit. The position of the finger PS is detected not only in a direction parallel to the display unit 109a of the input terminal device 1 (XY-axis direction) but also in a direction perpendicular to it (height Z-axis direction) to perform an input operation. The figure shows the relationship between the position F of the detected finger (position input object) PS and the display position f of the display unit 109a.

Three finger position detection sensors 106a, 106b, and 106c are provided to detect finger positions three-dimensionally. Each of the sensors 106a to 106c is composed of, for example, an infrared light emitting element and a light receiving element, and signals having levels corresponding to the distances La, Lb, and Lc to the finger PS are output from the sensors 106a to 106c. Based on the output signals from the sensors 106a to 106c, the finger position information acquisition unit 1022 detects a position input detection range (space) surrounded by positions ABCD-A'-B'-C'-D'. Obtain the position F of the finger PS at Pa. A position f in a display range Da surrounded by positions abcd of the display unit 109a is determined corresponding to the acquired position F in the XY axis direction, and a cursor or the like is displayed. On the other hand, in accordance with the acquired position F in the Z-axis direction, a plurality of display screens are switched as described later.

FIG. 20 is a diagram showing an example of screen display in the 3D finger operation mode. Here, a window screen (home screen) on which icons are arranged is displayed as an operation screen. The position in the height Z direction with respect to the display unit 109a is divided into a plurality of layers L0, L1, and L2 with Z0 to Z3 as boundaries. The window screen is switched and displayed according to which layer the finger PS is positioned. The display state in each layer is indicated by (S1) to (S3).

(S1): When the position F0 of the finger PS is in the range of the height direction Z0-Z1 (layer L0), an icon group 1901 corresponding to the applications A-D is displayed on the window screen.
(S2): When the position F1 of the finger PS is in the range of the height direction Z1-Z2 (layer L1), an icon group 1902 corresponding to the applications E-H is displayed on the window screen.
(S3): When the position F2 of the finger PS is in the range of the height direction Z2-Z3 (layer L2), an icon group 1903 corresponding to the applications I-L is displayed on the window screen.

After that, the mouse operation input process described in the third embodiment is executed. That is, clicking the displayed icon selects the application, double-clicking the icon executes the application, and dragging and dropping the icon moves the display position of the icon.

In this way, according to the 3D finger operation mode, the screen display for scanning can be switched for each of a plurality of layers, so that mouse operation input processing can be easily performed for many applications (icons).

FIG. 21 is a flowchart showing mouse operation input processing in the 3D finger operation mode. The same reference numerals are assigned to the same processes as in the third embodiment (FIG. 17), and repeated descriptions are omitted.

In S1501, the finger position information acquisition unit 1022 acquires the three-dimensional position F of the finger PS, and in S1502, the acquired position F in the XY direction is converted to the position f in the display range Da as described with reference to FIG. . Next, in S1520, it is determined which layer the acquired position F in the Z direction is within. In S1521, it is determined whether the determined layer LN is the same as the previously determined layer LNd stored in the temporary storage area of the RAM 103 or not.
As a result of the determination, if the determined layer LN is the same as the previously determined layer LNd (Yes), the process proceeds to S1503. The processing after S1503 is the same as in FIG. 17, and the description is omitted. As a result of the determination, if the determined layer LN is different from the last determined layer LNd (No), the process proceeds to S1522.

At S1522, it is determined whether the operation mode is M3 or M4. If the operation mode is M3 or M4 (Yes), the process proceeds to S1523, and if the operation mode is not M3 or M4 (No), the process proceeds to S1525. In S1525, the operation mode is set to M0, and the process proceeds to S1526. In S1526, the timer is reset/stopped, and the process proceeds to S1523.

In S1523, the determined layer LN is replaced with the previous determined layer LNd, stored in the temporary storage area of the RAM 103, and the process proceeds to S1524. In S1524, the screen of the display unit 109a is updated so as to display the window screen (home screen) corresponding to the determined layer LN, and the process proceeds to S1511. The processing after S1511 is mouse operation input processing in accordance with each of the operation modes M0 to M4, but is the same as in FIG. 17 and will not be described.

As described above, by detecting the three-dimensional position of the user's finger and performing an operation input, it is divided into a plurality of layers according to the position in the height Z direction, and an operation window screen ( Home screen) can be displayed. It is also possible to perform mouse operations such as clicking, double-clicking, and dragging and dropping on icons displayed in the windows of each layer.

In the above example, icons corresponding to different applications are displayed on the window screen for each layer. However, it is possible to display different operation surfaces for each layer.

For example, FIG. 22 is a diagram showing an example of screen display in key character input processing. In the key character input process, keyboards with different character types are displayed on the display unit for each layer.

(S1): When the position F0 of the finger PS is within the range of the layer L0, a keyboard for inputting alphabets and symbols is displayed.
(S2): If the position F1 is within the range of the layer L1, display a keyboard for inputting hiragana.
(S3): If the position F2 is within the range of the layer L2, display a keyboard for entering numbers.
Thereby, the user can easily change the character type to be input by changing the position of the finger PS in the height Z direction.

As described above, according to the fourth embodiment, it is possible to switch the screen display and operation contents for each of a plurality of layers in the 3D finger operation mode, so that various kinds of operations can be easily realized. Note that the number of layers may be appropriately set by the user depending on the type of operation to be used.

In a fifth embodiment, a configuration in which a transition area is provided between layers in order to smoothly switch between a plurality of layers in the 3D finger operation mode will be described.

FIG. 23 is a diagram showing an example of screen display when a transition area is provided between layers. The position of the finger PS in the height Z direction is divided into a plurality of layers L0 to L2, and layer transition regions T01 and T12 are provided at the boundaries of each layer. As in the fourth embodiment, different operation window screens (here, icon groups) are displayed corresponding to the respective layers L0 to L2. display the window screen. The change in screen display along with the movement of the finger PS in the height direction will be described below.

First, (S1) to (S5) describe the case where the position of the finger PS is moved from F0 to F4 (that is, from layer L0 to layer L2).

(S1): The position F0 of the finger PS is within the range of the layer L0, and the icon group 2101 corresponding to the applications A to D is displayed on the window screen.
(S2): When the position F1 of the finger PS moves to the layer transition area T01 between the layers L0 and L1, the window screen (icon group 2101) displayed on the previous layer L0 is continued to be displayed. At that time, the screen background is changed to the display 109T showing the transition area.

(S3): The position F2 of the finger PS is within the range of the layer L1, and the icon group 2102 corresponding to the applications E to H is switched and displayed on the window screen.
(S4): When the position F3 of the finger PS moves to the layer transition region T12 between the layers L1 and L2, the window screen (icon group 2102) displayed on the previous layer L1 continues to be displayed. At that time, the screen background is changed to the display 109T showing the transition area.

(S5): The position F4 of the finger PS is within the range of the layer L2, and the icon group 2103 corresponding to the applications I to L is switched and displayed on the window screen.

On the other hand, the case where the position of the finger PS is moved from F4 toward F0 (that is, from layer L2 to layer L0) will be described in (S5') to (S1').

(S5'): The position F4 of the finger PS is within the range of the layer L2, and the icon group 2103 corresponding to the applications I to L is displayed on the window screen.
(S4'): When the position F3 of the finger PS moves to the layer transition region T12 between the layers L1 and L2, the window screen (icon group 2103) displayed on the previous layer L2 is continued to be displayed. At that time, the screen background is changed to the display 109T showing the transition area.

(S3'): The position F2 of the finger PS is within the range of the layer L1, and the icon group 2102 corresponding to the applications E to H is switched and displayed on the window screen.
(S2'): When the position F1 of the finger PS moves to the layer transition area T01 between the layers L0 and L1, the window screen (icon group 2102) displayed on the previous layer L1 continues to be displayed. At that time, the screen background is changed to the display 109T showing the transition area.

(S1'): The position F0 of the finger PS is within the range of the layer L0, and the icon group 2101 corresponding to the applications A to D is switched and displayed on the window screen.

As described above, when the finger PS is positioned within the range of each layer, the window screen corresponding to each layer is displayed, and when the finger PS moves to the layer transition area, the window screen of the immediately preceding layer is displayed. is continuously displayed, and its background is changed and displayed.

In this manner, a transition area is provided between each layer, and the background of the window screen is changed when the finger PS moves to the transition area, thereby informing the user of the transition area. At that time, the window screen displayed immediately before is continued as it is, so there is no hindrance to the operation. This prevents the user from unintentionally moving to another layer. That is, when the background of the window screen changes, the finger PS should be moved in the direction of the original layer.

FIG. 24 is a flowchart showing mouse operation input processing when a transition area is provided between layers. The same processing as in the third embodiment (FIG. 17) and the fourth embodiment (FIG. 21) is denoted by the same reference numeral, and repeated description is omitted.

In S1501, the three-dimensional position F of the finger PS is acquired, and in S1502, the position of the position F in the XY direction is converted to the position f in the display range Da. Next, in S1520, a determination is made as to which layer or layer transition area the position F in the Z direction is within. In S1530, branch processing is performed according to the layer determination result.

In determination processing S1530, if the determined layer LN is within the layer transition area (LN=between layers), the process branches to S1531. In S1531, the background of the window display screen is set to the display 109T corresponding to the layer transition area, and the process proceeds to S1503.

In determination processing S1530, if the determined layer LN is the same as the layer LNd determined last time (LN=LNd), the process proceeds to S1503. The processing after S1503 is the same as in FIG. 17, and the description is omitted.

In determination processing S1530, if the determined layer LN is different from the layer LNd determined last time (LN≠LNd), the process proceeds to S1522. The processing after S1522 is the same as in FIG. 21, and the description is omitted.

As described above, according to the fifth embodiment, the three-dimensional position of the finger PS is detected, and windows corresponding to a plurality of layers corresponding to the position in the height direction Z are displayed. A transition area is provided. When the finger PS moves to the transition area, the user can be notified by changing the background of the window screen, thereby preventing the user from unintentionally moving to another layer. This allows smooth switching between layers.

In this embodiment, the background of the window screen is changed when the finger PS moves to the layer transition area, but the present invention is not limited to this. For example, any method that can be recognized by the user, such as changing the icon display, changing the cursor display, or generating sound or vibration, may be used. Also, when positioned in the transition area, the direction of the original layer (up/down) may be displayed.

In the description of each of the above embodiments, an example in which the finger position detection sensor 106 is composed of a light emitting/receiving element has been described. Any device that can detect the position of a position input object such as a pen or the like may be used. In addition, although a wristwatch type input terminal device has been described as an example, in smartphones, tablets, personal computers, etc., the position of a position input object such as a user's finger or pen is detected, and handwritten character input, mouse operation, etc. are performed. You may do so.

Further, in each of the above embodiments, as a condition for the specific operation for switching the operation mode, the position of the position input object such as the user's finger on the display screen is held within a predetermined distance for a predetermined time or more. However, it is not limited to this. For example, the condition may be that the amount of movement of the position of the position input object on the display screen within a predetermined time is within a predetermined distance. Alternatively, the condition may be that the moving speed of the position of the position input object on the display screen is equal to or less than a predetermined speed.

The embodiments of the present invention have been described above using several examples, but needless to say, the configuration for realizing the technology of the present invention is not limited to the above examples, and various modifications are conceivable. . For example, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. All of these belong to the scope of the present invention. Numerical values, messages, and the like appearing in the sentences and drawings are merely examples, and the effects of the present invention are not impaired even if different ones are used. Also, the programs described in each processing example may be independent programs, or a plurality of programs may constitute one application program. Also, the order of performing each process may be changed.

Some or all of the functions and the like of the present invention described above may be realized by hardware, for example, by designing an integrated circuit. Alternatively, the functions may be realized by software by having the microprocessor unit or the like interpret and execute an operation program for realizing each function. Hardware and software may be used together.

Also, the control lines and information lines shown in the drawings show what is considered necessary for explanation, and do not necessarily show all the control lines and information lines on the product. In practice, it may be considered that almost all configurations are interconnected.

1: input terminal device, 2: portable information terminal, 101: main control unit, 102: ROM, 103: RAM, 104: storage unit, 105: acceleration sensor, 106, 106a, 106b, 106c: finger position detection sensor, 108 : display control unit 109: touch panel 109a: display unit 109b: touch operation input unit 110: timer unit 111: operation input unit 1022: finger position information acquisition unit 1023: touch position information acquisition unit 1024: Finger operation processing control unit 1025: touch operation processing control unit PS: user's finger (position input object).

Claims (8)

A terminal device capable of inputting a user's operation via a position input object,
a position detection unit that detects the three-dimensional position of the position input object without contact;
a display unit that displays an object whose display icon can be selected based on the three-dimensional position detected by the position detection unit;
An operation processing control unit that executes corresponding operation processing based on the three-dimensional position detected by the position detection unit,
The display position of the object based on the three-dimensional position of the position input target detected by the position detection unit changes according to the position change of the position input target, and the icon displayed on the display unit changes the When the object is displayed
The operation processing control unit
When the state in which the object is displayed on the icon based on the three-dimensional position of the position input object continues for a predetermined time or longer, the icon is switched from a first operation mode in which no operation processing is performed on the icon . controlling to switch to a second operation mode for performing operation processing;
Further, among the three-dimensional position of the position input object with respect to the display unit, the position in the vertical direction with respect to the display unit is divided into a plurality of layers, and the vertical position of the position input object detected by the position detection unit is divided into a plurality of layers. controls to change the display contents of the display unit according to which layer of the plurality of layers is within the range,
A transition area is further set at the boundary of the plurality of layers, and when the position of the position input object moves within the transition area, the operation screen displayed immediately before is continued on the display unit. and displaying the position of the position input object to notify that the position of the position input object is within the transition area. The terminal device according to claim 1,
The terminal device, wherein the operation processing control unit notifies that the position of the position input target is within the transition area by changing a screen background. The terminal device according to claim 1 or 2,
Further, in the second operation mode, a plurality of different operation processes that are not performed in the first operation mode are performed according to the vertical position of the position input object in the three-dimensional position with respect to the display unit. A terminal device characterized by controlling. The terminal device according to any one of claims 1 to 3 ,
The operation control processing unit displays another icon instead of the icon when the vertical position of the position input object with respect to the display unit is within a predetermined first range in the second operation mode. A terminal device characterized by controlling the display unit at a time. The terminal device according to any one of claims 1 to 3 ,
The operation control processing unit is characterized in that, upon detecting a change in position of the position input object in the vertical direction with respect to the display unit in the second operation mode, the operation control processing unit performs control to change display contents of the display unit. terminal equipment. The terminal device according to any one of claims 1 to 5,
A terminal device according to claim 1, wherein the position detection section includes a light-emitting element and a light-receiving element. The terminal device according to any one of claims 1 to 5,
The terminal device according to claim 1, wherein the position detection unit includes a sensor using ultrasonic waves. The terminal device according to any one of claims 1 to 5,
A terminal device, wherein the position detection unit includes a camera.

Images