JP7354878B2 - Handwriting input device, handwriting input method, program, input system - Google Patents

Description

The present invention relates to a handwriting input device, a handwriting input method, a program, and an input system.

In a typical computer-controlled whiteboard device or an application that allows handwriting input (hereinafter referred to as a handwriting input device), the input means is limited to a pen or a finger. For this reason, an operation menu is provided so that the user can switch between pen functions, such as changing the color of characters, and editing functions, such as erasing characters. Normally, the pen function menu allows selection of color, thickness, etc., and the edit function menu allows selection of deletion, movement, scaling, rotation, cutting, copying, pasting, etc. (see, for example, Patent Document 1).

Patent Document 1 discloses a handwriting input device in which a menu of color settings, transparency settings, thickness settings, line type settings, stamp settings, and operation settings is displayed by pressing a pen button.

However, conventional handwriting input devices have a problem in that it is not easy to sign in. That is, in order for a user to sign in to a handwriting input device, for example, many operations are required, inputting a user name and password is not easy, or special hardware such as an IC card reader is required. .

SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to provide a handwriting input device that allows easy sign-in.

In view of the above problems, the present invention provides a handwriting input device that displays handwritten stroke data based on the position of an input means in contact with a touch panel, and which recognizes the stroke data and converts it into text data. a recognition control unit; an authentication control unit that authenticates a user based on the stroke data; and a display that displays a display component that accepts sign-in along with the text data when the authentication control unit determines that user authentication is successful. It is characterized by having a part.

It is possible to provide a handwriting input device that allows easy sign-in.

FIG. 7 is a diagram illustrating a comparative example of a sign-in operation method when a user signs in to a handwriting input device. FIG. 2 is a diagram illustrating an outline of signing in to a handwriting input device. It is a figure which shows an example of the perspective view of a pen. 1 is a diagram showing an example of an overall configuration diagram of a handwriting input device. 1 is an example of a hardware configuration diagram of a handwriting input device. It is a figure explaining the function which a handwriting input device and a pen have. It is a figure which shows an example of defined control data. It is a figure which shows an example of the dictionary data of a handwriting recognition dictionary part. It is a figure which shows an example of the dictionary data of a character string conversion dictionary part. It is a figure which shows an example of the dictionary data of a predictive conversion dictionary part. FIG. 3 is a diagram illustrating an example of operation command definition data and system definition data held by an operation command definition unit. FIG. 7 is a diagram illustrating an example of operation command definition data when there is a selected object selected by a handwritten object. FIG. 3 is a diagram illustrating an example of user-defined data held by an operation command definition section. FIG. 3 is a diagram showing an example of handwritten signature data held by a handwritten signature data storage unit. FIG. 3 is a diagram showing an example of handwritten input storage data stored in a handwritten input storage unit. FIG. 3 is a diagram illustrating pen ID control data stored by a pen ID control data storage unit. FIG. 7 is a diagram illustrating an example of an operation guide and selectable candidates displayed by the operation guide. FIG. 7 is a diagram illustrating the relationship between the position of an operation guide and the position of a handwritten object rectangular area display. FIG. 7 is a diagram showing an operation guide displayed above a handwritten object rectangular area display. FIG. 3 is an example of a diagram illustrating an example of specifying a selected object; FIG. FIG. 7 is a diagram illustrating a display example of operation command candidates based on operation command definition data when there is a handwritten object. FIG. 7 is a diagram illustrating a display example of operation command candidates based on operation command definition data when there is a handwritten object. FIG. 3 is a diagram illustrating a method of inputting 90 degree angle information. FIG. 7 is a diagram illustrating another method of inputting angle information. FIG. 3 is a diagram illustrating a method of registering handwritten signature data. FIG. 7 is a diagram illustrating an example of an operation guide displayed when the user handwrites "Suzuki", which is handwritten signature data registered by the user. FIG. 2 is a diagram illustrating a method of changing user-defined data. FIG. 7 is an example sequence diagram (part 1) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 7 is an example sequence diagram (Part 2) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 7 is an example sequence diagram illustrating a process in which the handwriting input device displays character string candidates and operation command candidates (Part 3); FIG. 12 is an example sequence diagram (part 4) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 6 is an example sequence diagram (part 5) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates; FIG. 6 is an example sequence diagram (part 6) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 7 is an example sequence diagram illustrating a process in which the handwriting input device displays character string candidates and operation command candidates (part 7); It is a figure which shows the other example of a structure of a handwriting input device. It is a figure which shows the other example of a structure of a handwriting input device. It is a figure which shows the other example of a structure of a handwriting input device. It is a figure which shows the other example of a structure of a handwriting input device. It is an example of a system configuration diagram of a handwriting input system (Example 2). 1 is an example of a hardware configuration diagram of an information processing system. 1 is an example of a functional block diagram showing functions of the handwriting input system in a block form. FIG. 3 is an example sequence diagram (Part 1) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 7 is an example sequence diagram (Part 2) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 7 is an example sequence diagram illustrating a process in which the handwriting input device displays character string candidates and operation command candidates (Part 3); FIG. 12 is an example sequence diagram (part 4) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 6 is an example sequence diagram (part 5) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates; FIG. 6 is an example sequence diagram (part 6) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates. FIG. 7 is an example sequence diagram illustrating a process in which the handwriting input device displays character string candidates and operation command candidates (part 7); FIG. 8 is an example sequence diagram (part 8) illustrating a process in which the handwriting input device displays character string candidates and operation command candidates;

DETAILED DESCRIPTION OF THE INVENTION Hereinafter, as an example of a mode for carrying out the present invention, a handwriting input device and a handwriting input method performed by the handwriting input device will be described with reference to the drawings.

<Comparative example of handwriting input device>
For convenience of explanation of the handwriting input device in this embodiment, first, a comparative example regarding sign-in to the handwriting input device will be briefly described.

FIG. 1 shows a comparative example of a sign-in operation method when a user signs in to the handwriting input device 2. FIG. 1(a) shows an operation screen 101 on the display. FIG. 1(b) shows the sign-in screen 102. FIG. 1C shows an enlarged view of the sign-in screen 102 and the soft keyboard 105. The sign-in screen 102 has a user name input field 103 and a password input field 104, and if the combination of user name and password entered by the user is registered in the handwriting input device 2, authentication is successful and the user The handwriting input device 2 can now be used in a state where the handwriting input device 2 is identified.

The handwriting input device 2 is equipped with a pen, and can be used to handwrite characters, numbers, symbols, alphabets, etc. (hereinafter simply referred to as characters), but when signing in, the pen is not displayed on the operation screen 101. The user must operate the soft keyboard 105 and press the user name and password with a pen. The keyboard is an input device that is easy to use when typing keys with both hands facing downward, and is not designed to make it easy to press each key on the soft keyboard 105 displayed on the operation screen 101 with a pen. In other words, the soft keyboard 105 is simply an imitation of a real keyboard so that the user can input keys even in devices that do not come with a keyboard.

Since the soft keyboard 105 displayed on the operation screen 101 is difficult to use, the user can also sign in with an IC card, as shown in FIG. 1(c). The user can sign in by simply pressing the "Use IC Card" button 106 and holding the IC card over the IC card reader. If you are a company that has already introduced IC cards, you can sign in using an IC card by simply installing an IC card reader, but IC cards hold different data for each company. Newly introducing telework in companies that already have telework in place will incur huge costs.

Taking advantage of the fact that the handwriting input device 2 allows handwriting input of characters, etc., it is sufficient to be able to sign in by handwriting the user name, etc. with a pen, but it is necessary to display a special frame for the user to sign in by hand. Additional operations were required. In other words, the user could not handwrite without distinguishing between inputting characters and the like and inputting a handwritten signature.

<Outline of sign-in of handwriting input device in this embodiment>
Therefore, the handwritten input device 2 of this embodiment authenticates the user using a handwritten user name or the like, without distinguishing it from inputting characters or the like. Stroke data for a user to sign in, such as a user name, is called handwritten signature data. The user does not need to perform any operations to sign in; the user can sign in by handwriting a user name, etc., in the same way as when inputting characters by hand.

FIG. 2 is a diagram illustrating an outline of sign-in to the handwriting input device 2 of this embodiment. FIG. 2(a) shows the operation screen 101. As shown in FIG. First, when a user signs in, the user handwrites the user's name on the operation screen 101. As with the case of handwriting any characters, it is sufficient to simply handwrite the user name, and there is no need to input a message indicating that a sign-in operation is to be performed or to display a sign-in screen. Handwritten signature data is registered in the handwritten input device 2 in advance.

FIG. 2(b) shows an example of the operation guide 500. When the user starts handwriting, the handwriting input device 2 displays an operation guide 500 below the handwritten object 504. In FIG. 2, the user name (Suzuki) is a handwritten object 504.

In addition, one or more selectable candidates 530 are displayed in the operation guide 500. In FIG. 2B, the selectable candidates 530 include an operation command 512 (an example of a display component) and character string candidates 539 (handwritten recognition character string/language character string candidates, conversion character string candidates, character strings/predictive conversion, etc., which will be described later). candidates) are displayed. The operation command 512 is an operation command for "handwritten signature", and is displayed when the stroke data "Suzuki" handwritten by the user matches the handwritten signature data registered in advance and the authentication is successful. In other words, it is displayed when user authentication is successful. When the user presses the operation command 512 with the pen 2500 or the like, the user can sign in to the handwriting input device 2. If the user authentication is not successful, the recognition result of the handwritten object 504 is displayed.

In addition to the operation command 512, various other operation commands are prepared in the handwriting input device 2. If the text data converted from handwritten stroke data partially matches (matches in part or in whole) a pre-registered character string for calling an operation command, the corresponding operation command is displayed. In other words, the user can call up the operation command 512 for signing in by handwriting the same way as when calling up other operation commands.

In this way, the handwriting input device 2 of this embodiment allows the user to handwrite without distinguishing between inputting characters and inputting a handwritten signature, and can input various operation commands and the operation command 512 for signing in. Users can call them by hand without distinguishing between them.

Furthermore, the handwriting input device 2 of this embodiment authenticates users simply by intuitive handwriting without using an on-screen soft keyboard or adding special hardware such as an IC card. becomes possible.

<About terms>
The input means may be any means that allows handwriting on a touch panel. Examples include a pen, a human finger or hand, and a rod-shaped member. Additionally, line of sight input may be possible.

Stroke data is a freely handwritten line. It has a continuous set of points and may be interpolated as appropriate.

The operation command is an instruction prepared for operating the handwriting input device 2 and instructing execution of a specific process. In this embodiment, editing, modification, input/output, and pen state operation commands are taken as examples, but all commands for operating the handwriting input device 2, such as screen reversal, page switching, and operation mode settings, are used as examples. is the target.

The handwritten signature data is stroke data for signing in by hand. It is not limited to the user name as long as it is stroke data registered in the handwritten signature data storage section.

Signing in means entering information that shows your identity into a computer and applying for connection and start of use. If the identity information stored on the computer matches, the computer can be used based on predetermined privileges. Also called login or logon.

The display component for accepting sign-in may be any soft key displayed for accepting sign-in, and may be not limited to an operation command but may be an icon, a button, or the like.

<Example of pen appearance>
FIG. 3 shows an example of a perspective view of pen 2500. FIG. 3 shows an example of a multifunctional pen 2500. A pen 2500 that has a built-in power supply and can send commands to the handwriting input device 2 is called an active pen (a pen that does not have a built-in power supply is called a passive pen). The pen 2500 in Figure 3 has one physical switch at the pen tip, one at the pen tip, and two physical switches on the side of the pen; the tip is for writing, the tip is for erasing, and the side of the pen is for assigning user functions. It is. The present embodiment further includes a nonvolatile memory, which stores pen IDs that do not overlap with other pens.

Note that if the pen is equipped with a switch, it is also possible to reduce the number of operating procedures for the handwriting input device 2 by the user. A pen with a switch mainly refers to an active pen, but it includes not only an active pen but also a passive pen using an electromagnetic induction method, since even a passive pen without a built-in power supply can generate power using only an LC circuit. Pens with optical, infrared, and capacitive switches other than electromagnetic induction are active pens.

It is assumed that the hardware configuration of the pen 2500 is similar to a general control system including a communication function and a microcomputer. The pen 2500 may be of an electromagnetic induction type, an active capacitive coupling type, or the like. It may also have functions such as pen pressure detection, tilt detection, and a hover function (displaying a cursor before the pen touches it).

<Overall configuration of the device>
The overall configuration of the handwriting input device 2 according to this embodiment will be described using FIG. 4. FIG. 4 is a diagram showing the overall configuration of the handwriting input device 2. As shown in FIG. FIG. 4A shows, as an example of the handwriting input device 2, the handwriting input device 2 used as a horizontally elongated electronic blackboard hung on a wall.

As shown in FIG. 4A, a display 220 as an example of a display device is installed above the handwriting input device 2. User U can handwrite (also referred to as input or drawing) characters and the like on display 220 using pen 2500.

FIG. 4(b) shows a handwriting input device 2 used as a vertically elongated electronic blackboard hung on a wall.

FIG. 4C shows the handwriting input device 2 placed flat on a desk 230. Since the handwriting input device 2 has a thickness of about 1 cm, there is no need to adjust the height of the desk even if it is placed flat on an ordinary desk. It can also be moved easily.

<Device hardware configuration>
Next, the hardware configuration of the handwriting input device 2 will be explained using FIG. 5. The handwriting input device 2 has the configuration of an information processing device or a computer as shown in the figure. FIG. 5 is an example of a hardware configuration diagram of the handwriting input device 2. As shown in FIG. As shown in FIG. 5, the handwriting input device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204. We are prepared.

Among these, the CPU 201 controls the operation of the handwriting input device 2 as a whole. The ROM 202 stores programs used to drive the CPU 201, such as the CPU 201 and an IPL (Initial Program Loader). RAM 203 is used as a work area for CPU 201. The SSD 204 stores various data such as programs for the handwriting input device 2.

The handwriting input device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a power switch 227, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I/O F223, a power control circuit 224, an AC adapter 225, and a battery 226.

The display controller 213 controls and manages screen display in order to output the output image to the display 220 or the like. The touch sensor 216 detects that the pen 2500, the user's hand, or the like (the pen or the user's hand serves as an input means) contacts the display 220. The touch sensor 216 also receives the pen ID.

Touch sensor controller 215 controls processing of touch sensor 216. The touch sensor 216 inputs and detects coordinates. For example, in the case of an optical system, two light receiving/emitting devices installed at both ends of the upper side of the display 220 emit a plurality of infrared rays in parallel to the display 220. This is a method of receiving light that is reflected by a reflecting member provided around the light receiving element 220 and returns along the same optical path as the light emitted by the light receiving element. The touch sensor 216 outputs the position information of the infrared rays emitted by the two light receiving and emitting devices blocked by the object to the touch sensor controller 215, and the touch sensor controller 215 identifies the coordinate position that is the contact position of the object. Furthermore, the touch sensor controller 215 has a communication unit 215a and can communicate with the pen 2500 wirelessly. For example, when communicating using a standard such as Bluetooth (registered trademark), a commercially available pen can be used. By registering one or more pens 2500 in the communication unit 215a in advance, the user can communicate with the handwriting input device 2 without performing connection settings for communicating the pen 2500 with the handwriting input device 2.

The power switch 227 is a switch for switching the power of the handwriting input device 2 on and off. The tilt sensor 217 is a sensor that detects the tilt angle of the handwriting input device 2. It is mainly used to detect whether the handwriting input device 2 is used in the installation state shown in FIG. 4(a), FIG. 4(b), or FIG. 4(c), and depending on the installation state, You can automatically change the thickness of characters, etc.

The serial interface 218 is an external communication interface such as a USB. Used for inputting information from outside. The speaker 219 is used for audio output, and the microphone 221 is used for audio input. The wireless communication device 222 communicates with a terminal carried by a user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), etc., but the communication standard does not matter. The wireless communication device 222 forms an access point, and when the user sets the obtained SSID (Service Set Identifier) and password on the terminal carried by the user, the user can connect to the access point.

Note that it is preferable that the wireless communication device 222 be provided with two access points.
a. Access point → Internet
b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the internal network but can use the Internet. Access point b is for internal users, who can use the internal network and the Internet.

The infrared I/F 223 detects the handwriting input device 2 placed next to it. Utilizing the straightness of infrared rays, only the handwriting input device 2 placed next to it can be detected. Preferably, one infrared I/F 223 is provided on each side, and it is possible to detect in which direction of the handwriting input device 2 another handwriting input device 2 is placed. The adjacent handwriting input device 2 can display handwritten information that has been handwritten in the past (handwritten information on another page with the width of one display 220 being one page).

A power supply control circuit 224 controls an AC adapter 225 and a battery 226, which are the power sources of the handwriting input device 2. The AC adapter 225 converts alternating current shared by commercial power sources into direct current.

If the display 220 is a so-called electronic paper, it consumes little or no power to maintain the image after it has been drawn, so it can also be driven by the battery 226. This allows the handwriting input device 2 to be used for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

Furthermore, the handwriting input device 2 includes a bus line 210. The bus line 210 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 201 shown in FIG. 5.

Note that the touch sensor 216 is not limited to an optical type, and may be a capacitive touch panel that identifies a touch position by detecting a change in capacitance, or a touch panel that identifies a touch position by a voltage change between two opposing resistive films. Various detection means may be used, such as a resistive film type touch panel or an electromagnetic induction type touch panel that detects electromagnetic induction generated when a touch object contacts the display section to specify the contact position. The touch sensor 216 may be of a type that does not require an electronic pen to detect the presence or absence of a touch with the pen tip. In this case, you can use your fingertips or a pen-shaped stick to perform touch operations. Note that the pen 2500 does not need to be an elongated pen type.

<About device functions>
Next, the functions of handwriting input device 2 and pen 2500 will be described using FIG. 6. FIG. 6A is an example of a functional block diagram showing functions of the handwriting input device 2 in a block form. The handwriting input device 2 includes a handwriting input section 21, a display section 22, a handwriting input display control section 23, a candidate display timer control section 24, a handwriting input storage section 25, a handwriting recognition control section 26, a handwriting recognition dictionary section 27, and a character string conversion section. Control unit 28, character string conversion dictionary unit 29, predictive conversion control unit 30, predictive conversion dictionary unit 31, operation command recognition control unit 32, operation command definition unit 33, pen ID control data storage unit 36, handwritten signature authentication control unit 38 , and a handwritten signature data storage section 39. Each function that the handwriting input device 2 has is a function that is realized when any of the components shown in FIG. or means.

The handwriting input unit 21 is realized by a touch sensor 216 and the like, and receives handwritten input from the user and receives a pen ID. The handwriting input unit 21 converts the user's pen input d1 into pen operation data d2 (pen up, pen down, or pen coordinate data) with a pen ID, and sends it to the handwriting input display control unit 23. Pen coordinate data is periodically transmitted as discrete values, and coordinates between the discrete values are calculated by interpolation.

The display unit 22 is realized by a display 220 or the like, and displays handwritten objects and operation menus. The display unit 22 converts the drawing data d3 written into the video memory by the handwriting input display control unit 23 into data according to the characteristics of the display 220, and transmits the data to the display 220.

The handwriting input display control unit 23 performs overall control regarding handwriting input and display. The handwriting input display control section 23 processes the pen operation data d2 from the handwriting input section 21 and transmits it to the display section 22 for display. Details of the processing of the pen operation data d2 and the display of strokes will be explained later with reference to FIGS. 28 to 34.

The candidate display timer control unit 24 is a timer for display control of selectable candidates. A timer is started or stopped to generate a timing to start displaying selectable candidates and a timing to erase the display. The selectable candidates are handwritten recognition character string/language character string candidates, conversion character string candidates, character string/predictive conversion candidates, and operation command candidates that are selectably displayed in the operation guide described later. The candidate display timer control unit 24 receives a timer start request d4 (or a timer stop request) from the handwriting input display control unit 23, and transmits a timeout event d5 to the handwriting input display control unit 23.

The handwritten input storage unit 25 has a storage function for storing user data (handwritten objects/character string objects). The handwritten input storage unit 25 receives the user data d6-1 from the handwritten input display control unit 23 and stores it in the handwritten input storage unit 25. The handwritten input storage section 25 receives the acquisition request d6-2 from the handwritten input display control section 23, and transmits the user data d7 stored in the handwritten input storage section 25. The handwritten input storage section 25 transmits the position information d36 of the confirmed object to the operation command recognition control section 32.

The handwriting recognition control unit 26 is a recognition engine that performs online handwriting recognition. Unlike general OCR (Optical Character Reader), characters (not only Japanese but also multiple languages such as English), numbers, symbols (%, $, &, etc.), figures (lines, etc.), , circles, triangles, etc.). Various algorithms have been devised for the recognition method, but in this embodiment, the details will be omitted because known techniques can be used.

The handwriting recognition control unit 26 receives the pen operation data d8-1 from the handwriting input display control unit 23, executes handwriting recognition, and holds handwritten recognition character string candidates. Further, the handwriting recognition control unit 26 uses the handwriting recognition dictionary unit 27 to hold language character string candidates converted from the handwritten recognition character string candidates d12. Separately, when receiving the acquisition request d8-2 from the handwriting input display control unit 23, the handwriting recognition control unit 26 transmits the held handwritten recognition character string candidates and language character string candidates d9 to the handwriting input display control unit 23. .

The handwriting recognition dictionary section 27 is dictionary data for language conversion for handwriting recognition. The handwriting recognition dictionary section 27 receives the handwritten recognition character string candidate d12 from the handwriting recognition control section 26, converts it into a linguistically likely linguistic character string candidate d13, and sends it to the handwriting recognition control section 26. For example, in the case of Japanese, hiragana is converted to kanji or katakana.

The character string conversion control unit 28 controls the conversion of converted character string candidates into character strings. The converted character string is a character string that is likely to be generated including a handwritten recognition character string or a language character string. The character string conversion control unit 28 receives the handwritten recognition character string and the language character string candidate d11 from the handwriting recognition control unit 26, converts it into a converted character string candidate using the character string conversion dictionary unit 29, and holds it. Separately, when an acquisition request d14 is received from the handwritten input display control unit 23, the held conversion character string candidate d15 is transmitted to the handwritten input display control unit 23.

The character string conversion dictionary section 29 is dictionary data for character string conversion. The character string conversion dictionary unit 29 receives the handwritten recognition character string and the language character string candidate d17 from the character string conversion control unit 28, and transmits the conversion character string candidate d18 to the character string conversion control unit 28.

The predictive conversion control unit 30 receives the handwritten recognition character string and the language character string candidate d10 from the handwriting recognition control unit 26, and receives the converted character string candidate d16 from the character string conversion control unit 28. The predictive conversion control unit 30 uses the predictive conversion dictionary unit 31 to convert each of the handwritten recognition character strings, language character string candidates, and conversion character string candidates into predicted character string candidates. The predicted converted character string is a character string that is likely to be generated including a handwritten recognition character string, a language character string, or a converted character string. Separately, when the acquisition request d19 is received from the handwritten input display control unit 23, the predicted character string candidate d20 is transmitted to the handwritten input display control unit 23.

The predictive conversion dictionary section 31 is dictionary data for predictive conversion. The predictive conversion dictionary unit 31 receives the handwritten recognition character string, language character string candidates, and conversion character string candidates d21 from the predictive conversion control unit 30, and transmits the predicted character string candidates d22 to the predictive conversion control unit 30.

The operation command recognition control unit 32 receives the handwritten recognition character string and the language character string candidate d30 from the handwriting recognition control unit 26, and receives the converted character string candidate d28 from the character string conversion control unit 28. The operation command recognition control unit 32 receives the predicted character string candidate d29 from the predictive conversion control unit 30. The operation command recognition control unit 32 transmits an operation command conversion request d26 to the operation command definition unit 33 for each of the handwritten recognition character string, language character string candidate, conversion character string candidate, and predicted character string candidate, and the operation command definition unit 33 The operation command candidate d27 is received from. The operation command recognition control unit 32 holds operation command candidates d27.

If the operation command conversion request d26 partially matches the operation command definition, the operation command definition unit 33 transmits the operation command candidate d27 to the operation command recognition control unit 32.

Further, the operation command recognition control section 32 receives pen operation data d24-1 from the handwriting input display control section 23. The operation command recognition control unit 32 transmits a position information acquisition request d23 of a confirmed object input in the past and confirmed to the handwritten input storage unit 25. The operation command recognition control unit 32 holds the confirmed object specified by the pen operation data as a selected object (including position information). The operation command recognition control unit 32 specifies the position of the pen operation data d24-1 and a selected object that satisfies predetermined criteria. Separately, when the acquisition request d24-2 is received from the handwriting input display control unit 23, the selected object d25 identified as the held operation command candidate is transmitted to the handwriting input display control unit 23.

The pen ID control data storage section 36 holds pen ID control data (it can also be called a storage means). Before the handwriting input display control section 23 transmits the display data to the display section 22, the pen ID control data storage section 36 transmits the pen ID control data d41 to the handwriting input display control section 23. The handwriting input display control unit 23 draws display data under operating conditions stored in association with the pen ID. Furthermore, before the handwriting recognition control unit 26 executes handwriting recognition, the pen ID control data storage unit 36 transmits the angle information d44 of the pen ID control data to the handwriting recognition control unit 26, and the handwriting recognition control unit 26 Handwriting recognition is executed by rotating the stroke using angle information stored in association with the ID.

Further, after the handwriting recognition control unit 26 recognizes a straight line for setting angle information when the user handwrites characters, etc., the handwriting recognition control unit 26 converts the angle information d43 of the pen ID control data into the pen ID control data. The angle information d43 is transmitted to the storage unit 36 and stored in association with the pen ID. Further, after the handwriting input display control unit 23 executes the operation command for setting the angle information, the handwriting input display control unit 23 transmits the pen ID control data d42 to the pen ID control data storage unit 36, and displays the result of the execution of the operation command. (angle information set by the user) is associated with the pen ID and saved. Thereafter, handwriting recognition is executed after the stroke of that pen ID is rotated by the set angle information.

Further, the handwriting recognition control unit 26 transmits stroke data d49 rotated clockwise based on the angle information of the pen ID control data to the handwritten signature authentication control unit 38. This makes it possible to authenticate a handwritten signature regardless of the user's position (from which direction the user handwrites with respect to the handwriting input device 2).

The handwritten signature data storage section 39 stores handwritten signature data. When the handwritten signature data storage unit 39 receives the handwritten signature data acquisition request d45 from the handwritten signature authentication control unit 38, the handwritten signature data storage unit 39 transmits the handwritten signature data d46 to the handwritten signature authentication control unit 38. The format of the handwritten signature data depends on the handwritten signature authentication algorithm of the handwritten signature authentication control unit 38. The data in the handwritten signature data storage section 39 will be explained with reference to FIG.

When the handwritten signature authentication control unit 38 receives the clockwise rotated stroke data d49 from the handwritten recognition control unit 26, it transmits a handwritten signature data acquisition request d45 to the handwritten signature data storage unit 39, and the handwritten signature data storage unit 39 receives the handwritten signature data storage unit 39. The signature data d46 is transmitted to the handwritten signature authentication control section 38.

The handwritten signature authentication control unit 38 performs user authentication based on handwritten signature data. Although various algorithms have been devised for user authentication based on handwritten signature data, this embodiment uses a technology that can be recognized with a recognition rate that does not pose a problem in practical use. For example, you can generate a feature vector whose elements are the coordinates, pen pressure, stroke writing time, etc. that make up the handwritten signature data, weight the elements appropriately, and then use the feature vector of the registered handwritten signature data when signing in. Compare feature vectors such as user names handwritten by users. If the degree of matching is greater than or equal to the threshold, it is determined that the authentication has been successful, and if it is less than the threshold, it is determined that the authentication has failed.

The handwritten signature authentication control unit 38 holds the handwritten signature authentication result, which is the result of comparing the stroke data d49 and the handwritten signature data d46, and separately holds the handwritten signature authentication result when receiving an acquisition request d48 from the handwritten input display control unit 23. The handwritten signature authentication result d47 is sent to the handwritten input display control section 23. The handwritten signature authentication result determines whether or not the stroke data d49 and handwritten signature data d46 can be considered to match, and if it is determined that they match, it has a SignatureId, which will be described later, that is associated with the matched handwritten signature data d46. .

When the handwriting recognition result of the handwriting recognition control unit 26 matches the operation command instructing execution of handwritten signature registration, the handwriting recognition control unit 26 stores the handwritten signature registration form (as described later) from the handwritten signature storage unit 25. The handwritten signature data d50 of the data d52 is transmitted to the handwritten signature authentication control section 38. The handwritten signature authentication control unit 38 transmits the received handwritten signature data d50 to the handwritten signature data storage unit 39 and registers it.

If the handwriting recognition result of the handwriting recognition control unit 26 is an instruction to cancel or register a handwritten signature, the handwriting recognition control unit 26 transmits a handwritten signature registration form deletion request d51 to the handwritten input storage unit 25 and saves the handwritten input. The handwritten signature registration form is deleted from section 25.

When the handwriting recognition result of the handwriting recognition control unit 26 is an instruction to change user-defined data, the handwriting recognition control unit 26 acquires the data d53 input into the user-defined data change form from the handwriting input storage unit 25. The handwriting recognition control unit 26 transmits the change value d54 of the data d53 to the operation command definition unit 33 to change the user-defined data. The user-defined data will be explained with reference to FIG.

If the handwriting recognition result of the handwriting recognition control unit 26 is an instruction to cancel or register the user-defined data change form, the handwriting recognition control unit 26 transmits a user-defined data change form deletion request d55 to the handwriting input storage unit 25, The user-defined data change form is deleted from the handwritten input storage section 25.

FIG. 6(b) is a functional block diagram showing the functions of the pen 2500 in block form. The pen 2500 has a pen event transmitter 41. The pen event transmitting unit 41 attaches a pen ID to event data of pen up, pen down, and pen coordinates and transmits the data to the handwriting input device 2.

<About predefined control data>
Next, defined control data used by the handwriting input device 2 for various processes will be described using FIG. 7. FIG. 7 shows an example of predefined control data. In the example of FIG. 7, control data is shown for each control item.

The selectable candidate display timer 401 defines the time until displaying selectable candidates (an example of a first time). This is because selectable candidates are not displayed during handwriting. In FIG. 7, it means that selectable candidates are displayed if pen down does not occur within TimerValue=500 [ms] from pen up. The selectable candidate display timer 401 is held by the candidate display timer control unit 24. The selectable candidate display timer 401 is used when starting the selectable candidate display timer in step S18-2 of FIG. 30, which will be described later.

The selectable candidate deletion timer 402 defines the time until the displayed selectable candidates are deleted (an example of a second time). This is to delete the selectable candidates when the user does not select the selectable candidates. In FIG. 7, it means that if a selectable candidate is not selected within TimerValue=5000 [ms] after the selectable candidate is displayed, the selectable candidate display data is deleted. The selectable candidate deletion timer 402 is held by the candidate display timer control unit 24. The selectable candidate deletion timer 402 is used at step S64 in FIG. 32 to start the selectable candidate display deletion timer.

A handwritten object neighborhood rectangular area 403 defines a rectangular area that is considered to be near the handwritten object. In the example of FIG. 7, the handwritten object vicinity rectangular area 403 is a rectangle obtained by enlarging the rectangular area of the handwritten object by 50% (Horizontal) of the estimated character size in the horizontal direction and by 80% (Vertical) of the estimated character size in the vertical direction. It becomes an area. In the example of FIG. 7, the estimated character size is expressed as a percentage (specified as a percentage), but if the unit is set to "mm" or the like, it is possible to set the length to be a fixed length. The handwritten object vicinity rectangular area 403 is held by the handwritten input storage unit 25. The estimated character size 405 is used in step S10 of FIG. 29 to determine the overlapping state of the handwritten object vicinity rectangular area and the stroke rectangular area.

Estimated writing direction/character size determination condition 404 defines constants for determining the writing direction and the measurement direction of character size. In the example in Figure 7, the difference between the time when the first stroke was added to the handwritten object rectangular area and the time when the last stroke was added is MinTime = 1000 [ms] or more, and the horizontal distance (width) of the handwritten object rectangular area If the difference between and the vertical distance (height) is MinDiff = 10 [mm] or more, and the horizontal distance is longer than the vertical distance, it means that the estimated writing direction is determined to be "horizontal writing" and the estimated character size is determined to be the vertical distance. do. If the horizontal distance is shorter than the vertical distance, it means that the estimated writing direction is determined to be "vertical writing" and the estimated character size is determined to be the horizontal distance. If the above conditions are not met, the estimated writing direction is determined to be "horizontal writing" (DefaultDir="Horizontal"), and the estimated character size is determined to be the longer distance between the horizontal distance and the vertical distance. The estimated writing direction/character size determination condition 404 is held in the handwritten input storage unit 25. The estimated writing direction/character size determination condition 404 is used in obtaining the estimated writing direction in step S59 of FIG. 32 and in obtaining the character string object font in step S81 of FIG.

Estimated character size 405 defines data for estimating the size of characters, etc. In the example of FIG. 7, the estimated font size determined by the estimated writing direction/character size determination condition 404 is the estimated font size 405 of a small character 405a (hereinafter referred to as the minimum font size) and a large character 405c (hereinafter referred to as the minimum font size). maximum font size). If the estimated character size is smaller than the minimum font size, the estimated character size is determined to be the minimum font size. If the estimated character size is larger than the maximum font size, the estimated character size is determined to be the maximum font size. Otherwise, the character size is determined to be medium character 405b. The estimated character size 405 is held in the handwritten input storage unit 25. The estimated character size 405 is used in obtaining the character string object font in step S81 of FIG.

Specifically, the handwritten input storage unit 25 compares the estimated character size determined by the estimated writing direction/character size determination condition 404 with the FontSize of the estimated character size 405, and uses the font with the closest size. For example, if the estimated font size is 25 [mm] (FontSize for small fonts) or less, "Small font" is used, and if the estimated font size is over 25 mm and 50 mm (FontSize for medium fonts) or less, "Medium font" is used. If the font size exceeds 100mm (FontSize for large fonts), it is determined as "large font". "Small font" 405a is a 25mm Mincho font (FontStyle="Mincho" FontSize="25mm"), "Medium font" 405b is a 50mm Mincho font (FontStyle="Mincho" FontSize= "50mm"), "Large font" 405c uses a Gothic 100mm font (FontStyle="Gothic" FontSize="100mm"). If you want to increase the number of font sizes or styles, you can increase the number of estimated font sizes 405.

The straddle line determination condition 406 defines data used to determine whether multiple objects have been selected. The handwritten object is a single stroke, and in the example in Figure 7, the length of the long side of the handwritten object is 100 [mm] or more (MinLenLongSide="100mm"), and the length of the short side is 50 [mm] or less. (MaxLenShortSide="50mm"), and if there is an object with an overlapping ratio of 80% or more (MinOverLapRate="80%") in the long side direction and short side direction with the handwritten object, multiple objects will be selected. (selected object). The straddle line determination condition 406 is held by the operation command recognition control unit 32. The straddle line determination condition 406 is used in the straddle line determination for determining the selected object in step S50 of FIG.

The enclosing line determination condition 407 defines data used to determine whether an object is an enclosing line. In the example of FIG. 7, the operation command recognition control unit 32 determines a confirmed object whose overlap rate in the long side direction and short side direction of the handwritten object is 100% or more (MinOverLapRate="100%") as the selected object. The enclosing line determination condition 407 is held by the operation command recognition control unit 32. The enclosing line determination condition 407 is used in enclosing line determination for determining the selected object in step S50 of FIG.

Note that either the straddling line determination condition 406 or the enclosing line determination condition 407 may be determined with priority. For example, if the straddle line judgment condition 406 is made loose (to make it easier to select the straddle line), and the enclosing line judgment condition 407 is made strict (the value is such that only the enclosing line can be selected), the operation command is recognized. It is preferable that the control unit 32 prioritizes the enclosing line determination condition 407 for determination.

<Example of dictionary data>
The dictionary data will be explained using FIGS. 8 to 10. FIG. 8 is an example of dictionary data of the handwriting recognition dictionary section 27. FIG. 9 is an example of dictionary data of the character string conversion dictionary section 29. FIG. 10 is an example of dictionary data of the predictive conversion dictionary section 31. Note that these dictionary data are used in steps S33 to S42 in FIG. 31, respectively.

In this embodiment, the conversion result using the dictionary data of the handwriting recognition dictionary unit 27 shown in FIG. 8 is used as a language string candidate, the conversion result using the dictionary data of the character string conversion dictionary unit 29 shown in FIG. The conversion result based on the dictionary data of the conversion dictionary unit 31 is called a predicted character string candidate. "Before conversion" of each dictionary data represents the character string to be searched for in the dictionary data, "after conversion" represents the character string after conversion corresponding to the character string to be searched, and "probability" represents the probability of the user's selection. The probability is calculated from past user selections of each string. Therefore, the probability may be calculated for each user. Although various algorithms have been devised as methods for calculating probability, it is sufficient to use an appropriate method to calculate the probability, and the details will be omitted. The present embodiment is characterized in that character string candidates are displayed in descending order of selection probability from the estimated writing direction.

In the dictionary data of the handwriting recognition dictionary unit 27 in FIG. 8, the handwritten "gi" is "gi" with a probability of 0.55, "waza" with a probability of 0.4, and the handwritten "gishi" is "technician" with a probability of 0.5. Indicates that the probability is 0.45 to convert to "Engineer". The same applies to other "before conversion" character strings. In FIG. 8, the character string "before conversion" is handwritten in hiragana, but anything other than hiragana may be registered in "before conversion."

The dictionary data of the character string conversion dictionary section 29 in FIG. 9 shows that the character string "Ki" is converted to "Minutes" with a probability of 0.95, and the character string "Waza" is converted to "Skill test" with a probability of 0.85. The same applies to other "before conversion" character strings.

In the dictionary data of the predictive conversion dictionary unit 31 in FIG. 10, the character string "minutes" is converted to "destination of minutes" with a probability of 0.65, and the character string "skills test" is converted to "approval of skill test" with a probability of 0.75. Indicates that the In the example of FIG. 10, the character strings before conversion are all kanji, but characters other than kanji may be registered.

Note that the dictionary data is language-independent, and any character strings may be registered before and after conversion.

<Operation command definition data held by the operation command definition section>
Next, the operation command definition data used by the operation command recognition control section 32 will be explained using FIGS. 11 and 12. FIG. 11 shows an example of operation command definition data and system definition data held by the operation command definition unit 33.

FIG. 11(a) shows an example of operation command definition data. The operation command definition data in FIG. 11A is an example of operation command definition data when there is no selected object selected by the handwritten object, and all operation commands for operating the handwriting input device 2 are targeted. The operation command in FIG. 11A includes an operation command name (Name), a character string (String) that partially matches the character string candidate, and an operation command string (Command) to be executed. “%~%” in the operation command character string is a variable, and is associated with system definition data as shown in FIG. 11(b). In other words, "%~%" is replaced with the system definition data shown in FIG. 11(b).

First, in the operation command definition data 701, the operation command name is "Read Minutes Template", the character string that partially matches the character string candidate is "Minutes" or "Template", and the operation command string to be executed is "ReadFile https ://%username%:%password%@server.com/template/minutes.pdf". In this example, the operation command string to be executed includes system definition data "%~%", indicating that "%username%" and "%password%" are replaced with system definition data 704 and 705, respectively. Therefore, the final operation command string to be executed will be "ReadFile https://taro.tokkyo:x2PDHTyS@server.com/template/minutes.pdf" and "https://taro.tokkyo:x2PDHTyS@ Indicates that the file "server.com/template/minutes.pdf" is to be read (ReadFile).

In the operation command definition data 702, the operation command name is "save in minutes folder", the character string that partially matches the character string candidate is "minutes" or "save", and the operation command string to be executed is "WriteFile https: //%username%:%password%@server.com/minutes/%machinename%_%yyyy-mm-dd%.pdf". Similar to the operation command definition data 701, "%username%", "%password%", and "%machinename%" in the operation command string are replaced with system definition data 704 to 706, respectively. Note that "%yyyy-mm-dd%" indicates replacement with the current date. For example, if the current date is September 26, 2018, this indicates that it will be replaced with "2018-09-26". The final operation command to be executed is "WriteFile https://taro.tokkyo:x2PDHTyS@server.com/minutes/%My-Machine_2018-09-26.pdf", and the minutes are written as "https://taro.tokkyo". :x2PDHTyS@server.com/minutes/%My-Machine_2018-09-26.pdf" (WriteFile).

In the operation command definition data 703, the operation command name is "print", the character string that partially matches the character string candidate is "print" or "print", and the operation command string to be executed is "PrintFile https://%username%". :%password%@server.com/print/%machinename%－"%yyyy－mm－dd%.pdf". If you replace the operation command string in the same way as the operation command definition data 702, the final operation command to be executed will be "PrintFile https://taro.tokkyo:x2PDHTyS@server.com/print/%My-Machine_2018-09-26 .pdf", indicating that the file "https://taro.tokkyo:x2PDHTyS@server.com/print/%My-Machine_2018-09-26.pdf" is to be printed (PrintFile). That is, the file is sent to the server. The user communicates the printer with the server, specifies a file, and the printer prints the file's contents on paper.

In this way, since the operation command definition data 701 to 703 can be identified from the character string candidates, the user can display the operation command by handwriting it. Additionally, if user authentication is successful, "%username%", "%password%", etc. in the operation command definition data are replaced with user information, making it possible to input and output files in association with the user.

If the user is not authenticated (including the case of authentication failure if the authentication fails but the user can use the handwriting input device 2), the handwriting input device 2 uses the preset handwriting input device 2. Replace with %username%”, “%password%”, etc. Therefore, it is possible to input and output files in association with the handwriting input device 2 without user authentication.

Operation command definition data 709, 710, and 711 are operation commands for changing the pen state. The pen state may also be referred to as the type of pen. The operation command names of the operation command definition data 709, 710, and 711 are "thin pen," "thick pen," and "marker," and the character strings that partially match the character string candidates are "thin," "pen," "thick," and "pen." ”, “marker” or “pen”, and the operation command character string is “ChangePen fine”, “ChangePen bold”, or “Changepen marking”. When this operation command is executed, the pen state is stored in the pen ID control data storage section 36, so the user can handwrite strokes in the set pen state.

The operation command definition data 712 is an operation command for aligning the direction of text data in a certain direction. The operation command name of the operation command definition data 712 is "Align text direction," the character string that partially matches the character string candidate is "Text," "Direction," or "Direction," and the operation command string is "AlignTextDirection." Text data written by a user from a direction other than the top and bottom has different orientations, making it difficult to read everything from one direction. When the user executes the operation command definition data 712, the handwriting input device 2 aligns the recognized handwritten character strings in the same direction (for example, vertically). Aligning in this case means rotating the text data by the angle information.

The operation command definition data 713 indicates that the operation command name is "Register handwritten signature," the character strings that partially match the character string candidate are "Sign" and "Register," and the operation command character string is "RegistSignature." When the RegistSignature command is executed, a handwritten signature registration form is added to the handwritten input storage section 25, and the handwritten signature registration form is displayed on the operation screen 101. An example of the handwritten signature registration form will be described later (see FIG. 25).

The operation command definition data 714 indicates that the operation command name is "handwritten sign in," the character string partially located in the character string candidate is "%signature%," and the operation command is "Signin." Here, "%signature%" is a reserved word in system definition data, and represents the fact that registered handwritten signature data matches stroke data such as a user name. In other words, if they match, the operation command 512 based on the operation command definition data 714 is displayed on the operation guide 500 (see FIGS. 2 and 26).

When the Singin command is executed, the AccountId of the user having the SignatureId of the matching handwritten signature data is stored in the pen ID control data of the pen 2500 that has handwritten stroke data such as the user name. As a result, the pen ID and AccountId are associated with each other, and the handwriting input device 2 can use the user-defined data specified by this AccountId (see FIG. 16(a)).

The operation command definition data 715 indicates that the operation command name is "handwritten sign out", the character string that partially matches the character string candidate is "sign" or "out", and the operation command is "Signout". When the Signout command is executed, AccountId is deleted from the pen ID control data of the pen 2500 that operated the handwritten signout. This eliminates the association between the pen ID and AccountId, allowing any user to use the pen 2500.

The operation command definition data 716 indicates that the operation command name is "change settings," the character string that partially matches the character string candidate is "setting" or "change," and the operation command is "ConfigSettings." When the ConfigSettings command is executed, a user-defined data change form is added to the handwritten input storage section 25, and the user-defined data change form is displayed on the operation screen 101. The user-defined data change form will be described later (see FIG. 27).

Next, operation command definition data when there is a handwritten object, that is, editing-related and modification-related operation command definition data will be explained. FIG. 12 shows an example of operation command definition data when there is a selected object selected by a handwritten object. The operation command definition data in FIG. 12 includes an operation command name (Name), a group name (Group) of operation command candidates, and an operation command character string (Command) to be executed.

The operation command definition data 707 defines editing-related operation commands (Group="Edit"), and is an example of the definition data of editing-related operation command names "erase," "move," "rotate," and "select." In other words, these operation commands are displayed for the selected object, and the user can select the desired operation command.

The operation command definition data 708 defines the operation commands of the decoration type (Group="Decorate"), and the definition data examples of the operation command names of the decoration type "thick", "thin", "large", "small", and "underline" are shown below. be. These operation commands are displayed for the selected object, and the user can select the desired operation command. In addition, color operation commands may be displayed.

Therefore, since the operation command definition data 707 and 708 are specified when the user selects the selected object with the handwritten object, the operation command can be displayed by handwriting the user.

<User-defined data>
Next, user-defined data will be explained using FIG. 13. FIG. 13 shows an example of user-defined data held by the operation command definition unit 33. The user definition data in FIG. 13 is an example of one user's definition data. AccountId of the user-defined data 717 is user identification information that is automatically numbered for each user, AccountUsername and AccountPassword are the user name and password, and SignatureId is the identification of handwritten signature data that is automatically numbered when handwritten signature data is registered. The information, username, password, and machinename are character strings set in the operation command definition data 701 to 703 instead of the system definition data 704 to 706, respectively. This allows execution of operational commands using user-defined data.

When a user signs in by handwriting a user name, etc., the pen 2500 used by the user is A character string of user-defined data having an AccountId associated with a pen ID is used when executing an operation command. After the user signs out, even if the user uses the pen 2500 used to sign in, the character string of the system definition data is used when executing an operation command.

User-defined data 718 is data used in the user-defined data change form. Name is the item name of AccountUsername, AccountPassword, username, password, or machinename of the user-defined data 717, and Data is a changed value of AccountUsername, AccountPassword, username, password, or machinename. In this example, the "Name" data is "%AccountName%", the "Password" data is "%AccountPassword%", the "Folder User Name" data is "%username%", and the "Folder Password" data is " The data for "%password" and "folder file name" is "%machinename", which corresponds to each item of the user-defined data 717. These items entered in the user-defined data change form are reflected in the user-defined data 717.

<Handwritten signature data>
Next, handwritten signature data will be explained using FIG. 14. FIG. 14 shows an example of handwritten signature data held by the handwritten signature data storage section 39. The handwritten signature data has Data representing a handwritten signature associated with the SignatureId. SignatureId is identification information that is automatically numbered when handwritten signature data is registered, and Data is data calculated from the stroke data received from the handwritten signature authentication control unit 38 by the handwritten signature authentication algorithm of the handwritten signature authentication control unit 38.

<Handwritten input saved data saved by the handwritten input storage section>
Next, handwritten input storage data will be explained using FIG. 15. FIG. 15 shows an example of handwritten input storage data stored in the handwritten input storage section 25. One line in FIG. 15 represents one stroke. One piece of handwritten input storage data has the following items: DataId, Type, PenId, Color, Width, Pattern, Angle, AccountId, StartPoint, StartTime, EndPoint, EndTime, Point, and Pressure.

DataId is stroke identification information. Type is the type of stroke. Types include stroke, group, and text. The type of handwritten input storage data 801 and 802 is Stroke, and the type of handwritten input storage data 803 is Group. Group means to group other strokes, and for handwritten input saved data of type Group, specify the strokes to be grouped by DataId. PenId, Color, Width, Pattern, Angle, and AccountId are transcribed pen ID control data described below. StartPoint is the coordinates of the start point of the stroke, and StartTime is the time of the start point of the stroke. EndPoint is the coordinate of the end point of the stroke, and EndTime is the time of the end point of the stroke. Point is a coordinate string from the start point to the end point, and Pressure is the pen pressure from the start point to the end point. As shown in Angle, handwritten input saved data 804 and 805 are rotated clockwise by 180 degrees and 270 degrees, respectively, and then handwritten recognition is performed. 802 and 805 indicate data input by the user with AccountId=1 in the user-defined data.

<Pen ID control data stored by the pen ID control data storage unit>
Next, pen ID control data will be explained using FIG. 16. FIG. 16 is a diagram illustrating pen ID control data stored by the pen ID control data storage section 36. One line in FIG. 16(a) indicates pen ID control data for one pen. Further, FIG. 16(b) is a diagram illustrating angle information when the user writes by hand on the handwriting input device 2. The angle information can be said to be the angle in the direction in which the user is present, the angle in the direction in which the pen is used, or the angle related to the rotation of characters handwritten by the user. The angle information for each user is 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees counterclockwise, with the predetermined direction (for example, the vertical direction) of the handwriting input device 2 being 0 degrees (reference). degree.

The user's angle information is the position of the user relative to the handwriting input device 2 when the handwriting input device 2 is placed flat. In other words, the user's angle information can be said to be information related to position. It is possible to specify in which direction the user is located when viewed from the handwriting input device 2. In addition to the angle information, the direction seen from the handwriting input device 2 is likened to a clock: 0 degrees: 6 o'clock direction, 45 degrees: between 4 o'clock direction and 5 o'clock direction, 90 degrees: 3 o'clock direction, 135 degrees: 1 o'clock direction 180 degrees: 12 o'clock direction, 225 degrees: between 10 o'clock direction and 11 o'clock direction, 270 degrees: between 9 o'clock direction, 315 degrees: between 7 o'clock direction and 8 o'clock direction. You may.

Note that the angle information is not automatically determined based on the user's position, but each user inputs (specifies) the angle information. The resolution of angle information that can be specified (45 degrees in FIG. 16) is just one example, and may be smaller, such as 5 degrees to 30 degrees. However, it is believed that users can read characters that are rotated at around 45 degrees.

The pen ID control data includes PenId, Color, Width, Pattern, Angle, and AccountId. PenId is identification information stored inside the pen. Color is the stroke color set for this pen (can be changed arbitrarily by the user). Width is the stroke thickness set for this pen (can be changed arbitrarily by the user). Pattern is the stroke line type set for this pen (can be changed arbitrarily by the user). Angle is the stroke angle information set for this pen (can be changed arbitrarily by the user). In the example of FIG. 16A, the angle information of each pen is 0 degrees, 90 degrees, 180 degrees, and 270 degrees counterclockwise. AccountId is user identification information. By associating the AccountId with the pen ID, the AccountId associated with the pen ID of the pen 2500 used by the user can be specified, and the operation command can be executed using user-defined data.

Pen ID control data 901 is control data with a pen ID of 1, the color is black, the thickness is 1 pixel (1 px), the pattern is solid, the angle information is 0 degrees, and AccountId = 1. be. The user with AccountId=1 is the user of the user defined data 717 in FIG. 13. Indicates that this user has signed in by handwriting the user name etc. with the pen with pen ID=1. Pen ID control data without AccountId indicates a signed out state (not associated with a user).

Similarly, the pen ID control data 902 has a pen ID of 2. The color is black. The thickness is 1 pixel. The pattern is solid. The angle information is 90 degrees. AccountId is None. The pen ID control data 903 has a pen ID of 3. The color is black. The thickness is 10 pixels. The pattern is solid. The angle information is 180 degrees. AccountId is None. The pen ID control data 904 is black in color. The thickness is 10 pixels. The pattern is halftone dots. The angle information is 270 degrees. AccountId is None.

This data includes step S5 in FIG. 28 (obtain pen ID control data), step S20 in FIG. 30 (save angle information of pen ID control data), and step S21 in FIG. 30 (obtain angle information of pen ID control data). , step S60 in FIG. 32 (obtain pen ID control data), and step S88 in FIG. 34 (save angle information of pen ID control data).

<Display example of selectable candidates>
FIG. 17 is an example of an operation guide and selectable candidates 530 displayed by the operation guide. The operation guide 500 is displayed by the user handwriting the handwritten object 504 (by the timeout of the selectable candidate display timer). The operation guide 500 includes an operation header 520, operation command candidates 510, handwritten recognition character string candidates 506, conversion character string candidates 507, character string/predictive conversion candidates 508, and handwritten object rectangular area display 503. . The selectable candidates 530 are an operation command candidate 510, a handwritten recognition character string candidate 506, a conversion character string candidate 507, and a character string/predictive conversion candidate 508. In this example, there is no language conversion string, but it may be displayed. Furthermore, the selectable candidates 530 excluding the operation command candidate 510 are referred to as character string candidates 539.

Operation header 520 has buttons 501, 509, 502, 505. A button 501 accepts an operation for switching between predictive conversion and kana conversion. In the example of FIG. 17, when the user presses the button 509 that is displayed as "prediction", the handwriting input section 21 accepts it, notifies the handwriting input display control section 23 to that effect, and the display section 22 displays "Kana". The display changes to button 509. After changing from predictive conversion to kana conversion, the character string candidates 539 are arranged in descending probability order of “kana conversion”.

A button 502 operates the page for displaying candidates. In the example of FIG. 17, there are three candidate display pages, and the first page is currently being displayed. Button 505 accepts deletion of operation guide 500. When the user presses the button 505, the handwritten input unit 21 accepts the button, notifies the handwritten input display control unit 23 of this, and the display unit 22 erases the display other than the handwritten object. Button 509 accepts batch display deletion. When the user presses the button 509, the handwritten input unit 21 accepts this and notifies the handwritten input display control unit 23 of this fact. The display unit 22 erases all the displays shown in FIG. 17, including the handwritten object. Users can restart their handwriting from the beginning.

A handwritten object 504 is the character "gi" handwritten by the user. A handwritten object rectangular area display 503 surrounding the handwritten object 504 is displayed. The display procedure will be explained using sequence diagrams in FIGS. 28 to 34. In the example of FIG. 17, a handwritten object rectangular area display 503 is displayed in a dotted line frame.

The handwritten recognition character string candidates 506, the conversion character string candidates 507, and the character string/predictive conversion candidates 508 are arranged in descending order of probability. “gi” in the handwritten recognition character string candidate 506 is a recognition result candidate. In this example, "gi" is correctly recognized.

A converted character string candidate 507 is a converted character string candidate converted from a language character string candidate. In this example, "skill test" is an abbreviation for "technical mass production trial." The character string/predicted conversion candidate 508 is a predicted character string candidate converted from a language character string candidate or a converted character string candidate. In this example, "Approval of skill test" and "Delivery address for minutes" are displayed.

The operation command candidate 510 is an operation command candidate selected based on the operation command definition data 701 to 703 and 709 to 716 in FIG. 11(a). In the example of FIG. 17, the bullet ">" 511 indicates that it is a candidate for an operation command. In FIG. 17, there is no selection object selected by the handwritten object 504 "gi", and "minutes", which is a character string candidate for "gi", partially matches the operation command definition data 701, 702 in FIG. 11(a). Therefore, it is displayed as an operation command candidate 510.

When the user selects "Load minutes template", the operation command defined in the operation command definition data 701 is executed, and when the user selects "Save in minutes folder", the operation defined in the operation command definition data 702 is executed. The command is executed. In this way, operation command candidates are displayed when operation command definition data that includes a converted character string are found, so they are not always displayed.

As shown in FIG. 17, since the character string candidate and the operation command candidate are displayed simultaneously (both), the user can arbitrarily select either the character string candidate he or she intended to input or the operation command.

<Relationship between the position of the operation guide and the position of the handwritten object rectangular area display>
The display unit 22 displays an operation guide 500 including text data at a position corresponding to the position of the stroke data. The display unit 22 displays an operation guide 500 including text data at a position on the screen based on the position of the stroke data. In this way, the position of the operation guide 500 is determined by the position of the stroke data.

FIG. 18 is a diagram illustrating the relationship between the position of the operation guide and the position of the handwritten object rectangular area display. First, the width A and height H1 of the operation guide 500 are constant. The right end of handwritten object rectangular area display 503 and the right end of operation guide 500 match.

The width B of the handwritten object rectangular area display 503 is determined by the length of the handwritten object 504 handwritten by the user. In FIG. 18(a), the width B of the handwritten object rectangular area display 503 is one character, and since A>B, the coordinates (x0, y0) of the upper left corner P of the operation guide 500 are calculated as follows. . Note that the coordinates of the upper left corner Q of the handwritten object rectangular area display 503 are (x1, y1). The height of the handwritten object rectangular area display 503 is assumed to be H2.
x0 = x1 - (A-B)
y0 = y1+H2
On the other hand, as shown in FIG. 18(b), when the width B of the handwritten object rectangular area display is larger than A, the coordinates (x0, y0) of the upper left corner P of the operation guide 500 are calculated as follows.
x0 = x1+(B-A)
y0 = y1+H2
Note that although the operation guide 500 is displayed below the handwritten object rectangular area display 503 in FIG. 18 and the like, the operation guide 500 may be displayed above the handwritten object rectangular area display 503.

FIG. 19 shows an operation guide 500 displayed above the handwritten object rectangular area display 503. The method for calculating x1 is the same as that in FIG. 18, but y FIG. 18 is a diagram illustrating the relationship between the position of the operation guide and the position of the handwritten object rectangular area display. First, the width A and height H1 of the operation guide 500 are constant. The right end of handwritten object rectangular area display 503 and the right end of operation guide 500 match.

The width B of the handwritten object rectangular area display 503 is determined by the length of the handwritten object 504 handwritten by the user. In FIG. 18A, the width B of the handwritten object rectangular area display 503 is one character, and A>B, so the coordinates (x ₀ , y ₀ ) of the upper left corner P of the operation guide 500 are calculated as follows. be done. Note that the coordinates of the upper left corner Q of the handwritten object rectangular area display 503 are (x ₁ , y ₁ ). The height of the handwritten object rectangular area display 503 is assumed to be _H2 .
x ₀ = x ₁ - (A-B)
y ₀ = y ₁ + H ₂
On the other hand, as shown in FIG. 18(b), when the width B of the handwritten object rectangular area display is larger than A, the coordinates (x ₀ , y ₀ ) of the upper left corner P of the operation guide 500 are calculated as follows. .
x ₀ = x ₁ + (B-A)
y0 = _y1 + _H2
Note that although the operation guide 500 is displayed below the handwritten object rectangular area display 503 in FIG. 18 and the like, the operation guide 500 may be displayed above the handwritten object rectangular area display 503.

FIG. 19 shows an operation guide 500 displayed above the handwritten object rectangular area display 503. The method for calculating x ₁ is the same as in FIG. 18, but the method for calculating y ₀ is different.
y ₀ = y ₁ - H ₁
Note that the operation guide 500 may be displayed on the right or left side of the handwritten object rectangular area display 503. Further, if there is no display space for the operation guide 500 because the user has handwritten on the edge of the display, the operation guide 500 is displayed on the side where there is display space.
The method of calculating 0 will change.

<Example of specifying selected object>
The handwriting input device 2 of this embodiment allows the user to designate a selected object by selecting the confirmed object by handwriting. The selected object is subject to editing or modification.

FIG. 20 is an example of a diagram illustrating an example of specifying a selected object. In FIG. 20, the handwritten object 11 is shown with a solid black line, the handwritten object rectangular area 12 is shown with gray shading, the confirmed object 13 is shown with a black line, and the selected object rectangular area 14 is shown with a dotted line. In addition, when distinguishing between each, a lowercase alphabet is added to the code. In addition, as a determination condition (whether or not there is a predetermined relationship) for determining a confirmed object as a selected object, the straddle line determination condition 406 or enclosing line determination condition 407 of the defined control data shown in FIG. 7 is used. ing.

FIG. 20(a) is an example in which the user specifies two horizontally written confirmed objects 13a and 13b with a straddling line (handwritten object 11a). In this example, the length H1 of the short side and the length W1 of the long side of the handwritten object rectangular area 12a satisfy the straddle line determination condition 406, and the overlap rate with the confirmed objects 13a and 13b is the straddle line determination condition. Since the condition 406 is satisfied, both the confirmed objects 13a and 13b of "minutes" and "giji" are designated as selected objects.

FIG. 20(b) is an example in which a horizontally written confirmed object 13c is designated with a surrounding line (handwritten object 11b). In this example, only the confirmed object 13c "Minutes" whose overlap rate between the confirmed object 13c and the handwritten object rectangular area 12c satisfies the enclosing line determination condition 407 is designated as the selected object.

FIG. 20(c) is an example in which a plurality of vertically written confirmed objects 13d and 13e are designated by a straddling line (handwritten object 11c). In this example, as in FIG. 20(a), the short side length H1 and the long side length W1 of the handwritten object rectangular area 12d satisfy the straddle line determination condition 406, and the "minutes" Since the overlap rate with each of the two confirmed objects 13d and 13e called "Giji" satisfies the straddle line determination condition 406, both the confirmed objects 13d and 13e of "Minutes" and "Giji" are selected as selected objects. specified.

FIG. 20(d) is an example in which a vertically written confirmed object 13f is designated with a surrounding line (handwritten object 11d). In this example, like FIG. 20(b), only the confirmed object 13f "minutes" is designated as the selected object.

<Example of displaying operation command candidates>
FIG. 21 shows an example of displaying operation command candidates based on the operation command definition data when the handwritten object shown in FIG. 12 is present. FIG. 21(a) shows candidates for editing-related operation commands, and FIG. 21(b) shows candidates for modification-related operation commands. Further, FIG. 21(a) shows an example in which a selected object is designated by the handwritten object 11a of FIG. 20(a).

As shown in FIGS. 21(a) and 21(b), the operation command candidate displayed following the bullet ">" 511 is the main menu 550. The main menu 550 displays the name of the last executed operation command or the name of the first operation command in the operation command definition data. The first bullet “》” 511a on the first line is a candidate for an editing operation command, and the first bullet “》” 511b on the second line is a candidate for a modification operation command.

“>” 512 at the end of the line indicates that there is a submenu (an example of a submenu button). ">" 512a in the first line displays a submenu (last selected) for editing operation command candidates. ">" 512b on the second line displays the remaining submenus regarding candidates for modification-type operation commands. When the user presses ">" 512, a submenu 560 is displayed on the right side. All operation commands defined in the operation command definition data are displayed in the submenu 560. In the display example of FIG. 21(a), a submenu 560 corresponding to ">" 512a on the first line from when the main menu is displayed is also displayed. It may be displayed by pressing ">" 512a on the first line.

When the user presses any operation command name with a pen, the handwriting input display control unit 23 executes the command of the operation command definition data associated with the operation command name on the selected object. That is, when "Delete" 521 is selected, "Delete" is selected, when "Move" 522 is selected, "Move" is selected, when "Rotate" 523 is selected, "Rotate" is selected, and when "Select" 524 is selected, " Execute "Select" respectively.

For example, when the user presses "Erase" 521 with a pen, "Minutes" and "Giji" can be deleted, and when the user presses "Move" 522, "Rotate" 523, and "Select" 524, the bounding box (the circumscribed rectangle of the selected object) can be deleted. ) are displayed, "Move" 522 and "Rotate" 523 allow you to move or rotate by dragging the pen, and "Select" 524 allows you to perform other bounding box operations.

Character string candidates other than operation command candidates, such as “1” 541, “1,” 542, “~” 543, “→” 544, and “⇒” 545, are the recognition results of the straddle line (handwritten object 11a), If the user intends to input a string rather than an operation command, a string candidate can be selected.

In FIG. 21(b), by pressing ">" 512b on the second line, the submenu of FIG. 21(b) is displayed. The display example in FIG. 21(b) also displays a main menu 550 and a submenu 560, similar to FIG. 21(a). Based on the operation command definition data in FIG. 12, when "thick" 531 is selected, "Thick" is selected, when "thin" 532 is selected, "Thin" is selected, and when "large" 533 is selected, "Large" is selected. When "small" 534 is selected, the handwriting input display control unit 23 executes "Small" and when the underline 535 is selected, "Underline" is executed for the selected object.

In addition, how thick should it be made when "Thick" 531 is selected, how thin should it be made when "Thin" 532 is selected, how large should it be made when "Large" 533 is selected, and "Small" Fixed values are separately defined for how much to reduce when 534 is selected, the line type when underline 535 is selected, etc. Alternatively, it is even better if a selection menu is opened separately when the submenu shown in FIG. 21(b) is selected so that the user can make adjustments.

When the user presses ``thick'' 531 with a pen, the handwritten input display control unit 23 thickens the lines that constitute the confirmed objects 13a and 13b, ``minutes'' and ``giji''. When you press "Thin" 532 with a pen, the handwriting input display control unit 23 can make the lines that make up the "minutes" and "giji" thinner, and when you press "Large" 533 with a pen, the handwriting input display control unit 23 can make the lines that make up the "minutes" and "giji" thinner. By pressing "Small" 534 with a pen, the handwritten input display control section 23 can be made smaller, and by pressing "Underline" 535 with a pen, an underline can be added to the handwritten input display control section 23.

FIG. 22 shows an example of displaying operation command candidates based on the operation command definition data when the handwritten object shown in FIG. 12 is present. The difference from FIGS. 21(a) and 21(b) is that FIGS. 22(a) and 22(b) show an example in which the selected object is designated by the handwritten object 11b (encircled line) in FIG. 20(b). As can be seen by comparing FIGS. 21 and 22, there is no difference in the operation command candidates displayed depending on whether the handwritten object is a line or an enclosing line. The purpose of the handwriting input display control section 23 is to display operation command candidates on the display section 22 when a selected object is specified. However, the handwritten object may be recognized and the operation command candidates to be displayed may be changed depending on the handwritten object. In this case, the recognized handwritten objects (one, circle, etc.) are associated with operation command definition data as shown in FIG.

In FIG. 22, character string candidates other than operation command candidates, such as "○" 551, "∞" 552, "0" 553, "00" 554, and "RO" 555, are the recognition results of the enclosing line (handwritten object 11b). Yes, and if the user intends to input a string rather than an operation command, the user can select a string candidate.

<Example of inputting angle information>
Next, a method for inputting angle information will be explained using FIG. 23. FIG. 23 is an example of a diagram illustrating a method of inputting angle information. In FIG. 23, a case will be described in which a user located at the 3 o'clock direction of the handwriting input device 2 inputs angle information. Since characters etc. handwritten from the 3 o'clock direction are correctly recognized when rotated 90 degrees clockwise, it is preferable to input 90 degree angle information.

In FIG. 23(a), when the angle information of the pen ID control data is 0 degrees (initial value), the user who is present in the 3 o'clock direction of the handwriting input device 2 has handwritten "gi", so the operation guide 500 is Indicates the displayed state. Since the handwriting input device 2 recognizes the character "gi" handwritten from the 3 o'clock direction with the angle information being 0 degrees, selectable candidates 530 that are different from the expected are displayed.

When inputting angle information, the user handwrites a straight line in the operation guide 500 from top to bottom as viewed from the user. FIG. 23(b) shows an example of this straight line 521. The angle α formed by the straight line 521 in the counterclockwise direction and the 6 o'clock direction where the angle information is 0 degrees is the angle information. That is, the angle α formed by the straight line 522 drawn from the starting point S in the 6 o'clock direction and the straight line 521 input by the user in a counterclockwise direction is the angle information. Simply put, the direction of the end point of the straight line 521 is the angle information. Therefore, the angle information input by the user in FIG. 23(b) is 90 degrees.

Note that to detect a straight line, for example, there is a method of converting the coordinates from the starting point S to the ending point E into a straight line using the least squares method, and comparing the obtained correlation coefficient with a threshold value to determine whether it is a straight line. .

Immediately after the user starts writing the straight line 521 (immediately after touching the starting point S of the straight line 521 with the pen 2500), the handwriting input device 2 erases the operation guide 500. Immediately after writing the straight line 521 (immediately after the pen 2500 is released from the end point E of the straight line 521), the handwriting input device 2 can write the lines at 45 degrees, 90 degrees, 135 degrees, 180 degrees, 215 degrees, 270 degrees, and 315 degrees. , 360 degrees, the angle α formed above searches for the closest value and determines it as the angle information. The formed angle α itself may be used as angle information. The determined angle information is set in Angle of the pen ID control data. Since the pen event transmitting unit 41 of the pen 2500 transmits the pen ID to the handwriting input device 2 when the pen tip is pressed for handwriting, etc., the handwriting input device 2 can associate the angle information with the pen ID control data. I can do it.

Note that only the operation guide 500 allows the user to input angle information by hand-drawing a straight line. As a result, when the user draws a straight line by hand with a device other than the operation guide 500, it is recognized as "1" or "1", and when the user draws a straight line by hand with the operation guide 500, angle information can be input. That is, the handwriting recognition control unit 26 detects a straight line from a predetermined range and converts stroke data handwritten outside the predetermined range into text data.

FIG. 23(c) shows the operation guide 500 immediately after the operation in FIG. 23(b). Since 90 degrees is set as the angle information (Angle) in the pen ID control data, the handwritten object (stroke data) is internally rotated clockwise by 90 degrees for handwriting recognition, and the operation guide 500 only rotates by 90 degrees. It is displayed rotated counterclockwise.

FIG. 24 is an example of a diagram illustrating another method of inputting angle information. In FIG. 24, it is assumed that the user is present in the 3 o'clock direction of the handwriting input device 2. In FIG. 24, when the angle information is 0 degrees (initial value), the user located in the 3 o'clock direction of the handwriting input device 2 has handwritten "gi", so the operation guide 500 and selectable candidates 530 are displayed. There is. The operation guide 500 in FIG. 24 has a rotation operation button 511 in an operation header 520.

Each time the user presses the rotation operation button 511 with the pen 2500, the rotation operation button 511 is a button that adds the angle information of the pen ID control data by 90 degrees and changes it to the remainder of dividing by 360 degrees. This remainder becomes angle information. Note that the angle added by pressing the rotation operation button 511 once may be 45 degrees.

<Example of registering handwritten signature data>
Next, an example of registering handwritten signature data will be described using FIG. 25. FIG. 25 is a diagram illustrating a method of registering handwritten signature data. First, FIG. 25(a) is an example of selectable candidates 530 displayed when the user handwrites "signature". Two operation commands 513 and 514, ``Register handwritten signature'' and ``Output handwritten signature'', based on operation command definition data 713 and 715, which partially match the character string ``Signature'', and ``Sign'' and ``Autograph session''. Character string candidates for "Signed" are displayed. The reason why the two operation commands 513 and 514 are displayed is because the Strings of the operation command definition data 713 and 715 in FIG. 11 have "sign".

When the user presses “Register handwritten signature” with the pen 2500, the handwritten signature registration form 561 shown in FIG. 25(b) is added to the handwritten input storage unit 25 and displayed on the operation screen 101. For example, the operation guide 500 in FIG. 25(a) is deleted, and the handwritten signature registration form 561 is displayed at the same location as the operation guide 500. The handwritten signature registration form 561 has, from the top, a name input field 561a, signature input fields 561b to 561d, and a registration confirmation field 561e. The user inputs a name text in the name input field 561a, a first handwritten signature, a second handwritten signature, and a third handwritten signature in the signature input fields 561b to 561d, and a check mark or cancellation mark in the registration confirmation field 561e. . The name text is the display name for this user and is converted to text data. The reason why the handwritten signature is input three times is because the feature quantities are registered on the assumption that the handwritten signature differs each time the user writes it and does not completely match.

Note that the handwritten signature is generally characters related to the user such as a user name, and in addition to the user name, it may also be a number such as an employee number, a nickname, a portrait, etc. Further, the handwritten signature is not limited to characters related to the user, but may be any handwritten object. For example, it may be a circle, triangle, square, symbol, or a combination thereof. Since coordinates are not the only characteristic data for handwritten signatures, even if two users with the same last name (for example, Mr. Suzuki) both register their handwritten signatures as "Suzuki," they can be authenticated correctly.

When the user writes handwritten notes on the handwritten signature registration form 561 as instructed, the handwritten signature registration form 561 shown in FIG. 25(c) is created. When the user handwrites a "check mark" in the registration confirmation field 561e, the handwritten signature data is registered in the handwritten signature data storage section 39 and the handwritten signature registration form 561 is deleted. A SignatureId is assigned a number upon registration. Similarly, the AccountId numbered and the text of the name in the name input field 561a are registered in the user-defined data in association with the SignatureId. When a user signs in by handwriting a user name, etc., the SignatureId associated with AccountId in the user-defined data is acquired and registered in the pen ID control data in association with the pen ID of the pen 2500 used for handwritten sign-in. . From then on, when the user uses the pen 2500, the pen ID is sent to the handwriting input device 2, so the AccountId associated with the pen ID is specified using the pen ID control data. It becomes possible to execute operation commands using user-defined data without the user being aware of it.

When the user handwrites an "x" in the registration confirmation field 561e, the handwritten signature registration is canceled and the handwritten signature registration form 561 is deleted. Note that if any error occurs regarding registration, a message to that effect is displayed in the system reserved area of the operation screen 101 or the like.

In this way, the handwritten input display control unit 23 can accept handwritten input without distinguishing between handwritten input on a form and handwritten input on a part other than the form.

<Example of handwritten sign-in>
Next, a method for the user to sign in after registering handwritten signature data will be explained using FIG. 26. FIG. 26 is an example of an operation guide 500 that is displayed when the user handwrites "Suzuki", which is handwritten signature data registered by the user. Since "Suzuki" is registered in the operation command definition unit 33 as handwritten signature data, "Suzuki" matches the handwritten signature data. Therefore, an operation command 512 "sign in by hand" is displayed.

Furthermore, since the handwritten signature data is suitable, the SignatureId representing "Suzuki" is specified, and the user-defined data having the AccountId associated with this SignatureId is specified.

When the user selects the operation command 512 "Sign in by handwriting", the AccountId of "Suzuki" is associated with the pen ID of the pen 2500 being used and added to the pen ID control data, and the user-defined data of "Suzuki" is added to the pen ID control data. is now used when using operational commands.

Since the registration of handwritten signature data using the handwritten signature registration form 561 in FIG. 26 is controlled as part of the processing for normal handwritten input of characters, etc., the handwritten signature registration form 561 is used as an operation screen for handwriting characters, etc. displayed on the same operation screen. There is no difference between handwriting operations inside and outside the handwritten signature registration form 561, and the user can complete the input to the handwritten signature registration form 561 by simply inputting handwriting into the area delimited by the ruled lines of the handwritten signature registration form 561.

<Example of changing user-defined data>
Next, a method for changing user-defined data will be explained using FIG. 27. FIG. 27 is a diagram illustrating a method of changing user-defined data. FIG. 27(a) is an example of an operation guide 500 that is displayed when the user handwrites "se". In the operation command definition data 716 shown in FIG. 11, "setting" is defined in String, and "setting" is found in the predicted character string of "se", so the operation command "change settings" is displayed.

When the user selects "Change settings" with the pen 2500 signed in by hand, the user's AccountId associated with the pen ID of this pen 2500 is specified in the pen ID control data. This identifies user-defined data for the signed-in user. The user-defined data change form 562 shown in FIG. 27(b) is added to the handwritten input storage section 25 and displayed on the operation screen 101. In the example of FIG. 27, the user-defined data change form 562 is created according to the user-defined data 718 of FIG. 13. The user-defined data change form 562 has a name field 562a, a password field 562b, a folder user name field 562c, a folder password field 562d, a folder file name field 562e, and a registration or cancellation field 562f.

Note that if the user has not signed in by hand in advance, the handwritten input device 2 will not be able to identify the user's Account ID, resulting in an error, and an error message will be displayed in the system reserved area of the operation screen 101.

In the user-defined data change form 562 in FIG. 27(b), enter the password in the password field 562b, the folder user name in the folder user name field 562c, the folder password in the folder password field 562d, and the folder file name in the folder file name field 562e. and a "check mark" or "x" in the registration or cancellation column 562f by hand, the user-defined data is changed, and the user-defined data change form 562 is deleted.

In this manner, the user can display the user-defined data change form 562 and arbitrarily change the user-defined data by handwriting stroke data that calls the user-defined data change form 562. The handwritten input display control unit 23 accepts handwritten input without distinguishing between handwritten input to a form and handwritten input other than the form.

Note that the AccountUsername included in the user-defined data is automatically displayed in the name column 562a. Further, the user-defined data change form 562 can be used not only for changes but also for registration.

Since the change of user-defined data using the user-defined data change form 562 in FIG. 27 is controlled as part of the process of normal handwritten input of characters, etc., the user-defined data change form 562 is an operation in which characters, etc. are handwritten. Displayed on the same operation screen as the screen. There is no difference between handwriting operations inside and outside the user-defined data change form 562, and the user completes input to the user-defined data change form 562 by simply inputting handwriting into the area delimited by ruled lines on the user-defined data change form 562. I can do it.

<Operating procedure>
The operation of the handwriting input device 2 will be explained using the above configuration and FIGS. 28 to 34. 28 to 34 are sequence diagrams illustrating an example of a process in which the handwriting input device 2 displays character string candidates and operation command candidates. The process in FIG. 28 starts when the handwriting input device 2 starts up (when the application starts up). Note that in FIGS. 28 to 34, the functions in FIG. 6 are indicated by symbols for space reasons.

S1: First, the handwritten input display control unit 23 transmits a handwritten object start signal to the handwritten input storage unit 25. The handwritten input storage unit 25 secures a handwritten object area (memory area for storing handwritten objects). The handwritten object area may be secured after the user contacts the handwriting input section 21 with a pen.

S2: Next, the user touches the handwriting input section 21 with a pen. The handwriting input section 21 detects pen down and transmits the detected pen down to the handwriting input display control section 23 .

S3: The handwriting input display control unit 23 transmits a stroke start signal to the handwriting input storage unit 25, and the handwriting input storage unit 25 secures a stroke area.

S4: When the user moves the pen while keeping it in contact with the handwriting input section 21, the handwriting input section 21 transmits pen coordinates to the handwriting input display control section 23.

S5: The handwriting input display control unit 23 specifies the pen ID received from the pen 2500 at the same time as inputting the coordinates, and acquires the current pen ID control data stored in the pen ID control data storage unit 36. Since the pen ID is transmitted when coordinates are input, the stroke and pen ID are associated with each other. The pen ID control data storage section 36 transmits pen ID control data (color, thickness, pattern, angle information) to the handwriting input display control section 23 . Note that the angle information is still at the initial value of zero.

S6: The handwritten input display control unit 23 transmits pen coordinate complementary display data (data for interpolating discrete pen coordinates) to the display unit 22. The display unit 22 interpolates pen coordinates using the pen coordinate complementary display data and displays a line.

S7: The handwritten input display control section 23 transmits the pen coordinates and the reception time to the handwritten input storage section 25. The handwritten input storage unit 25 adds pen coordinates to the stroke. While the user is moving the pen, the handwriting input section 21 periodically repeats sending the pen coordinates to the handwriting input display control section 23, so the processes of steps S4 to S7 are repeated until the pen is raised.

S8: When the user releases the pen from the handwriting input section 21, the handwriting input section 21 transmits pen up to the handwriting input display control section 23.

S9: The handwriting input display control unit 23 transmits the end of the stroke to the handwriting input storage unit 25, and the handwriting input storage unit 25 determines the pen coordinates of the stroke. Once the pen coordinates of a stroke are determined, it is no longer possible to add pen coordinates to the stroke.

S10: Next, the handwritten input display control unit 23 transmits to the handwritten input storage unit 25, based on the handwritten object nearby rectangular area 403, the overlapping state acquisition of the handwritten object nearby rectangular area and the stroke rectangular area. The handwritten input storage section 25 calculates the overlap situation and transmits the overlap situation to the handwriting input display control section 23 .

Subsequent steps S11 to S17 are executed when the handwritten object vicinity rectangular area and the stroke rectangular area do not overlap.

S11: If the handwritten object vicinity rectangular area and the stroke rectangular area do not overlap, one handwritten object is determined, and the handwritten input display control unit 23 sends a held data clear command to the handwritten recognition control unit 26.

S12 to S14: The handwriting recognition control unit 26 sends the held data clear to the character string conversion control unit 28, predictive conversion control unit 30, and operation command recognition control unit 32, respectively. Clears data related to character string candidates and operation command candidates that have been held by the handwriting recognition control unit 26, character string conversion control unit 28, predictive conversion control unit 30, and operation command recognition control unit 32. Note that at the time of clearing, the last handwritten stroke has not been added to the handwritten object.

S15: The handwritten input display control unit 23 transmits the end of the handwritten object to the handwritten input storage unit 25. The handwritten input storage unit 25 fixes the handwritten object. Confirmation of a handwritten object means that one handwritten object is completed (no more strokes are added).

S16: The handwritten input display control unit 23 transmits the handwritten object start to the handwritten input storage unit 25. In preparation for starting handwriting (pen down) of the next handwritten object, the handwritten input storage unit 25 secures a new handwritten object area.

S17: Next, the handwritten input display control unit 23 transmits a stroke addition request to the handwritten input storage unit 25 regarding the stroke completed in step S9. When steps S11 to S17 are executed, the stroke to be added is the first stroke of the handwritten object, and the handwritten input storage unit 25 adds stroke data to the handwritten object that is being started. If steps S11 to S17 are not performed, the added strokes are added to the handwritten object that is already being handwritten.

S18: Next, the handwriting input display control unit 23 transmits the stroke addition to the handwriting recognition control unit 26. The handwriting recognition control unit 26 adds stroke data to a stroke data holding area (an area where stroke data is temporarily stored) where character string candidates are stored.

S19: The handwriting recognition control unit 26 executes gesture handwriting recognition on the stroke data holding area. Gesture handwriting recognition refers to recognizing angular information from straight lines. Note that since gesture handwriting recognition is performed inside the operation guide 500, the handwriting recognition control unit 26 detects straight lines inside the operation guide 500. The position information of the operation guide 500 is transmitted to the handwriting recognition control unit 26 in step S67, which will be described later.

S20: When a straight line in the operation guide 500 is detected, the angle α between the straight line 522 drawn from the starting point of the straight line in the 6 o'clock direction and the straight line 521 input by the user in a counterclockwise direction is determined in units of 45 degrees. do. Then, the handwriting recognition control unit 26 stores the determined angle information in the pen ID control data storage unit 36 in association with the pen ID included in the stroke data of the straight line 521. Note that step S20 is executed when a straight line is detected within the operation guide 500.

S21: Next, the handwriting recognition control unit 26 specifies the pen ID received from the handwriting input unit 21 and acquires the angle information of the current pen ID control data from the pen ID control data storage unit 36.

S22: The handwriting recognition control unit 26 rotates clockwise using the angle information obtained from the stroke data in the stroke data holding area.

S23: The handwritten recognition control unit 26 transmits the rotated stroke data to the handwritten signature authentication control unit 38. In this way, whether the stroke data is a handwritten signature or not is always transmitted to the handwritten signature authentication control unit 38 in a state where it is unclear.

S24: The handwritten signature authentication control unit 38 receives the stroke data, and receives registered handwritten signature data from the handwritten signature data storage unit 39. Then, the stroke data and the handwritten signature data are compared (matched), and the handwritten signature authentication result is held so that it can be obtained in the subsequent step S61.

S25: Next, the handwriting recognition control unit 26 executes handwriting recognition on the stroke data, and if it is a "check mark" or "x" in the form registration or cancellation column, it processes the form, otherwise it processes the form normally. Performs handwriting recognition processing.

S26: If there is a "check mark" in the registration or cancellation field of the handwritten signature data registration form, the user responds to the handwritten signature registration form generated by the handwritten input display control unit 23 in the handwritten input storage unit 25 in step S86, which will be described later. The handwritten signature data (stroke data) inputted by the user is transmitted by the handwritten signature recognition control section 26 to the handwritten signature authentication control section 38 .

S27: The handwritten signature authentication control unit 38 registers the received handwritten signature data (stroke data) in the handwritten signature data storage unit 39. As a result, SignatureId is numbered. The SignatureId is returned to the handwriting recognition control unit 26. If the signature ID and the name input in the name input field 561a of the handwritten signature registration form 561 are not included in the user-defined data, the handwritten recognition control unit 26 adds new user-defined data and assigns an AccountId. The handwriting recognition control unit 26 stores the SignatureId in its user-defined data. If the name entered in the name input field 561a is in user-defined data, SignatureId is saved in the user-defined data. Through this process, AccountId and SignatureId are linked. Note that when new user-defined data is added, other values are not set, but the user can register and change them from the user-defined data change form 562.

S28: Upon registration of the handwritten signature data, the handwritten signature control unit 26 deletes the handwritten signature registration form 561 from the handwritten input storage unit 25.

S29: If there is a "check mark" in the registration or cancellation field of the user-defined data change form, the user-defined data change form 562 generated by the handwritten input display control unit 23 in the handwritten input storage unit 25 in step S86, which will be described later. The input change value is transmitted by the handwriting recognition control section 26 to the operation command definition section 33.

S30: Upon execution of the user-defined data change, the handwriting recognition control unit 26 deletes the user-defined data change form 562 from the handwritten input storage unit 25.

S31: If the registration or cancellation column of the form added in step S86, which will be described later, is marked “×”, the handwriting recognition control unit 26 deletes the form added in step S86 from the handwriting input storage unit 25.

S33: If it is not form processing, the handwriting recognition control unit 26 sends the handwriting recognition character string candidate, which is the execution result, to the handwriting recognition dictionary unit 27. The handwriting recognition dictionary section 27 sends linguistically likely linguistic character string candidates to the handwriting recognition control section 26 .

S34: The handwriting recognition control unit 26 transmits the handwritten recognition character string candidates and the received language character string candidates to the character string conversion control unit 28.

S35: The character string conversion control unit 28 transmits the handwritten recognition character string candidates and the language character string candidates to the character string conversion dictionary unit 29. The character string conversion dictionary unit 29 sends the converted character string candidates to the character string conversion control unit 28 .

S36: The character string conversion control unit 28 transmits the received conversion character string candidates to the predictive conversion control unit 30.

S37: The predictive conversion control unit 30 transmits the received conversion character string candidates to the predictive conversion dictionary unit 31. The predictive conversion dictionary section 31 transmits the predicted character string candidates to the predictive conversion control section 30.

S38: The predictive conversion control unit 30 transmits the received predicted character string candidate to the operation command recognition control unit 32.

S39: The operation command recognition control unit 32 transmits the received predicted character string candidate to the operation command definition unit 33. The operation command definition unit 33 sends the operation command candidates to the operation command recognition control unit 32. Thereby, the operation command recognition control unit 32 can obtain operation command candidates corresponding to the operation command definition data having a character string (String) that matches the predicted character string candidate.

Thereafter, similar processing is performed up to the transmission of operation command candidates explained in steps S40 to S47.
S40: The character string conversion control unit 28 transmits the received conversion character string candidate to the operation command recognition control unit 32.

S41: The operation command recognition control unit 32 transmits the received conversion character string candidate to the operation command definition unit 33. The operation command definition unit 33 sends the operation command candidates to the operation command recognition control unit 32. Thereby, the operation command recognition control unit 32 can obtain operation command candidates corresponding to the operation command definition data having a character string (String) that matches the converted character string candidate.

S42: The handwriting recognition control unit 26 transmits the handwritten recognition character string candidates and the language character string candidates to the predictive conversion control unit 30.

S43: The predictive conversion control unit 30 transmits the handwritten recognition character string candidates and the received language character string candidates to the predictive conversion dictionary unit 31. The predictive conversion dictionary section 31 transmits the predicted character string candidates to the predictive conversion control section 30.

S44: The predictive conversion control unit 30 transmits the received predicted character string candidate to the operation command recognition control unit 32.

S45: The operation command recognition control unit 32 transmits the received predicted character string candidate to the operation command definition unit 33. The operation command definition unit 33 sends the operation command candidates to the operation command recognition control unit 32. Thereby, the operation command recognition control unit 32 can obtain operation command candidates corresponding to the operation command definition data having a character string (String) that matches the predicted character string candidate.

S46: The handwriting recognition control unit 26 transmits the handwritten recognition character string candidates and the received language character string candidates to the operation command recognition control unit 32.

S47: The operation command recognition control unit 32 transmits the handwritten recognition character string candidates and the received language character string candidates to the operation command definition unit 33. The operation command definition unit 33 sends the operation command candidates to the operation command recognition control unit 32. Thereby, the operation command recognition control unit 32 can obtain operation command candidates corresponding to the operation command definition data having a string that matches the language character string candidate.

S48: Next, the handwriting recognition control unit 26 transmits stroke addition to the operation command recognition control unit 32.

S49: The operation command recognition control unit 32 transmits the acquisition of position information of the confirmed object to the handwritten input storage unit 25. The handwritten input storage section 25 transmits the position information of the confirmed object to the operation command recognition control section 32.

S50: In order to determine the selected object, the operation command recognition control unit 32 uses the stroke position information received from the handwriting recognition control unit 26 due to the stroke addition in step S48 and the position information of the confirmed object received from the handwriting input storage unit 25. It is determined whether or not there is a predetermined relationship based on the straddle line determination condition 406 and the enclosing line determination condition 407, and a confirmed object that can be determined to be selected is saved as a selected object. Furthermore, in this case, since the selected object is specified, candidates for input/output operation commands are obtained from the operation command definition unit 33.

In addition, the handwriting recognition control unit 26, the character string conversion control unit 28, the predictive conversion control unit 30, and the operation command recognition control unit 32 each include a handwritten recognition character string candidate, a language character string candidate, a converted character string candidate, and a predicted character string. Data related to candidates, operation command candidates, and selected objects are held so that they can be obtained in subsequent steps S55 to S58.

S18-2: Immediately after transmitting the stroke addition to the handwriting recognition control unit 26 in step S18, the handwriting input display control unit 23 transmits a selectable candidate display timer start command to the candidate display timer control unit 24. The candidate display timer control unit 24 starts this timer.

Subsequent steps S51 to S53 are executed if the pen down occurs before a certain period of time has elapsed (before the timer times out).

S51: If the user touches the handwriting input section 21 with a pen before the timer times out, the handwriting input section 21 sends a pen down (the same event as step S2) to the handwriting input display control section 23.

S52: The handwritten input display control unit 23 transmits a stroke start (same as step S3) to the handwritten input storage unit 25. The subsequent sequence is the same as step S3 and subsequent steps.

S53: Furthermore, the handwritten input display control unit 23 transmits a selectable candidate display timer stop command to the candidate display timer control unit 24. The candidate display timer control unit 24 stops the timer. This is because a timer is no longer necessary since pen-down has been detected.

Steps S54 to S89 are executed if pen-down does not occur before a certain period of time has elapsed (before the timer times out). Therefore, the operation guide 500 shown in FIG. 17 is displayed.

S54: If the user does not touch the handwriting input unit 21 with the pen while the selectable candidate display timer is starting, the candidate display timer control unit 24 transmits a timeout to the handwriting input display control unit 23.

S55: The handwriting input display control unit 23 transmits the handwriting recognition character string/language character string candidate acquisition to the handwriting recognition control unit 26. The handwriting recognition control unit 26 transmits the currently held handwritten recognition character string/language character string candidates to the handwriting input display control unit 23 .

S56: The handwritten input display control unit 23 transmits the conversion character string candidate acquisition to the character string conversion control unit 28. The character string conversion control unit 28 transmits the currently held converted character string candidates to the handwriting input display control unit 23.

S57: The handwritten input display control unit 23 transmits the prediction character string candidate acquisition to the prediction conversion control unit 30. The predictive conversion control unit 30 transmits the currently held predicted character string candidates to the handwriting input display control unit 23.

S58: The handwritten input display control unit 23 transmits the operation command candidate acquisition to the operation command recognition control unit 32. The operation command recognition control section 32 transmits the currently held operation command candidates and selected objects to the handwriting input display control section 23 .

S59: Further, the handwriting input display control unit 23 transmits the estimated writing direction acquisition to the handwriting input storage unit 25. The handwritten input storage section 25 determines the stroke addition time, horizontal distance, and vertical distance of the handwritten object rectangular area, and transmits the estimated writing direction to the handwritten input display control section 23 .

S60: Next, the handwriting input display control section 23 specifies the pen ID received from the handwriting input section 21 and acquires the angle information of the current pen ID control data from the pen ID control data storage section 36.

S61: The handwritten input display control unit 23 acquires the handwritten signature authentication result from the handwritten signature authentication control unit 38. As a result, the user's SignatureId is obtained, and the AccountId is registered in association with the pen ID when an operation command, which will be described later, is executed.

S62: The handwriting input display control unit 23 selects these handwritten recognition character string candidates (“gi” in FIG. 17), language character string candidates (not displayed in FIG. 17, but for example “Ki”), and conversion character string candidates (“gi” in FIG. 17). ``Minutes'', ``Skills test''), predicted character string candidates (``Settle skill test'', ``Send minutes'' in Figure 17), operation command candidates (``Load minutes template'', ``Send minutes'' in Figure 17), (save in the minutes folder), each selection probability, and the estimated writing direction to create selectable candidate display data as shown in FIG. The selectable candidate display data (operation guide 500) is rotated counterclockwise, and the rotated selectable candidate display data (operation guide 500) is transmitted to the display unit 22 to be displayed.

S63: Further, the handwriting input display control unit 23 rotates the handwritten object and the selected object's rectangular area display data (rectangular frame) (handwritten object rectangular area display 503 in FIG. 17) counterclockwise using the angle information acquired in step S60. The image is rotated and transmitted to the display unit 22 for display.

S64: The handwritten input display control unit 23 transmits a selectable candidate display deletion timer start command to the candidate display timer control unit 24 in order to delete the selectable candidate display data after a certain period of time from the display. The candidate display timer control unit 24 starts this timer.

Steps S65 to S70 are performed when the user erases the selectable candidate display displayed on the display unit 22 or when a change in the handwritten object occurs (that is, when a stroke of the handwritten object is added, (deleted, moved, transformed, or split), or if no candidate is selected by the timeout.

Further, steps S65 to S67 are executed when the candidate display is deleted or when a change in the handwritten object occurs.

S65: The handwriting input unit 21 transmits the occurrence of deletion of the selectable candidate display or change of the handwritten object to the handwriting input display control unit 23.

S66: The handwritten input display control unit 23 transmits a selectable candidate deletion timer stop command. The candidate display timer control unit 24 stops the timer. This is because the handwritten object is operated within a certain period of time, so the timer becomes unnecessary.

S67: The handwriting input display control unit 23 stores the position information of the operation guide 500 in the handwriting recognition control unit 26 so that it can be used in the gesture determination of gesture handwriting recognition in step S19. The position information is, for example, the coordinates of the upper left corner and the lower right corner, or equivalent coordinates. Thereby, the handwriting recognition control unit 26 can determine whether the straight line used for inputting angle information is within the operation guide 500.

S69: The handwritten input display control unit 23 sends the selectable candidate display data deletion command to the display unit 22 to erase the display.

S70: The handwritten input display control unit 23 sends a request to erase the rectangular area display data of the handwritten object and the selected object to the display unit 22, thereby erasing the display. Therefore, if the display of the operation command candidate is deleted under a condition other than that the operation command candidate is selected, the handwritten object remains displayed.

S68: On the other hand, if the selectable candidate display does not disappear or the handwritten object changes while the selectable candidate deletion timer is starting (if the user does not perform a pen operation), the candidate display timer control unit 24 times out. is transmitted to the handwritten input display control section 23.

After the selectable candidate display deletion timer times out, the handwriting input display control unit 23 similarly executes steps S69 and S70. This is because the selectable candidate display data and the rectangular area display data of the handwritten object and the selected object may be deleted after a certain period of time has elapsed.

If the user selects a selectable candidate while the selectable candidate deletion timer is starting, steps S71 to S89 are executed.

S71: When the user selects a selectable candidate while the selectable candidate deletion timer is starting, the handwriting input unit 21 transmits the selection of character string candidates or operation command candidates to the handwriting input display control unit 23.

S71-2: The handwritten input display control unit 23 transmits a selectable candidate display deletion timer stop command to the candidate display timer control unit 24. The candidate display timer control unit 24 stops this timer.

S72: Next, the handwriting input display control section 23 transmits a held data clear command to the handwriting recognition control section 26.

S73: The handwriting recognition control unit 26 transmits a held data clear command to the character string conversion control unit 28.

S74: The handwriting recognition control unit 26 transmits the held data clear to the predictive conversion control unit 30.

S75: The handwriting recognition control unit 26 transmits a held data clear command to the operation command recognition control unit 32.

The handwriting recognition control unit 26, the character string conversion control unit 28, the predictive conversion control unit 30, and the operation command recognition control unit 32 clear the data related to the character string candidates and operation command candidates that have been held so far.

S76: Next, the handwritten input display control unit 23 sends a command to delete selectable candidate display data to the display unit 22 to erase the display.

S77: The handwritten input display control unit 23 erases the display by transmitting a message to erase the rectangular area display data of the handwritten object and the selected object to the display unit 22.

S78: The handwritten input display control unit 23 erases the display by transmitting to the display unit 22 the message for erasing the handwritten object display data and erasing the pen coordinate complementary display data transmitted in step S6. This is because a handwritten object or the like is no longer necessary because a character string candidate or an operation command candidate has been selected.

S79: The handwritten input display control unit 23 transmits handwritten object deletion to the handwritten input storage unit 25.

If a character string candidate is selected, steps S80 to S82 are executed.

S80: If a character string candidate is selected, the handwritten input display control unit 23 sends the addition of a character string object to the handwritten input storage unit 25.

S81: Furthermore, the handwritten input display control unit 23 transmits the character string object font acquisition to the handwritten input storage unit 25. The handwritten input storage section 25 selects a defined font from the estimated character size of the handwritten object and transmits it to the handwritten input display control section 23 .

S82: Next, the handwritten input display control unit 23 uses the predefined font received from the handwritten input storage unit 25 to cause the display unit 22 to transmit character string object display data to be displayed at the same position as the handwritten object. Display.

If an operation command candidate is selected, steps S83 to S88 are executed. Further, if there is a selected object, steps S83 to S85 are executed.

S83: If a candidate for the operation command for the selected object is selected (if the selected object exists), the handwritten input display control unit 23 erases the display by sending a message to erase the selected object display data to the display unit 22. . This is to temporarily erase the original selected object.

S84: Next, the handwritten input display control unit 23 transmits an instruction to execute an operation command for the selected object to the handwritten input storage unit 25. The handwritten input storage section 25 transmits the display data (edited or modified display data) of the new selected object to the handwritten input display control section 23 .

S85: Next, the handwritten input display control unit 23 transmits selected object display data to the display unit 22 to redisplay the selected object after the operation command has been executed.

S86: When “Register handwritten signature” in the operation command definition data 713, which is an input/output operation command, or “Change settings” in the operation command definition data 716 is specified, the handwritten input display control unit 23 registers the handwritten signature. The form 561 or the user-defined data change form is added to the handwritten input storage section 25.

S87: If an input/output type operation command is selected, the handwriting input display control unit 23 executes the operation command character string (Command) of the operation command definition data corresponding to the operation command selected by the user.

When the operation command 512 for signing in is executed, the handwriting input display control section 23 acquires the pen ID received by the input means communication section 37 when the operation command 512 is executed. The handwriting input display control unit 23 identifies the user-defined data having the SignatureId acquired in step S61 and acquires the AccountId from the user-defined data. Then, the AccountId is registered in the pen ID control data in association with the pen ID. As a result, the pen 2500 and the user are linked, and the handwriting input device 2 can perform processing using user-defined data.

When the operation command is executed after the user signs in, the handwriting input display control unit 23 extracts the AccountId associated with the pen ID received by the input means communication unit 37 at the time of execution of the operation command from the pen ID control data. get. The handwriting input display control unit 23 identifies user-defined data using this AccountId, sets it to % to % of the operation command, and executes it.

S88: As shown in FIG. 24, when the user presses the rotation operation button 511 of the operation header 520, the handwriting input display control unit 23 receives angle information depending on the number of presses of the rotation operation button 511. The handwriting input display control unit 23 stores the received angle information in the pen ID control data storage unit 36 in association with the pen ID received from the pen 2500 when the rotation operation button 511 is pressed.

S89: For the next handwritten object, the handwritten input display control unit 23 transmits a handwritten object start signal to the handwritten input storage unit 25. The handwritten input storage unit 25 secures a handwritten object area. After this, the processes of steps S2 to S89 are repeated.

<Summary>
As described above, the handwriting input device 2 of this embodiment allows the user to handwrite without distinguishing between inputting characters and inputting handwritten signatures, and provides various operation commands and operation commands for signing in. 512 can be called by hand by the user without distinguishing between them.

Furthermore, the handwriting input device 2 of this embodiment allows user authentication simply by the user's intuitive handwriting without using an on-screen keyboard or adding special hardware such as an IC card. It becomes possible. Since handwriting is an intuitive task, it can be expected to reduce the cost for users to learn how to operate handwriting input devices. Similarly, you can sign out by simply handwriting predetermined characters. Users can also register handwritten signature data just by handwriting.

After signing in by hand, the user's identification information (AccountId) is associated with the pen used to sign in, and user-defined data can be used to execute operation commands. User-defined data can also be changed by hand.

Furthermore, the handwriting input device 2 of this embodiment does not require the operation of selecting an operation menu and selecting an operation from a list of buttons, and can input operation commands in the same way as when handwriting characters or the like. Since the operation command and the selectable candidates 530 are displayed simultaneously on the operation guide, the user can use the handwriting input device 2 without distinguishing between inputting characters and the like and selecting the operation command. Candidates for arbitrary operation commands can be displayed by the user handwriting a handwritten object or by enclosing a confirmed object with an enclosing line. Therefore, any function (editing function, input/output function, pen function, etc.) can be called from the handwritten state. This eliminates the need for a step-by-step operation to call up a desired function by pressing a menu button, and reduces the number of operating procedures from the user's handwriting to calling up a desired function.

<<Another example 1 of the configuration of the handwriting input device>>
Although the handwriting input device 2 of this embodiment is described as having a large touch panel, the handwriting input device is not limited to having a touch panel.

FIG. 35 is a diagram showing another example of the configuration of the handwriting input device. In FIG. 35, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to a handwriting input device. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which the user writes directly with a marker. Note that the whiteboard may be a blackboard, as long as it is a flat surface large enough to project an image.

The projector 411 has an ultra-short focus optical system and can project an image with little distortion onto the whiteboard 413 from a distance of about 10 cm. This video may be transmitted from the PC 400-1 connected wirelessly or by wire, or may be stored in the projector 411.

The user writes by hand on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has, for example, a light emitting part at its tip that is turned on and emits light when the user presses it against the whiteboard 413 for handwriting. The wavelength of the light is near-infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures an image of the light emitting unit, analyzes the image, and identifies the direction of the electronic pen 2501. Further, the electronic pen 2501 emits sound waves along with light emission, and the projector 411 calculates the distance based on the arrival time of the sound waves. The position of the electronic pen 2501 can be specified based on the direction and distance. A stroke is drawn (projected) at the position of the electronic pen 2501.

Since the projector 411 projects the menu 430, when the user presses a button with the electronic pen 2501, the projector 411 identifies the pressed button based on the position of the electronic pen 2501 and the ON signal of the switch. For example, when the save button 431 is pressed, the stroke (set of coordinates) handwritten by the user is saved by the projector 411. The projector 411 stores handwritten information in a predetermined server 412, USB memory 2600, or the like. Handwritten information is saved for each page. The coordinates are saved as they are, not the image data, so the user can re-edit them. However, in this embodiment, the menu 430 does not need to be displayed because operation commands can be called by hand.

<<Another example 2 of the configuration of handwriting input device>>
FIG. 36 is a diagram showing another configuration example of the handwriting input device 2. As shown in FIG. In the example of FIG. 36, the handwriting input device 2 includes a terminal device 600, an image projection device 700A, and a pen motion detection device 810.

The terminal device 600 is connected to the image projection device 700A and the pen motion detection device 810 by wire. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

The pen motion detection device 810 communicates with the electronic pen 820 and detects the motion of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating the point indicated by the electronic pen 820 on the screen 800 and transmits it to the terminal device 600.

The terminal device 600 generates image data of a stroke image input by the electronic pen 820 based on the coordinate information received from the pen motion detection device 810. The terminal device 600 causes the image projection device 700A to draw a stroke image on the screen 800.

Further, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the stroke image input by the electronic pen 820.

<<Another example 3 of the configuration of handwriting input device>>
FIG. 37 is a diagram illustrating a configuration example of a handwriting input device. The example in FIG. 37 includes a terminal device 600, a display 800A, and a pen motion detection device 810 as a handwriting input device.

Pen motion detection device 810 is placed near display 800A. Pen motion detection device 810 detects coordinate information indicating the point indicated by electronic pen 820A on display 800A, and transmits it to terminal device 600. Note that in the example of FIG. 37, the electronic pen 820A may be charged by the terminal device 600 via the USB connector.

Terminal device 600 generates image data of a stroke image input by electronic pen 820A based on the coordinate information received from pen motion detection device 810. It is displayed on the terminal device 600 display 800A.

<<Another example 4 of the configuration of handwriting input device>>
FIG. 38 is a diagram illustrating a configuration example of a handwriting input device. In the example of FIG. 38, the handwriting input device includes a terminal device 600 and an image projection device 700A.

The terminal device 600 performs wireless communication (for example, Bluetooth: registered trademark) with the electronic pen 820B, and receives coordinate information of the point indicated by the electronic pen 820B on the screen 800. Then, the terminal device 600 generates image data of a stroke image input by the electronic pen 820B based on the received coordinate information. The terminal device 600 causes the image projection device 700A to project a stroke image.

Further, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the stroke image input by the electronic pen 820.

As described above, each of the embodiments described above can be applied in various system configurations.

In this embodiment, a handwriting input system in the form of a system in which an information processing system on a network performs processing such as handwriting recognition and returns the processing results to the handwriting input device 2 will be described.

In the explanation of this embodiment, the components with the same reference numerals or the contents of the figures in Embodiment 1 have the same functions, so the explanation of the components that have already been explained will be omitted or only the differences will be explained. There are cases where

FIG. 39 is an example of a system configuration diagram of the handwriting input system 100. The handwriting input system 100 includes a handwriting input device 2 and an information processing system 10 that can communicate via a network N.

The handwriting input device 2 is placed in a facility such as a company, and is connected to a LAN or Wi-Fi installed within the facility. The information processing system 10 is located, for example, in a data center. The handwriting input device 2 is connected to the Internet i via a firewall 8, and the information processing system 10 is also connected to the Internet i via a high-speed LAN in a data center.

The handwriting input device 2 may be connected to the Internet i using wireless communication such as a telephone network. In this case, the wireless communication is 3G (3rd Generation), 4G (4rd Generation), 5G (5rd Generation), LTE (Long Term Evolution), WiMAX (Worldwide Interoperability for Microwave Access), or the like.

The information processing system 10 includes one or more information processing devices, and the one or more information processing devices serve as a server and provide services to the handwriting input device 2. A server is a computer or software that functions to provide information and processing results in response to requests from clients. As described later, the information processing system 10 receives pen coordinates from the handwriting input device 2, and transmits information necessary for displaying the operation guide 500 shown in FIG. 17 to the handwriting input device 2.

The server side system is sometimes called a cloud system. A cloud system is a system that uses cloud computing, and cloud computing refers to a usage form in which resources on a network are used without being aware of specific hardware resources. Cloud systems are not necessarily located on the Internet. Although the information processing system is placed on the Internet in FIG. 39, it may be placed on a local network (referred to as on-premises in this case).

Further, in some embodiments, information handling system 10 includes multiple computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network, shared memory, etc., to perform the processes disclosed herein.

The configuration of the handwriting input device 2 may be the same as that in the first embodiment, but in this embodiment, it is sufficient that it has a touch panel, a display, and a communication function. Handwriting input device 2 may include multiple computing devices configured to communicate with each other.

In this embodiment, a general information processing device such as a PC or a tablet can execute a web browser or a dedicated application. A web browser or a dedicated application communicates with the information processing system 10. When the web browser operates, the user inputs or selects the URL of the information processing system 10 to connect the handwriting input device to the information processing system 10. The handwriting input device 2 executes a web application provided by the information processing system 10 using a web browser. A web application refers to software or its mechanism that is executed by a program written in a programming language (e.g. JavaScript (registered trademark)) that runs on a web browser and a program on a web server that cooperates and is executed on a web browser. .

When a dedicated application operates, it connects to the URL of the information processing system 10 registered in advance. Since the dedicated application has a program and a user interface, the program sends and receives necessary information to and from the information processing system 10 and displays it on the user interface.

The communication method may be a general-purpose communication protocol such as HTTP, HTTPs, WebSocket, or a dedicated communication protocol.

<Hardware configuration example>
The hardware configuration of the handwriting input device 2 may be the same as that shown in FIG. 5. In this embodiment, an example of the hardware configuration of the information processing system 10 will be described.

FIG. 40 is a hardware configuration diagram of the information processing system 10. As shown in FIG. 40, the information processing system 10 is constructed by a computer, and includes a CPU 601, ROM 602, RAM 603, HD 604, HDD (Hard Disk Drive) controller 605, It is equipped with a display 606, an external device connection I/F (Interface) 608, a network I/F 609, a bus line 610, a keyboard 611, a pointing device 612, a DVD-RW (Digital Versatile Disk Rewritable) drive 614, and a media I/F 616. .

Among these, the CPU 601 controls the operation of the entire information processing system 10. The ROM 602 stores programs used to drive the CPU 601 such as IPL. RAM 603 is used as a work area for CPU 601. The HD 604 stores various data such as programs. The HDD controller 605 controls reading and writing of various data to the HD 604 under the control of the CPU 601. The display 606 displays various information such as a cursor, menu, window, characters, or images. External device connection I/F 608 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like. The network I/F 609 is an interface for data communication using a communication network. The bus line 610 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 601 shown in FIG. 40.

Further, the keyboard 611 includes a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 612 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 614 controls reading and writing of various data on a DVD-RW 613, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may be DVD-R or the like. The media I/F 616 controls reading or writing (storage) of data to a recording medium 615 such as a flash memory.

<About device functions>
Next, the functions of the handwriting input system 100 will be explained using FIG. 41. FIG. 41 is an example of a functional block diagram showing the functions of the handwriting input system 100 in block form. Note that in the explanation of FIG. 41, differences from FIG. 6 will be mainly explained. Note that the function of the pen 2500 may be the same as in the first embodiment.

In this embodiment, the handwriting input device 2 includes a display section 22 , a display control section 44 , a handwriting input section 21 , and a communication section 42 . Each function that the handwriting input device 2 has is a function that is realized when any of the components shown in FIG. It is a means.

The function of the handwriting input section 21 of this embodiment may be the same as that of the first embodiment. The handwriting input unit 21 converts the user's pen input d1 into pen operation data da (pen up, pen down, or pen coordinate data) and sends it to the display control unit 44.

The display control unit 44 controls the display of the handwriting input device 2. First, the display control unit 44 performs interpolation calculation of the coordinates between the discrete values of the pen coordinate data, which are discrete values, and transmits the pen coordinate data from pen down to pen up as one stroke db to the display unit 22.

Further, the display control unit 44 transmits pen operation data dc to the communication unit 42 and obtains various display data dd from the communication unit 42. The display data includes information for displaying the operation guide 500 in FIG. 17. The display control unit 44 transmits display data de to the display unit 22.

The communication unit 42 transmits pen operation data dc to the information processing system 10, receives various display data dd from the information processing system 10, and transmits it to the display control unit 44 (an example of a first communication unit). The communication unit 42 transmits and receives data in, for example, JSON format or XML format.

The function of the display section 22 may be the same as that in the first embodiment. The display section 22 displays the stroke db and display data de. The display unit 22 converts the stroke db and display data de written into the video memory by the display control unit 44 into data according to the characteristics of the display 220, and transmits the converted data to the display 220.

<<Functions of information processing system>>
The information processing system 10 includes a communication unit 43, a handwriting input display control unit 23, a candidate display timer control unit 24, a handwriting input storage unit 25, a handwriting recognition control unit 26, a handwriting recognition dictionary unit 27, a character string conversion control unit 28, and a character string conversion control unit 28. Column conversion dictionary section 29, predictive conversion control section 30, predictive conversion dictionary section 31, operation command recognition control section 32, operation command definition section 33, pen ID control data storage section 36, handwritten signature authentication control section 38, and handwritten signature A data storage section 39 is provided. Each function of the information processing system 10 is realized when any of the components shown in FIG. or means.

The communication unit 43 receives the pen operation data dc from the handwriting input device 2 and transmits the pen operation data df to the handwriting input display control unit 23 . The communication unit 43 receives the display data dd from the handwriting input display control unit 23 and transmits it to the handwriting input device 2 (an example of a second communication unit). The communication unit 43 transmits and receives data in, for example, JSON format or XML format.

The other functions are the same as those in the first embodiment, or even if they are different, there is no problem in explaining the present embodiment.

<Operating procedure>
The operation of the handwriting input system 100 will be explained using the above configuration and FIGS. 42 to 49. 42 to 49 are sequence diagrams illustrating an example of a process in which the handwriting input device 2 displays character string candidates and operation command candidates. The process in FIG. 42 starts when the handwriting input device 2 is activated (a web browser or a dedicated application is activated) and communication with the information processing system 10 is established. Note that the overall flow of FIGS. 42 to 49 may be the same as that of FIGS. 28 to 34.

S1: When communication is established, the handwriting input display control unit 23 transmits a handwriting object start signal to the handwriting input storage unit 25 in order to secure the memory area of the handwriting input device 2. The handwritten input storage unit 25 secures a handwritten object area (memory area for storing handwritten objects). The handwritten object area may be secured after the user contacts the handwriting input section 21 with a pen.

S2a: Next, the user touches the handwriting input section 21 with a pen. The handwriting input section 21 detects pen down and transmits the detected pen down to the display control section 44 .

S2b: The display control unit 44 transmits pen down to the communication unit 42 in order to notify the information processing system 10 that the pen is down.

S2c: The communication unit 42 transmits pen down to the information processing system 10.

S2d: The communication unit 43 of the information processing system 10 receives the pen down message and transmits it to the handwriting input display control unit 23.

S3: The handwriting input display control unit 23 transmits a stroke start signal to the handwriting input storage unit 25, and the handwriting input storage unit 25 secures a stroke area.

S4a: When the user moves the pen while keeping it in contact with the handwriting input section 21, the handwriting input section 21 transmits the pen coordinates to the display control section 44.

S4b: The display control unit 44 transmits the pen coordinates to the communication unit 42 in order to notify the information processing system 10 of the pen coordinates.

S4c: The communication unit 42 transmits the pen coordinates to the information processing system 10.

S4d: The communication unit 43 of the information processing system 10 receives the pen coordinates and transmits them to the handwriting input display control unit 23.

S6: The display control unit 44 transmits pen coordinate complementary display data (data for interpolating discrete pen coordinates) to the display unit 22. The display unit 22 interpolates pen coordinates using the pen coordinate complementary display data and displays a line. The processing in step S7 is the same as in the first embodiment.

S8a: When the user releases the pen from the handwriting input section 21, the handwriting input section 21 transmits a pen-up message to the display control section 44.

S8b: The display control unit 44 transmits the pen-up to the communication unit 42 in order to notify the information processing system 10 of the pen-up.

S8c: The communication unit 42 transmits the pen up to the information processing system 10.

S8d: The communication unit 43 of the information processing system 10 receives the pen-up and transmits it to the handwriting input display control unit 23.

The subsequent steps S9 to S17 and steps S18 to S50 are the same as in the first embodiment.

S51a: If the user touches the handwriting input section 21 with a pen before the timer times out, the handwriting input section 21 sends a pen down (the same event as step S2) to the display control section 44. The processing in steps S51b to S51d may be the same as steps S2b to S2d. Furthermore, the processing in steps S52 to S61 is the same as in the first embodiment.

S62a: The handwriting input display control unit 23 creates selectable candidate display data including each character string candidate, operation command candidate, each selection probability, and estimated writing direction shown in FIG. Selectable candidate display data consisting of command candidates is transmitted to the communication unit 43.

S62b: The communication unit 43 transmits selectable candidate display data to the handwriting input device 2.

S62c: The communication unit 42 of the handwriting input device 2 receives the selectable candidate display data and transmits it to the display control unit 44.

S62d: Since the display control unit 44 has received the selectable candidate display data, it transmits it to the display unit 22 to display it.

S63a: Furthermore, the handwritten input display control unit 23 transmits rectangular area display data (rectangular frame) of the handwritten object and the selected object (handwritten object rectangular area display 503 in FIG. 17) to the communication unit 43.

S63b: The communication unit 43 transmits the rectangular area display data to the handwriting input device 2.

S63c: The communication unit 42 of the handwriting input device 2 receives the rectangular area display data and transmits it to the display control unit 44.

S63d: Since the display control unit 44 has received the rectangular area display data, it is displayed by transmitting it to the display unit 22. The process in step S64 is the same as in the first embodiment.

S65a: When the user deletes the selectable candidates or additionally writes by hand on the handwritten object, the handwriting input unit 21 transmits the occurrence of deletion of the selectable candidate display or change in the handwritten object to the display control unit 44.

S65b: The display control unit 44 sends a message to the communication unit 42 in order to notify the information processing system 10 of the occurrence of deletion of the selectable candidate display or change of the handwritten object.

S65c: The communication unit 42 transmits the occurrence of deletion of the selectable candidate display or change of the handwritten object to the information processing system 10.

S65d: The communication unit 43 of the information processing system 10 receives the occurrence of selectable candidate display deletion or change in handwritten object, and transmits it to the handwritten input display control unit 23. The processing in steps S66, S67, and S68 is the same as in the first embodiment.

S69a: The handwritten input display control unit 23 transmits the selectable candidate display data deletion to the communication unit 43.

S69b: The communication unit 43 transmits the selectable candidate display data deletion to the handwriting input device 2.

S69c: The communication unit 42 of the handwriting input device 2 receives the selectable candidate display data deletion and transmits it to the display control unit 44.

S69d: Since the display control unit 44 has received the selectable candidate display data deletion request, it transmits the message to the display unit 22 to erase the selectable candidates.

S70a: The handwritten input display control unit 23 transmits to the communication unit 43 a request to erase the rectangular area display data of the handwritten object and the selected object.

S70b: The communication unit 43 transmits to the handwriting input device 2 an instruction to erase the rectangular area display data of the handwritten object and the selected object.

S70c: The communication unit 42 of the handwriting input device 2 receives the rectangular area display data deletion of the handwritten object and the selected object, and transmits it to the display control unit 44.

S70d: Since the display control unit 44 has received the request to erase the rectangular area display data of the handwritten object and the selected object, it transmits the request to the display unit 22 to erase the rectangular area of the handwritten object and the selected object. Therefore, if the display of the operation command candidate is deleted under a condition other than that the operation command candidate is selected, the handwritten object remains displayed.

If the user selects a selectable candidate while the selectable candidate deletion timer is starting, steps S71 to S89 are executed.

S71a: When the user selects a selectable candidate while the selectable candidate deletion timer is starting, the handwriting input unit 21 transmits the selection of character string candidates or operation command candidates to the display control unit 44.

S71b: The display control unit 44 sends a message to the communication unit 42 in order to notify the information processing system 10 of the selection of character string candidates or operation command candidates.

S71c: The communication unit 42 transmits the selection of character string candidates or operation command candidates to the information processing system 10.

S71d: The communication unit 43 of the information processing system 10 receives the selection of character string candidates or operation command candidates and transmits it to the handwritten input display control unit 23. The processing in steps S72 to S75 is the same as in the first embodiment.

S76a: Next, the handwritten input display control unit 23 transmits a selectable candidate display data deletion command to the communication unit 43.

S76b: The communication unit 43 transmits the selectable candidate display data deletion to the handwriting input device 2.

S76c: The communication unit 42 of the handwriting input device 2 receives the selectable candidate display data deletion and transmits it to the display control unit 44.

S76d: Since the display control unit 44 receives the selectable candidate display data deletion command, it causes the display unit 22 to delete the selectable candidates.

S77a: The handwritten input display control unit 23 transmits to the communication unit 43 a request to erase the rectangular area display data of the handwritten object and the selected object.

S77b: The communication unit 43 transmits rectangular area display data deletion to the handwriting input device 2.

S77c: The communication unit 42 of the handwriting input device 2 receives the rectangular area display data deletion and transmits it to the display control unit 44.

S77d: Since the display control unit 44 has received the request to erase the rectangular area display data, it causes the display unit 22 to erase the rectangular area.

S78a: The handwritten input display control unit 23 transmits handwritten object display data deletion to the communication unit 43.

S78b: The communication unit 43 transmits handwritten object display data deletion to the handwritten input device 2.

S78c: The communication unit 42 of the handwriting input device 2 receives the handwritten object display data deletion and transmits it to the display control unit 44.

S78d: Since the display control unit 44 has received the handwritten object display data deletion command, it causes the display unit 22 to delete the handwritten object and the pen coordinate complementary display data. The process in step S79 may be the same as in the first embodiment.

If a character string candidate is selected, steps S80 to S82 are executed. The processing in steps S80 and S81 may be the same as in the first embodiment.

S82a: Next, the handwritten input display control unit 23 uses the defined font received from the handwritten input storage unit 25 to transmit character string object display data to be displayed at the same position as the handwritten object to the communication unit 43.

S82b: The communication unit 43 transmits the character string object display data to the handwriting input device 2.

S82c: The communication unit 42 of the handwriting input device 2 receives the character string object display data and transmits it to the display control unit 44.

S82d: Since the display control unit 44 has received the character string object display data, it causes the display unit 22 to display the character string object.

If an operation command candidate is selected, steps S83 to S87 are executed. Further, if there is a selected object, steps S83 to S85 are executed.

S83a: If a candidate for the operation command for the selected object is selected (if the selected object exists), the handwritten input display control unit 23 transmits a command to erase the selected object display data to the communication unit 43. This is to temporarily erase the original selected object.

S83b: The communication unit 43 transmits selected object display data deletion to the handwriting input device 2.

S83c: The communication unit 42 of the handwriting input device 2 receives the selected object display data deletion and transmits it to the display control unit 44.

S83d: Since the display control unit 44 receives the selected object display data deletion command, it causes the display unit 22 to delete the selected object.

S84: Next, the handwritten input display control unit 23 transmits an instruction to execute an operation command for the selected object to the handwritten input storage unit 25. The handwritten input storage section 25 transmits the display data (edited or modified display data) of the new selected object to the handwritten input display control section 23 .

S85a: Next, the handwritten input display control section 23 transmits the selected object display data to the communication section 43.

S85b: The communication unit 43 transmits the selected object display data to the handwriting input device 2.

S85c: The communication unit 42 of the handwriting input device 2 receives the selected object display data and transmits it to the display control unit 44.

S85d: Since the display control unit 44 has received the selected object display data, it causes the display unit 22 to redisplay the selected object after the operation command has been executed. The processing in steps S86 to S89 may be the same as in the first embodiment.

In this way, even in a system configuration in which the handwriting input device 2 and the information processing system 10 communicate, the same effects as in the first embodiment can be achieved. Note that the processing flows shown in FIGS. 42 to 49 are examples, and may include processing caused by communication between the handwriting input device 2 and the information processing system 10, or may be omitted. good. Further, the handwriting input device 2 may perform part of the processing performed by the information processing system 10. For example, the handwriting input device 2 may perform processing related to deletion.

<Other application examples>
Although the best mode for carrying out the present invention has been described above using examples, the present invention is not limited to these examples in any way, and various modifications can be made without departing from the gist of the present invention. and substitutions can be added.

For example, although this embodiment has been described using an electronic whiteboard as an example, any information processing device having a touch panel can be suitably applied. Information processing devices equipped with touch panels include, for example, projectors (PJs), output devices such as digital signage, HUD (head up display) devices, industrial machinery, imaging devices, sound collection devices, medical equipment, network appliances, It may be a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, a desktop PC, or the like.

Further, in this embodiment, the coordinates of the pen tip are detected by a method of detecting the coordinates of the pen tip using a touch panel, but the coordinates of the pen tip may also be detected by ultrasonic waves. Further, the pen emits ultrasonic waves as well as emits light, and the handwriting input device 2 calculates the distance based on the arrival time of the ultrasonic waves. The location of the pen can be determined by direction and distance. The projector draws (projects) the trajectory of the pen as a stroke.

Furthermore, in the present embodiment, candidates for editing and modification operation commands are displayed when there is a selected object, and candidates for input/output operation commands are displayed when there is no selected object. However, candidates for editing and modification operation commands and candidates for input/output operation commands may be displayed simultaneously.

Further, the handwriting input device 2 does not need to have the user's handwritten signature data. It may be held on the cloud or in an in-house information processing device.

Further, in the configuration example shown in FIG. 6, etc., in order to facilitate understanding of the processing by the handwriting input device 2, the system is divided according to main functions. The present invention is not limited by the method of dividing the processing units or the names thereof. The processing of the handwriting input device 2 can also be divided into more processing units depending on the processing content. Furthermore, one processing unit can be divided to include more processing.

Moreover, each function of the embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" as used herein refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function explained above. This includes devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules.

The pen ID control data storage section 36 is an example of control data storage means. The display section 22 is an example of a display means according to claim 1. The handwriting recognition control unit 26 is an example of handwriting recognition control means. The communication unit 42 is an example of receiving means. The communication unit 43 is an example of a transmitting means. The operation command recognition control section 32 is an example of operation command recognition control means. The input means communication section 37 is an example of an input means communication means. The handwritten signature authentication control unit 38 is an example of authentication control means. The handwriting input unit 21 is an example of handwriting input means. Display 220 is an example of display control means.

2 Handwriting input device 21 Handwriting input section 22 Display section 23 Handwriting input display control section 24 Candidate display timer control section 25 Handwriting input storage section 26 Handwriting recognition control section 27 Handwriting recognition dictionary section 28 Character string conversion control section 29 Character string conversion dictionary section 30 Predictive conversion control section 31 Predictive conversion dictionary section 32 Operation command recognition control section 33 Operation command definition section 36 Pen ID control data storage section 38 Handwritten signature authentication control section 39 Handwritten signature data storage section

Japanese Patent Application Publication No. 2018-026185

Claims (19)

A handwriting input device that displays handwritten stroke data based on the position of an input means in contact with a touch panel,
handwriting recognition control means for recognizing the stroke data and converting it into text data;
authentication control means for authenticating a user based on the stroke data;
Display means for displaying a display component that accepts sign-in together with the text data when the authentication control means determines that the user authentication has been successful;
A handwriting input device comprising: comprising an operation command recognition control means for recognizing an operation command executed by the handwriting input device based on the text data converted by the handwriting recognition control means;
The handwriting input device according to claim 1, wherein the display means displays the display component as the operation command. When the text data converted by the handwriting recognition control means partially matches the character string of the operation command for registering the handwritten signature data,
3. The handwriting input device according to claim 2, wherein the display means displays the operation command for registering the handwritten signature data together with the text data. When the text data converted by the handwriting recognition control means partially matches the character string of the operation command for signing out,
4. The handwriting input device according to claim 2, wherein the display means displays the operation command for signing out together with the text data. an input means communication means communicating with the input means;
control data storage means for storing control data of the input means in association with identification information of the input means;
When the press of the display component that accepts sign-in is received, the authentication control means determines that the authentication has been successful in association with the identification information of the input means received by the input means communication means when the display component is pressed. The handwriting input device according to any one of claims 2 to 4, further comprising a control means for registering identification information of a user in the control data storage means. After accepting the press of the display component that accepts sign-in,
The control means acquires identification information of the user associated in the control data storage means with the identification information of the input means received by the input means communication means,
The handwriting input device according to claim 5, wherein the operation command is executed using user-defined data regarding the user specified by the user's identification information. In the user-defined data, a user name, password, or folder file name is defined for each user,
The handwriting according to claim 6, wherein the control means sets a user name, a password, or a folder file name specified by the user's identification information to the operation command and executes the operation command. Input device. When the text data converted by the handwriting recognition control means partially matches the character string of the operation command for changing the user-defined data,
8. The handwriting input device according to claim 6, wherein the display means displays the operation command for changing the user-defined data together with the text data. When the press of the operation command for registering the handwritten signature data is accepted,
The display means displays a form for registering handwritten signature data,
6. The handwriting input device according to claim 5, wherein the authentication control means registers the handwritten signature data input into the form. When registering the handwritten signature data, the authentication control means assigns a number to identification information for identifying the handwritten signature data, assigns a number to the identification information of the user, and adds the number to the user-defined data defining information regarding the user. 10. The handwriting input device according to claim 9, wherein identification information of handwritten signature data and identification information of the user are registered in association with each other. When the press of the operation command for changing the user-defined data is accepted,
The display means displays a form for accepting changes to the user-defined data,
9. The handwriting input device according to claim 8, wherein the handwriting recognition control means changes the user-defined data using a change value input in the form. If the press of the operation command to sign out is accepted,
The control means is characterized in that the user's identification information, which is associated with the identification information of the input means received by the input means communication means when the operation command is pressed, is deleted from the control data storage means. The handwriting input device according to claim 5. 12. The handwriting input device according to claim 9, wherein the control means accepts handwritten input without distinguishing between handwritten input to a form and handwritten input other than the form. 14. The handwriting input device according to claim 2, wherein the display means displays an operation guide including the operation command at a position corresponding to the position of the stroke data. The handwriting input device according to any one of claims 2 to 13, wherein the display means displays an operation guide including the operation command at a position on the screen based on the position of the stroke data. . The handwriting input device according to claim 1, wherein the authentication control means authenticates the user based on whether the stroke data matches handwritten signature data registered in advance. A handwriting input method in which a handwriting input device displays handwritten stroke data based on the position of an input means in contact with a touch panel, the method comprising:
a step in which the handwriting recognition control means recognizes the stroke data and converts it into text data;
an authentication control means authenticating the user based on the stroke data;
a step of displaying a display component that accepts sign-in along with the text data when the authentication control means determines that the user authentication is successful;
A handwriting input method characterized by having the following. A handwriting input device that displays handwritten stroke data based on the position of the input means in contact with the touch panel,
recognizing the stroke data and converting it into text data;
authenticating a user based on the stroke data;
If it is determined that the user has been successfully authenticated in the step of authenticating, displaying a display component that accepts sign-in along with the text data;
A program characterized by executing. An input system in which a handwriting input device and an information processing system communicate,
The handwriting input device includes:
a handwriting input means for receiving stroke data using the position of a pen or a user's finger in contact with the display;
display control means for displaying the stroke data received by the handwriting input means on a display means;
receiving means for transmitting the stroke data received by the handwriting input means to the information processing system and receiving text data converted from the stroke data and a display component that accepts sign-in;
The information processing system includes:
handwriting recognition control means that recognizes the stroke data transmitted from the handwriting input device and converts it into text data;
authentication control means for authenticating a user based on the stroke data;
If the authentication control means determines that user authentication has been successful, a transmission means for transmitting a display component that accepts sign-in together with the text data to the handwriting input device;
An input system with.

Images