WO2012008469A1

WO2012008469A1 - Information input auxiliary sheet, information processing system, information input device, information input system, handwriting input sheet, and handwriting input-output system using stream dots

Info

Publication number: WO2012008469A1
Application number: PCT/JP2011/065914
Authority: WO
Inventors: 吉田　健治
Original assignee: Yoshida Kenji
Priority date: 2010-07-12
Filing date: 2011-07-12
Publication date: 2012-01-19

Abstract

The provided simple, low cost information input auxiliary sheet and information processing system exhibit extremely high input accuracy and input efficiency, can be used with any piece of equipment, and use stream dot patterns for a variety of multimedia information output and/or operation commands, said stream dot patterns being arranged as follows: a step wherein a plurality of reference dots are placed continuously and linearly following predetermined rules; a step wherein a first virtual reference line is created comprising a straight line, a polygonal line, and/or a curved line connecting the plurality of reference dots; a step wherein at least one second virtual reference line is created comprising a straight line and/or a curved line, and defined in a predetermined position from the reference dots and/or the first virtual reference line; a step wherein a plurality of virtual reference points are created in predetermined positions on the second virtual reference line; and a step wherein an information dot having XY coordinate values and/or a code value defined by the distance and direction of the dot from a virtual reference point is arranged on the end point of a vector having an initial point at the virtual reference point. Further provided are a simple, low cost information input device using the stream dot patterns to enable the user to easily input information from content where links or the like are not set in advance, and a handwriting input-output system using the stream dot pattens to enable the input of characters, symbols and the like by handwriting.

Description

[Name of invention determined by ISA based on Rule 37.2] Information input auxiliary sheet and information processing system using stream dots

(A)
The present invention relates to a sheet on which a stream dot pattern is printed, and an information processing system using the sheet.

(B)
The present invention also provides a technique for inputting information by using a medium on which a dot pattern is printed, a display on which the dot pattern is displayed, or a transparent sheet on which the dot pattern is printed, and syntactic analysis and image analysis of electronic data. About.

(C)
The present invention also relates to a handwriting input / output system and a handwriting input sheet for inputting handwritten information such as characters and figures into a computer system.

The present invention relates to an information input system and an information input auxiliary sheet for assisting input to a computer system.

(A)
Conventionally, in the operation of a personal computer, a PDA, a handy terminal, etc., a technique for inputting directly on the screen with a fingertip or a touch pen is known (for example, Patent Document 1).

In these conventional techniques, for example, the coordinate operated by the touch pen is recognized by a pressure-sensitive or optical touch panel, and the control is notified to the control means.

However, conventional touch panel technology uses expensive parts and materials, and most of them are manufactured integrally with hardware. Therefore, devices such as personal computers and ATMs equipped with touch panels are very It was expensive.

In order to solve such a problem, a part of a touch panel for detachably mounting on a display has been developed (for example, Patent Document 2).

(B)
Conventionally, for example, when processing information related to contents displayed on a print medium or a screen is processed using a device such as a personal computer, a PDA, a handy terminal, or a television receiver, the user again sets the information to be processed. I had to enter them manually.

For example, if there is a phrase “Tokyo” on the print medium and the user searches for information related to the phrase “Tokyo” with a personal computer, the search engine on the Internet is called, the phrase “Tokyo” is input from the keyboard, and the screen The search result of the phrase “Tokyo” was displayed above.

Further, as described above, in order to save the user from inputting information to be processed by an input device such as a keyboard, as shown in FIGS. 58 and 59, content displayed on a print medium or a screen is displayed. When a specific dot code is pre-defined as a mask area, a specific dot code is embedded in the mask area, and the user reads the dot pattern of the mask area with a scanner or the like, the dot code corresponding to the dot pattern is recognized, A technique for searching for and displaying information associated with dot codes in advance is known (for example, Patent Document 3).

In addition, a technique for displaying an HTML document or the like on a personal computer or the like and setting a hyperlink for the content so that the user can search for information related to the displayed content is known.

(C)
Computers are also being used in every aspect of life. In addition to conventional uses such as creating documents and performing calculations, the functions and uses of computers have increased dramatically, such as drawing pictures and drawing buildings and machines.

Conventionally, in performing these actions, it has been necessary to perform operations such as character input and drafting using a keyboard, mouse, tablet, CAD system, and the like.

However, inputting characters with a keyboard or drawing with a tablet or CAD system involves many complicated operations and takes time to learn. In particular, there is a problem that operation is difficult for an elderly person or the like who has trouble in operating the device. In addition, the “digital divide” that creates a disparity in information and opportunities between those who can operate the computer by operating keyboards and mice and those who cannot operate the computer because they cannot operate them has become a problem. ing.

A system for inputting handwritten information to a computer has been proposed as a solution to such a problem. Such a system includes a base and a drawing device. When the drawing device is moved with respect to the base, graphical information (handwritten information) is generated. The base is provided with a position encoding pattern which is a fine dot, and position information on the base is registered in this position encoding pattern. When drawing is performed on the base using the drawing device, the drawing device reads the position coding pattern. The computer system records position information on the base from the drawing device as a graphical input. As a result, the drawn content can be input to the computer system and output to the display or stored in the memory (for example, Patent Document 4).

JP 10-171600 A JP 2001-92595 A WO2007 / 089021 Special table 2003-528387

(A)
However, when a touch panel is mounted on hardware that has already been used, it is expensive and has limited models and usage conditions, and cannot be used widely and simply. Moreover, most of the touch panels that are detachably mounted on the display are pressure-sensitive, so that the response due to touch panel input is not good, and the touch panel itself becomes dirty because it touches directly with a finger. The display has a drawback of poor recognition.

Furthermore, the conventional detachable touch panel has only been proposed to be mounted on a display, and no other touch panel has been proposed for use on printed materials or media.

Therefore, an information input auxiliary sheet and an information processing system that are simple, low cost, have extremely high input accuracy and input efficiency, and do not select a mounting target are proposed.

This information input auxiliary sheet is arranged on an existing display screen, for example. When the user touches an icon or the like on the display screen with a scanner pen through the information input auxiliary sheet, information corresponding to the icon or the like is output.

Moreover, this information input auxiliary sheet is used, for example, on an existing printed material. When the user touches a character or design on the printed material with a scanner pen through the information input auxiliary sheet, information corresponding to the character or design is output from a display, a speaker, or the like.
As described above, in the case of a conventional touch panel, since the icon is touched with a finger, the icon is also provided with a large tolerance for error during input, and the input accuracy of the touch panel need not be so high.

However, in the information input auxiliary sheet that is supposed to be used together with the existing screen display or printed matter, the display of the existing icon display or printed matter does not consider touch instructions, and the display is fine. When touching a printed pattern, high input accuracy may be required.

The present invention has been made in view of these points, and provides an information input auxiliary sheet and an information processing system that are simple and low-cost, have extremely high input accuracy and input efficiency, and do not select a mounting target. And a calibration method for using the information input auxiliary sheet that can accurately associate the aspect ratio of the information input auxiliary sheet with the screen or the printed matter and realize high input accuracy by a simple method. Providing is a technical issue.

(B)
However, when information is input to a device using the above-described conventional technology, in order to input specific information (hereinafter referred to as specific information) that the user desires to input to the device, a print medium, A mask area is defined in advance on the content displayed on the screen, and a dot pattern in which a dot code corresponding to specific information is embedded in the mask area is printed or displayed, and specific information corresponding to the dot code is defined. It took time and effort to create a dot code / specific information correspondence table.

In addition, when displaying related information of specific information on a device, a dot code / address correspondence table indicating addresses where related information corresponding to dot codes exists is created in advance instead of the dot code / specific information correspondence table. It took time and effort.

In addition, in the case of content described in HTML or the like displayed on the screen, for specific information for which no hyperlink is set, the user explicitly inputs specific information to be processed by keyboard input or the like. Unless that was the case, the specific information could not be processed.

The present invention has been made in view of such a point, and a mask area is defined in advance for a specific information that is printed or displayed on a screen and for which a hyperlink or the like is not set. Information input that allows the user to input specific information without embedding a dot code corresponding to the specific information or creating a dot code / specific information correspondence table that predefines specific information corresponding to the dot code It is a technical problem to provide a device.

(C)
However, the position coding pattern used in the system of Patent Document 4 has only position information registered. Therefore, the computer system can input only the graphical information drawn by the drawing device. For example, in order to input and record various information related to the graphical information such as the page number of the booklet, An area different from the above area must be provided separately. Therefore, the handwriting input / output system becomes complicated, and there is a problem that the user needs to operate in a plurality of areas. In addition, a reference table for inputting and recording various information related to the graphical information must be provided for all position information corresponding to the graphical information drawn by the drawing apparatus. In particular, when the drawing area is not a rectangle but an area surrounded by a polygon or a curve, a reference table for all position information in the area is required.

The present invention has been made in view of such a problem, and an object of the present invention is to provide a handwritten input / output system capable of inputting characters, figures and the like by handwriting in a simple and inexpensive manner. It is another object of the present invention to provide a handwriting input / output system that can input not only information on characters and figures but also information related to them in a lump.

(A)
(1) An information input auxiliary sheet according to the present invention is disposed on a printed surface of a printed material or a display screen of a display, and is read by an optical reading device in order to output information related to the image of the printed material or the display on the display. An information input auxiliary sheet provided with a dot pattern to be arranged, and for arranging various reference dots continuously in a line according to a predetermined rule for various multimedia information output and / or operation instructions; Providing a first virtual reference line composed of a straight line, a broken line and / or a curve connecting the plurality of reference dots; and defined at a predetermined position from the reference dot and / or the first virtual reference line. Providing at least one second virtual reference line comprising a straight line and / or a curve; and a predetermined value on the second virtual reference line Information in which an XY coordinate value and / or a code value is defined by a step of providing a plurality of virtual reference points at a position and a distance and direction from the virtual reference point at an end point expressed by a vector with the virtual reference point as a starting point And a dot pattern layer formed by arranging a plurality of stream dot patterns arranged in accordance with the step of arranging dots.

According to this, only the bright and clear dot pattern reflects the infrared rays irradiated from the infrared irradiation means without affecting the characteristics such as absorption and transmission of infrared rays on the medium surface on which the information input auxiliary sheet is arranged. The dot code can be analyzed accurately. In addition, a simple and low-cost information input auxiliary sheet can be provided.

Also, by forming a plurality of stream dot patterns with a fixed reference point interval, the XY coordinate values are defined in the information input auxiliary sheet without any gaps. In addition, characters, staffs, maps, figures, etc. are printed on the information input auxiliary sheet, and when operating on the line segment by tracing or touching with a scanner pen, a stream dot pattern is formed only along that line segment. Thus, the dot pattern can be arranged rationally. In addition, the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface Thus, it becomes possible to form a dot pattern by repeating a set of constant information.

(2) Further, in the information input assistance sheet according to the present invention, the stream dot pattern is used to define the second virtual reference line and / or the direction of the dot pattern and one XY coordinate value and / or In order to define the code value, a reference dot serving as a reference is provided at a predetermined position.
According to this, by setting a new reference point, it is possible to easily define the direction of the stream dot pattern and a set of constant information without using information dots, thereby suppressing unnecessary information reduction. It is done. Furthermore, the position of the virtual reference point that is the starting point of the information dot can be accurately indicated by the arrangement of the new reference point.

(3) Further, in the information input auxiliary sheet according to the present invention, the dot pattern layer reflects an infrared ray from one side or both sides and transmits infrared light, or one side or both sides. It is characterized in that it is provided on one side or both sides of an infrared diffuse reflection layer having the characteristics of diffusely reflecting infrared rays from the side and transmitting visible light.

According to this, when the infrared reflection layer is provided, the infrared irradiation means irradiates without affecting the characteristics such as infrared absorption / transmission on the medium surface on which the information input auxiliary sheet is disposed. Infrared light can be reflected, and only bright and clear dot patterns can be photographed, and dot codes can be analyzed accurately. When the infrared diffuse reflection layer is provided, the infrared ray can be reflected and diffused in the direction in which the infrared ray is irradiated by one layer, the information input auxiliary sheet is made thinner, and the information input auxiliary sheet is manufactured. The number of processes can be reduced and the cost can be reduced.

(4) Further, in the information input auxiliary sheet according to the present invention, the information input auxiliary sheet is provided with the infrared reflection layer, and transmits visible light between the infrared reflection layer and the dot pattern layer. An infrared diffusion layer for diffusing the infrared rays is provided.

According to this, even in a photographing region where the infrared ray irradiated from the infrared reflection layer cannot be incident on the lens due to specular reflection, the infrared ray can be uniformly incident on the lens by diffusely reflecting the irradiated infrared ray. .

(5) Further, in the information input auxiliary sheet according to the present invention, the information input auxiliary sheet is provided with the infrared reflecting layer, and is provided on one side or both sides of the infrared reflecting layer and is provided on the outer surface side of the dot pattern layer. Is characterized by being provided with an infrared diffusion layer that transmits visible light and diffuses infrared light.

According to this, after providing the dot pattern layer on the upper surface of the infrared reflection layer, the infrared diffusion layer can be provided, or after providing the dot pattern layer on the back surface of the infrared diffusion layer, the infrared reflection layer can be provided. The infrared diffusion layer also serves as a simple protective layer for the dot pattern layer.

(6) Further, in the information input auxiliary sheet according to the present invention, text, illustrations, photographs, or the like are superimposed on the infrared reflection layer or the anti-dot pattern layer side of the infrared diffuse reflection layer, or the dots The pattern layer or any other layer is characterized in that text, illustrations, photographs and the like are superimposed and printed with ink made of an infrared transmitting material.

According to this, since it is possible to superimpose and print icons such as operation buttons and patterns on the dot pattern of the information input auxiliary sheet, various types of fixed multimedia information output and / or operation instructions can be expressed explicitly. It is possible to widen usage and enhance convenience.

(7) Further, the information input auxiliary sheet according to the present invention is an information input auxiliary sheet in which the dot pattern is provided on one surface side, and an adhesive layer is provided on the outer surface side where the dot pattern is read by an optical reading device. It is provided.

According to this, the information input auxiliary sheet can be easily pasted on a display or the like.

(8) Further, in the information input auxiliary sheet according to the present invention, a protective layer having a property of transmitting infrared light and visible light is provided at least on the outer surface side where the dot pattern is read by the optical reading device. Features.

According to this, it is possible to prevent the dots from being worn and soiled and to use the sheet for a long time.

(9) Further, in the information input auxiliary sheet according to the present invention, a dot pattern on the medium is read by a dot pattern reading unit, and various multimedia information output and / or operation instructions are performed. It is processed into a bookmark or sticky note.

According to this, the information input auxiliary sheet can be processed into various forms to provide an information input auxiliary sheet that is more convenient and flexible.

(10) In the information processing system according to the present invention, the information input auxiliary sheet according to any one of claims 1 to 8 is disposed on a display screen of a display of the information processing apparatus, and has at least an XY coordinate value. By reading the defined dot pattern on the information input auxiliary sheet with the dot pattern reading unit, it is possible to input in the touch panel format, and to the information processing device based on the dot pattern read from the dot pattern reading unit The auxiliary input is performed.

According to this, a function as a touch panel can be provided simply by pasting an information input auxiliary sheet on which a dot pattern is formed on the display screen of the display and touching it with a dot pattern reading means, which is simple and low cost. An information processing system can be realized.

(11) In the content related information output system using the information input assisting sheet according to the present invention, the information input assisting sheet is a display screen of a display for displaying contents expressed by text, illustrations, photographs, etc. in the information processing apparatus. The XY coordinate value and / or code value of the dot pattern on the information input auxiliary sheet is read by the dot pattern reading means and converted into xy coordinates on the format information of the content displayed on the display. By doing so, it is possible to input in a touch panel format, to recognize content specifying information at a position touched by the dot pattern reading means, and to search and output content related information related to the content specifying information. .

According to this, an information input auxiliary sheet is arranged on the display, and a text, an illustration, a photograph, etc. on the display can be specified and a detailed description thereof can be displayed on the screen simply by touching with a dot pattern reading means. Can do. For this reason, even text, illustrations, photos, addresses, etc. that are not linked on the homepage can be accessed by touching them, making it more convenient for the user. . Note that the content specifying information may instruct an operation.

(12) In the information processing system according to the present invention, the information processing device is a personal computer, PDA, television receiver, front and rear projector, game device, karaoke device, mobile phone terminal device, POS terminal device, ATM, KIOSK. A terminal, a car navigation system, a pachinko, a watch, and a smartphone, which are arranged on a display screen or a display screen as a touch panel type input device.

According to this, an information processing system using an information input auxiliary sheet can be used for any device having a display and a screen.

(13) In the information processing system according to the present invention, the information input auxiliary sheet according to any one of claims 1 to 8 is used on a printed surface of a printed matter, and at least an XY coordinate value is defined. By reading the dot pattern on the information input auxiliary sheet by the dot pattern reading means and converting the recognized XY coordinates on the printed surface and the printed matter specifying information for specifying the printed matter into xy coordinates on the printed information Recognizing print information, information related to the print information is retrieved and output.

(14) Further, an icon for inputting an index for specifying a print surface of the printed matter on the information input auxiliary sheet or another sheet by the dot pattern reading unit is printed together with the dot pattern. Features.

According to this, since information such as the page number of the printed matter can be input by the dot pattern reading means, it is not necessary to input an index for specifying the page number with a keyboard or the like, and it can be used more generally. A simple system with high convenience can be provided.

In addition, an icon means an area in which a specific meaning is recognized by a text, an illustration, or a symbol in a rectangle, circle, or other figure.

(15) In the information processing system according to the present invention, the information input auxiliary sheet is arranged on a display screen of a plurality of displays connected to one or two or more information processing apparatuses, A code value for identifying a display is printed as a dot pattern, the code value and XY coordinate values read by the dot pattern reading means are transmitted to the information processing apparatus, and the display is recognized by the code value and the touch panel The information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.

According to this, the thin client system can be provided by using the information input auxiliary sheet. Multiple displays to which video and images are distributed from the server are inexpensive, enabling information input auxiliary sheets to be attached to general TV monitors and screens that do not have a built-in CPU, enabling touch-panel input using dot pattern reading means Since a simple input / output terminal can be realized, a thin client information processing system can be constructed at low cost.

(16) In the information processing system using the information input auxiliary sheet according to the present invention, the information input auxiliary sheet is arranged on a display screen of a plurality of displays connected to one or more information processing apparatuses. The XY coordinates that uniquely identify the display are printed as a dot pattern, and the XY coordinate values read by the dot pattern reading means are transmitted to the information processing device, and the touch panel format can be input by specifying the display The information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.

According to this, since the information input auxiliary sheet on which dot patterns having different XY coordinate values are printed is arranged on each display, the display can be identified only by the XY coordinate values.

(17) In the information processing system according to the present invention, a seal on which a code value for identifying each display is printed as a dot pattern is attached to the casing of the display, and the code read by the dot pattern reading unit A value is transmitted to the information processing apparatus, which display is recognized to enable touch-panel input, and the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display. The output information is displayed on the display.

According to this, since the display is identified by the sticker, the display can be identified even if the same information input auxiliary sheet is used.

The sticker on which the dot pattern is printed may be affixed on any place such as an information input auxiliary sheet or a display housing.
(B)

(1) In order to solve the above-mentioned problem, the information input device according to the present invention is DTP data, HTML, flash, CAD data, map data, etc., and is intended for two-dimensional format information for printing on a medium. Among information visualized as text, diagrams, illustrations, photographs, etc., information corresponding to a predetermined standard is formatted in advance as specific information, and the medium is used for various multimedia information output and / or operation instructions A plurality of reference dots arranged continuously in a line according to a predetermined rule, and a first virtual reference line comprising a straight line, a broken line and / or a curve connecting the plurality of reference dots. And providing at least one second of a straight line and / or a curved line defined at a predetermined position from the reference dot and / or the first virtual reference line A virtual reference line, a step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line, and an end point represented by a vector with the virtual reference point as a starting point, from the virtual reference point A plurality of stream dot patterns arranged in accordance with the step of arranging information dots in which XY coordinate values and / or code values are defined by the distance and the direction of the above, and on the medium on which the two-dimensional format information is printed, When the user specifies the visualized specific information by a predetermined operation, the two-dimensional input and the dot pattern reading unit that is used for touching the medium and reads the dot pattern at the touch position are used. Using identification information for identifying the format information, the identification information is stored in one or more of the two-dimensional format information stored in advance. A two-dimensional format information storage / selection unit for selecting the corresponding two-dimensional format information, and a coordinate value (Xt, Yt) of the dot pattern at the touch position in the XY coordinate system. The coordinate conversion means for converting the value into (xt, yt), and the xy coordinate system of the two-dimensional format information, in order to specify the position of the specific information, determined when the specific information is laid out and printed, The coordinate value (xi, yi) in the xy coordinate system of the two-dimensional format information is used as selection position information, and the xy of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position. The distance from the coordinate value (xt, yt) in the coordinate system to one or more selection position information within a predetermined distance is calculated, and the closest coordinates of the selection position information are obtained. The specific information, is characterized in that and a specific information selecting means for selecting as input to the own device.

2D format information refers to DTP (Desk Top Publishing) data, etc., which becomes the base electronic data when creating printed matter. In addition to DTP data, content data created by HTML, flash, or the like, CAD data, or map data may be used.

Specified information is information such as text, charts, illustrations, photographs, etc. included in 2D format information, and is a unit identified by syntax analysis and image analysis.

For example, when creating a printed matter with a word processor laying out a photo of Tokyo Tower, a building, and the sky, and the text “Hana no Miyako Tokyo”, the electronic data created by the word processor is two-dimensional format information. The photo and text included in the electronic data are specific information. In addition, the image is decomposed into Tokyo Tower, Building, and Sky by image analysis, and the text is decomposed into “Hana no Miyako”, “Tokyo”, “Hana”, “Miyako”, “East”, “Kyo”. , Each becomes specific information.

The position information for selection is information used to select specific information on the printed matter according to the position touched by the user. For example, the central coordinate value of the area where the phrase “Tokyo” is laid out on the DTP data. It is.

2D format information including specific information analyzed and set with position information for selection is stored in the 2D format information storage selection means.

The identification information is information for uniquely specifying the two-dimensional format information. For example, the file name of DTP data may be used, or a predetermined document ID may be used. It may also be a URL.

In the above example, a photograph and text are printed on the medium, and a dot pattern obtained by patterning XY coordinate values and / or code values is superimposed and printed thereon.

According to the above configuration, when the user touches on the information to be input on the medium using the dot pattern reading unit, the XY coordinate value and / or code value of the touch position is read. Next, the medium touched by the user is specified from the read XY coordinate values and / or code values, and the two-dimensional format information corresponding to the medium is extracted from the two-dimensional format information storage / selection means. Next, the coordinate conversion means calculates the xy coordinate value of the touch position on the two-dimensional format information from the extracted two-dimensional format information and the XY coordinate value on the medium of the touch position. Next, the specific information selection means uses the coordinates of the touch position represented by the xy coordinate system on the two-dimensional format information and the selection position information included in each specific information in the two-dimensional format information. The specific information of the touch position is selected by a predetermined algorithm and used as an input to the information input device. Therefore, a mask area is defined in advance for specific information that is not printed with hyperlinks, etc., or a dot code corresponding to the specific information is embedded in the mask area, or specific information corresponding to the dot code. Thus, there is an effect that the user can input the specific information without creating a dot code / specific information correspondence table in advance.

Also, by forming a plurality of stream dot patterns with a constant reference point interval, the XY coordinate values are defined in the medium without any gaps. In addition, when characters, staffs, maps, figures, etc. are printed on the medium and the operation is performed by tracing or touching the line segment with a scanner pen, by forming a stream dot pattern only along the line segment, A dot pattern can be arranged rationally. In addition, the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface Thus, it becomes possible to form a dot pattern by repeating a set of constant information.

(2) The information input device according to the present invention is a DTP data, HTML, flash, CAD data, map data, etc. for solving the above problems, and is intended for two-dimensional format information to be displayed on a display. Among information visualized as text, diagrams, illustrations, photographs, etc., information corresponding to a predetermined standard is preliminarily formatted as specific information, and the display is used for various multimedia information output and / or operation instructions A plurality of reference dots arranged continuously in a line according to a predetermined rule, and a first virtual reference line comprising a straight line, a broken line and / or a curve connecting the plurality of reference dots. The step of providing and at least a straight line and / or a curved line defined at a predetermined position from the reference dot and / or the first virtual reference line Both of the step of providing one or more second virtual reference lines, the step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line, and the end point represented by a vector using the virtual reference points as starting points In addition, a plurality of stream dot patterns arranged in accordance with the step of arranging information dots in which XY coordinate values and / or code values are defined by the distance and direction from the virtual reference point are formed side by side, and the two-dimensional format information is displayed. And a dot pattern reading for reading a dot pattern at a touch position, which is used for touching the display when the user designates the specific information visualized on the display by a predetermined operation. 1 and stored in advance using the identification information for identifying the two-dimensional format information that has been input. Or two-dimensional format information storage / selection means for selecting the two-dimensional format information corresponding to the identification information from two or more of the two-dimensional format information, and coordinate values in the XY coordinate system of the dot pattern at the touch position ( Xt, Yt) is converted to coordinate values (xt, yt) in the xy coordinate system of the two-dimensional format information, and the position of the specific information is specified in the xy coordinate system of the two-dimensional format information Therefore, the coordinate value (Xt, Yt) of the touch position is used by using (xi, yi) in the xy coordinate system of the two-dimensional format information, which is determined when the specific information is displayed on the display, as the position information for selection. Yt), a distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from Yt) to one or more selection position information within a predetermined distance Calculated, the specific information having coordinates nearest the selected position information, characterized in that and a specific information selecting means for selecting as input to the own device.

2D format information refers to content data that is electronic data of content displayed on a display.

The selection position information is information used to select specific information on the display according to the position touched by the user.

The content is displayed on the display, and a dot pattern obtained by patterning the XY coordinate value and / or the code value is displayed on the display.

According to the above configuration, when the user touches on the information to be input on the display using the dot pattern reading unit, the XY coordinate value and / or code value of the touch position is read. Next, the content touched by the user is specified from the read XY coordinate value and / or code value, and the two-dimensional format information of the content is retrieved from the two-dimensional format information storage / selection means. Next, the coordinate conversion means calculates the xy coordinate value of the touch position on the two-dimensional format information from the extracted two-dimensional format information and the XY coordinate value on the display of the touch position. Next, the specific information selection means uses the coordinates of the touch position represented by the xy coordinate system on the two-dimensional format information and the selection position information included in each specific information in the two-dimensional format information. The specific information of the touch position is selected by a predetermined algorithm and used as an input to the information input device. Therefore, for specific information that is displayed on the display and for which hyperlinks are not set, a mask area is defined in advance, a dot code corresponding to the specific information is embedded in the mask area, or dot code is supported. There is an effect that the user can input the specific information without creating a dot code / specific information correspondence table in which the specific information is defined in advance.

In addition, by forming a plurality of stream dot patterns with a constant reference point interval, XY coordinate values are defined on the display without any gaps. In addition, when characters, staves, maps, figures, etc. are displayed on the display and you operate by tracing or touching the line with a scanner pen, by forming a stream dot pattern only along that line, A dot pattern can be arranged rationally. In addition, the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface Thus, it becomes possible to form a dot pattern by repeating a set of constant information.

(3) In the information input device according to the present invention, the identification information is the code value of the dot pattern printed on the medium or displayed on the display, and the XY coordinate value of the dot pattern is printed or It is preferable to print or display in at least a part of the displayed area together with the XY coordinate value of the dot pattern, or to print or display in a different area from the area.

According to the above configuration, since the identification information is the code value, the identification information is read by the user's touch operation, so that it is possible to save the user from inputting the identification information separately.

Further, by adjusting the place where the identification information is printed, there is an additional effect that the read information can be adjusted depending on the position touched by the user.

(4) In the information input device according to the present invention, the identification information is an XY coordinate value of the dot pattern that uniquely identifies the two-dimensional format information, and the XY coordinate value is an XY coordinate of the dot pattern. Preferably, the value is printed over the entire area where it is printed or displayed.

According to the above configuration, if the vast XY coordinate plane is divided and the areas used by the respective DTP data are determined in advance so that the areas do not overlap with each other, the DTP depends on which XY coordinate value is used. Data can be identified.

The identification information is an XY coordinate value of the dot pattern that uniquely identifies the two-dimensional format information, and thus there is an additional effect that a process of including the identification information as a code value in the dot pattern becomes unnecessary.

(5) In the information input device according to the present invention, the display has a structure in which a dot portion of the dot pattern absorbs or transmits infrared rays irradiated from an infrared irradiation unit included in the dot pattern reading unit. The region excluding the portion is preferably a structure that diffusely reflects the infrared rays.

According to the above configuration, the region other than the dot portion of the display has an additional effect that reflection unevenness that occurs during specular reflection can be prevented by diffusing and reflecting infrared rays.

In addition, since the dot portion has a structure that absorbs or transmits infrared rays, the dot pattern reading means can increase the contrast ratio between the dots and the other portions so that the dot pattern can be read reliably. It has the additional effect of being able to.

(6) In order to solve the above-mentioned problem, the information input device according to the present invention has a linear shape on the surface of the information input auxiliary sheet according to a predetermined rule for various multimedia information output and / or operation instructions. Arranging a plurality of reference dots in succession, providing a first virtual reference line composed of a straight line, a broken line and / or a curve connecting the plurality of reference dots, and the reference dots and / or the first Providing at least one or more second virtual reference lines made of straight lines and / or curves defined at predetermined positions from one virtual reference line, and a plurality of positions at predetermined positions on the second virtual reference line An XY coordinate value and / or code value is defined by a step of providing a virtual reference point and a distance and direction from the virtual reference point at an end point represented by a vector with the virtual reference point as a starting point. Is printed by arranging a plurality of stream dot patterns arranged according to the process of arranging information dots to be printed, such as DTP data, HTML, flash, CAD data, map data, etc., for printing on a medium or displaying on a display Of the information visualized as text, charts, illustrations, photographs, etc., information that meets a predetermined standard is pre-defined as specific information and formatted, and the two-dimensional format information is printed When the user designates the information input auxiliary sheet placed or pasted on the displayed medium or the display to be displayed and the specific information visualized on the medium or the display by a predetermined operation Used to touch the medium or the display through the information input auxiliary sheet. One or more of the two-dimensional format information stored in advance using the dot pattern reading means for reading the dot pattern at the touch position and the input identification information for identifying the two-dimensional format information. 2D format information storage / selection means for selecting the 2D format information corresponding to the identification information from among the coordinates, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position In order to specify the position of the specific information in the xy coordinate system of the two-dimensional format information, the specific information is laid out in a coordinate conversion means for converting the coordinate value (xt, yt) in the xy coordinate system of the information (Xi, yi) in the xy coordinate system of the two-dimensional format information determined when printing or when the specific information is displayed on the display 1 or within a predetermined distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position. Specific information selection means for calculating a distance to two or more selection position information and selecting the specific information having the coordinates of the closest selection position information as an input to the own device;
It is characterized by having.

The information input auxiliary sheet is a transparent sheet, and a dot pattern is provided by printing or the like. It is used by being superimposed on a medium on which content is printed or a display on which content is displayed. In order to input the specific information on the medium or the display, the user touches the desired specific information on the information input auxiliary sheet by the dot pattern reading unit.

According to the above configuration, the user inputs information for specifying the printed or displayed content as necessary. Two-dimensional format information corresponding to the content is extracted from the two-dimensional format information storage / selection means. Next, when the user touches the information to be input through the information input auxiliary sheet using the dot pattern reading unit, the XY coordinate value and / or code value of the touch position is read. Next, the coordinate conversion means calculates the xy coordinate value of the touch position on the two-dimensional format information from the extracted two-dimensional format information and the XY coordinate values on the touch position information input auxiliary sheet. Next, the specific information selection means uses the coordinates of the touch position represented by the xy coordinate system on the two-dimensional format information and the selection position information included in each specific information in the two-dimensional format information. The specific information of the touch position is selected by a predetermined algorithm and used as an input to the information input device. Therefore, even if the dot pattern is not printed directly on a printed matter such as a book, magazine, or newspaper, a mask area can be defined or masked for specific information that is printed or displayed without hyperlinks set. The user can input specific information without embedding a dot code corresponding to the specific information in the area or creating a dot code / specific information correspondence table in which the specific information corresponding to the dot code is defined in advance. There is an effect.

(7) In the information input device according to the present invention, an index for identifying the two-dimensional format information printed on the medium or displayed on the display is provided on the information input auxiliary sheet by the user. A plurality of icons for input are printed together with the dot pattern, and the two-dimensional format information storage and selection unit uses the dot pattern reading unit to read the index input by touching the icon. It is preferable to convert the identification information and select the two-dimensional format information using the converted identification information.

According to the above configuration, since the index is provided on the information input auxiliary sheet, there is a further effect that the user can easily input the identification information.

(8) In the information input device according to the present invention, the predetermined operation is performed by the user touching or touching and tracing the specific information visualized by using the dot pattern reading unit. Preferably it is an operation.

(9) In the information input device according to the present invention, the predetermined operation is performed by a user touching the visualized specific information using the dot pattern reading unit and tracing an area where the specific information is displayed. It is preferable that the operation be surrounded.

According to the above configuration, the specific information selection unit only needs to select specific information having the enclosed selection position information. Therefore, when the specific information is selected based on the coordinate value of only one point obtained by the touch operation. In comparison, there is an additional effect that the specific information desired by the user can be selected more reliably.

(10) In the information input device according to the present invention, when the specific information is one word and the characters constituting the word are separated, the position for selection is set for each character. When there is information, and the specific information is one chart, illustration, photograph, etc. and there are a plurality of objects constituting these, it is preferable that the position information for selection is included for each object.

According to the above configuration, since the specific information has position information for selection for each character at a distant position, and for each chart, illustration, and photo, the additional effect that the specific information can be reliably selected is provided. Play.

(11) In the information input device according to the present invention, the selection position information is a phrase that is specific information or a central coordinate value of each character constituting the phrase, or a chart, illustration, or photo that is specific information Or the centroid coordinate values of the objects constituting them.

According to the above configuration, there is an additional effect that the specific information can be reliably selected by performing an operation in which the user designates the center of the specific information desired.

(12) In the information input device according to the present invention, the selection position information is a phrase that is specific information, or the phrase or the character at any position in the height direction of each character constituting the phrase. It is a combination of a start point coordinate value (xi1, yi1) and an end point coordinate value (xi2, yi2) of the line segment indicating the width, and the specific information selecting means calculates the line from the coordinate value (xt, yt) of the touch position It is preferable to select the specific information whose distance to the minute is within an arbitrary distance and has the closest line segment.

According to the above configuration, since the position information for selection is represented by a line segment, there is an additional effect that the calculation of the distance from the coordinate value of the touch position is simplified.

(13) In the information input device according to the present invention, the specific information selecting means is a text within an arbitrary distance from the coordinate value (xt, yt) of the touch position in the two-dimensional format information by a predetermined algorithm. 1 and / or two or more of the specific information is recognized, and the position information for selection of the specific information is predetermined. Preferably, the specific information is selected from the coordinate values (xt, yt), the recognized specific information, and the calculated position information for selection.

According to the above configuration, since the specific information selection unit acquires the two-dimensional format information and then performs syntax analysis / image analysis on a limited range close to the touch position in the content, the syntax analysis is performed in advance. -It has the further effect of saving the trouble of preparing two-dimensional format information in which image analysis is performed and selection position information is set.

Also, compared to the case where the entire two-dimensional format information is subjected to syntax analysis / image analysis, there is an additional effect that analysis can be performed in a short time.

(14) In the information input device according to the present invention, the specific information includes text analysis and / or file search and / or image analysis such as a figure, an illustration, and a photograph included in the two-dimensional format information. The position information for selection of the specific information is calculated in advance by a predetermined algorithm, and the specific information and the position information for selection are recorded in the two-dimensional format information. Preferably it is.

According to the above-described configuration, the two-dimensional format information is analyzed in advance, the position information for selection is set, the specific information is specified, and the specific information selection unit acquires the two-dimensional format information and then parses it. Since there is no need to perform image analysis, there is a further effect that the processing time from acquiring the two-dimensional format information to selecting specific information can be shortened.

(15) In the information input device according to the present invention, the two-dimensional format information includes text and / or a figure having a point, a line segment, a plane, and a three-dimensional object as the specific information drawn by CAD. It is preferable that

According to the configuration described above, there is an additional effect that content created by CAD, such as a CAD drawing, can also be used for input processing.

(16) In the information input device according to the present invention, the two-dimensional format information is map data, and the map data has specific information classified into categories such as text, road, feature, intersection, and the like. The specific information preferably includes a region surrounded by a road, a feature region, a road center line, and a feature centroid as selection position information.

According to the above configuration, there is a further effect that the map can also be used for input processing.

(17) An information input system according to the present invention comprises the information input device according to claim 1, a two-dimensional format information server, and a network connecting them in order to solve the problem. The information input device transmits the input identification information for identifying the two-dimensional format information to the two-dimensional format information server, instead of the two-dimensional format information storage and selection unit. Communication means for receiving two-dimensional format information corresponding to the identification information from the two-dimensional format information server, the two-dimensional format information server using the identification information received from the information input device, Two-dimensional format information storage selection means for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance, and the identification information Received from the serial data input device, and the selected the two-dimensional format information characterized by comprising: a communication unit for transmitting to the information input device.

The 2D format information server may be a server that provides 2D format information including specific information that is analyzed in advance and set with position information for selection, or may be a normal Web server. In addition, when using a normal Web server, since the position information for selection and specific information are not set in the two-dimensional format information acquired from the Web server, the specific information selecting unit acquires the two-dimensional format information, Analysis is performed so as to set position information for selection and specific information.

According to the above configuration, the information input device has a client / server configuration in which the two-dimensional format information corresponding to the identification information is acquired from the two-dimensional format information server on the network. There is an effect that the two-dimensional format information server can be shared.

Also, there is an effect that an existing Web server or the like can be used as a two-dimensional format information server.

(18) An information input system according to the present invention comprises an information input device according to claim 2, a two-dimensional format information server, and a network connecting them in order to solve the problem. The information input device transmits the input identification information for identifying the two-dimensional format information to the two-dimensional format information server, instead of the two-dimensional format information storage and selection unit. Communication means for receiving two-dimensional format information corresponding to the identification information from the two-dimensional format information server, the two-dimensional format information server using the identification information received from the information input device, Two-dimensional format information storage selection means for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance, and the identification information Received from the serial data input device, and the two-dimensional format information selected by comprising the communicating means for transmitting to said information input device.

(19) An information input system according to the present invention includes an information input device according to claim 1, a two-dimensional format information server, and a network connecting them in order to solve the problem. The information input device transmits the input identification information for identifying the two-dimensional format information to the two-dimensional format information server, instead of the two-dimensional format information storage and selection unit. Communication means for receiving two-dimensional format information corresponding to the identification information from the two-dimensional format information server, the two-dimensional format information server using the identification information received from the information input device, Two-dimensional format information storage selection means for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance, and the identification information Received from the serial data input device, and the two-dimensional format information selected by comprising the communicating means for transmitting to said information input device.

(20) In the information input system according to the present invention, the stream dot defines a second virtual reference line and / or defines the direction of the dot pattern and one XY coordinate value and / or code value. Therefore, it is preferable to provide a reference dot serving as a reference at a predetermined position.

According to this, by setting a new reference point, it is possible to easily define the direction of the stream dot pattern and a set of constant information without using information dots, thereby suppressing unnecessary information reduction. It is done. Furthermore, the position of the virtual reference point that is the starting point of the information dot can be accurately indicated by the arrangement of the new reference point.

(21) In the information input system according to the present invention, it is preferable that a part of the code value includes at least a code value for specifying the medium, the display, or the information input auxiliary sheet.
According to the above configuration, since the identification information is the code value, it is possible to recognize which medium or the display or the information input auxiliary sheet is being used by the user's touch operation. There is a further effect that it is possible to save the trouble of inputting information for identifying the.

(C)
<Handwritten input sheet>
(1) A handwriting input sheet according to the present invention includes a step of arranging a plurality of reference dots continuously in a line according to a predetermined rule in at least a part or a plurality of arbitrarily shaped regions; Providing a first virtual reference line composed of a straight line, a polygonal line and / or a curve connecting the reference dots, and a straight line and / or defined at a predetermined position from the reference dot and / or the first virtual reference line Alternatively, a step of providing at least one or more second virtual reference lines made of a curve, a step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line, and a vector starting from the virtual reference points Stream dots arranged in accordance with the process of arranging XY coordinate values or information dots in which XY coordinate values and code values are defined by the distance and direction from the virtual reference point at the represented end points. Characterized by comprising the medium having a plurality side by side writing area on which a pattern has been formed.

According to this, by forming a plurality of stream dot patterns having a constant reference point interval, the XY coordinate values are defined without any gaps in the writing area, and trajectory information can be generated, enabling handwritten input. In addition, when characters, staffs, maps, figures, etc. are printed on a handwritten input sheet, and a user scans or touches the line segment with a scanner pen, a stream dot pattern is formed only along that line segment. Thus, a dot pattern can be arranged rationally. In addition, when a dot pattern in which XY coordinates are defined is formed as a two-dimensional code (used as an index), it is not restricted by the shape of the rectangular area, and is matched to the information area visually formed on the medium surface. In addition, it is possible to form a dot pattern by repeating a set of constant information in a free shape.

(2) The stream dot is a reference that is further used as a reference in order to define a second virtual reference line and / or to define the direction of the dot pattern and one XY coordinate value and / or code value. You may provide a dot in a predetermined position.

According to this, by providing a new reference point, the direction of the stream dot pattern and the set of constant information can be easily defined without using information dots, and the reduction of unnecessary information can be suppressed. Furthermore, the position of the virtual reference point that is the starting point of the information dot can be accurately indicated by the arrangement of the new reference point.
<Handwriting input / output system>

(3) A handwriting input / output system according to the present invention is a handwriting input / output system using the handwriting input sheet according to

claim

1 or 2, wherein the stream dot pattern formed in the writing area is imaged, From the coordinate information, the coordinate information, and the code information by storing the image data of the stream dot pattern captured and transmitted by the imaging unit in the storage element and analyzing the code of the image data Dot pattern analysis means for obtaining the trajectory information, and processing command means for transmitting the trajectory information to the information processing means.
According to this, since the handwriting input / output system according to the present invention can define not only the coordinate information but also the code information in the dot pattern formed on the medium, the processing instruction means sends the information to the information processing unit. Thus, the trajectory information and processing instructions based on the trajectory information can be transmitted in a batch.

(4) The handwriting input / output system further includes a trajectory recognizing unit for recognizing the trajectory information of the characters and / or figures traced on the writing area surface based on the change of the analyzed coordinate information. It is preferable that the processing command means for transmitting a processing command based on the recognized information to the information processing means.

According to this, the handwriting input / output system according to the present invention can leave a locus in the vicinity of the imaging means on the writing area surface.

(5) In the medium, information specifying the medium is defined in at least a part of the code value, and / or specifying information specifying the writing area is included in at least a part of the code value. Operation instructions (program start, command input, data input / output / search, search target specification, etc.) meaning write processing are defined in at least a part of the code value. Specific information identifying a predetermined medium surface and / or the writing area surface in at least a part of the code value of the stream dot pattern formed in and / or around the writing area. And / or the operation instruction is defined and an icon representing the operation instruction is superimposed and printed, and / or the document The stream dot pattern formed in and / or around the print area is overlaid with the associated text, graphics, or photo, and / or braille in and / or around the writing area And / or a narrow convex guide along the outer periphery of the writing area so that the imaging means for imaging the dot pattern does not protrude when writing to the writing area. And / or the above-mentioned stream dot pattern, which is a sticker, sticky note, or other medium that can be applied and / or used on a monitor surface or a printed medium on which a writing area is displayed. It may be a transparent information input auxiliary sheet on which is formed.

According to the above feature, the handwriting input / output system according to the present invention can easily know what the user should write in the position of the writing area and the writing area.

Also, the user can easily know what meaning the code information has (what processing instruction is transmitted) by looking at the icon.

Moreover, it contributes to maintaining the same input efficiency as a healthy person when a visually handicapped person performs handwriting input.

In addition, a guide is provided so that the writing area can be accurately traced and drawn by the imaging means, which contributes to maintaining the same input efficiency as that of a healthy person when a visually handicapped person performs handwriting input.

Also, the flexibility of the layout of the writing area is further increased, and the search and output of information input by handwriting in the writing area can be freely executed according to the convenience and preference of the user.

Also, a touch panel with good handling can be provided. Alternatively, handwriting input is possible by placing the grid sheet on the medium surface without overlapping the grid sheet on the monitor surface.

(6) The medium forms a dot pattern in which code information for specifying the first medium surface and / or the writing area surface is repeatedly defined with the medium provided with the writing area as the first medium. The code information formed on the second medium is read by the imaging unit, and is printed on the print information on the first medium surface and / or the writing area surface. The trajectory information and / or information related to the trajectory information may be output.

According to the above feature, the handwriting input / output system according to the present invention prints code information for specifying a medium surface on another medium (such as a notebook) and touches it with an imaging unit, thereby handwriting input on the medium surface. Can be searched and output. In addition, it is possible to take a wide space in the writing area of the medium.

(7) The imaging means further includes an instruction protrusion and / or at least one pen tip, and characters and / or traced on the writing area surface with the instruction protrusion and / or at least one pen tip. When tracing a figure, the dot pattern on the locus in the vicinity of the pointing projection is imaged, and when the pointing projection and at least one pen tip are provided, the pen tip is stored in the pointing projection It is preferable that the pointing projection further includes a storage space for controlling and a button for controlling storage / removal of the pen tip.

According to the above feature, the handwriting input / output system according to the present invention makes it easy to visually recognize the position of the imaging means on the writing area surface.

Also, a locus near the imaging means can be left on the writing area surface.

In addition, it is possible to selectively use the case where it is desired to leave the locus near the imaging means drawn on the writing area surface and the case where it is not desired to leave it.

(8) The imaging means further includes an erasing means for erasing characters and / or figures drawn on the writing area surface on the instruction protrusion, and an area traced by the erasing means is formed on the writing area surface. At the same time as erasing, the dot pattern on the locus in the vicinity of the erasing means is imaged, and the processing instruction means outputs a processing instruction for erasing the locus information of the character and / or figure and the locus information. May be sent to.

According to the above feature, the handwriting input / output system according to the present invention can cope with not only writing information input on the writing area surface and the information processing means but also erasing the writing information.

(9) The trajectory information includes coordinate information of the imaging center of the imaging unit, or coordinate information of a position where the pointing protrusion or the pen tip provided near the imaging unit touches the writing area surface and the code Preferably, the information is obtained by interpolation calculation based on coordinate information defined in the dot pattern by the dot pattern analysis means.

According to the above feature, the handwriting input / output system according to the present invention matches the actual trajectory by correcting the trajectory information even when the trajectory information recognized by the trajectory recognition unit is different from the trajectory on the medium. Or it can be approximated to such an extent that there is no trouble in practical use.

(10) The trajectory recognition unit further includes a timing unit, and the timing unit measures a time when the imaging unit touches the writing area surface and a time away from the writing area surface, and the imaging unit touches the writing area surface. It is also possible to record the order of the traces drawn and the touching time, and transmit them to the information processing means.

According to the above feature, the handwriting input / output system according to the present invention grasps the time required for the handwriting input by the information processing means and compares it with the normal time, so that the character's personality, age, gender, etc. Various research is possible.

(11) The processing command means may specify, based on the code information and / or the recognition information, specific information and / or an operation instruction (program activation, command input, data input / output / search, It may be possible to uniquely input a search target designation or the like.

According to the above feature, in the handwriting input / output system according to the present invention, the processing command means can give a unique instruction to the information processing means based on the code information and / or the recognition information. This is the most important function in the present invention, and it is possible to easily specify which writing area is always written or what operation instruction is given to which writing area.

(12) The trajectory recognition unit may further recognize the trajectory information with high accuracy by referring to the character and / or graphic information patterns classified in advance regarding the contents to be written in the writing area based on the code information. preferable.

According to the above feature, the handwriting input / output system according to the present invention refers to the character and / or graphic information pattern table previously classified based on the code information when recognizing the trajectory information. It can be recognized with accuracy.

(13) After recognizing the trajectory information, the trajectory recognizing means further determines the character and / or the figure, or a phrase composed of one or more of the characters, based on the change in the analyzed coordinate information. The semantic information is converted into a semantic information code by referring to a predefined semantic information code table (dictionary) corresponding to the code information, and the processing instruction means is based on the semantic information code and the semantic information. Preferably, the processing instruction is transmitted to the information processing means.

According to the above feature, the handwriting input / output system according to the present invention recognizes the trajectory information with high accuracy by referring to the semantic information code table at the time of trajectory recognition, and further converts it into a semantic information code. A processing command can be transmitted as that meaning.

(14) The processing instruction means may be a first operation for inputting the recognition result of the trajectory information by the trajectory recognition means as a document and / or a graphic, or a second operation for inputting as a comment, or a document and The information processing unit may be instructed as a third operation for editing the document and / or the figure and / or the comment based on the edit symbol and / or the figure and / or the comment that edits the comment. .

According to the above feature, the handwriting input / output system according to the present invention can write a document or draw a picture by handwriting input.

Also, comment documents can be input in addition to documents and pictures input by handwriting input.

Also, documents and images (including those not based on the handwriting input / output system) can be edited by handwriting input.

(15) In the handwriting input / output system according to the present invention, at least the imaging optical axis of the imaging means that is in contact with the medium surface is tilted and tilted by being tilted with respect to a vertical line of the medium surface. And an angle measuring means for measuring the inclination and / or an action of tilting the imaging means with reference to the inclination and the direction measured when the imaging means is in contact with the medium surface, that is, a grid tilt action. Or an angle change recognizing means for analyzing a predetermined change in the direction, wherein the processing instruction means may transmit a predetermined processing instruction to the information processing means based on the result of the analysis. Good.

According to the above feature, the handwriting input / output system according to the present invention can give an operation instruction by the grid tilt operation of the imaging means.

(16) In the medium, the dot pattern arranged at a predetermined angle with respect to the direction of the writing area is formed in the writing area, and the angle measuring unit and the angle change recognizing unit are configured by the imaging unit. Based on the direction of the dot pattern read when contacting the surface of the writing area, a predetermined change in the direction of tilting and / or the inclination with respect to the direction of the writing area may be analyzed.

According to the above feature, the handwriting input / output system according to the present invention can recognize the inclination and the change thereof with reference to the direction of the dot pattern in addition to the change of the angle.

(17) The handwriting input / output system is configured such that the imaging optical axis of the imaging unit is substantially vertical with respect to the writing area surface, and the direction of the dot pattern read when the writing area surface contacts the writing area surface. A rotation angle reading means for reading a rotation angle formed by the angle of the imaging means, and an operation for rotating the imaging means with reference to the rotation angle measured when the imaging means comes into contact with the medium surface. In other words, rotation angle change recognizing means for analyzing a predetermined change in the rotation angle by rotating the imaging optical axis in a predetermined rotation direction by a grid turn operation, and the processing command means includes: Based on the result of the analysis, a predetermined processing command may be transmitted to the information processing means.

According to the above feature, the handwriting input / output system according to the present invention can give an instruction by a grid turn operation.

(18) The handwriting input / output system further captures the writing area by the imaging means, obtains coordinate information and code information by the dot pattern analysis means, and then performs the writing area by the imaging means by a predetermined operation. Referring to a voice recording means for recording voice instead of tracing or drawing, and a predefined voice recognition information table (dictionary) corresponding to the code information, the recorded voice is recognized and converted into character information. Voice recognition means for converting and transmitting to the information processing means may be further provided.

According to the above feature, the handwriting input / output system according to the present invention can perform voice input instead of handwriting input.

(A)
According to the present invention, a device equipped with any display means, such as a general-purpose computer, PDA, or mobile phone, can be used as a touch panel type input means very easily.

In addition, it can be attached to printed materials such as books and catalogs, and any media, and without printing any code on the media itself, the text, illustrations, It can output multimedia information such as photos, sounds, and moving pictures, and can operate devices.

(B)
As described above, the information input apparatus according to the present invention is capable of outputting various multimedia information and / or operation instructions by formatting the specific information visualized as text, diagrams, illustrations, photographs, and the like. Therefore, when the user designates the specific information visualized on the printed medium together with the dot pattern obtained by patterning the XY coordinate value and / or the code value by a predetermined algorithm by a predetermined operation, the medium 1 or 2 stored in advance by using dot pattern reading means for reading the dot pattern at the touch position, and input identification information for identifying the two-dimensional format information. A 2D document for selecting the 2D format information corresponding to the identification information from the 2D format information. Information storage selection means and coordinates for converting the coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position to the coordinate value (xt, yt) in the xy coordinate system of the selected two-dimensional format information In the xy coordinate system of the conversion means and the two-dimensional format information, in order to specify the position of the specific information, the position information for selection included in the specific information is used, and the coordinate value of the dot pattern of the touch position From the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from (Xt, Yt), the specific information having the coordinates of the position information for selection that is within an arbitrary distance and is closest And a specific information selecting means for selecting as an input to the own apparatus.

Further, in order to solve the above-described problem, the information input device according to the present invention can output various multimedia information and two-dimensional format information obtained by formatting specific information visualized as text, diagrams, illustrations, photographs, and the like. In order to instruct the operation, the user displays a display to be displayed together with a dot pattern obtained by patterning XY coordinate values and / or code values by a predetermined algorithm, and the specific information visualized on the display by a user. When specifying by operation, using the dot pattern reading means for reading the dot pattern at the touch position, which is used for touching the display, and the input identification information for identifying the two-dimensional format information From one or more of the two-dimensional format information stored in advance Two-dimensional format information storage / selection means for selecting the two-dimensional format information corresponding to the identification information, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position are selected as the selected two-dimensional format information. In the xy coordinate system, the coordinate information is converted into coordinate values (xt, yt), and the specific information has the specific information in order to specify the position of the specific information in the xy coordinate system of the two-dimensional format information. Within the arbitrary distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position using the position information for selection. Specific information selection means for selecting specific information having the coordinates of the closest position information for selection as input to the own apparatus is provided.

In addition, as described above, the information input device according to the present invention prints a dot pattern in which XY coordinate values and / or code values are patterned by a predetermined algorithm for various multimedia information output and / or operation instructions. An information input auxiliary sheet in which two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted is placed or pasted on a printed medium or a display to be displayed; When the user designates the specific information visualized on the medium or the display by a predetermined operation, it is used to touch the medium or the display through the information input auxiliary sheet. , Dot pattern reading means to read the dot pattern at the touch position, and input Using the identification information for identifying the two-dimensional format information, the two-dimensional format information corresponding to the identification information is selected from one or more of the two-dimensional format information stored in advance. And a coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position, and a coordinate value (xt, yt) in the xy coordinate system of the selected two-dimensional format information. In the xy coordinate system of the two-dimensional format information and the coordinate conversion means for converting into the two-dimensional format information, the position information for selection included in the specific information is used to specify the position of the specific information, and the dot of the touch position From the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the pattern, the closest selection position information within the arbitrary distance. Specific information having mark, is characterized in that a specific information selecting means for selecting as input to the own device.

In addition, as described above, the information input method according to the present invention can output various multimedia information and / or operations using two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted. For instruction, when the user designates the specific information visualized on the printed medium together with the dot pattern obtained by patterning the XY coordinate value and / or the code value by a predetermined algorithm by a predetermined operation, In order to touch on the medium, using a dot pattern reading unit, a dot pattern reading step for reading a dot pattern at a touch position, and input identification information for identifying the two-dimensional format information, Corresponds to the identification information from one or more of the two-dimensional format information stored in advance A two-dimensional format information storing and selecting step for selecting the two-dimensional format information, and a coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position, and the xy coordinate system of the selected two-dimensional format information In the coordinate conversion step for converting to the coordinate value (xt, yt) in, and the xy coordinate system of the two-dimensional format information, the position information for selection possessed by the specific information for specifying the position of the specific information The coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position is within an arbitrary distance and closest. And a specific information selection step of selecting specific information having the coordinates of the position information for selection as an input to the own apparatus.

In addition, as described above, the information input method according to the present invention can output various multimedia information and / or operations using two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted. For instruction, when the user designates the specific information visualized on the displayed display together with the dot pattern obtained by patterning the XY coordinate value and / or the code value by a predetermined algorithm by a predetermined operation, In order to touch on the display, using a dot pattern reading unit, a dot pattern reading step for reading a dot pattern at a touch position, and input identification information for identifying the two-dimensional format information, From one or more of the two-dimensional format information stored in advance A two-dimensional format information storage selection step for selecting the two-dimensional format information corresponding to the identification information, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position are selected as the selected two-dimensional format information. In the coordinate transformation step for transforming into coordinate values (xt, yt) in the xy coordinate system, and in the xy coordinate system of the two-dimensional format information, the specific information includes the specific information. Within the arbitrary distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position using the position information for selection. And a specific information selection step of selecting specific information having the coordinates of the closest position information for selection as an input to the own apparatus.

Further, as described above, the information input method according to the present invention prints a dot pattern in which XY coordinate values and / or code values are patterned by a predetermined algorithm for various multimedia information output and / or operation instructions. An information input auxiliary sheet in which two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted is placed or pasted on a printed medium or a display to be displayed; When the user designates the specific information visualized on the medium or the display by a predetermined operation, it is used to touch the medium or the display through the information input auxiliary sheet. , Using dot pattern reading means to read the dot pattern at the touch position The dot pattern reading step for reading the dot pattern at the touch position and the input identification information for identifying the two-dimensional format information are used to store one or more of the two-dimensional format information stored in advance. The two-dimensional format information storage and selection step for selecting the two-dimensional format information corresponding to the identification information from the inside, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position are selected. In order to specify the position of the specific information in the coordinate conversion step for converting into coordinate values (xt, yt) in the xy coordinate system of the dimensional format information and the xy coordinate system of the two-dimensional format information, the specific information is present. The xy coordinates of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position using the selected position information for selection. A specific information selection step of selecting, as an input to the own apparatus, specific information having coordinates of the position information for selection that is within an arbitrary distance and is closest from a coordinate value (xt, yt) in the system. It is characterized by.

Therefore, a mask area is defined in advance for the specific information that is printed / displayed and no hyperlinks are set, or a dot code corresponding to the specific information is embedded in the mask area. There is an effect that the user can input the specific information without creating a dot code / specific information correspondence table in which the specific information is defined in advance.

(C)
The present invention provides a handwriting input / output system that has various features described above and can give a user various and new added value and convenience that are completely different from those of the prior art. Can do.

It is a figure (1) explaining the stream dot pattern used by this invention. It is a figure (2) explaining the stream dot pattern used by this invention. It is a figure (3) explaining the stream dot pattern used by this invention. It is FIG. (4) explaining the stream dot pattern used by this invention. It is a figure (5) explaining the stream dot pattern used by this invention. It is a figure (6) explaining the stream dot pattern used by this invention. It is the figure which used the grid sheet (information input auxiliary sheet) concerning the present invention for the general-purpose computer system which is an information processor. It is sectional drawing (1) explaining a diffuse reflection. It is sectional drawing (2) explaining a diffuse reflection. It is sectional drawing which showed the various structures of the grid sheet. It is sectional drawing which showed the structure of the grid sheet | seat which provided the dot pattern layer on both surfaces. It is the figure which showed the specific example which used the grid sheet | seat which has a dot pattern layer only on one side for printed matter. It is the front view and sectional drawing of a grid sheet used in FIG. It is the figure which showed the specific example which used the grid sheet | seat which has a dot pattern layer on both surfaces for printed matter. It is the front view and sectional drawing of a grid sheet used in FIG. It is the figure which showed the system for a user to touch a grid sheet and to display information on a display. It is the figure which showed the other form of the system for a user to touch a sheet | seat and to display information on a display. It is explanatory drawing shown about the Example which sticks a grid sheet | seat on a display, recognizes the content specific information touched with the scanner, and searches and outputs related information automatically. It is sectional drawing of the grid sheet stuck and used for a display. It is the figure shown about the case where a grid sheet is used for a projector system. It is the figure shown about the case where a grid sheet is used for a rear projector system. (A) is the figure shown about the case where a grid sheet is used for a portable terminal, (b) is the figure shown about the case where a grid sheet is used for a car navigation system. It is a block diagram of a thin client system using a grid sheet. It is the figure which showed an example of the display and sticker which are used for a thin client system. It is the figure explaining the grid sheet used for the calibration method and calibration with respect to a display. It is the figure explaining the other Example of the calibration method with respect to a display. It is the figure explaining the grid sheet used for the calibration method with respect to printed matter, and calibration. It is the figure explaining the other Example of the calibration method with respect to printed matter. BRIEF DESCRIPTION OF THE DRAWINGS The outline | summary of the 1st Embodiment of this invention is shown, FIG. 1 (a) is a figure which shows the mode of operation, FIG.1 (b) is a figure which shows a mode that specific information is selected. . FIG. 8A shows an example of a dot code format. FIG. 8A shows an example in which the dot code includes an XY coordinate value, a code value, and a parity. FIG. 8B shows a dot code 103 depending on where the dot pattern 103 is provided. FIG. 18C shows an example in which the format is changed, and FIG. 18C shows an example in which the dot code includes XY coordinate values and parity. FIG. 31A is a cross-sectional view of the printed material 101a, and FIG. 31A is a diagram illustrating a state in which infrared rays are diffused and reflected on a portion of the printing medium 101c where nothing is printed, and FIG. FIG. 31C is a diagram showing a state where the infrared rays hit the dot pattern 103 and is absorbed. FIG. 2 is a diagram illustrating a structure of a portion of a scanner 104 that reads a dot pattern 103 and how the dot pattern 103 on a printed material 101a is read. It is a functional block diagram of the information input device 105 which concerns on 1st this embodiment. It is a figure which shows the example in the case of using the center coordinate value or centroid coordinate value of an object as position information for selection, FIG.34 (a) is a case where specific information is a text, FIG.34 (b) The specific information is a chart, and FIG. 34 (c) is a case where the specific information is an illustration / photo. It is a figure which shows the example of the other position information for selection, Fig.35 (a) is an example which uses the starting point coordinate and height of a word as position information for selection, FIG.35 (b) is the starting point coordinate of a word FIG. 35C shows an example in which the start point coordinates, the end point coordinates, and the height of each character are selected when the characters constituting the word are separated. This is an example of information. It is a figure which shows the example of operation which touches one point of the specific information which a user wants to designate. It is a figure which shows the example of operation which traces on the specific information which a user wants to designate. It is a figure which shows the example of operation which traces the circumference | surroundings of the specific information which a user wants to designate. FIG. 39A is a diagram illustrating an example of a method for selecting specific information, and FIG. 39A illustrates a case where a perpendicular foot drawn from a touch position with respect to a line segment does not intersect the line segment, and FIG. This is a case where a perpendicular foot drawn from the touch position with respect to the line segment intersects the line segment. FIG. 40A is a diagram illustrating an example of a method for selecting specific information. FIG. 40A illustrates a case where the touch position is within the mask area 102b, and FIG. 40B illustrates that the touch position is within the mask area 102b. This is also the case where it does not enter the mask region 102b ′. It is a figure which shows the example of the method of selecting specific information. 10 is a flowchart of a procedure of information input processing performed by the information input device 105. It is a flowchart explaining in detail the selection process of specific information performed in process S40. It is a functional block diagram of the information input device 105a which concerns on 2nd Embodiment. It is a functional block diagram of information processor 112 concerning a 3rd embodiment. 10 is a flowchart for describing a procedure of information processing performed by the information processing apparatus 112. It is a figure which shows the mode of the information processing which information processing apparatus 112 performs. It is a figure which illustrates the outline | summary of 4th Embodiment. FIG. 49A is a cross-sectional view of an organic EL display provided with a dot portion, FIG. 49A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed, and FIG. It is a figure which shows a mode that infrared rays inject into other light emitting elements, and are diffusely reflected. FIG. 50A is a diagram illustrating an example in which a dot portion is provided on a screen of a front projector or a rear projector, and FIG. 50A is a diagram illustrating a state in which infrared light incident on the dot portion is transmitted or absorbed, and FIG. ) Is a diagram showing a state in which infrared rays are incident on a screen portion other than the dot portion and diffusely reflected. It is a functional block diagram of information processor 112b concerning a 4th embodiment. It is a flowchart explaining the procedure of the information processing which the information processing apparatus 112b performs. It is a figure which shows the outline | summary of 5th Embodiment. It is a figure which shows the outline | summary of 6th Embodiment. It is a figure which shows the outline | summary of 7th Embodiment. It is a functional block diagram of the television receiver 112e which concerns on 7th Embodiment. It is a flowchart explaining the procedure of the information processing which the television receiver 112e performs. It is a figure which shows a prior art. It is a figure which shows a prior art. It is an external view which shows the use condition of the handwriting input / output system concerning this invention. It is a block diagram which shows the structure of the 1st Example of the handwriting input / output system concerning this invention. It is a block diagram which shows the structure of the 2nd Example of the handwriting input / output system concerning this invention. It is a block diagram which shows the structure of the 3rd Example of the handwriting input / output system concerning this invention. It is a block diagram which shows the structure of the 4th Example of the handwriting input / output system concerning this invention. It is a block diagram which shows the structure of the 5th Example of the handwriting input / output system concerning this invention. It is a block diagram which shows the structure of the 6th Example of the handwriting input / output system concerning this invention. It is a figure explaining the format of the stream dot pattern used for the handwriting input / output system concerning this invention. It is an external view which shows the whole structure of the imaging part used for the handwriting input / output system concerning this invention. It is sectional drawing which shows the front-end | tip part of the imaging part used for the handwriting input / output system concerning this invention. It is FIG. (2) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (3) explaining the function of the image pick-up part used for the handwriting input / output system concerning this invention. It is a figure (4) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (5) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (7) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (8) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (9) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (10) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (11) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (12) explaining the function of the imaging part used for the handwriting input / output system concerning this invention. It is a figure (1) explaining the function of the medium in which the writing area | region used for the handwriting input / output system concerning this invention was formed. It is a figure (2) explaining the function of the medium in which the writing area | region used for the handwriting input / output system concerning this invention was formed. It is a figure (3) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (4) explaining the function of the medium in which the writing area | region used for the handwriting input / output system concerning this invention was formed. It is a figure (5) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (6) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (7) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (8) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (9) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (10) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (11) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (12) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (13) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (14) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure explaining the discriminating method of the operation | movement which touches and the operation | movement which jumps of the imaging part used for the handwriting input / output system concerning this invention. It is a figure explaining the 1st operation in the handwriting input / output system concerning the present invention. It is FIG. (1) explaining the 2nd operation in the handwriting input / output system concerning this invention. It is FIG. (2) explaining the 2nd operation in the handwriting input / output system concerning this invention. It is FIG. (3) explaining the 2nd operation in the handwriting input / output system concerning this invention. It is FIG. (1) explaining the 3rd operation in the handwriting input / output system concerning this invention. It is FIG. (2) explaining the 3rd operation in the handwriting input / output system concerning this invention. It is FIG. (3) explaining the 3rd operation in the handwriting input / output system concerning this invention. It is FIG. (4) explaining the 3rd operation in the handwriting input / output system concerning this invention. It is FIG. (5) explaining the 3rd operation in the handwriting input / output system concerning this invention. It is FIG. (6) explaining the 3rd operation in the handwriting input / output system concerning this invention. It is a figure explaining the preservation | save method of the file change result by rotation operation | movement of an imaging part in the handwriting input / output system concerning this invention. It is a figure explaining the cancellation method of the file content change by rotation operation | movement of an imaging part in the handwriting input / output system concerning this invention. It is a figure explaining the starting method of the application by rotation operation | movement of an imaging part in the handwriting input / output system concerning this invention. It is a figure explaining the completion | finish method of the application by rotation operation | movement of an imaging part in the handwriting input / output system concerning this invention. It is a figure (1) explaining the operation switching method by the inclination operation | movement to four directions of an imaging part in the handwriting input / output system concerning this invention. It is a figure (2) explaining the operation switching method by the inclination operation | movement to four directions of an imaging part in the handwriting input / output system concerning this invention. It is a figure explaining the touch operation | movement of an imaging part in the handwriting input / output system concerning invention. It is a block diagram explaining the structure of the audio | voice recording part and the audio | voice output part in the handwriting input / output system concerning this invention. It is a figure explaining the operation | movement which the imaging part bounces in the handwriting input / output system concerning this invention. It is a figure (15) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure (7) explaining the dot pattern used for the handwriting input / output system concerning this invention. It is a figure (8) explaining the dot pattern used for the handwriting input / output system concerning this invention.

<Description of stream dots>
First, an example of a method for forming a stream dot, which is a dot pattern used in the present invention, will be described with reference to FIGS.

1 and 2 sequentially show an example of a process for forming a stream dot pattern.

The dot pattern according to the present invention is different from the conventional dot pattern. First, in step 1, corresponding to the visible information on the surface of the medium, the reference dot 1 is continuously arranged in a line at a position where information is input / output. Place multiple.

In FIG. 1A, the reference dots 1 are arranged in a curved line. However, the arrangement of the reference dots 1 is not limited to this, and is composed of a straight line and a curved line or a plurality of line segments. Various modifications can be made to form the dot pattern in a shape that matches the area where information is input and output, such as a polygonal line.

In addition, the reference dot 1 may be arranged on a real line visually formed on the medium surface, or the reference dot 1 may be arranged according to a predetermined rule along the real line. The real line here is a concept with respect to a virtual line, and includes all the lines that actually exist. For example, a solid line, a broken line, a dotted line, a straight line, a curved line, and the like can be given. Absent. In addition, the dot pattern may be unevenness such as printing, display display, or a hole or groove on a metal or plastic.

In addition, it is desirable to arrange the reference dots at equal intervals from the viewpoint of improving reading accuracy, but the reference dots are not limited to this, and a plurality of intervals are mixed to define a set of constant information of the dot pattern, It is also possible to define both a set of constant information of a dot pattern and a direction of the dot pattern by arrangement intervals of three different reference dots in the set of fixed information.

Next, as step 2, a first virtual reference line 2 that connects the reference dots 4 arranged in a line is provided. In FIG. 1B, the first virtual reference line 2 is provided by a curved line, but the first virtual reference line 2 is not limited to this, and the reference virtual dots 1 arranged in a curved line are not limited thereto. The straight first virtual reference line 2 may be provided, or the curved first virtual reference line 2 may be provided for the reference dots 1 arranged in a straight line. That is, the second virtual reference line 3, virtual reference point 4, and information dot 5 in Step 3 to Step 5 to be described later are arranged at positions where the reference dots are connected to form a first line consisting of a straight line, a broken line and / or a curve. One virtual reference line 2 can be freely defined.

In addition, as shown in an example in FIG. 3, the first virtual reference line 2 in the case of a curve is preferably a Bezier curve.

That is, first, the reference dots 4 on the first virtual reference line 2 are P0 and P3, and P1 and P2 are given control points. Next, points P4, P5, and P6 are obtained by dividing the three line segments P0-P1, P1-P2, and P2-P3 obtained by connecting the control points in order at a one-to-one ratio. Then, points P7 and P8 are obtained by dividing the two line segments P4-P5 and P5-P6 obtained by connecting these points in order at a one-to-one ratio.

Finally, a point P9 that further divides the line segment connecting the two points, P7-P8, at a ratio of 1: 1 is obtained, and this point becomes a point on the Bezier curve.

By repeating this procedure, a Bezier curve with control points P0, P1, P2, and P3 is obtained.

The first virtual reference line 2 may be provided by using various algorithms such as a spline curve obtained by using a spline function, an nth order polynomial, an elliptic arc, and the like, not limited to a Bezier curve.

Also in the second virtual reference line 9, it is possible to define a curve using this method in the same manner as the first virtual reference line 2.

Next, as step 3, a second virtual reference line 3 defined at a predetermined position from the reference dots 1 and / or the first virtual reference line 6 arranged in a line is provided. In FIG. 1 (c), the second virtual reference line 3 is adjacent to the reference dot 1 adjacent to the predetermined position on the vertical line with respect to the tangent line of the first virtual reference line 3 at the midpoint between the adjacent reference dots 1. However, the second virtual reference line 3 is not limited to this, and a virtual reference point is provided in accordance with an area in which information is input / output by a dot pattern as will be described later. In addition, it can be defined by various methods.

Further, the second virtual reference line 3 may be provided only on one side with respect to the first virtual reference line 2 to define the direction of the dot pattern, or may be provided on both sides to increase the amount of information. .

Next, as step 4, a plurality of virtual reference points 4 are provided at predetermined positions on the second virtual reference line 3. In FIG. 2 (a), the virtual reference point 4 is an isosceles triangle having an intersection of the second virtual reference lines 3, that is, a straight line connecting adjacent reference dots 1 as a base and the second virtual reference line 3 as an opposite side. However, the position of the virtual reference point 4 is not limited to this, and is provided at the midpoint of the second virtual reference line 3 or instead of the second virtual reference line 3. Various modifications such as provision on the dot 1 are possible.

Then, in step 5, the information dot 5 is arranged at the end point expressed by a vector with the virtual reference point 4 as the start point. In FIG. 2B, one information dot 5 is arranged for one virtual reference point 4 so that the vector direction from the virtual reference point 4 is eight directions and the distance from the virtual reference point 4 is equal. However, the arrangement of the information dots 5 is not limited to this, and the information dots 5 are arranged on the virtual reference point 4, the vector direction is set to 16 directions, or two are arranged for one virtual reference point 4. It is possible to arrange a plurality of elements in any direction and in any length.

FIG. 4 is an enlarged view showing an example of a dot pattern information dot and a bit display of data defined therein.

Information dot 5 is a dot for recognizing various information. The information dot 5 is arranged at the end point represented by a vector with the virtual reference point 4 as the start point. For example, as shown in FIG. 4, the information dot 5 is rotated by 45 degrees in the clockwise direction because a dot 0.1 mm away from the virtual reference point 5 has a direction and a length expressed by a vector. These are arranged in 8 directions to represent 3 bits.

In the illustrated example, 3 bits are expressed by arranging in 8 directions, but the present invention is not limited to this, and 4 bits can be expressed by arranging in 16 directions. Of course, it can be arranged in length.

Further, in FIG. 2B, in all the virtual reference points 4, the information dot 5 is arranged at the end position with the virtual reference point 4 as a starting point. However, the present invention is not limited to this. Information may be defined depending on whether or not is arranged. For example, information can be defined as “1” if a dot is arranged on a virtual reference point, and “0” if no dot is arranged.

FIG. 5 is an example of information dots and bit display of data defined therein, and shows other forms.

Further, regarding the information dot 5, if two types of short (upper part in FIG. 5) and long (lower part in FIG. 5) are used from the virtual reference point 5 derived from the reference dot 1 and the vector direction is eight directions, then 4 bits. Can be expressed. At this time, it is desirable that the longer one is about 25 to 30% of the distance between adjacent virtual reference points 5, and the shorter one is about 15 to 20%. However, it is desirable that the center interval between the long and short information dots 3 is longer than the diameter of these dots.

The information dot 5 is preferably one dot in consideration of its appearance. However, if the appearance is ignored and the amount of information is to be increased, a large amount of information can be obtained by assigning 1 bit to each vector and expressing the information dot 3 with a plurality of dots. For example, the vector of concentric eight directions, an information dot 3 defined from the reference dots 4 can represent information of 2 ^8, and 2 ⁶⁴ in chunks of the information dot eight constant information 1.

As described above, the stream dot pattern according to the present invention is continuous with a line including a curve unlike the reference dot formed in a two-dimensional lattice pattern in the conventional dot pattern proposed by the present inventor. Are formed based on the reference dots arranged in this manner.

FIG. 6 is a diagram illustrating an example of a state where stream dot patterns are arranged in the vertical direction.

In the figure, key dots and side dots are arranged in addition to the reference dots and information dots. Key dots are dots arranged at both ends of a set of fixed information. This key dot is a representative point of the dot pattern 1 for one region representing a group of information dots. The side dots are dots arranged on the positive and negative extension lines of the shift of the key dot 2.

(B) in FIG. 6 has reference dots and stream dot patterns arranged at equal intervals. In this way, by forming a plurality of stream dot patterns with a constant reference point interval, the XY coordinate values are defined without any gaps in the writing area. Is possible). However, the stream dot pattern according to the present invention is not limited to this, and the interval between the dot patterns may be arbitrarily set as shown in FIG. Also, the interval between the reference dots can be set arbitrarily.

As a result, the dot pattern in which the XY coordinates are defined is formed in the information area visually formed on the surface of the medium without being restricted by the shape of the rectangular area when used as a two-dimensional code (used as an index). It is possible to form a dot pattern by repeating a set of constant information in a combined free shape.

Note that the virtual reference line and the virtual reference point according to the present invention are not actually printed and formed on the surface of the medium, but only when the dot pattern is arranged on the image memory of the computer or when the dot pattern is read. Is set virtually.

By using this stream dot pattern, it is possible to form a dot pattern on a curved surface body such as a globe, a three-dimensional structure such as a human body model, a three-dimensional map, etc., and the present invention is not limited to a planar map or picture book. It is possible to use the input / output device according to the above.

(A)

FIGS. 7 to 8 are explanatory views showing an embodiment in which the grid sheet 7 which is an information input auxiliary sheet according to the present invention is mounted on a display and used.

FIG. 7 is a diagram in which a grid sheet is used in a general-purpose computer system that is an information processing apparatus. In this embodiment, a grid sheet is pasted on a display screen such as a liquid crystal display (LCD) such as a personal computer or a CRT (CRT). The grid sheet is formed of a transparent film, and is printed with a dot pattern. Although details will be described later, this dot pattern is obtained by patterning XY coordinate values and / or code values with a predetermined algorithm. A scanner which is a dot pattern reading unit is connected to the computer main body. The user touches the grid sheet with the scanner in accordance with the instructions on the fluoroscopic screen. The scanner reads the dot pattern and transmits it to the personal computer via the USB cable. The central processing unit (CPU) of the personal computer analyzes the dot pattern to obtain the XY coordinate value on the grid sheet, and further, xy on the display. Conversion to coordinate values enables input in touch panel format from the coordinate value information.

According to this, it becomes possible to input a touch panel format by this sheet, and it is possible to provide a touch panel that is inexpensive and excellent in convenience. Further, when browsing a site on the Internet, it is possible to search and refer to related information even if the link information is not set.

In FIG. 7, a personal computer is used as the information processing device. However, the present invention is not limited to this, and the information processing device is a personal computer, a PDA, a television receiver, a front and rear projector, a game device, a karaoke device, a portable device. A telephone terminal device, a POS terminal device, an ATM, a KIOSK terminal, a car navigation system, a pachinko machine, a watch, a smartphone, or the like may be used. In these, a grid sheet is arranged on a display screen or a display screen as a touch panel type input device.

FIG. 8 is a cross-sectional view showing the internal structure of the grid sheet described in FIG.

(A) is a longitudinal sectional view showing a state where the grid sheet is touched by the scanner.

As shown in the figure, the grid sheet has a structure in which an infrared reflection layer, a dot pattern layer, and a protective transparent sheet (protective layer) are laminated from the display device side.

The infrared reflective layer has a configuration in which an infrared reflective material is vapor-deposited on a transparent sheet for vapor deposition made of a material that transmits visible light, such as vinyl, envipet, and polypropylene. The infrared reflection layer reflects the infrared ray irradiated from the infrared irradiation means of the scanner and transmitted through the protective transparent sheet to the scanner and transmits visible light. At the same time, infrared rays from the display device are blocked. Thereby, only the infrared light irradiated from the infrared irradiation means of the scanner can be used as the irradiation light, and only a bright and clear dot pattern can be photographed, and the dot code can be analyzed accurately.

The dot pattern layer is printed with a dot pattern in which dots made of an infrared absorbing material such as carbon ink are arranged according to a predetermined rule as will be described later.

Protective transparent sheet is made of a material that transmits visible light and infrared rays, such as vinyl, envipette, and polypropylene. When the dot pattern is repeatedly touched with the scanner, the dots are worn out, causing a problem that the dot pattern cannot be read accurately. Therefore, by providing a protective transparent sheet, it is possible to prevent dot wear and dirt and to use the sheet for a long period of time.

The scanner incorporates an IR-LED that is an infrared irradiation means, an IR filter that cuts a predetermined wavelength component of the reflected light, a C-MOS sensor that is an image sensor, and a lens. The scanner captures reflected light of the irradiation light irradiated on the grid sheet. As described above, since the dot pattern is printed with ink having the characteristic of absorbing infrared rays, only the dot portion is captured black in the captured image by the C-MOS sensor.

In this case, the reflection layer that is specularly reflected by the infrared reflection layer is photographed in a state where the reflected light is not incident on the lens and the center is black as shown in FIG. End up. Therefore, it is impossible to photograph the dot pattern without omission. Therefore, it is necessary to diffusely reflect infrared light and to enter the lens. That is, as shown in FIG. 9A, an infrared diffusion layer is provided between the dot pattern printing layer and the infrared reflection layer. This infrared diffusion layer is made of a transparent or translucent material. As a result, as shown in FIG. 5B, the infrared light irradiated from the IR-LED is specularly reflected by the infrared reflecting layer and diffused by the infrared diffusing layer, and the reflected light of all the imaging regions is incident on the lens.

In the figure, the grid sheet is provided with an infrared diffusing layer. However, the present invention is not limited to this, and a filter made of an infrared diffusing material may be provided at the photographing port of the scanner.

FIG. 10 is a cross-sectional view showing various structures of the grid sheet.

(A) of FIG. 1 is a grid sheet including an infrared reflection layer having a characteristic of reflecting infrared light and transmitting visible light, and a dot pattern layer. As shown in the figure, the infrared rays irradiated from the infrared irradiation means are first absorbed by the portion of the dot pattern layer where the dots are formed (hereinafter referred to as the dot portion) and transmitted through the other regions. Next, the transmitted infrared ray is specularly reflected by the infrared reflection layer and passes through the dot pattern layer except for the dot portion.

(B) of the figure is a grid sheet in which an infrared diffusion layer that transmits visible light and diffuses the infrared light is provided between the infrared reflection layer and the dot pattern layer. As shown in the figure, the infrared rays irradiated from the infrared irradiation means are first absorbed by the dot portions of the dot pattern layer and transmitted in other regions. Next, the transmitted infrared rays are diffused by the infrared diffusion layer, then specularly reflected by the infrared reflection layer, diffused again by the infrared diffusion layer, and transmitted through the dot pattern layer except for the dot portion.

(C) in the figure is a grid sheet in which an infrared diffusion layer that transmits visible light and diffuses infrared light is provided on one side of the dot pattern layer, that is, on the opposite side of the infrared reflection layer. As shown in the figure, the infrared rays irradiated from the infrared irradiation means are first diffused by the infrared diffusion layer, then absorbed by the dot portion of the dot pattern layer, and transmitted in the other regions. Next, the transmitted infrared rays are specularly reflected by the infrared reflection layer, and are again transmitted through the dot pattern layer except for the dot portion and diffused by the infrared diffusion layer.

FIG. 4D is a grid sheet composed of an infrared diffuse reflection layer having a characteristic of diffusing and reflecting infrared light from one side and transmitting visible light, and a dot pattern layer. As shown in the figure, the infrared rays irradiated from the infrared irradiation means are first absorbed by the dot portions of the dot pattern layer and transmitted in other regions. Next, the transmitted infrared light is diffusely reflected by the infrared diffuse reflection layer, and passes through the dot pattern layer except for the dot portion.

(E) in the figure is a grid sheet in which a protective layer having a property of transmitting infrared light and visible light is provided on the outer surface of the grid sheet shown in the figure (a) on the dot pattern layer side. By providing such a protective layer, it is possible to prevent dot wear and dirt and to use the sheet for a long period of time.

It should be noted that the protective layer may be provided not only on the figure (a) but also on the outer surface of the grid sheet shown in (b) to (d) on the dot pattern layer side.

(F) in the figure is a grid sheet having a characteristic of reflecting infrared rays from the side opposite to the dot pattern layer in the grid sheet shown in (a). By having such characteristics, it is possible to block infrared rays from a display screen or a screen on which the grid sheet of the present invention is mounted, and to use only infrared rays emitted from infrared irradiation means as irradiation light. Therefore, only a bright and clear dot pattern can be photographed, and the dot code can be analyzed accurately.

Note that the grid sheets shown in (b) to (e) in addition to (a) in the figure may have a characteristic of reflecting infrared rays from the side opposite to the dot pattern layer (the other side). .

(G) in the figure is a grid sheet in which an adhesive layer is provided on the anti-dot pattern layer side of the infrared reflecting layer in the grid sheet shown in (a). The adhesive layer is made of a removable material. The pressure-sensitive adhesive layer is necessary only when the grid sheet is used by being attached to a display device or a medium, and is not necessary when it is placed on a printed material or sandwiched as described later. By providing such an adhesive layer, the grid sheet can be easily attached to a display or the like.

The pressure-sensitive adhesive layer may be provided not only on the same figure (a) but also on the anti-dot pattern layer side of the infrared reflecting layer of the grid sheets shown in (b) to (e).

FIG. 11 shows another embodiment of the grid sheet. In this embodiment, a dot pattern layer is provided on both sides of the sheet.

The figure (a) from the infrared reflective layer which has the characteristic which permeate | transmits visible light while reflecting the infrared rays to each surface in each direction, and the dot pattern layer provided on both surfaces of the infrared reflective layer, It is a grid sheet. Thereby, since the sheet can be used on both sides, convenience is improved.

FIG. 4B is a grid sheet in which an infrared diffusion layer that transmits visible light and diffuses infrared light from each direction is provided between the infrared reflective layer and the dot pattern layer on each surface. It is.

(C) is a grid sheet in which an infrared diffusion layer is provided on the outer surface of the infrared reflection layer and the dot pattern layer on each side.

FIG. 4D is a grid sheet composed of an infrared diffuse reflection layer and dot pattern layers provided on both sides of the infrared diffuse reflection layer.

The figure (e) is an information input auxiliary sheet in which a protective layer having a property of transmitting infrared light and visible light from each direction is provided on both outer surfaces of the grid sheet shown in the figure (a). is there.

Note that the protective layer may be provided on both outer surfaces of the grid sheets shown in (b) to (d) as well as in FIG.

The figure (f) is a grid sheet in which an adhesive layer is provided on each anti-dot pattern layer side of the infrared reflection layer, that is, both outer surfaces of the grid sheet, in the grid sheet shown in the figure (a). is there.

It should be noted that the adhesive layer may be provided on both outer surfaces of the grid sheet shown in (b) to (e) as well as in FIG.

The description of each layer and the description of reflection in the grid sheets in FIGS. 9A to 9F are the same as those described above, and thus the description thereof is omitted here.
In the grid sheet shown in the figure, different coordinate values and / or code values may be patterned in the dot pattern layer on one side and the other side of the infrared reflecting layer or infrared diffuse reflecting layer.

In the grid sheets shown in FIGS. 10 to 11, text, illustrations, photographs, etc. may be superimposed and printed on the dot pattern layer or other layers with ink made of an infrared transmitting material or an infrared reflecting material.

12 to 17 are diagrams for explaining another embodiment of the present invention. In this embodiment, a grid sheet is used on a printed material.

FIG. 12 is a diagram showing a case where a grid sheet on which a dot pattern is printed only on one side is used.

Users use grid sheets on printed materials such as books, magazines, and newspapers. Even if it is used for the page on the right side as shown in FIG. 11A, the user can use the sheet even if it is used for the page on the left side as shown in FIG. Use with the front side facing up.

FIG. 13A is a front view showing the surface of the grid sheet. On the grid sheet, a dot pattern that means XY coordinates is printed on almost the entire area, and at the bottom, icons with numbers from 0 to 9 and characters “page input”, “decision”, and “stop” are written, A dot pattern that means a code value is overprinted. This icon is an icon for inputting an index for specifying a printed matter using a scanner.

(B) is a longitudinal sectional view of the grid sheet. This grid sheet is configured from the back side as a layer of a protective transparent sheet, a transparent infrared diffuse reflection layer, a dot pattern + graphic printing layer, and a protective transparent sheet. Here, the graphic print layer is a visible print layer.

FIG. 14 is a diagram showing a case where a grid sheet having a dot pattern printed on both sides is used. In this case, the grid sheet can be used while being sandwiched. That is, when a grid sheet is used on the right page, the grid sheet is put on the right page as shown in FIG. Then, the side on which the icon is printed on the right side is up. Next, when the grid sheet is used for the left page, the grid sheet is turned to the left side with the grid sheet being sandwiched as shown in FIG. Then, the side on which the icon is printed on the left side is up. The icon is printed on both sides with ink.

FIG. 15A is a front view showing the front surface of the grid sheet, and FIG. 15B is a front view showing the back surface. On the surface of the grid sheet, a dot pattern that means XY coordinates is printed on almost the entire area, and numbers from 0 to 9 and characters “page input”, “decision”, and “stop” are written on the right side. An icon and a dot pattern representing a code value are printed in a superimposed manner. This icon is an icon for inputting an index for specifying a printed matter using a scanner. On the back surface of the grid sheet, a dot pattern that means XY coordinates is printed on almost the entire area, and the same icon and dot pattern as the front surface are superimposed and printed on the left side.

(C) is a cross-sectional view of the grid sheet. This grid sheet is constituted by a protective transparent sheet, a dot pattern + graphic print layer, an infrared diffuse reflection layer, a dot pattern + graphic print layer, and a protective transparent sheet from the back side.

FIG. 16 is a diagram showing a system for displaying information on the display by the user touching the grid sheet.

As shown in FIG. 5A, the user puts a grid sheet on the printed material and touches the grid sheet with a scanner. A central processing unit (CPU) in the scanner analyzes the dot pattern, converts it into coordinate values and / or code values, and transmits them to a personal computer via a USB cable. It should be noted that the central processing unit in the personal computer may be analyzed and converted into coordinate values and / or code values.

The coordinate value means the value of the XY coordinate in the dot coordinate system. In addition, an index (page, title, etc.) for specifying the printed paper surface is input by touching the icon printed on the grid sheet, or the index code value input with the keyboard is, for example, the number of pages on the printed paper surface. I mean. Therefore, it is assumed that n pages are designated. If you set in advance a mask that specifies print information such as text, illustrations, and photos with XY coordinate values and multimedia information that corresponds to the mask number, the XY coordinates of the dot pattern of the grid sheet read by the scanner Based on the value, the central processing unit in the personal computer uses the dot coordinate-mask No. stored in the hard disk. Refer to the correspondence table ((b) in the figure). The table is provided for each page. In this embodiment, the dot coordinates of n pages-mask No. Refer to the correspondence table. Then, the touch position mask No. Is detected. For example, when the crossed portion shown in FIG. Is 1. Next, the page No. shown in FIG. -Mask No. References the multimedia information file correspondence table. In the table, page No. And mask No. A multimedia information file is registered correspondingly. For example, a web address is registered in the n-page mask 1, and a local drive and an execution file are registered in the n-page mask 2. From the table, page No. And mask no. Search and output multimedia information files corresponding to. That is, page No. Is n and mask no. Since 1 is 1, the Web page at the registered Web address is displayed.

FIG. 17 is a diagram showing another form of a system for displaying information on the display by the user touching the sheet.

In the figure, as an example, the case where the character “Tokyo” is touched in the printed matter described as “Tokyo” in the text will be described.

As shown in the figure, touch the icon printed on the grid sheet and enter the index to identify the printed paper surface, or enter the index with the keyboard to identify the printed paper surface, and the user specifies the printed paper surface on the printed material. And touch the dot pattern of the part marked “Tokyo” with the scanner. Then, the central processing unit (CPU) in the scanner analyzes the dot pattern, converts it into coordinate values or / and code values, and dot XY coordinates, and transmits them to a personal computer via a USB cable. It should be noted that the central processing unit in the personal computer may be analyzed and converted into coordinate values and / or code values.

The central processing unit (CPU) of the personal computer converts the printed paper surface ID and the dot XY coordinates obtained as described above into DTPxy coordinates (xy coordinates on the DTP surface, which is the original data of the printed material).

Next, the central processing unit (CPU) of the personal computer accesses the document management server via the network (NW). The document management server stores a DTP file corresponding to each printed sheet. The central processing unit (CPU) takes out the DTP file of the corresponding printed paper (for example, ID = n) from the document management server. Then, in the DTP file with ID = n, the DTP coordinate (xt, yt) closest to the touch position (x, y) is selected to specify “Tokyo”. The DTP coordinates (xt, yt) are the coordinates of the center position of the text area. When the DTP coordinates are searched, the dictionary server is accessed and a data file indicating the specified text, illustration, photo, etc. is searched. The dictionary server stores related information such as explanations, URLs, and multimedia information corresponding to data files indicating text, illustrations, photographs, and the like. The central processing unit (CPU) of the document management server searches for “Tokyo” from the dictionary server based on the program, and automatically transmits the related information to the personal computer via the network (NW). As a result, the explanation regarding “Tokyo” is output and displayed on the display of the personal computer.

According to this, it is possible to display detailed explanations such as texts, illustrations and photographs on the printed matter on the screen simply by placing the grid sheet on the printed matter and touching it with the scanner.

Conventionally, text, illustrations, photographs, and dot patterns are superimposed on a printed material, and the user touches the printed material with a scanner to output information corresponding to the touched character. However, this method cannot handle a medium that has already been printed, and requires new printing. In the present invention, since a printed material is covered with a sheet, the existing printed material can be used. Further, since a single sheet can be used in common for a plurality of printed materials, convenience is enhanced.

FIG. 18 is an explanatory diagram showing an embodiment in which a grid sheet is attached to a display, content specifying information touched by a scanner is recognized, and related information is automatically searched and output.

In the figure, as an example, a case where the character “Tokyo” displayed on the display screen is touched will be described.

As shown in the figure, a grid sheet is pasted on the display screen of the personal computer. The user touches the dot pattern of the portion displayed as “Tokyo” with the scanner. Then, the central processing unit (CPU) in the scanner analyzes the dot pattern, converts it into XY coordinates, and transmits it to the personal computer via the USB cable. The XY coordinates are XY coordinate values of the touch position in the dot coordinate system. In addition, you may analyze and convert into a coordinate value with the central processing unit in a personal computer.

Next, the central processing unit (CPU) of the personal computer accesses the Web server of each content displayed on the display via the network (NW). Each content file is stored in the Web server. The central processing unit retrieves the displayed content file (HTML, XML, flash, etc.). Then, in the extracted content file, the XY coordinate value of the touch position in the dot coordinate system is converted into the xy coordinate value in the format information of the content such as HTML, XML, and flash.

If the displayed content file is on a personal computer, there is no need to access the Web server and retrieve the content file.

Next, the coordinate value (xt, yt) of the content specifying information such as text, illustration, photo, etc. displayed on the display closest to the touch position coordinate value (x, y) is searched. The coordinate values (xt, yt) of the content specifying information are the coordinates of the center position of the content specifying information. When the content identification information is recognized, various multimedia information such as audio information and image information corresponding to the content identification information such as a data file indicating the recognized text, illustration, photo, etc. is stored in an electronic dictionary or a multimedia corresponding beforehand. Search from the database where information is set. The dictionary server stores content-related information such as explanations, URLs, and multimedia information corresponding to data files indicating text, illustrations, photographs, and the like. The central processing unit (CPU) retrieves “Tokyo” from the dictionary server, and automatically outputs the content related information to the personal computer via the network (NW). As a result, the explanation about “Tokyo” is displayed on the display of the personal computer.

According to this, it is possible to display detailed explanations such as text, illustrations, and photos on the screen simply by placing a grid sheet on the display.

FIG. 19 is a longitudinal sectional view of a grid sheet used in such an example. The grid sheet is composed of an adhesive layer, an infrared diffuse reflection layer, a dot pattern + graphic printing layer, and a protective transparent sheet from the back side.

FIG. 20 is a diagram illustrating a case where the grid sheet is used in the front projector system.

In this embodiment, a grid sheet is pasted on the screen. A predetermined image is projected and displayed on the screen by the projector. The projector is connected to a personal computer or a video distribution system (not shown). When a desired position on the screen is touched using the scanner connected to the personal computer, the dot pattern at that position is read into the scanner and the personal computer uses the scanner. Converted to coordinate values. A hard disk device or video distribution server of a personal computer refers to an index table in which coordinate values are associated with instructions, addresses, etc., and outputs multimedia information and operation instructions defined in the corresponding addresses.

FIG. 21 shows an example using a rear projector panel, which is a system for projecting an image from the rear projector.

A grid sheet is affixed to the surface of the rear projector panel. The structure of the grid sheet is the same as that shown in FIG.

A predetermined image is projected and displayed on the rear projector panel by the rear projector. The rear projector is connected to a personal computer or video distribution system (not shown), and when a desired position on the rear projector panel is touched using the scanner connected to the personal computer, the dot pattern at that position is read into the scanner. Are converted into coordinate values by a personal computer. A hard disk device or video distribution server of a personal computer refers to an index table in which coordinate values are associated with instructions, addresses, etc., and outputs multimedia information and operation instructions defined in the corresponding addresses.

FIG. 22A shows an example in which a grid sheet is used for a mobile terminal. A grid sheet is attached to the display of the portable terminal. Then, the grid sheet is touched with a scanner connected by wireless or cable. Then, the dot pattern at that position is read into the scanner and converted into coordinate values in the scanner or portable terminal. In the memory of the portable terminal or in the portable dot code management server, an index table in which coordinate values are associated with commands or addresses is stored. The central processing unit in the portable terminal refers to the index table, and outputs multimedia information and an operation instruction specified by the corresponding address.

FIG. 22B shows an example in which a grid sheet is used for the car navigation device.

In this way, when used in a car navigation device, it is preferable that communication between the scanner and the car navigation device is performed by wireless communication using Bluetooth or the like. Further, the scanner may be connected to a mobile terminal and transmit data corresponding to the read dot pattern to the server via the mobile terminal network.

In addition, the car navigation shown in the figure may be used in combination with a map such as a tourist guide on which a dot pattern is printed.

FIG. 23 to FIG. 24 are diagrams for explaining an embodiment in which the grid sheet according to the present invention is used in the thin client system.

Thin client is a general term for a system in which a client terminal used by a user has only a minimum function and manages resources such as application software and files on the server side.

As shown in FIG. 23, in this embodiment, a plurality of displays are connected to one server (information processing apparatus) via a network. As this display, various types of displays other than a general television monitor and a personal computer display which do not have a built-in CPU can be used. An information input auxiliary sheet is affixed on the display screen of each display. The server and each display are connected by a wired system or a wireless system such as RDP, USB, BNC (Bayonet Neil Concelman). The server and the scanner are connected by a wired method or a wireless method.

When the user touches the dot pattern on the grid sheet with the scanner, the dot pattern at that position is read into the scanner and converted into XY coordinate values within the scanner. In the hard disk device of the server, an index table in which XY coordinate values are associated with commands or addresses is stored. The central processing unit of the server refers to the index table, and outputs multimedia information and an operation instruction specified by the corresponding address.

In this way, by attaching a grid sheet on the display screen, it is possible to input in a touch panel format with a simple terminal system with only a display.

In the dot pattern, a code value may be registered together with the XY coordinate value. The code value is for identifying each display. In this case, when the scanner reads the dot pattern, it transmits the code value and the XY coordinate value to the server (information processing apparatus). The server recognizes which display from the code value. Then, various multimedia information and / or operation instructions corresponding to each display are recognized, and output information is displayed on the display.

In the dot pattern, an XY coordinate value for uniquely identifying the display may be printed as a dot pattern. In this case, XY coordinates obtained by dividing a large area in advance are printed on the sheet, and this sheet is used separately for each display. That is, the XY coordinate values are different for each display. When the XY coordinate value read by the scanner is transmitted to the server (information processing apparatus), the server specifies which display is operated from the XY coordinate value. Then, various multimedia information output and / or operation instructions corresponding to each display are recognized, and the output information is displayed on the display.

Also, as shown in FIG. 24, a sticker on which a code value for identifying each display is printed as a dot pattern may be attached to the display. The user touches the grid sheet after touching the seal. The server specifies which display is operated by the code value registered in the dot pattern of the seal. Next, the touch position of the user is recognized from the XY coordinate values transmitted from the dot pattern of the grid sheet, various multimedia information and / or operation instructions corresponding to each display are recognized, and output information is displayed on the display. To display. The seal may be made of the same material as that used for the adhesive sticker on paper on which a normal dot pattern is printed or the grid sheet of the present invention.

By adopting the thin client system in this way, it is possible to use various types of displays other than a general television monitor and a personal computer display without a built-in CPU, and thus a system is provided at low cost. It becomes possible.

<Calibration>
FIG. 25 to FIG. 28 are diagrams for explaining calibration performed when the grid sheet is used.

When the user touches the display or the printed material, the positional relationship between the display or the printed material and the grid sheet needs to be matched in order to output information that accurately reflects the instruction displayed at the touch position. . Therefore, calibration is performed to properly associate the coordinate system of the display or printed material with the coordinate system of the grid sheet.

FIG. 25 to FIG. 26 are diagrams illustrating a case where calibration is performed on the display screen of the display.

FIG. 25 is a diagram illustrating a case where calibration marks are provided on the grid sheet.

As shown in FIG. 5A, calibration marks are provided in the vicinity of the four corners on one side of the grid sheet. The calibration mark is not necessarily provided near the four corners, and may be provided at two or more corners or may be provided at the center. Moreover, you may provide in two or more corner parts and the center.

As shown in FIG. 5B, the user moves the cursor to the calibration mark and left-clicks the mouse. The central processing unit of the personal computer recognizes the clicked position. Then, the display coordinate system and the grid sheet coordinate system are appropriately associated with each other. As a result, calibration is performed.

In the figure, the calibration mark is formed on a detachable transparent seal. After the calibration is completed, the user peels off the seal from the grid sheet as shown in FIG.

The calibration mark may be directly printed on the grid sheet in addition to the case where it is formed on the transparent sticker. Moreover, it is provided in a removable state on the grid sheet, and may be removed from the grid sheet after calibration is completed.

FIG. 26 is a diagram illustrating a case where a calibration mark is displayed on the display screen of the display.

As shown in FIG. 4A, calibration marks are displayed near the four corners of the display. Note that the calibration mark is not necessarily displayed near the four corners, and may be displayed at two or more corners, or may be displayed at the center. Moreover, you may display on both the corner part and center of 2 or more.

The user touches the grid sheet at the position where the calibration mark is displayed as shown in FIG. The scanner reads the dot pattern on the grid sheet and transmits it to the personal computer. The central processing unit of the personal computer recognizes the dot XY coordinates (X ₁ , Y ₁ ) of the touch position from the transmitted dot pattern, and performs calibration to properly associate the display coordinate system and the grid sheet coordinate system. Do.

Note that the calibration mark will not be displayed after calibration.

FIGS. 27 to 28 are diagrams illustrating a case where calibration is performed on a printed material.

FIG. 27 is a diagram illustrating a case where calibration marks are provided on both the printed material and the grid sheet for the printed material.

As shown in FIG. 5A, calibration marks are printed in the vicinity of the four corners of the grid sheet and the printed material. Note that the calibration mark is not necessarily printed in the vicinity of the four corners, and may be printed in two or more corner portions.

The user puts the grid sheet on the printed matter by matching both calibration marks. As a result, calibration for properly associating the coordinate system of the printed material with the coordinate system of the grid sheet is performed.

FIG. 28 is a diagram illustrating a case where a calibration mark is printed only on a printed material.

As shown in FIG. 4A, calibration marks are printed in the vicinity of the four corners of the printed material. Note that the calibration mark is not necessarily printed in the vicinity of the four corners, and may be printed in two or more corner portions.

The user puts a grid sheet on the printed material. Then, the scanner is matched with the mark. The scanner reads the dot pattern on the grid sheet and transmits it to the personal computer. The central processing unit of the personal computer recognizes the XY coordinates (X ₁ , Y ₁ ) of the touch position from the transmitted dot pattern, and performs calibration for appropriately associating the coordinate system of the printed material with the coordinate system of the grid sheet. .

By performing such calibration, the positional relationship between the coordinate position of the grid sheet and the image of the display or printed matter is matched, and information corresponding to the image or character touched by the user is output accurately. Become.

In each of the above-described embodiments, when a grid sheet is used for a display screen or the like, it may be used by other methods such as using a grid sheet attached from the upper part of the display screen, in addition to being attached with an adhesive.

(B)

[First Embodiment]
One embodiment of the information input device according to the present invention using a medium on which a dot pattern is printed together with content will be described with reference to FIGS. 29 to 45 as follows.
<Overview>

FIG. 29 shows an outline of the present embodiment. FIG. 29A is a diagram showing a state of operation in the present embodiment, and FIG. 29B is a diagram showing a state of selecting specific information.

As shown in FIG. 29A, first, the user inputs information to be input to the information input device 105 on the printed material 101a (medium) on which the dot pattern 103 (details will be described later) is printed together with the content. Touch using the dot pattern reading means.

In this example, since the user wants to input the word “Tokyo” in the place field of the input form displayed on the screen of the information input device 105, the word “Tokyo” 102 a printed on the printed material 101 a is used as the scanner 104. Is touching.

In the dot pattern 103, an XY coordinate value on the printed material 101a and a document ID (identification information) for specifying the printed material 101a are coded in advance, and when the user touches the printed material 101a with the scanner 104, The coordinate value (Xt, Yt) in the XY coordinate system on the printed material 101a at the touch position and the document ID of the printed material 101a are read into the information input device 105.

The information input device 105 uses the read XY coordinate values, the document ID, and DTP (Desk Top Publishing) data 101b (two-dimensional format information), which is electronic data based on the prepared printed matter 101a. The touch position recognizes that the word “Tokyo” 102 a is present (details will be described later), and the word “Tokyo” 102 a is input to the information input device 105.

In the example shown in FIG. 29A, the phrase “Tokyo” 102a input to the information input device 105 is entered in the location column of the input form displayed on the screen.

As shown in FIG. 29B, the DTP data 101b has its own xy coordinate system. Selection position information used to select these is set in text, charts, illustrations, photographs, etc. (specific information) included in the DTP data 101b.

For example, in the phrase “Tokyo” 102a, a corresponding mask area 102b and / or a center coordinate value (xi, yi) is set.

FIG. 29B shows the coordinate value (xt, yt) of the touch position on the DTP data 101b converted from the coordinate value (Xt, Yt) of the touch position on the printed material 101a touched by the scanner 104, and a phrase. The example of the positional relationship with the mask area | region 102b corresponding to "Tokyo" 102a and center coordinate value (xi, yi) is shown.

For example, when the coordinate value (xt, yt) of the touch position is included in the mask area 102b, the phrase “Tokyo” 102a among many specific information included in the DTP data 101b is input to the information input device 105. Selected.

Note that the method for selecting the specific information is that the coordinate value (xt, yt) of the touch position is greater than the distance from the coordinate value (xt, yt) of the touch position to the center coordinate value (xi, yi) of the other specific information. ) To the center coordinate value (xi, yi) of the phrase “Tokyo” 102a is shorter, so the phrase “Tokyo” 102a may be selected as an input. Details of the method for selecting the specific information will be described later.
<About dot code format>

The dot code and its format example will be described with reference to FIG. A dot code is information recorded in the dot pattern 103.

As shown in FIG. 6, the dot pattern is a dot pattern composed of unit areas of 2 rows × 4 columns of information dots in which a set of constant information is defined, and is partitioned into C ₀ to C ₃₁ within this unit. ing. The dot code format of each area is shown in (a) to (c).

30A is an example of a format in which the dot code includes an XY coordinate value, a code value, and parity, and FIG. 30B is an example in which the format is changed depending on where the dot pattern 103 is provided. 30 (c) is an example of a format in which the dot code includes an XY coordinate value and a parity.

In the format example shown in FIG. 30A, the X coordinate value at the position where the dot pattern 103 is provided is expressed using 8 bits of bits C0 to C7, and the Y coordinate value is expressed using bits C8 to C15. The The code value is then represented using 14 bits, bits C16 to C29. This code value can be used to represent arbitrary information in accordance with the purpose of use of the dot pattern 103. In the present embodiment, it can be used to express a document ID. Finally, two bits C30 and C31 are used as the parity of the dot code. Note that the parity calculation method may be a generally known method, and a description thereof will be omitted.

In the format example shown in FIG. 30B, the format is changed depending on where the dot pattern 103 is provided. In this example, a place where the dot pattern 103 is provided is divided into an XY coordinate area and a code value area. In the XY coordinate area, the XY coordinate area format is used, and in the code value area, the code value area format is used.

In the XY coordinate area format, the X coordinate is expressed using 15 bits of bits C0 to C14, and the Y coordinate is expressed using 15 bits of C15 to C29. In the code value area format, the code value is expressed using 30 bits from C0 to C29.

In order to distinguish whether the read information represents an XY coordinate value or a code value, the bit string representation is made so that the bit strings representing the XY coordinate value and the code value do not overlap. It is good to decide rules.

In this way, in the format example shown in FIG. 30B, more bits can be assigned to the XY coordinate value and code value than in the format example shown in FIG. Values and more code values can be represented.

In the format example shown in FIG. 30 (c), the same format as the XY coordinate area format in FIG. 30 (b) is used. Since there is no code value column in the format, the document ID cannot be directly recorded in the dot pattern 103, and the document ID is indirectly distinguished by dividing the XY coordinate value assignment area.

For example, an area where the X coordinate is in the range of 0 to 100 and the Y coordinate is in the range of 0 to 100 is allocated exclusively for DTP data of document ID = 1. Similarly, an area having an X coordinate in the range of 101 to 200 and a Y coordinate in the range of 0 to 100 is assigned to the DTP data with the document ID = 2. By performing such allocation in advance, for example, when the XY coordinate value read from the dot pattern 103 is (150, 50), the coordinate value is in the range of 101 to 200, and the Y coordinate Is included in the region in the range of 0 to 100, the document ID of the printed material 101a provided with the dot pattern 103 can be specified as 2.
<About the document ID input method>

As described above, the document ID indicating which DTP data the printed material 101a corresponds to is included in the dot code together with the XY coordinate value in at least a part of the dot pattern 103 printed on the printed material 101a. Alternatively, the printed matter 101a may be divided into a dot code area that includes only XY coordinate values and a dot code area that includes only the document ID.

The document ID may be manually input to the information input device 105 by the user using an input unit such as a keyboard.
<About the structure of printed matter>

The structure of the printed material 101a will be described with reference to FIG. FIG. 31 is a cross-sectional view of the printed material 101a. FIG. 31 (a) is a diagram showing a state in which infrared rays hit and diffusely reflect a portion of the print medium 101c where nothing is printed, and FIG. FIG. 31C is a diagram showing a state in which infrared rays hit the content printing portion of the printed matter 101a and is diffusely reflected, and FIG. Here, for ease of illustration, the ink is shown so as to swell on the surface of the print medium, but this arrangement is the case where the print medium 101c has a characteristic of not absorbing ink, The print medium 101c has a characteristic of absorbing ink, but the viscosity of the ink is extremely high. Usually, the print medium 101c is printed so as to bleed on the print medium.

As shown in FIG. 31 (a), it is desirable that the surface of the print medium 101c on which the content and the dot pattern 103 are printed has a characteristic of diffusing and reflecting infrared rays. The reason is that when the surface of the print medium 101c has a characteristic of specularly reflecting infrared rays, the reflected light from a part of the surface read by the scanner 104 does not enter the lens (described later) properly, and one of the read images This is because the image is dark and the image analysis becomes difficult.

In FIG. 31 (b), the infrared light hits the part printed using the infrared diffuse reflection ink 110, and the infrared light hits the part printed using the infrared transmission ink 111. After the transmission, the state of diffuse reflection on the surface of the print medium 101c is shown.

In FIG. 31 (c), in order from the top, on the surface on which the dot pattern 103 and the infrared diffuse reflection ink 110 are superimposed and printed, the state in which the infrared rays incident on the dot pattern 103 are absorbed, and only the dot pattern 103 is present. Absorption of infrared rays incident on the dot pattern 103 on the printed surface, and absorption of infrared rays incident on the dot pattern 103 on the surface on which the dot pattern 103 and the infrared transmission ink 111 are superimposed and printed. It shows how it is done.

Thus, when the dot pattern 103 and the infrared diffuse reflection ink 110 or the infrared transmission ink 111 are superimposed and printed, if the viscosity is extremely high, the infrared diffuse reflection ink 110 is formed in a donut shape around the previously printed dot pattern 103. Alternatively, infrared transmissive ink 111 is disposed. Ordinary viscous inks will blend into each other.

Since the scanner 104 irradiates the reading surface on which the dot pattern 103 of the printed material 101a is printed with infrared rays and shoots the reflected light as an image (details will be described later), the dot pattern 103 printed with the infrared absorbing ink is In the image, the portion that is black and diffusely reflects infrared light is white.

Note that the infrared diffuse reflection ink 110 and the infrared transmission ink 111 have a characteristic of reflecting or absorbing a specific wavelength of visible light, and therefore printed when a person sees the printed material 101a printed using these inks. Content can be read in the same way as normal printing.

In this way, by printing the dot pattern 103 on the print medium 101 c and further switching the ink used when the content is superimposed and printed on the print medium 101 c between the infrared diffuse reflection ink 110 and the infrared transmission ink 111, a variety of A combination of dot pattern printing and content printing can be realized.
<About Scanner 104>

32 shows the structure of a portion for reading the dot pattern 103 in the structure of the scanner 104 and how the dot pattern 103 on the printed material 101a is read. The scanner 104 includes an IR-LED that is a means for irradiating infrared rays, a lens that collects reflected light reflected from the surface of the printed material 101a by the infrared rays irradiated from the IR-LED, and reflected light that has passed through the lens. An IR filter that cuts a predetermined wavelength component and a C-MOS sensor that is an image sensor are incorporated.

The scanner 104 irradiates the printed matter 101 a with infrared rays, and images reflected light that is diffusely reflected from the surface of the printing medium 101 c and a portion printed by the infrared diffuse reflection ink 110. As described above, since the dot pattern 103 is printed with ink having a characteristic of absorbing infrared rays, only the dot portion of the dot pattern 103 is captured black in the image captured by the C-MOS sensor.

In addition to the above configuration, the scanner 104 includes a central processing unit (not shown). The central processing unit analyzes the dot pattern 103 input from the C-MOS sensor with analysis software, Convert to dot code and output.

Note that the scanner 104 may include a hardware circuit having similar functions instead of the central processing unit and the analysis software.

In addition, the process of analyzing the dot pattern 103 and converting it into a dot code may be performed on a device connected to the scanner 104, for example, a personal computer, instead of being performed by the central processing unit on the scanner 104.

The scanner 104 and a device to which the scanner 104 is connected may be connected by a wired method such as a USB cable, or may be connected by a wireless method.
<Configuration of Information Input Device 105>

33 shows a functional block diagram of the information input apparatus 105 according to the present embodiment. The information input device 105 includes a scanner 104, a document management unit 120 (two-dimensional format information storage selection unit), a coordinate conversion unit 130 (coordinate conversion unit), and a specific information selection unit 140 (specific information selection unit). ing.

The scanner 104 reads the dot pattern 103 at the position touched by the user using the scanner 104 on the printed material 101a, converts the dot pattern 103 into a dot code, and outputs the document ID portion in the dot code to the document management unit 120. Further, the scanner 104 outputs the XY coordinate value (xt, yt) portion in the dot code to the coordinate conversion unit 130. Here, a case where the dot code includes an XY coordinate value and a code value will be described.

The document management unit 120 uses the document ID input from the scanner 104 to search the document DTP data stored in the document management unit 120 in advance. The DTP data 101b corresponding to the input document ID is output to the coordinate conversion unit 130 and the specific information selection unit 140.

The coordinate conversion unit 130 uses the XY coordinate values (Xt, Yt) input from the scanner 104 and the DTP data 101b input from the document management unit 120 to convert the touch position coordinates into a coordinate system on the printed material 101a. XY coordinate value (Xt, Yt) at, is converted into an xy coordinate value (xt, yt) in the coordinate system on the DTP data 101b, and the converted xy coordinate value (xt, yt) is output to the specific information selection unit 140. .

The specific information selection unit 140 uses the DTP data 101b input from the document management unit 120 and the xy coordinate values (xt, yt) input from the coordinate conversion unit 130 to display texts, charts, and diagrams included in the DTP data 101b. Among the specific information such as illustrations and photographs, the specific information considered to be instructed by the touch operation by the user is selected and output. Although the details of the selection method of specific information will be described later, for example, there is a method of selecting specific information at a position closest to the xy coordinate value (xt, yt) of the touch position.

The specific information output from the specific information selection unit 140 is used as an input in another device or the like.

Specific examples of the information input device 105 include personal computers, PDAs, television receivers, front projectors, rear projectors, game devices, karaoke devices, mobile phone terminal devices, POS terminal devices, ATMs, KIOSK terminals, car navigation systems, and pachinko machines. , Watches, smartphones and the like.
<About DTP data>

In this embodiment, DTP data is cited as an example of two-dimensional format information. More specifically, the DTP data is electronic data created by DTP software such as a word processor, authoring software that creates files in a format such as CAD software, HTML, XML, flash, PDF, etc. For example, in addition to the text body, the type and size of the font (vector font) used when displaying or printing the text, layout information, and the like are included.

In addition, when the DTP data includes charts, illustrations, photographs, etc., it also includes the name of the file in which each image data is stored and the location information of the area occupied when displaying or printing. It is.

DTP data is managed so that it can be uniquely identified by the document ID. When there are a plurality of pages when the DTP data is printed, that is, when the printed material 101a is composed of a plurality of pages, it is desirable that the document ID is configured so that each page can be distinguished.

The DTP data may be map data.
<About position information for selection>

For specific information such as text, charts, illustrations, and photos in the DTP data, it is determined from the positional relationship with the xy coordinates (xt, yt) of the touch position which specific information has been designated by the user. Therefore, position information for selection is provided.

The timing for setting the selection position information for each specific information may be set in advance when the DTP data is stored in the document management unit 120, or the specific information selection unit 140 may receive the DTP data from the document management unit 120. It may be set by the specific information selection unit 140 after receiving the data 101b.

Further, the specific information selection unit 140 may analyze the entire DTP data 101b received from the document management unit 120 at a time, and set selection position information for all the specific information included in the DTP data 101b. In the DTP data 101b, only the specific information in the vicinity of the coordinate value (xt, yt) of the touch position may be analyzed to set the position information for selection.

As the analysis method of the specific information, when the specific information is text, a generally known ordinary syntax analysis method may be used. In the case of charts, illustrations, and CAD data (points, line segments, planes, solid objects, etc.), in which the specific information is represented as a set of hierarchized objects, the hierarchical structure and the software that can handle the objects Analysis can be performed. If the specific information is an illustration or a photograph represented by a bitmap, the bitmap data is acquired from the file name of the specific information included in the DTP data, and a generally known ordinary image is obtained. The bitmap may be analyzed using an analysis method.

FIG. 34 shows an example in which the center coordinate value or centroid coordinate value of an object is used as position information for selection of specific information. 34A shows an example in which the specific information is text, FIG. 34B shows an example in which the specific information is a chart (pie chart), and FIG. 34C shows the specific information. This is an example of illustrations / photos. In this figure, a plus mark is used to indicate the position of the center coordinate value or centroid coordinate value of the object, but this mark is shown for explanation, and is printed or displayed. Is not to be done.

In the example shown in FIG. 34A, the phrases “Hana no Miyako”, “Osaka”, and “Tokyo Tower” are shown as specific information. For the phrase “Tokyo Tower”, the center of the display area is set as selection position information.

On the other hand, in the case of the phrase “Hana no Miyako”, the spaces between the characters constituting the phrase are separated, and when only the center coordinate value of the display area is set as the position information for selection, the user sets the phrase “flower flower”. When touching the vicinity of the character “flower” to select “city”, the phrase “flower city” is not selected because the center coordinate value of the word “flower city” is far from the coordinate value of the touch position. Probability is high.

Therefore, when the characters constituting the phrase are spaced apart, it is preferable to set the center coordinate value of each character as selection position information for each character constituting the phrase.

When the user touches the vicinity of the character “flower”, the specific information selection unit 140 cannot determine whether the phrase that the user wants to specify is the phrase “flower capital” or the phrase “flower”. There is a case. In that case, the specific information selection unit 140 may present both the phrase “flower capital” and the phrase “flower” to the user as selection candidates, and the user may select a desired phrase from the selection candidates.

Also, in the case of the phrase “Osaka”, the line between the character “Dai” and the character “Osaka” is separated by a line feed. Also in this case, the user can appropriately select the word “Osaka” by setting the character center coordinate value for each character as the position information for selection.

In the example shown in FIG. 34 (b), the pie chart and the pie constituting the pie chart are shown as the specific information. Center position coordinates of a circle are set as position information for selection of the entire pie chart, and centroid coordinates of 27%, 18%, and 55% of each pie (objects A, B, and C, respectively) A value is set as position information for selection of each object.

In the example shown in FIG. 34C, contours of illustrations of Kawama (object D) and bear (object E) are shown as specific information. The centroid coordinate value of each object is set as selection position information. Note that the centroid coordinate value to be set does not need to be calculated accurately, and may be an approximate value that can appropriately select an object.

As described above, when one large object is composed of a plurality of small objects, the user can give more specific information instructions by setting the position information for selection for each small object. .

FIG. 35 shows another example of position information for selection when text and phrases are specified information. FIG. 35A is an example in which the starting point coordinates and height of a word are used as selection position information, and FIG. 35B is an example in which the starting point coordinates, end point coordinates, and height of a word are used as selection position information. FIG. 35C shows an example in which the start point coordinates, end point coordinates, and height for each character are used as position information for selection when the characters constituting the phrase are separated.

In the example shown in FIG. 35A, the height hi of the phrase “Tokyo” and the coordinate value (xi, yi) of the midpoint of the left side (length hi) of the rectangular area occupied by the phrase “Tokyo” are The position information for selection is used.

In the example shown in FIG. 35 (b), the start point coordinates (xi1, yi1) and end point coordinates of a line segment at an arbitrary position in the height direction indicating the height hi of the phrase “Tokyo” and the width of the phrase “Tokyo”. (Xi2, yi2) is used as position information for selection.

In the example shown in FIG. 35C, since the characters constituting the phrase “Osaka” are separated from each other, for each of the characters “Dai” and “Osaka”, as in the example shown in FIG. The position information for selection is set by line segments indicating the height and width of the characters. That is, the character “Large” has the height hi1, the start point coordinates (xi11, yi11), and the end point coordinates (xi12, yi12) as position information for selection, and the character “Osaka” has the height hi2. It has start point coordinates (xi21, yi21) and end point coordinates (xi22, yi22) as selection position information.

Note that the position, height, and width of the area occupied by the text, the phrase, and the character can be known by using various selection position information shown in FIG. 35 for the text, the phrase, and the character. Mask areas for these texts, phrases, and characters in the coordinate system can be easily calculated.

Further, the selection position information may be composed of a set of all coordinate values (xia, yia) included in the mask area and a set of coordinate values (xie, yie) of the contour part of the mask area.

Even in the case where charts, illustrations, photographs, etc. are used as specific information, by defining the contours of these objects, a set of all coordinate values (xia, yia) included in the objects and the coordinate values of the contours of the objects A set of (xie, yie) may be used as position information for selection.

When the DTP data is map data, the area surrounded by the road, the feature area, the road center line, and / or the feature centroid are the position information for selection. In the map data, the specific information is classified into categories such as text, road, feature, and intersection. When selecting specific information, one of the categories is selected to recognize the specific information. Naturally, it may be selected from candidates for specific information, or may be touched after a category is selected in advance.
<About touch operation>

36 to 38, an operation performed using the scanner 104 when the user designates specific information on the printed material 101a will be described.

FIG. 36 shows an example of an operation in which the user touches one point of the specific information using the scanner 104. The coordinate value (Xt, Yt) of the touch position is recognized by the specific information selection unit 140 as a point coordinate.

FIG. 37 shows an example of an operation in which the user touches one point of the specific information and then traces on the specific information to be specified using the scanner 104. The coordinate information (Xt, Yt) of the locus traced by touching is recognized by the specific information selection unit 140 as a line segment.

Using this operation method, for example, when the entire phrase “Tokyo” 102a is traced, the phrase “Tokyo” 102a is recognized by the specific information selection unit 140 as the specific information to be input, whereas the character “East” ], Only the character “East” can be recognized by the specific information selection unit 140 as input specific information.

FIG. 38 shows an example of an operation of tracing and surrounding the specific information to be designated after the user touches one point of the specific information using the scanner 104. The coordinate information (Xt, Yt) of the locus traced by touching is recognized by the specific information selection unit 140 as a closed curve.

If a complete closed curve is not formed by the tracing operation, the specific information selection unit 140 may form a complete closed curve by correcting the start point and the end point of the trace traced by touching.

When this operation method is used, an accurate instruction of specific information is given to the information input device 105 so as to distinguish between the designation of the phrase “Tokyo” 102a and the designation of the character “East”, similarly to the touching and tracing operation. I can do it.

It should be noted that the enclosing operation here includes a case where the enclosing is completely included so that the specific information to be specified is included therein, and an enclosing operation where the traced trace is overlapped on the specific information. In the case where the traced trace is enclosed so as to overlap the specific information, at least the position represented by the selection position information may be enclosed. If at least the position represented by the position information for selection is not surrounded, the specific information corresponding to the position of the position information for selection closest to the plurality of coordinate values (xt, yt) constituting the trace is selected. Is done.
<How to select specific information>

39 to 41, a method for selecting specific information from the coordinate values (xt, yt) of the touch position and the position information for selection provided in each specific information will be described. In addition, since there are three types of touch operations by the user and three types of position information for selection, a method for selecting specific information will be described below for nine cases as combinations thereof.

Case 1: When the trajectory of the touch operation is a point and the position information for selection is the center point, the specific information selection unit 140 has the center coordinate (xi, yi) closest to the point of the coordinate value (xt, yt) of the touch position. Alternatively, specific information having centroid coordinates (xi, yi) is selected.

Case 2: When the locus of the touch operation is a point and the position information for selection is a line segment In FIG. 41, the user designates specific information by the touch operation, and the phrase “Tokyo” 102a is used as the position information for selection. An example of a method for selecting specific information when a line segment is present will be described.

FIG. 39 (a) shows an example in which the perpendicular foot drawn from the point of the coordinate value (xt, yt) of the touch position does not intersect the line segment. In this case, the distance d from the point of the coordinate value (xt, yt) to the line segment is the coordinate value (xt, yt) and the start point coordinate value (xi1, yi1) or end point coordinate value (xi2, yi2) of the line segment. The shorter of the distances.

FIG. 39 (b) shows an example in which a perpendicular foot drawn from the point of the coordinate value (xt, yt) of the touch position intersects the line segment. In this case, the distance d from the point of the coordinate value (xt, yt) to the line segment is the length of the perpendicular drawn from the point of the coordinate value (xt, yt) to the line segment.

The specific information selection unit 140 obtains the distance d between the line segment representing each specific information and the point of the coordinate value (xt, yt) of the touch position by the above method, and selects the specific information with the shortest distance d. .

Case 3: When the locus of the touch operation is a point and the position information for selection is a mask area In FIG. 40, the user designates specific information by the touch operation, and the phrase “Tokyo” 102a is used as the position information for selection. An example of a method for selecting specific information when the mask area 102b is provided will be described.

In this method, if the coordinate value (xt, yt) of the touch position matches any of the coordinate values (xia, yia) in the mask area, the specific information selection unit 140 specifies the specific information having the matching coordinate value. Select. If there is no coincident coordinate value, the specific information having the coordinate value (xie, yie) of the mask region outline portion closest to the coordinate value (xt, yt) of the touch position may be selected.

The selection method using this mask area can be used not only for text but also for objects such as charts, illustrations, and photographs that are specific information.

FIG. 40A shows an example in which the coordinate value (xt, yt) of the touch position is within the mask area 102b. In this case, the specific information selection unit 140 selects the phrase “Tokyo” 102a, which is specific information corresponding to the mask area 102b.

FIG. 40B shows a case where the coordinate value (xt, yt) of the touch position does not enter the mask area 102b of the phrase “Tokyo” 102a nor the mask area 102b ′ of the phrase “Osaka” 2a ′. It is an example. In this case, the specific information selection unit 140 obtains distances d1 and d2 from the coordinate values (xt, yt) of the touch position to the mask area 102b and the mask area 102b ', and compares these distances. Since the distance d1 is shorter, the specific information selection unit 140 selects the phrase “Tokyo” 102a, which is specific information corresponding to the mask area 102b.

Case 4: When the trajectory of the touch operation is a line segment and the position information for selection is the center point, the specific information selection unit 140, like the case 2, the center coordinates (xi, yi) or the centroid coordinates (xi, yi) ) To the specific information with the shortest distance d to the line segment of the trajectory.

Case 5: When the touch operation trajectory is a line segment and the selection position information is a line segment The specific information selection unit 140, for example, the start point coordinate value (xt1, yt1) of the line segment that is the trajectory and the end point coordinate value Select specific information that minimizes the area of the quadrangle formed by (xt2, yt2), the start point coordinate value (xi1, yi1) of the line segment that is the position information for selection, and the end point coordinate value (xi2, yi2) To do.

Case 6: When the locus of the touch operation is a line segment, and the position information for selection is a mask area The specific information selection unit 140 is the most of the plurality of coordinate values (xt, yt) constituting the line segment that is the locus. Specific information having a mask area including many coordinate values (xt, yt) is selected.

Case 7: When the locus of the touch operation is a closed curve and the position information for selection is the center point In FIG. 43, the user touches and traces to specify specific information, and the phrase “Tokyo” 102a is selected. An example of a method for selecting specific information when the center position value (xi, yi) is used as the position information for use will be described. In this case, the specific information selection unit 140 selects the specific information having the position information for selection surrounded by the trajectory input by the tracing operation.

Case 8: When the trajectory of the touch operation is a closed curve and the position information for selection is a line segment The specific information selection unit 140, as in Case 6, has a plurality of coordinate values ( xi, yi), the specific information having the largest number of coordinate values included in the closed curve as the locus is selected.

Case 9: When the locus of the touch operation is a closed curve and the position information for selection is a mask region The specific information selection unit 140 selects specific information that maximizes the area of overlap between the closed curve that is the locus and the mask region.
<Procedure for information input processing>

42, the procedure of the information input process performed by the information input device 105 will be described based on the flowchart shown in FIG. In the following, one dot code includes both an XY coordinate value and a code value, and the specific information in the DTP data 101b corresponding to the phrase “Tokyo” 102a on the printed material 101a includes selection position information. A case where the mask region 102b is set in advance will be described.

First, when the user performs an operation of specifying desired specific information using the scanner 104, the scanner 104 reads the dot pattern 103 printed at a location on the printed matter 101a touched by the user using the scanner 104. . The read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104. The scanner 104 sends the document ID portion of the converted dot code to the document management unit 120 and sends the XY coordinate value portion to the coordinate conversion unit 130 (step 10, hereinafter abbreviated as S10).

Next, using the document ID received from the scanner 104, the document management unit 120 selects DTP data 101b corresponding to the document ID from the DTP data stored in advance. The selected DTP data 101b is sent to the coordinate conversion unit 130 and the specific information selection unit 140 (S20).

Next, the coordinate conversion unit 130 receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104, and the coordinate value (xt, yt) in the xy coordinate system in the DTP data 101b received from the document management unit 120. (S30).

Next, the specific information selection unit 140 selects appropriate specific information from the specific information in the DTP data 101b received from the document management unit 120 based on the coordinate values (xt, yt) received from the coordinate conversion unit 130. (S40). The specific information selection unit 140 outputs the selected specific information so as to be input to another device or to be input for the next arbitrary process performed in the information input device 105.

43. The specific information selection process performed in S40 will be described in detail based on the flowchart shown in FIG.

First, the specific information selection unit 140 determines whether or not the locus of the coordinate value (xt, yt) of the touch position obtained by the user's touch operation is a point (S41a).

If it is a point, the process proceeds to S42a.

If not, the process proceeds to S41b.

Next, the specific information selection unit 140 determines whether or not the coordinate value (xt, yt) of the point matches any of the coordinate values (xia, yia) in the mask area 102b (S42a).

If they match, the process proceeds to S43a.

If they do not match, the process proceeds to S44a.

In S42a, when the coordinate value (xt, yt) of the point matches any of the coordinate values (xia, yia) in the mask area 102b, the specific information selection unit 140 determines that the coordinate value (xt, yt) of the point is The specific information having the matching coordinate value (xia, yia) is selected and the process is terminated (S43a).

In S42a, if the coordinate value (xt, yt) of the point does not match any of the coordinate values (xia, yia) in the mask area 102b, the specific information selection unit 140 sets the coordinate value (xt, yt) of the point. The closest specific information having the coordinate value (xie, yie) of the contour portion is selected, and the process ends (S44a).

In S41a, when the locus of the coordinate value (xt, yt) of the touch position is not a point, the specific information selecting unit 140 next selects the line of the locus of the coordinate value (xt, yt) of the touch position. Whether or not (S41b).

If it is a line segment, the process proceeds to S42b.

If it is not a line segment, the process proceeds to S41c.

In S41b, when the locus of the coordinate value (xt, yt) of the touch position is a line segment, the specific information selection unit 140 determines whether at least a part of the line segment is included in the mask region 102b. (S42b).

When at least a part of the line segment is included in the mask area 102b, the process proceeds to S43b.

If the line segment is not included in the mask area 102b, the process proceeds to S44b.

In S42b, when at least a part of the line segment is included in the mask region 102b, the specific information selection unit 140 selects specific information including the largest number of coordinate values (xt, yt) in the mask region, and performs processing. The process ends (S43b).

In S42b, when the line segment is not included in the mask area 102b, the specific information selection unit 140 selects specific information having the coordinate value (xie, yie) of the contour portion closest to the line segment, and ends the process. (S44b).

In S41b, when the locus of the coordinate value (xt, yt) of the touch position is not a line segment, the specific information selection unit 140 then selects the locus of the coordinate value (xt, yt) of the touch position as a closed curve. Whether or not (S41c).

If it is a closed curve, the process proceeds to S42c.

If it is not a closed curve, error processing is performed as necessary, and the process is terminated.

In S41c, when the locus of the coordinate value (xt, yt) of the touch position is a closed curve, the specific information selection unit 140 determines whether at least a part of the area surrounded by the closed curve overlaps with the mask area 102b. (S42c).

If they overlap, the process proceeds to S43c.

If they do not overlap, the process proceeds to S44c.

In S42c, when at least a part of the region surrounded by the closed curve overlaps with the mask region 102b, the specific information selection unit 140 displays the specific information corresponding to the mask region 102b having the largest overlap with the region surrounded by the closed curve. The process is terminated (S43c).

In S42c, when the region surrounded by the closed curve does not overlap any mask region 102b, the specific information selection unit 140 specifies specific information having the coordinate value (xie, yie) of the contour portion closest to the region surrounded by the closed curve. Is selected and the process is terminated (S44c).
[Second Embodiment]

One embodiment of the information input device according to the present invention that uses a medium on which a dot pattern is printed together with content and acquires DTP data from a server on the network will be described with reference to FIG. is there.

In the following, only portions different from the first embodiment will be described, and description of similar portions will be omitted.
<Configuration of Information Input Device 105a and Information Processing Input System>

44 shows a functional block diagram of the information input device 105a according to the present embodiment. The information input device 105a includes a scanner 104a, a coordinate conversion unit 130a, a specific information selection unit 140a, and a communication unit 150 (communication means).

In addition, the information input system according to the present embodiment includes an information input device 105a, a document management server 113 (two-dimensional format information server), and a network connecting them.

In the information input device 105a, the scanner 104a reads the dot pattern 103 at the touch position, sends the document ID portion of the dot code to the communication unit 150, and sends the XY coordinate value portion of the dot code to the coordinate conversion unit 130a.

The communication unit 150 transmits the document ID received from the scanner 104a to the document management server 113, and receives the DTP data 101b corresponding to the document ID received from the document management server 113 as the coordinate conversion unit 130a and the specific information selection unit 140a. Send to.

The specific information selection unit 140a selects specific information in the DTP data 101b using the xy coordinate values received from the coordinate conversion unit 130a and the DTP data 101b received from the communication unit 150, and outputs the selected specific information To do.

The document management server 113 includes a document management unit 120a (two-dimensional format information storage selection unit) and a communication unit 151 (communication unit). The function of the document management unit 120a is the same as that of the document management unit 120 in the first embodiment.

In the configuration described above, the document management unit 120a is provided only on the document management server 113. However, as in the first embodiment, the document management unit 120 may also be provided in the information input device 105a. Good.

In this case, if the DTP data in the document management unit 120 is searched first and the necessary DTP data 101b is searched, there is no need to access the document management server 113 on the network again, thus reducing the traffic on the network. In addition, the processing speed can be increased. Further, the document management unit 120 may cache the DTP data 101b acquired from the document management unit 120a.
<Procedure for information input processing>

The procedure of information input processing performed by the information input device 105a will be described. The processes in steps S10a to S40a in the following description correspond to S10 to S40 in the flowchart shown in FIG. Hereinafter, a case where one dot code includes both an XY coordinate value and a code value will be described.

First, when the user performs an operation of designating desired specific information using the scanner 104a, the scanner 104a reads the dot pattern 103 printed at a location on the printed matter 101a touched by the user using the scanner 104a. . The read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104a. The scanner 104 sends the document ID part of the converted dot code to the document management part 120a on the document management server via the

communication parts

150 and 151, and sends the XY coordinate value part to the coordinate conversion part 130 (S10a). ).

Next, the document management unit 120a uses the document ID received from the scanner 104a to select DTP data 101b corresponding to the document ID from the DTP data stored in advance. The selected DTP data 101b is sent to the coordinate conversion unit 130a and the specific information selection unit 140a via the communication units 151 and 50 (S20a).

Next, the coordinate conversion unit 130a receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104a, and the coordinate value (xt, yt) in the xy coordinate system in the DTP data 101b received from the document management unit 120a. (S30a).

Next, the specific information selection unit 140a selects appropriate specific information from the specific information in the DTP data 101b received from the document management unit 120a based on the coordinate values (xt, yt) received from the coordinate conversion unit 130a. (S40a). The specific information selection unit 140a outputs the selected specific information so as to be input to another device or to be input for the next arbitrary process performed in the information input device 105a.
<About content and dot pattern printing>

The information input device 105 a may further include a printing unit (not shown) (printing unit) so that the newly acquired DTP data 101 b can be printed on the medium together with the dot pattern 103. The dot code of the dot pattern 103 desirably includes a document ID for identifying the newly acquired DTP data 101b in addition to the XY coordinate values.

According to this configuration, the printed material 101a used for input processing can be created from the newly acquired DTP data 101b.
[Third Embodiment]

Of the information processing apparatus according to the present invention, using a medium on which a dot pattern is printed together with content, obtaining DTP data from a server on the network, inputting specific information, and searching for related information of the specific information One embodiment of the information processing apparatus to be displayed will be described below with reference to FIGS.

In the following, only portions different from the first and second embodiments will be described, and description of similar portions will be omitted.
<Configuration of Information Processing Device 112 and Information Processing System>

45 shows a functional block diagram of the information processing apparatus 112 according to the present embodiment. The information processing apparatus 112 includes a scanner 104a, a coordinate conversion unit 130a, a specific information selection unit 140b, a communication unit 150b, a related information search unit 160 (related information search unit), and a display unit 170. A specific example of the information processing apparatus 112 is a personal computer, for example.

The information processing device 112 is a PDA, a television receiver, a front projector, a rear projector, a game device, a karaoke device, a mobile phone terminal device, a POS terminal device, an ATM, a KIOSK terminal, a car navigation system, a pachinko machine, a watch, a smartphone, or the like. May be.

In addition, the information processing system according to the present embodiment includes an information processing apparatus 112, a document management server 113, a dictionary server 114, and a network connecting them.

In the information processing apparatus 112, the scanner 104a reads the dot pattern 103 at the touch position, sends the document ID portion of the dot code to the communication unit 150b, and sends the XY coordinate value portion of the dot code to the coordinate conversion unit 130a.

The communication unit 150b transmits the document ID received from the scanner 104a to the document management server 113, and receives the DTP data 101b corresponding to the document ID received from the document management server 113 as the coordinate conversion unit 130a and the specific information selection unit 140b. Send to. Further, the communication unit 150b transmits the specific information received from the related information search unit 160 to the dictionary server 114, and the related information of the specific information searched in the dictionary server 114 based on the transmitted specific information. And pass it to the related information search unit 160.

The specific information selection unit 140b selects specific information in the DTP data 101b using the xy coordinate values received from the coordinate conversion unit 130a and the DTP data 101b received from the communication unit 150b, and associates the selected specific information with the related information. The information is output to the information search unit 160.

The related information search unit 160 sends the specific information received from the specific information selection unit 140b to the dictionary server 114 via the communication unit 150b, receives the related information as a search result from the dictionary server 114 via the communication unit 150b, The received related information is output to the display unit 170.

In the present embodiment, the related information search unit 160 is configured to search one or more dictionary servers 114 on the network. However, as a function to replace the dictionary server 114 on the network, a preset link table is set. A database and / or an electronic dictionary may be provided in the information processing apparatus 112, and the related information search unit 160 may search related information without accessing the network.

Note that examples of the dictionary server 114 include a general-purpose link table, a general-purpose database, and a general-purpose electronic dictionary.

The display unit 170 presents the related information received from the related information search unit 160 to the user using a display or the like.

In the above configuration, the related information search unit 160 provided in the information processing apparatus 112 communicates with the dictionary server 114 to acquire the related information, but instead of this configuration, the document management server 113 The related information search unit 160 may be provided. In this case, the specific information selection unit 140b transmits the selected specific information to the related information search unit 160 on the document management server 113 via the communication unit. The related information search unit 160 communicates with the dictionary server 114, acquires related information of the selected specific information, and transmits it to the information processing apparatus 112.

The dictionary server 114 stores explanations corresponding to data files indicating text, illustrations, photographs, and the like, URLs, and related information such as multimedia information. The dictionary server 114 searches for data files indicating related information such as text, illustrations, and photographs that are the selected specific information. The dictionary server 114 may be a search engine such as Wikipedia, Yahoo, Google, or a specialized database for membership.
<Information processing procedure>

The information processing procedure performed by the information processing apparatus 112 will be described with reference to the flowchart shown in FIG. In the following, one dot code includes both an XY coordinate value and a code value, and the specific information in the DTP data 101b corresponding to the phrase “Tokyo” 102a on the printed material 101a includes selection position information. A case where the center coordinate value (xi, yi) is set in advance will be described.

First, when the user performs an operation of designating desired specific information using the scanner 104a, the scanner 104a reads the dot pattern 103 printed at a location on the printed matter 101a touched by the user using the scanner 104a. . The read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104a. The scanner 104a sends the document ID part of the converted dot code to the document management part 120a on the document management server 113 via the

communication parts

150b and 151, and sends the XY coordinate value part to the coordinate conversion part 130 ( S10a).

Next, the document management unit 120a uses the document ID received from the scanner 104a to select DTP data 101b corresponding to the document ID from the DTP data stored in advance. The selected DTP data 101b is sent to the coordinate conversion unit 130a and the specific information selection unit 140b via the

communication units

151 and 150b (S20a).

Next, the coordinate conversion unit 130a receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104a, and the coordinate value (xt, yt) in the xy coordinate system in the DTP data 101b received from the document management unit 120a. To the specific information selection unit 140b (S30a).

Next, the specific information selection unit 140b selects appropriate specific information from the specific information in the DTP data 101b received from the document management unit 120a based on the coordinate values (xt, yt) received from the coordinate conversion unit 130a. To do. The specific information selection unit 140b passes the selected specific information to the related information search unit 160 (S40a).

For example, from the coordinate value (xt, yt) of the touch position rather than the distance from the coordinate value (xt, yt) of the touch position to the center coordinate value (xi, yi) of other specific information in the DTP data 101b, Since the distance to the central coordinate value (xi, yi) of the phrase “Tokyo” 102 a is shorter, the phrase “Tokyo” 102 a is selected as an input to the information processing apparatus 112.

Next, the related information search unit 160 transmits the specific information received from the specific information selection unit 140b to the dictionary server 114 via the communication unit 150b, and the related information of the specific information searched in the dictionary server is Received from the dictionary server 114 via the communication unit 150b. Then, the related information search unit 160 presents the received related information to the user via the display unit 170 (S50a).

In the example shown in FIG. 47, the phrase “Tokyo” 102a is transmitted to the dictionary server 114 via the network. On the dictionary server 114, an explanatory sentence “What is Tokyo?” Corresponding to the headword “Tokyo” is searched and returned to the information processing apparatus 112. On the screen of the information processing apparatus 112, an explanatory note “What is Tokyo?”, Which is related information corresponding to the phrase “Tokyo” 102a, is displayed on the screen.

Note that the related information may be a data file indicating an illustration or a photograph in addition to the text.
[Fourth Embodiment]

One embodiment of the information processing apparatus according to the present invention that uses a display on which a dot pattern 103 is displayed together with content will be described with reference to FIGS. 48 to 52 as follows.

Note that the printed material 101a is used in the information input device 105 in the first embodiment, the information input device 105a in the second embodiment, and the information processing device 112 in the third embodiment. In the present embodiment, in describing an example in which a display is used instead of the printed material 101a, the information processing apparatus 112b will be described as an example of a modification of the information processing apparatus 112. Since the method of transforming the information input device 105 and the information input device 105a into a configuration using a display instead of the printed material 101a is the same, the description is omitted.

Moreover, below, only a different location from 3rd Embodiment is demonstrated, and description is abbreviate | omitted regarding the same location.
<Overview>

FIG. 48 illustrates an outline of the present embodiment.

As shown in FIG. 48, first, the user downloads appropriate content from the Web server 113b (two-dimensional format information server) and displays it on the screen 15 of the information processing apparatus 112b.

Next, the user uses the scanner 104b to touch information to be searched using the information processing apparatus 112b on the display 115 on which the dot pattern 103 is displayed together with the content.

In this example, the user wants to search related information related to the phrase “Tokyo” 102a among the contents displayed on the display 115, and therefore touches the phrase “Tokyo” 102a displayed on the display 115 with the scanner 104b. ing.

In the dot pattern 103, the XY coordinate value of the display 115 is coded in advance, and when the user touches the display 115 with the scanner 104b, the coordinate value (Xt, Yt) in the XY coordinate system on the display 115 at the touch position. ) Is read into the information processing apparatus 112b.

The information processing apparatus 112b recognizes the specific information at the touch position using the read XY coordinate value and the content data 1d (two-dimensional format information) of the content downloaded and displayed from the Web server 113b.

As shown in the figure, the content data in the content file has its own xy coordinate system. For specific information such as text, charts, illustrations, and photographs included in the content data, selection position information used to select them is set.

For example, when the coordinate value (xt, yt) of the touch position is included in the mask area 102b, the phrase “Tokyo” 102a is selected from a lot of specific information included in the content data 1d.

The information processing apparatus 112b searches the dictionary server 114 using the selected phrase “Tokyo” 102a. The related information that is the search result is returned from the dictionary server 114 to the information processing apparatus 112 b and displayed on the display 115.

In the example shown in FIG. 50, an explanatory note “What is Tokyo?”, Which is related information of the phrase “Tokyo” 102a, is displayed on the display 115 as a search result.

As described above, even specific information in which no hyperlink or the like is set in the content can be input to the information processing apparatus 112b by the user touching and instructing with the scanner 104b. In addition, related information of the input specific information can be searched and displayed.
<About dot code format>

The dot code in this embodiment may include only XY coordinate values. The content ID corresponds to the document ID in the above embodiment.

In the case of the information processing apparatus 112b having a plurality of displays 115, a dot code including a display ID can also be used as a code value in order to distinguish the displays 115.
<About the structure of the display>

The display 115 may display the dots of the dot pattern 103 by adjusting the RGB values using a general display, or display a dot portion representing the dot pattern 103 separately from the pixels of the display 115. 115 may be arranged on 115.

In the description of the present embodiment and the like, the expression “display” is used for dots and the like. However, as described above, this expression includes “a dot portion arranged on the display 115”. The case is also included.

49 shows a cross-sectional view of an organic EL display in which a dot portion is provided separately from a pixel. FIG. 49A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed, and FIG. 49B is a diagram in which infrared rays are incident on light emitting elements other than the dot portion and diffusely reflected. It is a figure which shows a mode.

Thus, the dot pattern can be expressed by making the infrared reflection characteristics different between the light emitting element portion and the dot portion.

FIG. 50 shows an example in which a dot portion is provided on the screen of a front projector or a rear projector as a modification of the display 115. FIG. 50A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed, and FIG. 50B is a diagram in which infrared rays are incident on a screen portion other than the dot portion and diffusely reflected. It is a figure which shows a mode.
<About content data>

Examples of content data include data created by HTML, XML, flash, or the like. Of course, the DTP data exemplified in the above embodiment can also be handled as content data.
<Configuration of Information Processing Device 112b and Information Processing System>

FIG. 51 shows a functional block diagram of the information processing apparatus 112b according to the present embodiment. The information processing apparatus 112b includes a scanner 104b, a coordinate conversion unit 130b, a specific information selection unit 140c, a communication unit 150c, a related information search unit 160, a display unit 170b, a display 115, and a content acquisition unit 180. I have. A specific example of the information processing apparatus 112b is a personal computer, for example.

In addition, the information processing system according to the present embodiment includes an information processing apparatus 112b, a Web server 113b, a dictionary server 114, and a network connecting them.

In the information processing apparatus 112b, the content acquisition unit 180 transmits a content request to the Web server 113b via the communication unit 150c, and receives and receives the content data 1d returned from the Web server 113b via the communication unit 150c. The content data 1d is presented to the user via the display unit 170b and the display 115.

The scanner 104b reads the dot pattern 103 at the touch position and sends the XY coordinate value of the dot code to the coordinate conversion unit 130b.

The coordinate conversion unit 130b uses the XY coordinate values (Xt, Yt) received from the scanner 104b and the coordinate values (Xt, Yt) of the touch position in the xy coordinate system on the content data 1d using the content data 1d received from the content acquisition unit 180. xt, yt), and the converted coordinate value (xt, yt) is sent to the specific information selection unit 140c.

The specific information selection unit 140c selects the specific information in the content data 1d using the xy coordinate value received from the coordinate conversion unit 130b and the content data 1d received from the content acquisition unit 180, and selects the selected specific information. The information is output to the related information search unit 160.

The related information search unit 160 sends the specific information received from the specific information selection unit 140c to the dictionary server 114 via the communication unit 150c, receives the related information as a search result from the dictionary server 114 via the communication unit 150c, The received related information is presented to the user via the display unit 170b and the display 115.

The communication unit 150c transmits the content request received from the content acquisition unit 180 to the Web server 113b, and displays the content data 1d corresponding to the content request received from the Web server 113b, the display unit 170b, the coordinate conversion unit 130b, and the identification The information is sent to the information selection unit 140c. In addition, the communication unit 150c transmits the specific information received from the related information search unit 160 to the dictionary server 114, and the related information of the specific information searched in the dictionary server 114 based on the transmitted specific information. And pass it to the related information search unit 160.

The display unit 170b presents the content data 1d received from the content acquisition unit 180 and the related information received from the related information search unit 160 to the user via the display 115.

In this configuration example, since a dot portion that absorbs infrared rays is provided on the screen of the display 115, a functional block for displaying the dot pattern 103 by adjusting RGB values is not provided. .
<Information processing procedure>

The procedure of information processing performed by the information processing apparatus 112b will be described based on the flowchart shown in FIG. In the following, the center coordinate value (xi, yi) is set in advance as the position information for selection in the specific information in the content data 1d corresponding to the phrase “Tokyo” 102a displayed on the display 115. Will be described.

First, the content acquisition unit 180 transmits a content request to the Web server 113b via the communication unit 150c, and receives the content data 1d corresponding to the content request from the Web server 113b via the communication unit 150c. The content acquisition unit 180 displays the received content data 1d on the display 115 via the display unit 170b (S5). The content acquisition unit 180 also outputs the content data 101d of the content being displayed to the coordinate conversion unit 130b and the specific information selection unit 140c.

Specifically, the process of S5 represents a state where the user is browsing the content data downloaded from the Web server 113b by surfing the Internet using a general Web browser.

Next, when the user performs an operation of designating desired specific information using the scanner 104b, the scanner 104b displays the dot pattern 103 set at a location on the display 115 touched by the user using the scanner 104b. read. The read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104b. The scanner 104b sends the XY coordinate value of the converted dot code to the coordinate conversion unit 130b (S10b).

Next, the coordinate conversion unit 130b receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104b, and the coordinate value (xt, yt) in the xy coordinate system in the content data 1d received from the content acquisition unit 180. To the specific information selection unit 140c (S30a).

Next, the specific information selection unit 140c selects appropriate specific information from the specific information in the content data 1d received from the content acquisition unit 180 based on the coordinate values (xt, yt) received from the coordinate conversion unit 130b. To do. The specific information selection unit 140c passes the selected specific information to the related information search unit 160 (S40b).

For example, from the coordinate value (xt, yt) of the touch position rather than the distance from the coordinate value (xt, yt) of the touch position to the center coordinate value (xi, yi) of other specific information in the content data 1d, Since the distance to the central coordinate value (xi, yi) of the phrase “Tokyo” 102a is shorter, the phrase “Tokyo” 102a is selected as an input to the information processing apparatus 112b.

Next, the related information search unit 160 transmits the specific information received from the specific information selection unit 140c to the dictionary server 114 via the communication unit 150c, and the related information of the specific information searched in the dictionary server is Received from the dictionary server 114 via the communication unit 150c. Then, the related information search unit 160 presents the received related information to the user via the display unit 170b and the display 115 (S50a).

In the example shown in FIG. 48, the phrase “Tokyo” 102a is transmitted to the dictionary server 114 via the network. On the dictionary server 114, an explanatory sentence “What is Tokyo?” Corresponding to the headword “Tokyo” is searched and returned to the information processing apparatus 112 b. On the screen of the information processing apparatus 112b, an explanatory note “What is Tokyo” which is related information corresponding to the phrase “Tokyo” 102a is displayed on the screen.
[Fifth Embodiment]

One embodiment of the information processing apparatus according to the present invention, in which a transparent sheet on which a dot pattern 103 is printed, is placed on a printing surface of a normal printed material and used is described below with reference to FIG. is there. Hereinafter, this transparent sheet is referred to as a grid sheet 101g (information input auxiliary sheet).

In the following description, a medium on which a dot pattern is printed together with content is used, DTP data is acquired from a server on the network, specific information is input, related information of the specific information is searched and displayed. Only the differences from the third embodiment will be described.
<Overview>

FIG. 53 shows an outline of the present embodiment.

First, the user inputs an index for specifying a normal printed matter 101e on which the dot pattern 103 is not printed, to the information processing apparatus 112c. The input method may be a method in which the user touches an icon (described later) printed on the grid sheet 101g to input an index for specifying the printed material 101e, or a method in which the user inputs an index using a keyboard or the like.

The index may be converted into a document ID in the document management server 113, and the DTP data on the document management server 113 may be searched using the converted document ID.

Next, the user accurately overlays the grid sheet 101g on the printing surface of the printed material 101e.

Next, using the scanner 104a, the user touches the phrase “Tokyo” 102a, which is specific information on the print surface that the user wants to search for related information, through the grid sheet 101g.

Next, the scanner 104a reads the coordinate values (Xt, Yt) in the XY coordinate system on the grid sheet 101g at the touch position.

The process after the index and the coordinate value (Xt, Yt) of the touch position are input to the information processing apparatus 112c is the same as the process in the third embodiment.

In this way, by covering the printed material 101e with the grid sheet 101g and touching the grid sheet 101g with the scanner 104a, detailed descriptions (related information) such as text, illustrations, and photos on the printed material 101e can be displayed on the screen. I can do it.

In the first to third embodiments, text, illustrations, photographs, and the like and the dot pattern 103 are superimposed and printed on the printed material 101 a, and the user touches the printed material 101 a with the scanner 104 and corresponds to the touched specific information. Information was displayed.

However, with this method, it is not possible to use a normal printed material 101e on which the dot pattern 103 is not printed for inputting the specific information. In the present embodiment, since the printed sheet 101e is used while being covered with the grid sheet 101g, the existing printed object 101e can be targeted. In addition, the grid sheet 101g can be used in common for a plurality of printed materials 101e, so that convenience is enhanced.
[Sixth Embodiment]

One embodiment of the information processing apparatus according to the present invention in which the grid sheet 101g on which the dot pattern 103 is printed is arranged and used over a normal display screen will be described with reference to FIG. .

In the following description, only different points from the fourth embodiment using a display on which a dot pattern is displayed together with contents will be described.
<Overview>

FIG. 54 shows an outline of the present embodiment.

First, the user places the grid sheet 101g on the display screen of the information processing apparatus 112d. The placement may be performed by sticking the grid sheet 101g to the display. It is desirable that a calibration operation is performed after the grid sheet 101g is arranged.

Next, the user downloads a desired content file from the Web server 113b to the information processing apparatus 112d using a Web browser or the like. The content in the downloaded content file is displayed on the screen of the information processing apparatus 112d.

Next, the user touches the phrase “Tokyo” 102a, which is specific information for searching related information on the screen, using the scanner 104a through the grid sheet 101g arranged on the display screen.

The information processing apparatus 112d recognizes the specific information at the touch position using the read XY coordinate value and the content data 1d of the content downloaded and displayed from the Web server 113b.

The processing after recognizing the specific information is the same as the processing in the fourth embodiment.

As described above, by covering the display screen with the grid sheet 101g and touching the grid sheet 101g with the scanner 104a, it is possible to display detailed descriptions (related information) such as text, illustrations, and photos on the display screen. I can do it.
[Seventh Embodiment]

An embodiment using a television receiver having a display on which a dot pattern 103 is displayed together with content in the information processing apparatus according to the present invention will be described with reference to FIGS. 55 to 57 as follows. .

Note that the printed material 101a is used in the information input device 105 in the first embodiment, the information input device 105a in the second embodiment, and the information processing device 112 in the third embodiment. In this embodiment, in describing an example in which a display of a television receiver is used instead of the printed material 101a, the television receiver 112e will be described as an example in which the information processing apparatus 112 is modified. Since the method of transforming the information input device 105 and the information input device 105a into a configuration using a display instead of the printed material 101a is the same, the description is omitted.

Moreover, below, only a different location from 4th Embodiment is demonstrated, and description is abbreviate | omitted regarding the same location.
<Overview>

FIG. 55 shows an overview of the present embodiment.

As shown in FIG. 55, the user first receives a broadcast radio wave in which the dot pattern 103 is superimposed on a normal image using the television receiver 112e, and displays the received image on the display 115. Then, the user uses the scanner 104c to touch information to be input to the television receiver 112e on the display 115 on which the dot pattern 103 is displayed together with the received image.

In this example, the user wants to search related information related to the phrase “Tokyo” 102a in the received image displayed on the display 115, and therefore touches the phrase “Tokyo” 102a displayed on the display 115 with the scanner 104c. is doing.

In the dot pattern 103, the XY coordinate value of the display 115 and the content ID for specifying the received image are coded. When the user touches the display 115 with the scanner 104c, the XY coordinate on the display 115 at the touch position is displayed. The coordinate value (Xt, Yt) and content ID in the system are read into the television receiver 12e.

The television receiver 112e uses the read content ID, and the content corresponding to the content ID among the content data in which the specific information and the position information for selection are set in advance from the content server 113c (two-dimensional format information server). Download data 1d. Then, the specific information at the touch position is selected using the read XY coordinate values and the downloaded content data 1d.

The television receiver 12e searches the dictionary server 114 using the selected phrase “Tokyo” 102a. Related information as a search result is returned from the dictionary server 114 to the television receiver 12e and displayed on the display 115.

In the example shown in FIG. 55, an explanatory note “What is Tokyo?”, Which is related information of the phrase “Tokyo” 102a, is displayed on the display 115 as a search result.

In this way, even in the information received from the broadcast wave, even specific information for which a hyperlink or the like is not set can be input to the television receiver 12e by the user touching and instructing with the scanner 104c. . In addition, related information of the input specific information can be searched and displayed.
<About dot code format>

In the above description, the dot code of the dot pattern 103 in the present embodiment includes the content ID for identifying the content received by the television receiver 112e as a broadcast wave and the XY coordinate value. In addition to this, a dot code and a dot pattern 103 are generated from an XY coordinate value generated by the television receiver 112e itself and a content ID acquired from EPG (Electronic Program Guide) data obtained by the television receiver 112e, and the display 115 may be displayed on the screen 115.

Further, the dot code of the dot pattern 103 displayed on the display 115 may include only XY coordinate values. In that case, the dot pattern 103 may have a configuration in which the dot portion of the dot pattern 103 is fixedly provided in advance alongside the pixels on the display 115 instead of the configuration represented by RGB pixels.

In the configuration in which the dot code of the dot pattern 103 displayed on the display 115 does not include the content ID, the content ID is a corresponding portion of a print medium such as a television guide in which the content ID is printed as the dot pattern 103. It may be configured to be read by touching, or may be configured to be input manually by a user using a paper keyboard and the scanner 104c.

The content ID displayed on the display 115 may be displayed together with the XY coordinate value of the dot pattern 103 in at least a part of the region where the XY coordinate value of the dot pattern 103 is displayed, or is different from the region. It may be displayed in the area.

Further, the content ID displayed on the display 115 is the XY coordinate value of the dot pattern 103 that uniquely identifies the content data. This XY coordinate value is an area where the XY coordinate value of the dot pattern 103 is displayed. The configuration may be displayed entirely.
<About content data>

Examples of content data include moving image data such as MPEG, and data created by HTML, XML, flash, and the like. Of course, the DTP data exemplified in the above embodiment can also be handled as content data.

The content data basically includes two-dimensional format information, but in addition to this, MPEG data or the like may have three-dimensional format information including a time axis. Further, if it is three-dimensional stereoscopic image data, it may have three-dimensional format information including the Z axis in the front-rear direction toward the display 115. Furthermore, as long as it is moving image data of a three-dimensional stereoscopic image, it may have four-dimensional format information having an XYZ axis and a time axis.
<Configuration of Television Receiver 112e and Information Processing System>

FIG. 56 shows a functional block diagram of the television receiver 112e according to the present embodiment. The television receiver 112e includes a scanner 104c, a coordinate conversion unit 130c, a specific information selection unit 140d, a communication unit 150d, a related information search unit 160, a display unit 170c, a display 115, and a television image reception unit 190. I have.

In addition, the information processing system according to this embodiment includes a television receiver 112e, a content server 113c, a dictionary server 114, and a network connecting them.

In the television receiver 112e, the television receiver 190 receives broadcast waves. A dot pattern 103 in which a content ID included in a broadcast wave or obtained from an EPG and an XY coordinate value on a display screen is combined is added to the received image, and the user is notified via the display unit 170c and the display 115. Present. Note that the dot pattern 103 may be added to the received image only when the user uses the scanner 104c.

The scanner 104c reads the dot pattern 103 at the touch position, sends the XY coordinate value of the dot code to the coordinate conversion unit 130c, and sends the content ID of the dot code to the content server 113c via the communication unit 150d. The content ID may be directly sent from the television receiver 190 to the content server 113c when the scanner 104c reads the XY coordinate values instead of being converted into the dot pattern 103 and read by the scanner 104c.

The coordinate conversion unit 130c uses the XY coordinate values (Xt, Yt) of the touch position received from the scanner 104c and the xy coordinates on the content data 1d using the content data 1d received from the content server 113c via the communication unit 150d. The coordinate value (xt, yt) of the touch position in the system is converted, and the converted coordinate value (xt, yt) is sent to the specific information selection unit 140d.

The specific information selection unit 140d selects specific information in the content data 1d using the xy coordinate values received from the coordinate conversion unit 130c and the content data 1d received from the content server 113c via the communication unit 150d, The selected specific information is output to the related information search unit 160.

The related information search unit 160 sends the specific information received from the specific information selection unit 140d to the dictionary server 114 via the communication unit 150d, receives related information as a search result from the dictionary server 114 via the communication unit 150d, The received related information is presented to the user via the display unit 170c and the display 115.

The communication unit 150d transmits the content ID received from the scanner 104c to the content server 113c, and transmits the content data 1d corresponding to the content ID received from the content server 113c to the coordinate conversion unit 130c and the specific information selection unit 140d. . Further, the communication unit 150d transmits the specific information received from the related information search unit 160 to the dictionary server 114, and the related information of the specific information searched in the dictionary server 114 based on the transmitted specific information. And pass it to the related information search unit 160.

The display unit 170c presents the received image to which the dot pattern 103 is added received from the television receiver 190 and the related information received from the related information search unit 160 to the user via the display 115.
<Information processing procedure>

The information processing procedure performed by the television receiver 112e will be described based on the flowchart shown in FIG. In the following, the center coordinate value (xi, yi) is set in advance as the position information for selection in the specific information in the content data 1d corresponding to the phrase “Tokyo” 102a displayed on the display 115. Will be described.

First, the television receiver 190 receives a broadcast wave, and superimposes the dot pattern 103 including the content ID acquired from the broadcast wave or the Internet and the XY coordinate value on the screen of the display 115 on the received image to display the display unit. The image is displayed on the display 115 via 170c (S5b).

Next, when the user performs an operation of designating desired specific information using the scanner 104c, the scanner 104c displays the dot pattern 103 set at a location on the display 115 touched by the user using the scanner 104c. read. The read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104c. The scanner 104c transmits the XY coordinate value of the converted dot code to the coordinate conversion unit 130c, and transmits the content ID of the converted dot code to the content server 113c via the communication unit 150d (S10c).

Next, the content server 113c uses the content ID received from the scanner 104c to select content data 1d corresponding to the content ID from the content data stored in advance. Then, the selected content data 1d is sent to the coordinate conversion unit 130c and the specific information selection unit 140d via the communication unit 150d (S20b).

Next, the coordinate conversion unit 130c converts the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104c to the coordinate value (xt, yt) in the xy coordinate system in the content data 1d acquired from the content server 113c. The data is converted and passed to the specific information selection unit 140d (S30a).

Next, the specific information selection unit 140d selects appropriate specific information from the specific information in the content data 1d received from the content server 113c based on the coordinate values (xt, yt) received from the coordinate conversion unit 130c. . The specific information selection unit 140d passes the selected specific information to the related information search unit 160 (S40b).

For example, from the coordinate value (xt, yt) of the touch position rather than the distance from the coordinate value (xt, yt) of the touch position to the center coordinate value (xi, yi) of other specific information in the content data 1d, Since the distance to the center coordinate value (xi, yi) of the phrase “Tokyo” 102a is shorter, the phrase “Tokyo” 102a is selected as an input to the television receiver 112e.

Next, the related information search unit 160 transmits the specific information received from the specific information selection unit 140d to the dictionary server 114 via the communication unit 150d, and the related information of the specific information searched in the dictionary server 114 is obtained. And received from the dictionary server 114 via the communication unit 150d. Then, the related information search unit 160 presents the received related information to the user via the display unit 170c and the display 115 (S50a).

In the example shown in FIG. 55, the phrase “Tokyo” 102a is transmitted to the dictionary server 114 via the network. On the dictionary server 114, an explanatory sentence “What is Tokyo” corresponding to the headword “Tokyo” is searched and returned to the television receiver 112 e. On the screen of the television receiver 112e, an explanatory note “What is Tokyo” which is related information corresponding to the phrase “Tokyo” 102a is displayed on the screen.

In the above description, the content data is analyzed in advance and the specific information and the position information for selection are already set. However, the present invention is not limited to this configuration. When the user touches the dot pattern 103 on the display 115 with the scanner 104c, the specific information selection unit 140d is a text, figure, illustration, photograph, etc. within a predetermined distance from the coordinate value (xt, yt) of the touch position for the first time. The content data may be analyzed to recognize specific information and set position information for selection.

In the present embodiment, the television receiver 112e is described as an example of the information processing apparatus. However, the television receiver is a normal one, and is combined with a set-top box to be additionally installed. The function of the machine 112e may be realized.
[Additional notes]

In the above embodiment, in order to indicate the position, the user touches the medium on which the dot pattern is printed, the display on which the dot pattern is displayed or provided, or the grid sheet on which the dot pattern is provided with the scanner. However, the input method of the coordinate position is not limited to these, and a method of touching a normal touch panel or touch screen provided on a normal display with a finger or a method using a normal mouse operation may be used.

The information input device uses the coordinate value (xt, yt) of the touch position to search for the specific information, based on the selection position information setting method according to any one of the above, One or two or more information search methods may be assigned, and one of the search methods may be selected by a predetermined algorithm to search for the specific information.

The information input device selects one or more specific information from a plurality of the specific information searched within an arbitrary distance from the coordinate value (xt, yt), and stores related information of the specific information in advance. A search may be performed from a set link table, database, or electronic dictionary, and the related information may be output.

In the information input device, the coordinate value (xt, yt) of the touch position among all the specific information searched within an arbitrary distance from the coordinate value (xt, yt) is the total coordinate value in the mask. When at least one of the specific information retrieved by being identical to any coordinate value of (xia, yia) is included, the position information for selecting the specific information is used as the centroid of the mask, and the touch position The specific information having the shortest distance from the coordinate value (xt, yt) of the specific information to the position information for selection of the plurality of searched specific information is searched, or candidates for the specific information are searched in order of the shortest distance The selection may be a feature.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

Finally, each block of the

information input devices

105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e, particularly the coordinate conversion units 130 to 130c, the specific information selection units 140 to 140d, and the related information search unit 160, You may comprise by a hardware logic and may implement | achieve by software using CPU as follows.

In other words, the

information input devices

105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e each have a CPU (central processing unit) that executes instructions of a control program that realizes each function, and a ROM (read that reads the program). only (memory), RAM (random access memory) for expanding the program, and a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to provide program codes (execution format program, intermediate code program) of control programs for the

information input devices

105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e, which are software for realizing the functions described above. , A source program) recorded on a computer readable medium is supplied to the

information input devices

105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e, and the computer (or CPU or MPU) records them. This can also be achieved by reading and executing the program code recorded on the medium.

Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

The

information input devices

105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

(C)

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Description of overall system configuration>
FIG. 60 is an external view showing an example of a usage state of the handwriting input / output system 201 according to the present invention. According to the drawing, when writing is performed on the medium 202 using the scanner 203, the document as written is displayed on the display 206. This is realized by the scanner 203 taking an image of the dot pattern on the written locus and analyzing the computer 204 (accessing the server 205 as necessary). However, the computer 204 may be built in the scanner 203.

FIG. 61 is a block diagram showing the configuration of the first embodiment of the handwriting input / output system 201 according to the present invention.

The handwriting input / output system 201 includes at least a medium 202 (handwriting input sheet), an imaging unit 207 (imaging unit), a dot pattern analysis unit 208 (dot pattern analysis unit), a locus recognition unit 209 (trajectory recognition unit), and a processing instruction unit. 210 (processing instruction means) provides various instructions to the information processing unit 211 and accesses the various servers 205 as necessary.

FIG. 62 is a block diagram showing a configuration of the second embodiment of the handwriting output / input system 1 according to the present invention. In the second embodiment, in addition to the configuration provided in the first embodiment, an imaging device 212, an irradiation unit 213, and a light control unit 214 are further provided.

Here, as indicated by a broken line in the figure, the imaging device 212 and the irradiation unit 213 are integrated with the imaging unit 207, and the light control unit 214 is integrated with the dot pattern analysis unit 208.

FIG. 63 is a block diagram showing the configuration of the third embodiment of the handwriting input / output system 201 according to the present invention. In the third embodiment, in addition to the configuration provided in the first embodiment, a timer 215 is further provided.

Here, as indicated by a broken line in the figure, the time measuring unit 215 is integrated with the locus recognizing unit 209.

FIG. 64 is a block diagram showing a configuration of the fourth embodiment of the handwriting input / output system 201 according to the present invention. In the fourth embodiment, in addition to the configuration provided in the first embodiment, an angle measuring unit 216 and an angle change recognizing unit 217 are further provided.

Here, as indicated by a broken line in the figure, the angle measurement unit 216 and the angle change recognition unit 217 are integrated with the dot pattern analysis unit 208.

FIG. 65 is a block diagram showing the configuration of the fifth embodiment of the handwriting input / output system 201 according to the present invention. In the fifth embodiment, in addition to the configuration provided in the first embodiment, a rotation angle reading unit 218 and a rotation angle change recognition unit 219 are further provided.

Here, as indicated by a broken line in the figure, the rotation angle reading unit 218 and the rotation angle change recognition unit 219 are integrated with the dot pattern analysis unit 208.

FIG. 66 is a block diagram showing the configuration of the sixth embodiment of the handwriting input / output system 201 according to the present invention. In the sixth embodiment, in addition to the configuration provided in the first embodiment, a voice recording unit 220, a voice recognition unit 221, and a voice output unit 222 are further provided.

<Description of stream dot pattern format>
FIG. 67 is a diagram for explaining the format of the dot pattern formed on the medium 202 in the handwriting input / output system 201 according to the present invention.

As shown in FIG. 4D, the dot pattern is a dot pattern composed of unit areas of 2 rows × 5 columns of information dots in which a set of constant information is defined, and C ₀ to C ₂₉ in this unit. It is divided into. The dot code format of each area is shown in (a) to (c).

FIG. 67 (a) shows a format in which a page code, an X coordinate, and a Y coordinate are registered. The page code means the number of pages of the medium 202 when the medium 202 is in the form of a booklet, and the X coordinate and the Y coordinate each mean a position (coordinate value) on the medium. Is. Thus, when the user reads the medium 202 with the scanner 203, the number of pages of the medium and the coordinate position of the read position are simultaneously recognized by one reading operation.

(B) is a format in which page code, action code, X coordinate and Y coordinate are registered. The action code means an operation instruction. For example, it is defined only in the area where icons indicating operations such as “enlarge” and “reduce” provided on the page are drawn, and is mainly included in the dot pattern used for the icon area described later. ing. When an action code is defined over the entire writing area of the page and necessary processing is performed during or after handwriting input, the command processing may be associated with the action code and executed. For example, the command processing refers to processing for recognizing characters during handwriting input or after handwriting input, or transmitting to another information processing apparatus. Although the use as a page code has been described here, a plurality of specific write area codes in parentheses may be provided in one medium surface, and these write areas must be defined by unique codes. This makes it possible to determine where the data is written instantaneously. Of course, it may be defined together with the page code so that it can be determined which specific write area of which page. Furthermore, when the content input by handwriting belongs to a predetermined category, the recognition rate is remarkably improved by collation with the dictionary of the category. Needless to say, the contents input by handwriting may be not only characters but also symbols and illustrations. Of course, the action code may be defined together with the specific write area code and the instruction process may be executed.

In FIG. 5C, by providing a writing area flag, if the dot pattern formed on the medium surface is imaged by the imaging means and analyzed by the dot pattern analysis unit 208, it can be immediately determined whether the area is the writing area. If it is not the writing area, information output and / or command processing associated with the code value and / or coordinate value is executed, and if it is the writing area, predetermined processing of handwriting input is executed by the locus recognition means. Of course, this write area flag may be accompanied by the page code (specific write area code) and action code shown in FIGS.

Thus, the dot pattern of the present invention can register various code information such as page code and action code and XY coordinates in one format. In addition, it is possible to register only XY coordinates or only code information, and a flexible format is possible.

<Imaging unit>
68 to 81 are diagrams for explaining the imaging unit 207 used in the handwriting input / output system 201 according to the present invention.

Note that the imaging unit 207 preferably has a size and shape suitable for a person to hold, and most preferably has a pen shape as shown in FIG.

FIG. 9A is a diagram showing the most basic embodiment of the imaging unit 207 having a pen shape. In such an embodiment, an instruction projection 228 and a transmission unit 223 are provided, and a C-MOS sensor or the like (not shown) is generally built on the axis of the pen. The transmission unit 223 is for the imaging unit 207 to transmit a signal to another member using a method such as infrared communication or wireless communication, and is illustrated for convenience of explanation. It is not necessary to provide it at the position.

FIG. 6B is a diagram showing another embodiment of the imaging unit 207 having a pen shape. In such an embodiment, two buttons for accepting an operation by the user are provided. The two buttons are a send button 224 and a re-send button 225, and the user can instruct the user to send a signal from the imaging unit 207 to another member. Here, the re-transmission button 225 is used to re-transmit the same signal when the transmission of the signal by the transmission button 224 cannot be transmitted to the receiving member due to the presence of a shield or the like.

FIG. 10C is a diagram showing another embodiment of the imaging unit 207 having a pen shape. In this embodiment, a microphone 226 that accepts voice input by the user and an audio playback button 227 are further provided. The microphone 226 is used for recording a voice used for voice recognition described later, and the voice playback button 227 is used for playing a voice guide described later.

FIG. 4D is a diagram showing another embodiment of the imaging unit 207 having a pen shape. In such an embodiment, the pen 229 is stored in the internal space of the instruction projection 228, and the pen 229 comes out when the pen storage button 230 is pressed.

FIG. 69 is a cross-sectional view showing the distal end portion of a pen-type scanner 231 that is one embodiment of the imaging unit 207 constituting the handwriting input / output system 201 according to the present invention. The pen type scanner 231 has a light guide 232 having a hollow portion formed therein and having an opening at the tip thereof, a lens 233 disposed at a position facing the opening of the light guide 232, and the same as the lens 233. An LED 234 (irradiation unit 213) serving as a light source for irradiating light of a predetermined wavelength onto a medium surface on which a dot pattern is formed, and a C-MOS sensor 235 (imaging) disposed at a position retracted from the lens 233 An element 212) and a PCB 236 at a further retracted position and a CPU (not shown) installed therein are provided. The hollow portion of the light guide 232 is formed in a tapered shape that gradually increases in diameter from the lens 233 toward the opening. Here, the CPU has any or all of the functions of the dot pattern analysis unit 208, the trajectory recognition unit 209, the processing command unit 210, and the information processing unit 211 described above.

The light guide 232 is made of transparent or milky white resin, and the inside functions as a light guide. Irradiation light from the LED 234 travels inside the light guide 232 and is irradiated to the medium 202 from the opening. When the light guide 232 is made of milky white resin, the light emitted from the LED 234 is appropriately diffused when traveling through the light guide 232, so that the light irradiating the medium 202 from the opening is made more uniform. I can do it.

<Attaching the pen>
The pen-type scanner 231 is equipped with a pen 229 as a writing instrument. When printing with a writing instrument such as a normal pen, since it is used in an inclined state, the outer periphery of the pen-type scanner 231 is formed so as to be inclined up to 45 degrees. In this case, by picking up an image of the medium surface in the vicinity of the tip of the pen 229, it is possible to accurately analyze the handwriting, the moving distance, and the like when printing the pen 229 at a position away from the image pickup center. Further, the handwriting can be analyzed more accurately by offset correction described later.

The pen 229 as a writing instrument can be stored by various methods so that no handwriting is left on the medium. As a storing method, a technique used in a general pen such as a ballpoint pen can be used as it is. For example, a knock type or rotary type storage method may be employed, or a cap may be put on.

Also, the pen can be changed like a general three-color ballpoint pen. In this case, not only the pens of different colors may be replaced, but an eraser or a correction liquid may be used instead of the pen.

Furthermore, instead of the pen 229, an instruction protrusion 228 (stylus pen) may be provided to image the vicinity of the handwriting of the instruction protrusion 228. A storage space may be provided in the instruction protrusion 228 and the pen 229 may be stored to be used as an instruction protrusion.

<Offset correction>
As shown in FIG. 69, in the pen-type scanner 231, the XY coordinate value defined by the dot pattern captured by the imaging unit 207 is different from the XY coordinate value defined by the dot pattern on the locus actually written by the pen 229. Furthermore, the XY coordinate value defined by the dot pattern on the locus needs to be converted into the xy coordinate value in the writing area on the medium surface to perform writing input and operation instruction.

In the handwriting input / output system 201 according to the present invention, a method of correcting the dot pattern on the locus actually written by the pen 229 with the XY coordinate value of the dot pattern captured by the imaging unit 207, and the XY coordinate of the dot pattern coordinate system A method for converting the value into the xy coordinate value of the writing area coordinate system will be described below.

<Relationship between writing area coordinate system and dot pattern coordinate system>
FIG. 70 shows an example of the write area.

The writing area coordinate system is (0,0) in the lower left corner. The xy coordinate value of the writing area is expressed in mm. For example, a coordinate position of (16, 40) indicates a position moved 16 mm to the right and 40 mm upward from the lower left corner of the writing area.

In the writing area, at least dot pattern units that define at least one set of XY coordinate values in the dot pattern coordinate system are continuously formed. The XY coordinate value indicates the position of the center point of the unit. In order to distinguish the coordinate value of the dot pattern coordinate system from the coordinate value of the writing area coordinate system, it is expressed as [X, Y]. The unit is a stream dot in which a set of constant information is defined, and the center of the first virtual reference line connecting the reference dots arranged in a line is the center of the unit.

If the lower left corner of the writing area is the origin (0,0) of the coordinate system, and the lower left corner of the dot pattern unit with [0,0] defined matches the origin (0,0), the XY in the dot pattern coordinate system As shown in FIG. 71, a dot pattern is formed such that the lower left corner is [−0.5, −0.5] in any writing area as shown in FIG. In this case, a unique code value is defined for the dot pattern formed in each writing area, and each writing area is specified. If there is no need to specify each other, the same code value may be defined.

Further, as shown in FIG. 73, a dot pattern in which XY coordinate values are defined is formed on the entire surface of the paper, and a unique code value is defined in the writing area in addition to the XY coordinate values. In this case, the XY coordinate value of the lower left corner of the writing area is [Xn0, Yn0] (n is a number specifying the writing area).

Next, assuming that one side of each unit is Lmm, the XY coordinate values are converted to the writing area coordinate system as follows.

As shown in FIG. 71, when the dot pattern is formed with the XY coordinate value of the origin (0,0) as [−0.5, −0.5], the coordinates [X, Y] in the dot pattern coordinate system are the writing area coordinate system. Then
(X × L + L / 2, Y × L + L / 2).

As shown in FIG. 72, when the lower left corner of the writing area is the origin (0,0) of the coordinate system and the center point of the dot pattern unit defining [0,0] is made to coincide with the origin,
(X × L, Y × L).

The relationship between the writing area coordinate system and the dot pattern coordinate system in the example of FIG. 72 is the same as the example of the enlarged view portion of FIG.

As shown in FIG. 73, when the XY coordinate value of the lower left corner of the writing area is [Xn0, Yn0], the coordinates [X, Y] in the dot pattern coordinate system are
((X−Xn0) × L, (Y−Yn0) × L).

<Calculation of XY coordinate values [Xc, Yc] in the dot pattern coordinate system of the imaging center C>
A calculation method of the XY coordinate values [Xc, Yc] of the imaging center C will be described with reference to FIG.
The captured image is temporarily stored in a frame buffer having a predetermined resolution, and image analysis is performed to obtain the number M of pixels constituting the center interval between adjacent units. Since the center interval of the unit is equal to the length L of one side of the unit, if the size per pixel of the captured image with respect to L is a value k in the dot pattern coordinate system,
k = L / M.

Similarly, by image analysis, the number u of pixels constituting the interval between the unit center P [Xp, Yp] closest to the imaging center C and the imaging center C is obtained. The distance CP from the imaging center C to the nearest unit center P is
CP = k × u.

Next, when the dot pattern is formed on the paper so that the direction of the dot pattern (the direction of the writing area) is the upward direction of the writing area, the angle between the direction of the dot pattern and the PC vector is θ ′ (left Rotation is positive direction)
[Xc, Yc] = [Xp-CP × sinθ ′, Yp + CP × cosθ ′]
Is obtained.

<Calculation of xy coordinate value Q (xq, yq) in the writing area coordinate system of the pen tip (when the pen is perpendicular to the paper surface)>
As shown in FIG. 75, the coordinate value of the imaging center C in the writing area coordinate system is (xc, yc), and the coordinate value on the medium surface of the pen tip is Q (xq, yq). As shown in FIG. 76, if the difference between the x-coordinate value and the y-coordinate value of the point C and the point Q is Δxq and Δyq, respectively,
Δxq = xq -xc
Δyq ＝ yq－yc
Is obtained.

When the dot pattern is formed so that the direction of the dot pattern is in the upward direction of the writing area, the direction of the CQ vector starting from the imaging center C and ending at the pen tip is the direction of the pen scanner 231. .

The angle between the direction of the dot pattern and the direction of the pen-type scanner 231 is θ (left rotation is the positive direction).

In addition, as shown in FIG. 77, when the distance from the imaging center C to the pen tip Q in a state where the pen type scanner 231 is perpendicular to the paper surface is s,
Δxq and Δyq are
Δxq = −s × sinθ
Δyq ＝ s × cosθ
It becomes.

Therefore, xq and yq are
xq = xc -s × sinθ
yq = yc + s × cosθ
It becomes.

<Calculation of xy coordinate value Q '(xq', yq ') in the writing area coordinate system of the pen tip (when the pen is inclined with respect to the paper surface)>
As shown in FIG. 78, when the pen-type scanner 231 is inclined with respect to the paper surface, the distance from the imaging center C to the pen tip Q changes.

When the inclination from the normal direction of the paper surface of the pen-type scanner 231 body (hereinafter referred to as “pen inclination”) is α, and the distance from the imaging center to the pen tip is s ′, the above equation is replaced by s → s'
xq '= xc -s' × sinθ
yq '= yc + s' × cosθ
It becomes.

Also,
s' = s / cosα
Therefore, if this is substituted into the formula,
xq '= xc-s' × sinθ / cosα
yq '= yc + s' × cosθ / cosα
It becomes.

<Distance from the imaging center to the pen tip when the pen tip is stored in the pen body>
As shown in FIG. 79, when the pen tip is stored, the pen type scanner 231 approaches the paper surface by ΔL, but does not affect s and s ′, and the calculation formula is the same.

<Calculation method of pen inclination α>
In order to detect the pen inclination α, the pen body may be provided with an inclination measuring means such as an inclination sensor.

Alternatively, the brightness BLi, j = 1, p of a plurality of predetermined p positions in the captured image is obtained, and the pen inclination is calculated from the relational expression α = f (ΔBL) using the difference ΔBL in brightness between the brightest and darkest areas. α can also be obtained.

Α can also be obtained using a Fourier function or a sin function with BLi, j as a parameter, such as α = f (BLi, j = 1, p).

Also, a table that associates ΔBL with α may be created in advance and α may be calculated. The direction in which the pen is tilted can be obtained by measuring the direction of the brightest part from the darkest part of BLi, j = 1, p.

<Media 202>
80 to 93 are diagrams for explaining the medium 202 used in the handwriting input / output system 201 according to the present invention.

FIG. 80 is a diagram showing the medium 202 that is printed (formed) on the entire surface to form a writing area.

This dot pattern defines XY coordinate values and page codes. Here, the page code means a code for specifying a medium using code information.

FIG. 81 is a diagram showing a medium 202 on which another dot pattern further defining an action code is printed in an area where a dot pattern defining a page code and an XY coordinate value is printed.

Here, the action code means a code for instructing the information processing unit 211 to perform a predetermined operation (especially an operation of an application) with respect to a handwritten input to a specified medium by using code information. Thereby, security can be improved and an erroneous operation can be reduced.

This media consists of multiple sheets, and a different page code is assigned to each sheet. For example, the page code is “01” for the first page, “02” for the second page, and “03” for the third page.

In the area where the dot pattern defining the action code is printed, text, graphics, photos, and icons can be superimposed and printed to explain to the user what the action code means. it can. For example, an action code in an area printed with a text icon “start” superimposed thereon instructs the information processing unit 211 to start an application for handwriting input / output. The action code of the area printed with the text icon “Cancel” superimposed thereon instructs the information processing unit 211 to discard (cancel) the input result by the handwriting input / output application, or the information processing unit 211 When determining whether or not to execute a predetermined process, the information processing unit 211 is instructed to stop the execution of the process. The action code in the area printed with the text icon “save” superimposed instructs the information processing unit to save the input result by the handwriting input / output application. The action code in the area printed with the text icon “End” superimposed thereon instructs the information processing unit 211 to end the handwriting input / output application.

FIG. 82 is a diagram showing a medium 202 in which an area where a dot pattern defining XY coordinate values and a page code is printed and an area where a dot pattern defining an action code and a page code are printed are separated. In this case, various commands can be executed only for the page specified by the page code without selecting which page of the medium is the target. Thereby, security can be improved and an erroneous operation can be reduced.

FIG. 83 is a diagram illustrating a first medium on which a dot pattern in which XY coordinate values and page codes are defined is printed, and a second medium that is a paper controller on which a dot pattern in which only action codes are defined is printed. . In this case, various commands for handwritten input can be executed on any page of the medium. When the writing area and the icon are provided separately, as shown in FIG. 85, in addition to the paper controller, the writing area and the icon may be affixed to another medium as a sticker, sticky note, or card.

It should be noted that the action code of the area printed with the text icon “text input” superimposed is used to instruct the information processing unit 211 to input characters or figures as the text in the operation by the handwriting input / output application. The action code of the area printed with the text icon “comment input” superimposed thereon instructs the information processing unit 211 to input a character or a figure to be a comment in the operation by the handwriting input / output application. The action code in the area printed with the text icon “edit” superimposed thereon instructs the information processing unit 211 to edit the character or figure in the operation by the handwriting input / output application (that is, handwriting input / output). It is also possible to edit files such as documents and images created by an application other than the application for use.) The action code of the area printed with the text icon “determine” superimposed is used to execute the process for the information processing unit 211 when the information processing unit 211 determines whether to execute the predetermined process. Command to decide.

FIG. 84 shows a first medium on which a dot pattern defining XY coordinate values and page code A is printed, and a second medium which is a paper controller on which a dot pattern defining action code and page code A is printed. It is a figure which shows what became one booklet.

In this case, a dot pattern in which the same XY coordinates of each page are registered may be printed in the writing area. This is because, when performing a writing operation, the writing area can be specified by using the icon of the controller area in which the action code and page code are registered. That is, it can be said that the position and page of the writing area can be changed without changing the XY coordinate values. In this case, various commands can be executed only for the page specified by the page code without selecting which page of the medium is the target. Thereby, security can be improved and an erroneous operation can be reduced.

In addition to the above, a dot pattern defining an action code for instructing the information processing unit to start an application for handwriting input / output may be printed over the entire writing area. In this case, when writing is performed on the medium surface, the handwriting input / output application is automatically activated.

Also, the number of write areas on the medium is not limited to one, and a plurality of write areas can be defined.
(Regarding the case where multiple writing areas are provided)

Hereinafter, the case where a plurality of write areas are provided will be described with reference to FIGS. 85 to 89 using four examples.

<First example>
In the first example, as shown in FIG. 85, a dot pattern in which XY coordinate values based on different dot pattern coordinate systems and code values for specifying a writing area are formed is formed for each of a plurality of writing areas. .

In this figure, for each writing area, a dot pattern defining XY coordinate values in the dot pattern coordinate system and code values m and n for specifying the writing area is formed.

The code value can include not only the writing area but also the classification number that classifies the page number by defining multiple information.

When the page number is included, it is possible to determine which writing area of which page, and there are selection items such as information, processing instructions, and alternatives in the writing area, and when processing as a common code, the defined page number In addition, information specifying a selection item can be combined and defined as a code value.

Note that, as shown in FIG. 86, different code values can be defined by dividing the area in the write area. In this figure, the code values are n1, n2, n3, and n4. Thereby, a plurality of selection items can be set in one writing area.

This first example has the advantage that it is easy to handle because the lower left corner of each area is always 0 (origin).

<Second example>
In the second example, as shown in FIG. 87, a dot pattern in which the XY coordinate value is defined by one dot pattern coordinate system is formed on the entire surface of the predetermined area, and the code value is defined together with the XY coordinate in the dot pattern of the writing area. This is the case.

In this figure, a dot pattern in which XY coordinate values are defined in one dot pattern coordinate system with the lower left corner of the predetermined area as the origin is formed on the entire surface of the predetermined area. The two writing areas in the predetermined area are defined with XY coordinate values whose lower left corners are [Xm0, Ym0] and [Xn0, Yn0] and code values m and n for specifying the writing area, respectively. ing.

As with the first example, code values can be defined by dividing the area within the write area.

Note that writing cannot be performed in an area where a dot pattern other than the writing area is formed.

According to the second example, by reading the XY coordinate values, it is easy to know where the writing area is laid out within the predetermined area.

<Third example>
As shown in FIG. 88, the third example is a case where a write area is further inserted into the write area.

In this figure, a dot pattern defining an XY coordinate value in one dot pattern coordinate system with the origin at the lower left corner of a predetermined area and a code value L for specifying a writing area is formed on the entire predetermined area. Yes. Two write areas in the predetermined write area are defined by XY coordinate values whose lower left corners are [Xm0, Ym0] and [Xn0, Yn0] and code values m and n for specifying the write area, respectively. Has been.

According to this example, it becomes easy to arrange a small writing area such as a selection item column in a large writing area in which a sentence or a figure is freely written.

If this is to be realized by the first example, it is necessary to provide a writing area with holes when forming a further writing area in the writing area, and the formation of the dot pattern is a complicated operation.

Furthermore, if the writing area can be formed in the writing area in one coordinate system, the writing mode in the writing area can be ignored and the writing operation can be performed by changing the input mode. It becomes possible. For example, when writing to a writing area in the writing area, a setting that is effective only when the input mode is switched or a specific symbol is drawn may be provided.

As with the first example, the code value can also be defined by dividing the area within the write area.

In this example, it is easy to know where the writing area is laid out by reading the XY coordinate values.

<Fourth example>
In the fourth example, as shown in FIG. 89, one dot pattern coordinate system is defined over the entire predetermined area, and the XY coordinate value in this coordinate system and a code value for specifying the write area are defined in the write area. This is a case where a pattern is formed.

In this figure, one coordinate system with the lower left corner as the origin is defined in a predetermined area, and the lower left corner is [Xm0, Ym0] and [Xn0, Yn0] in the two writing areas in this area, respectively. A dot pattern in which a certain XY coordinate value and code values m and n for specifying a writing area are defined is formed.

In this example, unlike the second example, no dot pattern is formed in a range other than the writing area.

In this example, if there is a block of printed material such as graphic or text and a block pattern of dot pattern, the area of the writing area is automatically laid out by arranging the mask according to the coordinate system of the block of printed material. It is possible to create a dot pattern block by performing a mask process, and a dot pattern can be easily issued.

Also, when the dot pattern is read, the XY coordinate value is read to find where the writing area is laid out within the predetermined area. It becomes possible to use for.

In the above description, the dot pattern coordinate system is used for all description, but it is natural that the writing area coordinate system converted from the dot pattern coordinate system is the same.

FIG. 90 is a diagram showing a document as a specific example in which a plurality of write areas are defined on a medium.

(A) of the figure shows a document in which a specific code for instructing the origin of the XY coordinate values and the writing area is defined in each writing area.

In FIG. 6A, the “name” writing area has the “phonetic” writing area, the “address” writing area the “zip code” writing area, and the “neighboring map” writing area. Write areas for “walking (time)”, “train (use time)”, and “bus (use time)” are defined.

The “sex” writing area is divided into “male” and “female” writing areas, and the “birth date” writing area is divided into “year”, “month”, and “day” writing areas. .

Such a layout of the writing area is possible by assigning different specific codes to each.

(B) of the drawing shows that in such a document, the origin of the XY coordinate values is not defined in each writing area, and only a specific code for instructing specification of the writing area is defined.

With such a configuration, the position of the write information on the entire medium surface can be specified.

FIG. 91 is a diagram showing a state in which handwriting input is actually performed on the document of FIG.

(A) of FIG. 9 is a diagram showing the document of FIG. 90 (a) in which handwriting input is actually performed.

(B) in the figure is a diagram showing a state in which the result of the process performed by the information processing unit 211 is received on the display 6 or output by printing in response to the handwritten input in the figure (a).

Although the details will be described later, the handwriting input / output system 201 according to the present invention performs output as shown in FIG. 5B by recognizing handwritten input characters and figures by several methods.

<Print a new dot pattern>
In the handwriting input / output system 201 of the present invention, handwriting input can be performed, the information processing unit 211 can reflect the handwriting input, and handwriting input can be performed again.

FIG. 92A is a diagram showing the medium 202 printed by superimposing the trajectory information recognized by the trajectory recognition unit 209 and the dot pattern. Although not shown, a dot pattern different from the dot pattern printed on the medium 202 on which handwriting input was first performed may be newly printed on the medium 202 shown in FIG.

FIG. 92 (b) is a diagram showing a medium 202 on which a result of processing performed by the information processing unit 211 is printed with a dot pattern superimposed thereon. Although not shown, a dot pattern different from the dot pattern printed on the medium 202 on which handwriting input is first performed may be newly printed on the medium 202 shown in FIG. In this case, the XY coordinates are the same, and only the page code may be changed and printed.

<Braille / Bank>
The medium 202 may further be provided with a bank guide 38 (narrow convex portion) or Braille 39.

FIG. 93 shows a bank guide 38 provided on the outer periphery of the writing area, and a braille 39 protrusion along with a dot pattern in the writing area. By providing the bank guide 38 in this way, the position of the writing area can be known even when a visually impaired user uses handwritten input / output, and by providing the braille 39 and the dot pattern in the same area, the visually impaired user When the user uses handwritten input / output, the same input efficiency as that of a healthy person can be maintained.

<Dot pattern analysis unit>
The dot pattern analysis unit 208 obtains XY coordinate information and code information defined by the dot pattern by performing code analysis on the image data in accordance with the information input / output method using the dot pattern in the stream dots described above.

In the handwriting input / output system 201 according to the present invention, since the dot pattern analysis unit 208 sequentially performs code analysis on the trajectory written (traced) on the medium by the imaging unit 207, the XY coordinate information and the code information are the trajectory. (Trajectory information).

<Track recognition unit>
The trajectory recognition unit 209 recognizes characters and graphics written on the medium based on changes in trajectory information.

The trajectory recognition method is a method that considers characters and figures as graphics, and refers to the XY coordinate patterns of the characters and figures in the database, like a general OCR (optical character reader).

Here, in the present invention, in particular, the contents to be written in the writing area can be designated in advance by code information. That is, the trajectory information can be recognized with high accuracy by referring to the character and / or graphic information pattern table classified in advance. As a specific example, if it is instructed in advance to describe only Arabic numerals, the recognition rate increases because the trajectory recognition unit 209 refers only to a table of Arabic numerals. If recognition is not possible, all characters and figures in the database can be referenced as usual.

Furthermore, after performing the above-described trajectory recognition, the semantic information of a character and / or figure written in the writing area or a phrase composed of one or more characters is defined as predefined semantic information corresponding to the code information. By referring to the code table (dictionary), it can be converted into a semantic information code. In other words, for example, when “Bunkyo-ku, Tokyo” is written, the trajectory recognition unit 209 first performs pattern recognition one character at a time, “East”, “Kyo”, “Miyako”, “Kyo”, “Ku”. Referring to the semantic information code table (dictionary), the semantic information code defined as corresponding to “Tokyo” and the semantic information code defined as corresponding to “Bunkyo-ku” are converted. Thus, the information processing unit can recognize that the place names “Tokyo” and “Bunkyo-ku” have been input.

Referring to the semantic information code table (dictionary) also assists character pattern recognition. For example, if the user has written the character “east” of “Tokyo” dirty, and it is difficult for the locus recognition unit 209 to recognize whether the user has written “east” or “car”, the semantic information If the idiom "Tokyo" is defined in the code table and the idiom "Kakkyo" is not defined (the idiom "Kakkyo" usually does not exist), the user wrote the word "east" The trajectory recognition unit 209 can recognize as follows.

<Timekeeping section>
The handwriting input / output system 201 according to the present invention may further include a timer unit 215. The timer unit 215 has a function of detecting time information.

Here, the time information is
(1) The order of writing history drawn by the imaging unit 207 touching the writing area surface is used.

(2) A set of time until the imaging unit 207 touches the writing area surface and leaves, and ignores the time when the imaging unit 207 is away from the start of writing. That is, the time when the imaging unit 207 is away is omitted, and the handwriting input time and its order are described from the start and end times of (1). As a result, the time for writing the character is known, the accuracy of character recognition is improved, and the character input speed of the target person is known.

(3) By recording all the time when the imaging unit 207 touches or leaves the writing area surface, it is possible to know how much time was required for questionnaires and product selection, and to compare with the normal time This makes it possible to grasp difficulty, thinking state, writer's personality, age, gender, and the like.

(4) Also, by grasping the moving speed of the pen, it is possible to analyze the difficulty of writing and the difficulty of writing, and also to estimate the writer's psychological state and health state.

<Angle measurement unit and angle change recognition unit>
The handwriting input / output system 201 according to the present invention may further include an angle measurement unit 216 and an angle change recognition unit 217. The angle measurement unit 216 and the angle change recognition unit 217 measure and recognize the inclination of the imaging unit 207 with respect to the medium surface.

As the method for angle measurement and angle change recognition, the above-described method for calculating the pen inclination α can be used as it is. In this case, it is possible to measure and recognize the tilt even if the imaging unit 207 does not include a pen. Further, the slope calculation method disclosed in the invention of the information output apparatus according to Japanese Patent Application No. 2005-267565 (PCT / SG2006 / 000267) can be used. However, other methods may be used.

Note that the angle change with respect to the medium surface of the image capturing unit 207, which is a reference for measuring and recognizing the inclination, is recognized not only by the angle when the image capturing unit 207 is set up vertically with respect to the medium surface, but also by the image capturing unit 207 The angle when touching the surface can be used as a reference.

<Rotation angle reading unit / Rotation angle change recognition unit>
The handwriting input / output system 201 according to the present invention may further include a rotation angle reading unit 218 and a rotation angle change recognition unit 219. The rotation angle reading unit 218 and the rotation angle change recognizing unit 219 read and recognize the rotation angle when the imaging optical axis of the imaging unit 207 is set to be substantially vertical and rotated.

The method for obtaining the angle between the direction of the dot pattern and the direction of the imaging unit 207 has already been described in the section of offset correction, but this method can be used for a method of reading and recognizing the rotation angle regardless of the presence or absence of a pen. it can. That is, since the direction of the dot pattern is always constant, the angle θ _{1 formed by} the direction of the dot pattern and the direction of the image capturing unit 207 when the image capturing unit 207 is set up vertically with respect to the medium surface for the first time is set to a predetermined value. It is possible to read and recognize the difference between the direction of the dot pattern after the elapse of the time and the angle θ ₂ (the left rotation is a positive direction) formed by the direction of the imaging unit 207 as the rotation angle.

<Distinction method of touch action and bouncing action>
In the handwriting input / output system 201 according to the present invention, a method of discriminating between an operation of touching the imaging unit 207 on the writing area (touching operation) and an operation of bouncing from the state of touching the writing area (bounce operation) will be described with reference to FIG.

As shown in FIG. 5A, when the imaging unit 207 touches the writing area, the dot pattern in the writing area can be captured at the moment when the imaging unit 207 touches the writing area. When the imaging unit 207 is moved away from the writing area, the irradiation light from the LED 234 is not reflected on the medium surface, and the C-MOS sensor 235 does not detect the irradiation light from the LED 234 (reflection light from the medium surface). As shown in (c), the dot pattern cannot be imaged.

As shown in FIG. 5B, when the image pickup unit 207 bounces from the state where it touches the writing area, the dot pattern in the writing area can be picked up at the moment when the image pickup unit 207 touches the writing area. The operations up to are the same as the operation of touching the writing area. When the imaging unit 207 bounces from the state where it touches the writing area, the tip of the imaging unit 207 is still directed to the writing area for a predetermined time, so that the irradiation light from the LED 234 is reflected on the medium surface. The sensor 235 temporarily captures the dot pattern on the medium surface. However, since the moving speed of the imaging unit 207 is fast, the dot pattern cannot be analyzed blurry as shown in FIG.

That is, the image capturing unit 207 touches the writing area to capture a dot pattern, and analyzes the captured image data after a predetermined time (eg, 0.2 seconds later) with the analyzed moment as a reference time, When the pattern is not imaged (reflected light is not detected), the imaging unit 207 is preliminarily defined as a touch operation, and the processing command unit 210 is pre-defined as corresponding to the touch operation with respect to the information processing unit 211 Send processing instructions. When a dot pattern is imaged (reflected light is detected) but the dot pattern is blurred and cannot be analyzed, the image capturing unit 207 performs a jumping operation, and the processing command unit 210 instructs the information processing unit 211 to Then, a processing command defined in advance as corresponding to the bouncing action is transmitted. As shown in FIG. 5E, when a dot pattern can be analyzed, it is nothing but a normal handwriting input operation.

<Processing instruction part>
The processing command unit 210 transmits a processing command based on the information recognized by the trajectory recognition unit 209 to the information processing unit 211 together with the trajectory information.

As described above, the instruction specifying the page based on the page code, the instruction specifying a certain writing area based on the specific code, and the various instructions based on the action code defined by the dot pattern printed on the medium surface are the processing instruction unit 210. Is actually performed on the information processing unit 211.

Further, as will be described in detail later, the processing command unit 210 commands the information processing unit 211 to execute various operations in the handwriting input / output application.

Furthermore, predetermined processing instructions based on the measurement / recognition result by the angle measurement unit 216 / angle change recognition unit 217 and the read / recognition result by the rotation angle reading unit 218 / rotation angle change recognition unit 219 are sent to the information processing unit 211. Command against.

<Information Processing Department>
The information processing unit 211 performs various processes in response to processing instructions performed by the handwriting input / output system 201 according to the present invention.

<Server>
The handwriting input / output system 201 according to the present invention may access the server 5 in which various information is recorded as necessary.

<Track recognition server>
The server 5 can store information for locus recognition, for example. As employed in a general OCR (optical character reader), XY coordinate patterns are defined and stored in a plurality of characters and figures.

Also, a classification code for classifying these characters and figures is defined and stored.

Further, the meaning information code for outputting a phrase or figure consisting of one or a plurality of characters, or a combination thereof is defined and stored in the characters and figures by referring to the meaning information code table (dictionary). Has been.

In the handwriting input / output system 201 according to the present invention, since the writing area can be specified using not only the XY coordinate information but also the code information, it is easily managed even if the amount (number) of the writing area is enormous. can do. This is particularly effective when writing information is stored in a shared global server that anyone can use.

<Other servers>
Needless to say, the server 5 can store voice, document, image, video, application, and other various information in addition to the information for recognizing the trajectory. For example, a handwriting input / output application can be stored. That is, when the information processing unit 211 does not have a handwriting input / output application, it can search the server and execute the handwriting input / output application on the server. However, the handwriting input / output application on the server may be executed with priority.

In addition, it may replace with the server 5 and the application for handwritten input / output of the other information processing part 211 may be performed.

Of course, it is assumed that the server 5 can be implemented with various modifications other than the above embodiment.

<Handwriting input / output application>
The handwriting input / output application executed in the handwriting input / output system 201 according to the present invention will be described below.

Such an application is a system in which writing on a medium surface is reflected on a computer by executing various operations.

The writing on the medium surface may be performed while viewing the display 6 on which the processing result is displayed by instructing the information processing unit 211 in real time by the processing command unit 210 or writing without looking at the display 6. It is assumed that the processing instruction unit 210 may instruct the information processing unit 211 after writing.

<Write operation>
Hereinafter, the writing operation in the handwriting input / output system 201 according to the present invention will be described.

The writing operation is performed by writing the character or figure to be input into the writing area on the medium surface using the imaging unit 207.

The imaging unit 207 captures a dot pattern printed in a superimposed manner at a predetermined time interval while the imaging unit 207 moves on the medium surface. The subsequent steps are as already described.

Hereinafter, each operation executed by the handwriting input / output system 201 will be described.

<Character and figure recognition method>
FIG. 95 is an image diagram for explaining a first operation for inputting text characters by handwriting.

When characters are written in the writing area using the image pickup unit 207 as shown in FIG. 11A, the image pickup unit 207 sequentially picks up the dot patterns on the locus as shown in FIG. The unit 208 acquires trajectory information composed of XY coordinate information and code information.

For convenience of explanation, the interval between the imaging positions of the dot pattern on the locus is wide in FIG. 5B, but actually it is narrower.

As shown in FIG. 5C, based on the result of recognition based on the trajectory information, the information processing unit 211 executes the first operation and outputs the result to the display 6.

As shown in FIG. 4D, when the figure is recognized, there is no change in the recognition method as when the character is recognized, and the recognition result is displayed on the display 6 as shown in FIG.

However, the XY coordinate information obtained by the dot pattern analysis unit 208 may be input / output as it is without recognizing characters and figures. In this case, the handwriting input / output system 201 according to the present invention functions as a pen tablet.

<Second operation>
As shown in FIG. 96, the second operation is an operation of inputting a comment character by handwriting. In the figure, for convenience of explanation, the comment part is surrounded by a broken line, but it is not necessary to actually write the broken line.

The recognition method of characters or figures is the same as the first operation method.

The character of the comment input in the second operation is recognized as a comment different from the text in the information processing unit 211 as well.

97, the comment entered between the lines may be reflected in the information processing unit 211 as it is as shown in FIG. 97. However, as shown in FIG. 98, by specifying the part in the text where the comment is to be inserted with an underline, Comments may be listed in the processing unit 211 beside the text.

<Third operation>
The third operation is an operation for editing a handwritten character.

Each editing is performed by writing, using the imaging unit 207, characters or figures that are defined in advance as editing symbols in the information processing unit 211.

FIG. 99 is a diagram for explaining editing symbols used in the editing operation which is the third operation of the present invention.

A line forming one closed area means an edit symbol for selecting a character or figure in the area.

二 Two sets of lines in parentheses that do not close the area mean edit symbols that select characters or figures between the two sets of lines.

レ The red dot means an edit symbol that specifies the cut target when cutting and pasting.

二重 Double line means delete (erase) edit symbol.

“Toru” means an edit symbol for deletion (deletion).

“Iki” means an edit symbol for canceling the edit symbol “Toru”.

】 Zigzag line more than 3 round trips means edit symbol for canceling edit.

双方向 A bidirectional arrow means an edit symbol that replaces selected characters or figures.

矢印 Arrows in only one direction mean edit symbols that directly specify the paste destination.

Letters (A), (B), (C). . . Means an edit symbol that copies (cuts) a selected character or figure with a flag and designates a paste destination at a distant position.

However, it goes without saying that edit symbols other than the above may be defined.

FIG. 100 is a diagram showing the medium 202 before the third operation is executed. The medium 202 is filled with a document input by handwriting by executing the first operation or a document printed as a result of handwriting input.

FIG. 101 is a diagram for explaining an editing state on the medium 202 in the third operation.

When writing a line that closes the area as shown in FIG. 5A, the handwritten input / output application recognizes the area closed by the line as one editing target.

When writing a bracket-like line that does not close the area as shown in FIG. 5B, in the subsequent area, draw a corresponding parenthesis-closed line as shown in FIG. Is recognized by the handwriting input / output application as the editing target. This is especially used when the editing target extends over several lines.

As shown in FIG. 4D, when “Toru” is drawn on the line indicating the editing target in a state where the editing target is selected, the editing target character is deleted in the handwriting input / output application.

When drawing an arrow that specifies from the editing target to a predetermined position with the editing target selected, as shown in FIG. 5E, in the handwriting input / output application, cut and paste or copy to the area where the editing target is specified. & Paste. Here, the paste destination is not only inserted between characters (or at the beginning or end of the line), but as shown in (f) in FIG. It is also possible to do.

In this case, as shown in FIG. 5G, when a check mark, which is an edit symbol for deletion, is added on the line indicating the copy source edit target, it is processed as a cut and paste in the handwriting input / output application and cut. The target is deleted. As shown in (h) in the figure, when a check mark is not drawn, it is processed as copy and paste in the handwriting input / output application.

As shown in FIG. 6 (i), when an edit symbol that is a flag such as (A) or (B) is added to the edit target, the edit target is copied with the flags (A) and (B). Thereafter, the paste can be made any number of times. Furthermore, as shown in FIG. 6J, it is possible to paste at a distant position such as a different page.

If a double line is drawn on the character as shown in FIG. 4 (k), the character with the double line is deleted in the handwriting input / output application.

As shown in (l) of the figure, when two editing objects are connected by an editing symbol of a bidirectional arrow, the editing objects are exchanged in the handwriting input / output application.

FIG. 102 is a diagram showing a sentence in which the result of each editing in the third operation performed on the medium 202 is reflected in the handwriting input / output application.

FIG. 103 is a diagram for explaining an operation of canceling the editing and returning to the state before the editing after the third operation shown in FIG. 101 is edited.

If a zigzag line, which is a cancellation edit symbol, is added to a region once deleted with a check mark, which is an edit symbol, as shown in FIG. Cancel and restore the area.

As shown in Fig. 5 (b), once the copy symbol (A) is added and the copy and paste operation is performed, a zigzag line is added to the arrow portion that is the edit symbol that specifies the paste destination. In the input / output application, copying and pasting is performed to return to the state before inserting the character.

FIG. 104 is a diagram showing a sentence in which the result of performing the edit cancellation in the third operation on the medium 202 is reflected in the handwriting input / output application.

<Hierarchical relationship of each operation>
Hereinafter, the hierarchical relationship of each operation will be described.

Each operation may be a parallel operation of the third operation with the first and second operations.

In this case, the text and comment text are edited simultaneously during the third operation.

On the other hand, the third operation may be a sub-operation subordinate to the first operation and the second operation.

In this case, only the text of the body is edited during the third operation as a sub-operation subordinate to the first operation, and the comment character is edited during the third operation as a sub-operation subordinate to the second operation. Edit only.

Needless to say, the handwriting input / output system 201 according to the present invention can be modified and implemented as necessary in addition to the first to third operations.

<Save to computer>
As shown in FIG. 105, the processing command unit 210 rotates the image pickup unit 207 rightward by a predetermined rotation angle (for example, 10 degrees or more) while touching the medium surface, so that the processing command unit 210 receives the input character and document structure result (file The information processing unit 211 is instructed to save the content change result. When the imaging unit 207 main body has a function capable of recording an editing history, the result of the document configuration is transmitted to the information processing unit 211.

As shown in FIG. 106, the processing command unit 210 discards the input character and the result of the document structure by rotating the imaging unit 207 to the left by a predetermined rotation angle (eg, 10 degrees or more) while touching the medium surface. To instruct the information processing unit 211 to cancel the change of the file contents.

For the recognition of the rotation of the imaging unit 207, the above-described rotation angle reading / rotation angle change recognition method is used.

<Starting and exiting the application>
As shown in FIG. 107, while the image pickup unit 207 is touched on the medium surface, the imaging unit 207 is rotated to the right by a predetermined rotation angle (eg, 10 degrees or more), and the predetermined rotation angle (eg, within 1 second) is directly maintained. (Example: 10 degrees or more) By rotating counterclockwise, the processing command unit 210 commands the information processing unit 211 to start a handwriting input / output application.

As shown in FIG. 108, the image pickup unit 207 is touched on the medium surface and is rotated to the left by a predetermined rotation angle (eg, 10 degrees or more), and is left as it is within a predetermined time (eg, within 1 second). (Example: 10 degrees or more) By rotating clockwise, the processing command unit 210 instructs the information processing unit 211 to end the handwriting input / output application.

<Operation switching operation-Method 1>
The first method for switching operations will be described below with reference to FIG.

This method uses the angle measurement / angle change recognition method described above.

As shown in FIG. 5A, the image pickup unit 207 is set up on the medium surface, and a predetermined angle (for example, 15 degrees or more) is set on the left side of the medium surface with reference to the angle formed by the medium surface and the image pickup unit 207. By defeating, the processing instruction unit 210 instructs the information processing unit 211 to start the first operation for inputting the text by handwriting.

As shown in FIG. 5B, the image pickup unit 207 is set up on the medium surface, and a predetermined angle (for example, 15 degrees or more) is set on the right side of the medium surface on the basis of the angle formed by the medium surface and the image pickup unit 207. By defeating, the processing command unit 210 commands the information processing unit 211 to start a second operation for inputting a comment by handwriting.

As shown in FIG. 6C, the image pickup unit 207 is set up on the medium surface, and a predetermined angle (for example, 15 degrees or more) is set above the medium surface by using the angle formed by the medium surface and the image pickup unit 207 as a reference. By defeating, the processing command unit 210 commands the information processing unit 211 to start a third operation for editing the text and comments.

As shown in FIG. 4D, the image pickup unit 207 stands on the medium surface, and the lower side of the medium surface by a predetermined angle (for example, 15 degrees or more) on the basis of the angle formed by the medium surface and the image pickup unit 207. The processing command unit 210 instructs the information processing unit 211 to end the currently executed operation and start a preset normal operation (startup) operation.

FIG. 110 is a plan view seen from directly above, supplementing the explanation of the first method for switching operations.

<Operation switching operation-Method 2>
The second method for switching operations will be described below.

The operation switching may be performed by an input means such as a button provided on the imaging unit 207 main body.

In addition to the imaging unit 207, the operation may be switched by an external input unit such as a mouse or a keyboard.

<Operation switching operation-Method 3>
Below, the 3rd method of switching operation is demonstrated.

In the imaging unit 207 according to an embodiment that can store and switch a plurality of pen tips inside, the operation may be switched in conjunction with the pen tip switching operation.

In this case, since it is possible to color-code each operation, it is easy to distinguish each operation on the medium.

In addition, when an eraser or a correction liquid is attached instead of the pen tip, the information processing unit 211 may erase characters or figures instead of writing.

<Decision / Cancellation>
When the information processing unit 211 determines whether or not to execute a predetermined process, as shown in FIG. 111A, the imaging unit 207 is set up vertically on the medium surface, and once touched on the medium surface. When the medium surface is touched again within a predetermined time (eg, within 1 second), the processing command unit 210 instructs the information processing unit 211 to determine execution of the processing.

As shown in FIG. 5B, when the imaging unit 207 is set up vertically on the medium surface, and once touched on the medium surface, when no touch is made within a predetermined time (eg, within 1 second), a processing command The unit 210 instructs the information processing unit 211 to stop the execution of the process.

<Other usage methods>
The above configuration is premised on leaving a writing history on the medium 202, but the handwriting input / output system 201 according to the present invention performs handwriting input while looking at the display 6 without leaving a writing history on the medium 202. As described above, it may be possible.

<Voice recognition>
Although the present invention is a handwriting input / output system, speech recognition and voice guidance may be performed as a method for assisting handwriting input.

<Audio recording part>
As shown in FIG. 112, the audio recording unit 220 is configured around a central processing unit (MPU) 240. That is, under the control of the central processing unit (MPU) 240, audio information input from the microphone 241 is processed by the A / D converter 243 and the compression circuit 244 via the amplifier 242 and recorded as digital audio information.

<Voice recognition unit>
The voice recognition unit 221 converts the recorded voice information into corresponding text information, and the processing command unit 210 transmits a processing command based on the text information to the information processing unit 211 together with the text information.

Here, before and after voice recognition, it is possible to improve the accuracy of voice recognition by imaging a dot pattern defining a classification code. That is, the accuracy of speech recognition is improved by referring to the text information in the speech recognition information table corresponding to the classification code without referring to all the text information when searching the database and outputting the text information.

For example, if you say “Ueno”, it ’s difficult to know if it ’s Ueno or Ueno, but after the classification code that means “address” has been captured, When input as information, the voice information of “Ueno” is transmitted to the information processing unit 211 together with the address input category code.

<Switch to audio recording mode>
When the handwriting input / output system 201 according to the present invention performs voice recording instead of handwriting input, it is necessary to switch to the voice recording mode by performing a predetermined operation. A method for switching to the audio recording mode will be described below.

<Trace specific symbols>
As a first method of switching to the sound recording mode, there is a method of switching to the sound recording mode by the imaging unit 207 writing (tracing or drawing) a specific symbol that means switching to the sound recording mode. The trajectory recognition unit 209 recognizes the trajectory of a specific symbol that means switching to the voice recording mode and transmits it to the voice recording unit 220 (or by transmitting it via the processing command unit 210), thereby recording the voice. The unit 220 starts audio recording.

<Bounce the pen>
As a second method of switching to the audio recording mode, there is a method of switching to the audio recording mode by performing an operation of bouncing the imaging unit 207 while touching the writing area on the medium surface, as shown in FIG.

<Switching by voice recognition>
As a third method of switching to the voice recording mode, there is a method of recording a predetermined voice in the voice recording unit 220 and switching to the voice recording mode when the voice recognition unit 221 recognizes the predetermined voice.

The processing command unit 210 commands the information processing unit 211 to perform voice input instead of handwriting input in the handwriting input / output application when an operation by the above-described various methods is performed.

<Audio output unit>
The audio output unit 22 reproduces a predetermined audio file in response to an instruction from the processing command unit 210. The audio file that has been digitized and stored in advance is output from the speaker 245 or the earphone 246 via the A / D converter 42 and the amplifier 242.

<Voice guide>
In the handwriting input / output system 201 according to the present invention, voice guidance may be given to a user who performs handwriting input by using code information.

In other words, a voice guide code that instructs the information processing unit 211 to reproduce voice is defined in the dot pattern printed in the writing area, so that handwriting input is started (the imaging unit 207 changes the dot pattern). When the image is captured), the sound output unit 22 reproduces the sound file stored in association with the sound guide code in advance.

However, the audio guide code and the corresponding audio file may be used for purposes other than the audio guide.

Further, by using the above-described grid sheet as the medium 202, the medium can be pasted on the monitor surface of the display 6 to form a touch panel. However, means other than the grid sheet may be adopted.

Further, in addition to using the grid sheet as the medium 202, the grid sheet may be used by being superimposed on the medium 202. Such a usage method is such that when a document or image printed (described) on a medium that is not for handwriting input / output is first edited using the handwriting input / output system 201 according to the present invention, the image pickup unit is used. This is particularly effective when the user feels resistance to pressing 207 on the monitor surface.

<About the shape of the writing area>
As described above, the writing area is rectangular, but when the writing area is specified using the code information as in the present invention, the shape of the writing area is, for example, a polygon as shown in FIG. , Circular, elliptical, heart-shaped, star-shaped, or when an illustration and a dot pattern are overprinted, the same analysis efficiency can be achieved regardless of the shape along the actual line of the illustration. be able to.

Conventionally, when an attempt is made to specify a writing area using only coordinate information, coordinates indicating the boundary of the writing area are previously defined by a table or function, and the input coordinate value falls within the category of the table or function. It was necessary to judge sequentially. For this reason, the analysis is complicated, and when the writing area is specified only using the coordinate information, the writing area needs to be rectangular in order to minimize the decrease in the analysis efficiency such as the memory used and the calculation speed. In the case of a rectangle, the minimum value and the maximum value of the XY coordinate values of the rectangular area are stored in a table, and it is determined whether the input coordinate value is within the range. Therefore, even for a rectangular shape, extra comparison calculation is required four times to determine whether the used memory is four times or more in the writing area for specifying the writing area based on the code information. Furthermore, when there are a plurality of write areas, it is necessary to determine which area is used, and it is predicted that a large amount of used memory and calculation speed are required. According to the present invention, since the code information directly specifies the writing area regardless of the shape of the writing area, the writing area can be specified without using extra memory or calculating.

<Distinction by combination of arrangement direction>
Another method for automatic operation discrimination will be described.

When using a dot pattern that defines information by arranging information dots with a distance and direction from the reference position, such as a stream dot (first numerical information), the information dots are placed on the + direction line (up and down Information can be defined for the combination of information dot arrangement directions (second digitized information), which is arranged on the horizontal direction line) or on the x direction line (on the diagonal direction line). When the information dot is arranged on the x-direction line is “0” and the information dot is arranged on the + direction line is “1”, the second digitization in the dot pattern of FIG. The information is “1010010110100101”, and the second numerical information in the dot pattern of FIG. 116B is “0000000011111111”.

In addition to the direction in which the information dots are arranged, the distance (arrangement distance) at which the information dots are arranged from the virtual reference point is set to two or more patterns so that the information dots are arranged at a distance from the virtual reference point. The second digitized information can be defined also by the combination.

Of course, the second digitized information may be defined by a combination of both the information dot arrangement direction and the arrangement distance.

That is, the second digitized information means information defined by the information dot arrangement rule itself, such as the information dot arrangement direction and arrangement distance.

The inventor has so far proposed a dot pattern format that defines both code information and coordinate information. In order to increase the number of information dots, the dot pattern in such a format increases the dot pattern distribution area. Accordingly, the optical reading unit of the pen scanner needs to have a large reading area in order to read the dot pattern.

For this reason, it has been difficult to use a format that defines both code information and coordinate information for toys that have an optical reading unit with low reading accuracy, calculation speed, and calculation procedure. Even with a small dot pattern with a small distribution area, a large amount of information can be defined.

(A)
The present invention can be used as a touch panel by sticking on a screen of any display such as a personal computer, PDA, bank ATM or the like. Moreover, by covering the printed material and outputting information related to the information described in the printed material on the screen, it can be used for a dictionary, a catalog of mail order, a learning material, and the like.

(B)
An information input device according to the present invention includes a personal computer, a PDA, a television receiver, and a front projector that can easily input text, diagrams, illustrations, photographs, etc. printed on a print medium or displayed on a screen. , Rear projectors, game devices, karaoke devices, mobile phone terminal devices, POS terminal devices, ATMs, KIOSK terminals, car navigation systems, pachinko machines, pachislot machines, watches, smartphones, and the like.

In addition, the information processing apparatus according to the present invention is a personal computer, PDA, and television receiver that can easily retrieve related information such as text, diagrams, illustrations, and photos that are printed on a print medium or displayed on a screen. It can be used as a device, a front projector, a rear projector, a game device, a karaoke device, a mobile phone terminal device, a POS terminal device, an ATM, a KIOSK terminal, a car navigation system, a pachinko machine, a pachislot machine, a watch, a smartphone, and the like.

(C)
The handwriting input / output system according to the present invention has the same feeling as that a person who has not mastered the technique of using an existing input device such as a mouse, a keyboard, and a tablet used for computer operation writes on a paper medium. The present invention has industrial applicability as an input device that can be used in a computer. Furthermore, it has industrial applicability as toys, especially educational toys, mainly for children.

Secondly, since the handwriting input / output system according to the present invention can specify the writing area by the code information read simultaneously with the coordinate information, if the content input by handwriting belongs to a predetermined category, the dictionary of the category The recognition rate is greatly improved by collating with.

Thirdly, the handwriting input / output system according to the present invention determines whether or not it is a writing area simply by reading the dot pattern without any special operation by the operator, and a writing instrument that everyone is familiar with is used. Intuitive handwriting input as you use it.

1 reference dot 2 first virtual reference line 3 second virtual reference line 4 virtual reference point 5 information dot 101a printed matter (medium)
101b DTP data (two-dimensional format information)
101c Print media (medium)
101d Content data (two-dimensional format information)
101e Print 101g Grid sheet (information input auxiliary sheet)
102a Word (specific information)
102b Mask area (selection position information)
103 dot pattern 104 to 104c scanner (dot pattern reading means)
105 to 105a Information input device 110 Infrared diffuse reflection ink 111 Infrared transmission ink 112 to 112d Information processing device 112e Television receiver (information processing device)
113 Document management server (2D format information server)
113b Web server (two-dimensional format information server)
113c Content server (two-dimensional format information server)
114 Dictionary server 115 Display 120 to 120a Document management unit (two-dimensional format information storage selection means)
130 to 130b Coordinate converter (coordinate converter)
140-140c Specific information selection unit (specific information selection means)
150 to 150c communication unit (communication means)
151 Communication unit (communication means)
160 Related Information Search Unit (Related Information Search Means)
DESCRIPTION OF SYMBOLS 170 Display part 180 Content acquisition part 190 Television receiver part 201 Handwriting input / output system 202 Medium 203 Scanner 204 Computer 205 Server 206 Display 207 Imaging part 208 Dot pattern analysis part 209 Trajectory recognition part 210 Processing instruction part 211 Information processing part 212 Imaging element 213 Irradiation unit 214 Light control unit 215 Timing unit 216 Angle measurement unit 217 Angle change recognition unit 218 Rotation angle reading unit 219 Rotation angle change recognition unit 220 Audio recording unit 221 Audio recognition unit 222 Audio output unit 223 Transmission unit 224 Transmission button 225 Retransmission Button 226 Microphone 227 Audio playback button 228 Instruction projection 229 Pen 230 Pen storage button 231 Pen scanner 232 Light guide 233 Lens 234 LED
235 C-MOS sensor 236 PCB
237 Pen 238 Bank guide 239 Braille 240 Central processing unit (MPU)
241 Microphone 242 Amplifier 243 A / D converter 244 Compression circuit 245 Speaker 246 Earphone

Claims

An information input auxiliary sheet provided on a printed surface of a printed material or a display screen of a display and provided with a dot pattern that is read by an optical reading device in order to output information related to the image of the printed material or the display on the display. And
In order to output various multimedia information and / or instructions for operation, a step of arranging a plurality of reference dots in a line in accordance with a predetermined rule, and a straight line, a broken line and / or a curve connecting the plurality of reference dots A first virtual reference line comprising: at least one second virtual consisting of a straight line and / or a curve defined at a predetermined position from the reference dot and / or the first virtual reference line A step of providing a reference line; a step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line; and a distance from the virtual reference point to an end point represented by a vector using the virtual reference point as a starting point A plurality of stream dot patterns arranged in accordance with the step of arranging information dots in which XY coordinate values and / or code values are defined by the direction and direction are formed. And Ttopatan layer, the information input help sheet made of.
The stream dot pattern further includes a reference dot for defining the second virtual reference line and / or for defining the direction of the dot pattern and one XY coordinate value and / or code value. The information input assisting sheet according to claim 1, wherein is provided at a predetermined position.
The dot pattern layer reflects infrared light from one side or both sides and transmits visible light, or diffuses and reflects infrared light from one side or both sides and transmits visible light. The information input auxiliary sheet according to claim 1 or 2, which is provided on one side or both sides of an infrared diffuse reflection layer having characteristics.
An information input auxiliary sheet provided with the infrared reflecting layer,
The information input auxiliary sheet according to claim 3, wherein an infrared diffusion layer that transmits visible light and diffuses the infrared light is provided between the infrared reflection layer and the dot pattern layer.
An information input auxiliary sheet provided with the infrared reflecting layer,
4. The information input assist according to claim 3, wherein an infrared diffusion layer that transmits visible light and diffuses infrared light is provided on one side or both sides of the infrared reflection layer and on the outer surface side of the dot pattern layer. Sheet.
On the anti-dot pattern layer side of the infrared reflective layer or the infrared diffuse reflective layer, text, illustrations, photographs, etc. are superimposed and printed,
6. The information input auxiliary sheet according to claim 3, wherein text, illustrations, photographs, etc. are superimposed and printed with ink made of an infrared transmitting material on the dot pattern layer or any other layer. .
An information input auxiliary sheet provided with the dot pattern on one side,
The information input auxiliary sheet according to claim 1, wherein an adhesive layer is provided on an outer surface side where the dot pattern is read by an optical reading device.
3. The information input auxiliary sheet according to claim 1, wherein a protective layer having a property of transmitting infrared light and visible light is provided at least on an outer surface side where the dot pattern is read by an optical reading device.
The processing is performed into a seal shape, a card shape, a bookmark shape, or a sticky note shape for performing various multimedia information output and / or operation instructions by reading a dot pattern on the medium with a dot pattern reading unit. The information input auxiliary sheet according to any one of 8.
The information input auxiliary sheet according to any one of claims 1 to 8, is disposed on a display screen of a display of an information processing device,
By reading the dot pattern on the information input auxiliary sheet with at least XY coordinate values defined by the dot pattern reading means, it is possible to input in touch panel format,
An information processing system for performing auxiliary input to the information processing apparatus based on a dot pattern read from the dot pattern reading means.
The display displays content expressed in text, illustrations, photos, etc.
By reading the XY coordinate value of the dot pattern on the information input auxiliary sheet or the XY coordinate value and the code value by the dot pattern reading means, and converting it into the xy coordinate on the format information of the content displayed on the display, Enable touch panel input,
Recognizing the content specifying information at the position touched by the dot pattern reading means,
The information processing system according to claim 10, wherein content related information related to the content specifying information is searched and output.
The information processing device is a personal computer, PDA, television receiver, front and rear projector, game device, karaoke device, mobile phone terminal device, POS terminal device, ATM, KIOSK terminal, car navigation system, pachinko, watch, smartphone,
The information processing system according to claim 10 or 11, which is arranged on a display screen or a display screen as a touch panel type input device.
The information input auxiliary sheet according to any one of claims 1 to 8 is used by being placed on a printed surface of a printed matter,
The dot pattern reading means reads the dot pattern on the information input auxiliary sheet, at least the XY coordinate value is defined, and recognizes the XY coordinate on the printed surface and the printed material specifying information for specifying the printed material. Recognize print information by converting to xy coordinates in
An information processing system that retrieves and outputs information related to the print information.
On the information input auxiliary sheet or other sheet,
The information processing system according to claim 13, wherein an icon for inputting an index for specifying a printing surface of the printed matter by the dot pattern reading unit is superimposed and printed together with the dot pattern.
The information input auxiliary sheet is arranged on a display screen of a plurality of displays connected to one or more information processing apparatuses, and a code value for identifying each display is printed as a dot pattern together with an XY coordinate value. And
The code value and XY coordinate value read by the dot pattern reading means are transmitted to the information processing apparatus, and which display is recognized by the code value to enable input in a touch panel format,
The information processing system according to claim 10, wherein the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.
The information input auxiliary sheet is arranged on a display screen of a plurality of displays connected to one or more information processing devices, and XY coordinates that uniquely identify the display are printed as a dot pattern,
The XY coordinate value read by the dot pattern reading means is transmitted to the information processing apparatus, the display is specified, and touch panel format input is enabled.
The information processing system according to claim 10, wherein the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.
A sticker on which a code value for identifying each display is printed as a dot pattern is attached to the housing of the display,
Sending the code value read by the dot pattern reading means to the information processing device, recognizing which display is possible, enabling touch panel format input,
The information processing system according to claim 10, wherein the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.
DTP data, HTML, flash, CAD data, map data, etc. that are targeted for two-dimensional format information for printing on media, and that are standardized among the information visualized as text, diagrams, illustrations, photographs, etc. In order to output various multimedia information and / or for operation instructions, a plurality of reference dots are continuously formed in a line according to a predetermined rule on the surface of the medium. A first virtual reference line comprising a straight line, a polygonal line and / or a curve connecting the plurality of reference dots, a predetermined distance from the reference dot and / or the first virtual reference line Providing at least one or more second virtual reference lines defined by positions, each consisting of a straight line and / or a curve, and a predetermined position on the second virtual reference line. An information dot in which an XY coordinate value and / or code value is defined by a distance and a direction from the virtual reference point at an end point expressed by a vector with the virtual reference point as a starting point, and a step of providing a number of virtual reference points A plurality of stream dot patterns arranged according to the arranging step are formed side by side, and when the user designates the specific information visualized on the medium on which the two-dimensional format information is printed by a predetermined operation, the medium Dot pattern reading means used for touching the top and reading the dot pattern at the touch position;
Using the input identification information for identifying the two-dimensional format information, the two-dimensional format information corresponding to the identification information is stored in one or more of the two-dimensional format information stored in advance. 2D format information storage selection means to select;
Coordinate conversion means for converting the coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position into the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information;
In the xy coordinate system of the two-dimensional format information, in order to specify the position of the specific information, coordinate values (xi, yi) as the position information for selection,
One or more selections within a predetermined distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position Specific information selection means for calculating the distance to the position information for use and selecting the specific information having the coordinates of the position information for selection closest as an input to the own device;
An information input device comprising:
DTP data, HTML, flash, CAD data, map data, etc. that are targeted for 2D format information to be displayed on the display. Among the information visualized as text, diagrams, illustrations, photographs, etc. In order to output various multimedia information and / or operation instructions, information corresponding to the above is formatted in advance as specific information, and a plurality of reference dots are continuously arranged in a line according to a predetermined rule on the surface of the display. A first virtual reference line comprising a straight line, a polygonal line and / or a curve connecting the plurality of reference dots, a predetermined distance from the reference dot and / or the first virtual reference line A step of providing at least one or more second virtual reference lines, each of which is defined by a position and comprising a straight line and / or a curve, on the second virtual reference line XY coordinate values and / or code values are defined by a step of providing a plurality of virtual reference points at a predetermined position and a distance and direction from the virtual reference point at the end point expressed by a vector with the virtual reference point as a starting point. A plurality of stream dot patterns arranged in accordance with the step of arranging information dots to be arranged and a display on which the two-dimensional format information is displayed;
A dot pattern reading unit that is used for touching the display when the user designates the specific information visualized on the display by a predetermined operation, and reads a dot pattern at a touch position;
Using the input identification information for identifying the two-dimensional format information, the two-dimensional format information corresponding to the identification information is stored in one or more of the two-dimensional format information stored in advance. 2D format information storage selection means to select;
Coordinate conversion means for converting the coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position into the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information;
In the xy coordinate system of the two-dimensional format information, in order to specify the position of the specific information, (xi, yi) in the xy coordinate system of the two-dimensional format information determined when the specific information is displayed on the display. Use as location information for selection,
One or more selections within a predetermined distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position Specific information selection means for calculating the distance to the position information for use and selecting the specific information having the coordinates of the position information for selection closest as an input to the own device;
An information input device comprising:
The identification information is
The code value of the dot pattern printed on the medium or displayed on the display;
Printed or displayed together with the XY coordinate value of the dot pattern in at least a part of the area where the XY coordinate value of the dot pattern is printed or displayed;
The information input device according to claim 18, wherein the information input device is printed or displayed in an area different from the area.
The identification information is
An XY coordinate value of the dot pattern that uniquely identifies the two-dimensional format information;
The XY coordinate value is
The information input device according to claim 18 or 19, wherein the XY coordinate value of the dot pattern is printed or displayed over an entire area to be printed or displayed.
The display is
It is a structure in which the dot part of the dot pattern absorbs or transmits the infrared ray irradiated from the infrared irradiation unit provided in the dot pattern reading unit,
The area excluding the dot portion is
The information input device according to claim 19, wherein the information input device has a structure that diffusely reflects infrared rays.
In order to output various multimedia information and / or instructions for operation, a step of arranging a plurality of reference dots continuously in a line according to a predetermined rule on the surface of the information input auxiliary sheet, and the plurality of reference dots Providing a first virtual reference line comprising a straight line, a polygonal line and / or a curve, and a straight line and / or a curve defined at a predetermined position from the reference dot and / or the first virtual reference line A step of providing at least one or more second virtual reference lines, a step of providing a plurality of virtual reference points at a predetermined position on the second virtual reference line, and an end point represented by a vector using the virtual reference points as start points The stream dots arranged in accordance with the step of arranging information dots in which XY coordinate values and / or code values are defined by the distance and direction from the virtual reference point. Be printed is formed by arranging a plurality of patterns,
DTP data, HTML, flash, CAD data, map data, etc., for 2D format information to be printed on media or displayed on a display, and for information visualized as text, diagrams, illustrations, photographs, etc. Among them, information corresponding to a predetermined standard is specified and formatted as specific information in advance,
The information input auxiliary sheet, wherein the two-dimensional format information is arranged or pasted on a printed medium or a display to be displayed,
When the user designates the specific information visualized on the medium or the display by a predetermined operation, it is used to touch the medium or the display through the information input auxiliary sheet. , Dot pattern reading means for reading the dot pattern at the touch position,
Using the input identification information for identifying the two-dimensional format information, the two-dimensional format information corresponding to the identification information is stored in one or more of the two-dimensional format information stored in advance. 2D format information storage selection means to select;
Coordinate conversion means for converting the coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position into the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information;
In the xy coordinate system of the two-dimensional format information, it is determined when the specific information is laid out and printed in order to specify the position of the specific information, or is determined when the specific information is displayed on the display. Using (xi, yi) in the xy coordinate system of the two-dimensional format information as selection position information,
One or more selections within a predetermined distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position Specific information selection means for calculating the distance to the position information for use and selecting the specific information having the coordinates of the position information for selection closest as an input to the own device;
An information input device comprising:
In the information input auxiliary sheet,
A plurality of icons for a user to input an index for specifying the two-dimensional format information printed on the medium or displayed on the display are printed together with the dot pattern,
The two-dimensional format information storage selection means includes:
The index input by touching the icon using the dot pattern reading unit is converted into the identification information, and the two-dimensional format information is selected using the converted identification information. The information input device according to claim 23.
The predetermined operation is:
24. The operation according to claim 18, wherein the user touches the specific information visualized by using the dot pattern reading unit, or the user touches and traces the specific information. Information input device according to the above.
The predetermined operation is:
24. The operation in which the user touches the visualized specific information using the dot pattern reading unit and traces and surrounds the area where the specific information is displayed. The information input device according to any one of the above.
The specific information is
When the specific information is one word and the characters constituting the word are separated,
For each character, it has the position information for selection,
When the specific information is one chart, illustration, photo, etc. and there are a plurality of objects constituting these,
24. The information input device according to claim 18, wherein the selection position information is included for each object.
The position information for selection is
Whether it is specific information or a central coordinate value of each character constituting the word,
The information input device according to claim 18, wherein the information input device is a centroid coordinate value of a chart, an illustration, a photograph, or the like that is specific information, or an object that constitutes the specific information.
The position information for selection is
Start point coordinate value (xi1, yi1) and end point coordinate value of a line segment indicating the width of the word or the character at the center or at an arbitrary position in the height direction of each word constituting the word or the specific information A combination of (xi2, yi2),
The specific information selecting means includes
20. The specific information having a distance from the coordinate value (xt, yt) of the touch position to the line segment within an arbitrary distance and having the closest line segment is selected. 24. The information input device according to any one of 23.
The specific information selecting means includes
In the two-dimensional format information, a predetermined algorithm is used to parse text within an arbitrary distance from the coordinate value (xt, yt) of the touch position and / or search for files such as graphics, illustrations, and photos, and / or Or by recognizing one or more specific information by performing image analysis,
The position information for selection of the specific information is calculated by a predetermined algorithm,
24. The specific information is selected from the coordinate value (xt, yt), the recognized specific information, and the calculated position information for selection. Information input device.
The specific information is
By performing file search and / or image analysis such as text analysis and / or graphics, illustrations, photos, etc. included in the two-dimensional format information,
The selection position information of the specific information is:
Calculated in advance by a predetermined algorithm,
The two-dimensional format information includes
24. The information input device according to claim 18, wherein the specific information and the selection position information are recorded.
The two-dimensional format information is
24. The graphic according to claim 18, wherein the specific information is a text and / or a figure having a point, a line segment, a plane, or a three-dimensional object drawn by CAD. Information input device.
The two-dimensional format information is
Map data,
The map data has specific information classified into categories such as text, roads, features, intersections, etc.
The specific information includes an area surrounded by a road, a feature area, a road centerline, and a feature centroid as selection position information. Information input device.
An information input system comprising the information input device according to claim 18, a two-dimensional format information server, and a network connecting them,
The information input device, instead of the two-dimensional format information storage selection means,
The input identification information for identifying the 2D format information is transmitted to the 2D format information server, and the 2D format information corresponding to the identification information is received from the 2D format information server. With communication means,
The two-dimensional format information server
Two-dimensional format information for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance using the identification information received from the information input device Storage selection means;
An information input system comprising: communication means for receiving the identification information from the information input device and transmitting the selected two-dimensional format information to the information input device.
20. An information input system comprising the information input device according to claim 19, a two-dimensional format information server, and a network connecting them, wherein the information input device is replaced with the two-dimensional format information storage selection means. And
The input identification information for identifying the 2D format information is transmitted to the 2D format information server, and the 2D format information corresponding to the identification information is received from the 2D format information server. With communication means,
The two-dimensional format information server
Two-dimensional format information for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance using the identification information received from the information input device Storage selection means;
An information input system comprising: communication means for receiving the identification information from the information input device and transmitting the selected two-dimensional format information to the information input device.
An information input system comprising the information input device according to claim 18, a two-dimensional format information server, and a network connecting them,
The information input device, instead of the two-dimensional format information storage selection means,
The input identification information for identifying the 2D format information is transmitted to the 2D format information server, and the 2D format information corresponding to the identification information is received from the 2D format information server. With communication means,
The two-dimensional format information server
Two-dimensional format information for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance using the identification information received from the information input device Storage selection means;
An information input system comprising: communication means for receiving the identification information from the information input device and transmitting the selected two-dimensional format information to the information input device.
The stream dot is
In order to define a second virtual reference line and / or to define the direction of the dot pattern and one XY coordinate value and / or code value, a reference dot serving as a reference is provided at a predetermined position. The information input device according to claim 18, or the information input system according to any of claims 34 to 36.
Part of the code value is
The information input device according to any one of claims 18 to 19, or the information input device according to any one of claims 34 to 36, comprising at least a code value for specifying the medium or the display or the information input auxiliary sheet. Information input system.
A step of arranging a plurality of reference dots continuously in a line according to a predetermined rule in at least a part or a plurality of arbitrarily shaped regions, and a straight line, a broken line and / or a curve connecting the plurality of reference dots A first virtual reference line comprising: at least one second virtual consisting of a straight line and / or a curve defined at a predetermined position from the reference dot and / or the first virtual reference line A step of providing a reference line; a step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line; and a distance from the virtual reference point to an end point represented by a vector using the virtual reference point as a starting point Writing formed by arranging a plurality of stream dot patterns arranged in accordance with the process of arranging information dots in which XY coordinate values or XY coordinate values and code values are defined by and direction Handwriting input sheet characterized by comprising a medium having a frequency band.
The stream dot is
In order to define the second virtual reference line and / or to define the direction of the dot pattern and one XY coordinate value and / or code value, a reference dot serving as a reference is further provided at a predetermined position. 40. The handwritten input sheet according to claim 39.
A handwriting input / output system using the handwriting input sheet according to claim 39 or 40,
Imaging means for imaging and transmitting the stream dot pattern formed in the writing area;
Dots for obtaining coordinate information and trajectory information made up of coordinate information and code information by storing image data of the stream dot pattern imaged and transmitted by the imaging means in a storage element and performing code analysis on the image data Pattern analysis means;
A handwriting input / output system comprising processing instruction means for transmitting the trajectory information to information processing means.
The handwriting input / output system includes:
Furthermore, based on the change of the analyzed coordinate information, the locus recognition means for recognizing the locus information of the characters and / or figures traced on the writing area surface,
42. The handwriting input / output system according to claim 41, further comprising processing command means for transmitting a processing command based on the recognized recognition information to the information processing means.
The medium is
Whether information specifying the medium is defined in at least a part of the code value,
And / or whether at least part of the code value is defined with specific information for identifying the write area,
And / or whether at least a part of the code value defines an operation instruction (program activation, command input, data input / output / search, designation of a search target, etc.) meaning write processing,
And / or specific information for specifying a predetermined medium surface and / or the writing region surface in at least a part of the code value of the stream dot pattern formed in and / or around the writing region and / or Whether the operation instruction is defined and an icon representing the operation instruction is superimposed and printed,
And / or the stream dot pattern formed in and / or around the writing area is overlaid with associated text, graphics, or photographs,
And / or further comprising braille in and / or around the writing area,
And / or further comprising a narrow convex guide along the outer periphery of the writing area so that the imaging means for imaging the dot pattern does not protrude when writing to the writing area,
And / or stickable stickers, sticky notes, other media,
The transparent information input auxiliary sheet on which the stream dot pattern is formed, which is used in a superimposed manner on a monitor surface on which a writing area is displayed or on a printed medium. 42. The handwriting input / output system according to claim 41.
The medium is
A second medium on which a dot pattern in which code information for specifying the first medium surface and / or the writing area surface is repeatedly defined is further defined by using the medium provided with the writing area as a first medium. Prepared,
When the code information formed on the second medium is read by the imaging unit, the printing information on the first medium surface and / or the locus information drawn on the writing area surface, and / or the locus 43. The handwriting input / output system according to claim 41, wherein information related to the information is output.
The imaging means includes
Further comprising an indicator projection and / or at least one nib,
When tracing the character and / or figure traced with the pointing protrusion and / or at least one pen tip on the writing area surface, image the dot pattern on the locus near the pointing protrusion,
When the indicator projection and at least one nib are provided,
A storage space for storing the pen tip in the pointing protrusion;
43. The handwriting input / output system according to claim 41 or 42, wherein the pointing projection further includes a button for controlling storage / removal of the pen tip.
The imaging means includes
And further comprising an erasing means for erasing characters and / or figures drawn on the writing area surface on the instruction protrusion,
At the same time as erasing the area traced by the erasing means on the writing area surface, image the dot pattern on the locus near the erasing means,
The processing instruction means includes
46. The handwriting input / output system according to claim 45, wherein a processing command for deleting the trajectory information of the character and / or figure and the trajectory information are transmitted to an information processing means.
The trajectory information includes coordinate information of the imaging center of the imaging unit, or coordinate information of the position where the pointing protrusion or the pen tip provided near the imaging unit touches the writing area surface and the code information. ,
46. The handwriting input / output system according to claim 45, wherein the handwritten input / output system is obtained by interpolation calculation based on coordinate information defined in the dot pattern by the dot pattern analysis means.
The locus recognition means includes
It further includes a timing means,
The time measuring means measures the time when the imaging means touches the writing area surface and the time away from it,
43. The handwriting input / output system according to claim 42, wherein the imaging means records the order of the trace drawn by touching the writing area surface and the touching time, and transmits the recorded time to the information processing means.
The processing instruction means includes
Based on the code information and / or the recognition information, specific information and / or operation instructions (program activation, command input, data input / output / search, specification of search target, etc.) for specifying the writing area are uniquely specified 43. The handwriting input / output system according to claim 41 or 42, wherein input is possible.
The locus recognition means includes
Furthermore, based on the code information,
Regarding the contents to be written in the writing area,
Refer to pre-classified character and / or graphic information patterns,
The handwriting input / output system according to claim 42, wherein the trajectory information is recognized with high accuracy.
The trajectory recognizing means further converts the semantic information of the character and / or the figure, or one or more words composed of the characters, in advance corresponding to the code information, based on the analyzed change of the coordinate information. Refer to the defined semantic information code table (dictionary), convert it to semantic information code,
The processing instruction means includes
43. The handwriting input / output system according to claim 42, wherein the semantic information code and the processing instruction based on the semantic information are transmitted to the information processing means.
The processing instruction means includes
The result of recognition of the trajectory information by the trajectory recognition means is
The first operation to be entered as a document and / or figure,
Or the second operation to be entered as a comment,
Or as an edit symbol to edit documents and / or figures and / or comments,
As a third operation for editing a document and / or a figure and / or a comment based on the edit symbol,
44. The handwriting input / output system according to claim 43, wherein the information processing means is instructed.
An angle measuring means for measuring an inclination and a direction in which the imaging optical axis of the imaging means in contact with at least the medium surface is tilted with respect to a vertical line of the medium surface;
With reference to the tilt and the direction measured when the imaging means contacts the medium surface,
An angle change recognizing means for analyzing a predetermined change in the tilt and / or the direction due to an operation of tilting the imaging means, that is, a grid tilt operation;
The processing instruction means includes
43. The handwriting input / output system according to claim 41 or 42, wherein a predetermined processing instruction is transmitted to the information processing means based on the result of the analysis.
In the medium, the dot pattern arranged at a predetermined angle with respect to the direction of the writing area is formed in the writing area.
The angle measuring means and the angle change recognizing means are configured to be tilted with respect to the direction of the writing area based on the direction of the dot pattern read when the imaging means is in contact with the writing area surface. 54. The handwriting input / output system according to claim 53, wherein a predetermined change in the inclination is analyzed.
The handwriting input / output system includes:
The imaging optical axis of the imaging means is set substantially vertically with respect to the writing area surface, and is formed by the direction of the dot pattern read when contacting the writing area surface and the angle of the imaging means. Rotation angle reading means for reading the rotation angle,
Based on the rotation angle measured when the imaging means contacts the medium surface,
A rotation angle change recognizing means for analyzing a predetermined change in the rotation angle by rotating the imaging optical axis in a predetermined rotation direction by an operation of rotating the imaging means, that is, a grid turn operation; ,
The processing instruction means includes
43. The handwriting input / output system according to claim 41 or 42, wherein a predetermined processing instruction is transmitted to the information processing means based on the result of the analysis.
The handwriting input / output system includes:
Further, after imaging the writing area by the imaging means and obtaining the coordinate information and code information by the dot pattern analyzing means, instead of tracing or drawing characters in the writing area by the imaging means by a predetermined operation Audio recording means for recording audio;
Speech recognition means for referring to a predefined speech recognition information table (dictionary) corresponding to the code information, recognizing the recorded speech, converting it into character information, and transmitting it to the information processing means; 43. The handwriting input / output system according to claim 41 or 42.