JP2001282785A - Device and method for processing document image and computer-readable recording medium with document image processing program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decorate the image of an arbitrary character string in a document image by drawing a handwriting line segment corresponding to this arbitrary character string to make the appearance better. SOLUTION: Only by drawing the handwriting line segment through a line input part 3 corresponding to the image of the arbitrary character string desired to draw a line in the document image displayed on a display part 2, the line segment is drawn as a straight line by a curve correcting part 5 and a transmission processing part 71 while being overlapped on the image of this arbitrary character string so that this arbitrary character string image can be displayed transmissibly through the straight line. Thus, only by drawing the line segment by handwriting corresponding to the desired arbitrary character string, a user can beautifully draw the handwriting line segment as a straight line without damaging visibility of the arbitrary character string. Therefore, an effect similar to the case of overwriting the straight line on the arbitrary character string with a fluorescent pen while using a ruler can be easily provided.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image processing apparatus and a document image processing method for displaying a line segment input by hand with respect to an image of a document including characters, and a computer which stores a document image processing program. Possible recording medium, in particular, a document image processing apparatus and a document image processing method for processing and displaying a line segment input by handwriting corresponding to a desired character string, and a computer reading a document image processing program It relates to a possible recording medium.

[0002]

2. Description of the Related Art An apparatus having an editing function of adding an underline to a desired portion of a read document image is disclosed in Japanese Patent Application Laid-Open No. 61-273.
There is an image processing device described in Japanese Patent Application Laid-Open No. 070/070. This image processing apparatus has a display unit, and the image of the read document is displayed on the display unit. When the user draws an underline along a desired character string in a displayed document image, the user first designates this desired character string with a mouse or the like, and then prepares in advance "draw underline". Enter an edit command. As a result, the desired character string is underlined.

A device for adding an underline to an image of a character string developed into a dot image is disclosed in
There is a document processing device described in -30250. In this document processing apparatus, when an underline is drawn along the entire character string of one line in which the size and type of characters are different, a rectangular area surrounding each character of the character string and a rectangular area surrounding the entire character string Is used to underline the desired position at an arbitrary distance from the rectangular area surrounding the entire character string.

Conventionally, in a text printed on paper, a pen using a transparent water-based fluorescent pigment for ink (hereinafter, referred to as a pen) in order to allow a user to pay special attention to a desired character string in the text. The desired character string may be painted out using a highlighter pen. In this way, even a desired character string painted with ink can be confirmed by penetrating the applied ink.

However, conventionally, at most, as a method of modifying a desired character string image in a read document image for attention, an underlining method as described above is used. There has not been proposed any device that modifies a desired character string for attention with a handwritten line segment in various forms such as having the same effect as painting with a pen.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a document image processing apparatus and method capable of modifying the image of an arbitrary character string by drawing handwritten lines corresponding to the arbitrary character string in the document image and improving the appearance. And a computer-readable recording medium on which a document image processing program is recorded.

[0007]

According to an aspect of the present invention, there is provided a document image processing apparatus configured as follows. That is, the document image processing apparatus includes a display unit that displays a document image, and a line segment input unit that inputs a line segment that is handwritten to correspond to an image of an arbitrary character string in the document image displayed on the display unit. And a processing unit that processes a handwritten line segment input by the line segment input unit, wherein the processing unit is determined by a direction determining unit that determines a direction of the handwritten line segment, and is determined by the direction determining unit. A line extracting unit that extracts a line region including the image of the arbitrary character string in the region of the document image based on the direction and one or more character images in the document image; A line segment correcting unit that corrects a line that is drawn by being superimposed on the area of the line extracted by the line extracting unit; and an arbitrary character string that is drawn by superposing the straight line obtained by the correction by the line segment correcting unit. The image Transmitting to a and a transmission means for processing to be displayed on the display unit.

[0008] Therefore, by simply drawing a line segment by hand through the line input means corresponding to an image of an arbitrary character string to be drawn in the document image, the line segment correcting means and the transmissive means allow the line segment to be drawn. The arbitrary character string image is drawn as a straight line so as to be superimposed on the image of the character string and to be displayed through the straight line. In this way, the user can draw the handwritten line segment as a straight line without compromising the visibility of the arbitrary character string only by handwriting the line segment corresponding to the desired arbitrary character string. Therefore, the same effect as when a straight line is overwritten on the arbitrary character string using a ruler with a highlighter can be easily obtained.

In the above-described document image processing apparatus, the processing section corrects the handwritten line segment into a rectangular straight line surrounding an image of an arbitrary character string in the area of the line extracted by the line extracting means and displays it on the display section. May be further provided.

Accordingly, by simply drawing a line segment by hand through the line input means corresponding to an image of an arbitrary character string that the user wants to enclose in a rectangle in the document image, the image of the arbitrary character string can be obtained by the rectangle correcting means. Is modified to a straight rectangle so that it is displayed. In this manner, the user draws the handwritten line segment as a straight line surrounding the arbitrary character string only by handwriting a line segment corresponding to a desired arbitrary character string without impairing the visibility of the arbitrary character string. be able to.

In the above-described document image processing apparatus, the above-mentioned straight line has an arbitrarily set constant line width. Therefore, a line segment input by handwriting can be corrected to a straight line by designating its line width, so that an image of an arbitrary character string can be overlapped with a desired line width to draw a straight line, or a rectangle having a desired line width can be formed. The image of an arbitrary character string can be modified in a desired appearance.

In the above-described document image processing apparatus, the constant line width of the straight line may be set according to the size of an image of an arbitrary character string in the line area. Therefore, the line segment input by handwriting can be corrected to a straight line having a line width corresponding to the image of the arbitrary character string. Can be overlapped to draw a straight line, or surrounded by a rectangle having a line width corresponding to the size of the arbitrary character string image, without impairing visibility, and improving the appearance of the arbitrary character string. Images can be modified.

A document image processing apparatus according to another aspect of the present invention is configured as follows. Display means for displaying an image of a document, line input means for inputting a line handwritten so as to surround an image of an arbitrary character string in the document image displayed on the display means, and input by the line input means Processing means for processing the handwritten line segment. The processing means includes: a direction discriminating means for discriminating a direction of the handwritten line segment;
A line extracting means for extracting a line area including an image of an arbitrary character string in a document image area based on one or more character images, and a handwritten line segment; A line segment correction display unit that corrects the image into a straight line surrounding the image of the arbitrary character string and displays the straight line on the display unit.

Therefore, by simply drawing a line segment by hand through the line input means corresponding to an image of an arbitrary character string that the user wants to enclose in a rectangle in the document image, the image of the arbitrary character string is drawn by the rectangle correcting means. Is modified to a straight rectangle so that it is displayed. In this manner, the user draws the handwritten line segment as a straight line surrounding the arbitrary character string only by handwriting a line segment corresponding to a desired arbitrary character string without impairing the visibility of the arbitrary character string. be able to.

In the above-described document image processing apparatus, the processing means may further include a processing area designating means for designating a processing area for processing a handwritten line segment in the area of the document image.

Therefore, since the processing area designating means is provided, the user does not need to perform an operation on the document image such as designating the processing area in advance, and the operability is improved.
Further, since not all areas of the document image are to be processed, but only specified processing areas are to be processed, the processing means can execute the processing accurately and at high speed.

The line extracting means of the above-described image processing apparatus may include a candidate area extracting means and a line area determining means. In the candidate area extracting means, one or more candidate areas which are candidates for a line area are extracted based on the direction determined by the direction determining means and one or more character image areas in the document image area. Is done. The line area determining means determines a line area including an arbitrary character string in the extracted one or more candidate areas.

Therefore, even if a plurality of regions are extracted by the candidate region extracting unit, the line region determining unit determines which of the plurality of candidate regions a handwritten line segment has been drawn. . Therefore, it is not necessary for the user to perform an operation of designating a line area including an arbitrary character string with a handwritten line segment, thereby improving operability.

The line segment correcting means of the above-mentioned document image processing apparatus has an adjusting means. The adjusting means adjusts the length of the specified line segment to be optimal based on one or more character areas in the line area extracted by the line extracting means.

Therefore, utilizing one or more character regions in the region of the extracted line, that is, according to the region of the arbitrary character string, the length of the straight line is adjusted so as to be optimal, and the appearance is improved. A document image is obtained.

In the above-described document image processing apparatus, the line segment correcting means applies a straight line position correcting unit when one or more straight lines have already been drawn in the line area extracted by the line extracting means. You. In the straight line position correcting unit, the position of the straight line corresponding to the handwritten line segment may be corrected such that the straight line is drawn continuously with one or more straight lines already drawn in the area of the corresponding line.

Therefore, if a straight line has already been drawn in the same line area containing an arbitrary character string, the straight line formed by the newly input handwritten line segment is corrected so as to be continuous with the already drawn straight line. Since the document image is displayed after being displayed, a more attractive document image can be obtained.

In the above-described document image processing apparatus, the line segment correcting means may include, when one or more handwritten line segments are input in the line area extracted by the line extracting means, the handwritten line segment position correcting means. Is applied. The handwritten line segment position correcting means adjusts the position of a straight line corresponding to each of one or more handwritten line segments to correct the straight line into one continuous straight line.

Therefore, when a plurality of handwritten line segments are input in the same line area, the plurality of straight lines corresponding to these handwritten line segments are corrected and displayed as one continuous straight line. A good-looking document image is obtained.

In the above-mentioned document image processing apparatus, the line segment correcting means has a ruby line identifying means for identifying that the area of the line extracted by the line extracting means corresponds to a ruby line. When the ruby line identification unit identifies that the line region corresponds to the ruby line, the handwritten line segment is corrected to a straight line drawn along the ruby line.

Therefore, even if an arbitrary character string corresponds to a ruby line, it is possible to draw a straight line properly,
A good-looking document image is obtained.

According to still another aspect of the present invention, a document image processing method includes a display step of displaying a document image, and a handwritten line corresponding to an image of an arbitrary character string in the displayed document image. And a processing step of processing the input handwritten line segment. The processing step includes: a direction determining step of determining a direction of the handwritten line segment; a determined direction;
Based on one or more character images in the document image,
A line extracting step of extracting a line region including an arbitrary character string in a region of the document image; a line segment correcting step of correcting a handwritten line segment into a straight line drawn so as to overlap with the extracted line region; And a transmission step of processing an image of an arbitrary character string on which a straight line obtained by the correction in the correction step is drawn so as to be displayed through the straight line.

According to still another aspect of the present invention, a document image processing method includes a display step of displaying a document image, and a handwriting method surrounding an image of an arbitrary character string in the document image displayed by the display step. And a processing step of processing a handwritten line segment input in the line segment input step.
The processing step includes: a direction determining step of determining a direction of the handwritten line segment; and an arbitrary character in an area of the document image based on the direction determined by the direction determining step and one or more character images in the document image. A line extracting step of extracting a line region including a column image, and correcting the handwritten line segment into a straight line surrounding an image of an arbitrary character string in the line region extracted by the line extracting means, and displaying the straight line on the display unit According to still another aspect of the present invention having a line segment correction display step, a computer-readable recording medium storing a document image program for causing a computer to execute each of the above-described document image processing methods is provided. Is done.

[0029]

Embodiments of the present invention will be described below in detail with reference to the drawings.

(First Embodiment) FIGS. 1A and 1B show a functional configuration and a hardware configuration of a document image processing apparatus according to each embodiment of the present invention, respectively. 1A, the document image processing apparatus has an image input unit 1, a display unit 2, a line input unit 3, a line extraction unit 4, and a width setting unit 61 for inputting image data of a document including characters. And a curve correction unit 5 and a transmission processing unit 71. The row extracting unit 4 and the curve correcting unit 5 are referred to as a row extracting unit 4A and a curve correcting unit 5A in each embodiment described later. The curve correction unit 5 includes an adjustment unit 53 described in the first embodiment, a ruby line recognition unit 50 described below in a second embodiment, a correction data presence / absence determination unit 54 and a position correction unit 55, and Multiple free curve presence / absence determination unit 56 and multiple line position correction unit 5 shown in a third embodiment to be described later.
7 is further included.

In FIG. 1B, a document image processing apparatus has a CPU for centrally monitoring and controlling the apparatus itself.
The CPU 30 includes a scanner 1A as an example of the image input unit 1, a display unit 2, a tablet 3A as an example of the line input unit 3, and a program storage unit for storing various programs. Storage unit 3 including 31P and data storage unit 31D for storing various data
1. Communication I / F for connecting the communication network 33
An interface storage 32 and an external storage medium 35 storing various programs and various data are set, and an external storage medium driver 34 for accessing the external storage medium 35 is connected. As the communication network 33, for example, any communication network such as the Internet can be applied.

FIG. 1B shows an example of the storage contents of the data storage section 31D according to the first embodiment. The details will be described later.

In the operation of the document image processing apparatus shown in FIG. 1A, the image data of the document input by the image input unit 1 is developed and displayed in a bitmap font on the display unit 2, and the data storage unit 31D Is stored as image data 36D. While viewing the displayed document image, the user operates the line input unit 3 to input a freehand handwritten line segment (hereinafter, referred to as a free curve) in a manner superimposed on a character string at a desired position in the document image. The free curve includes a straight line that is not inclined, a straight line that is inclined, a polygonal line, and the like. The input free curve data is input data 36A.
Are sequentially stored in the data storage unit 31D. next,
The image data 36D and the input data 36
A, the discrimination line area data 36B is extracted using
It is provided to the curve correction unit 5. Discrimination line area data 36B
Is information indicating an area of a line where a free curve is drawn in the document indicated by the image data 36D. In the curve correcting unit 5, the given discrimination line area data 36B and the input data 3
6A, the input data 36A is corrected as described later, and the corrected data 36C is used as the corrected data 36C.
To be stored.

From the image data 36D input by the image input unit 1 and the correction data 36C obtained by the curve correction unit 5, the transmission processing unit 71 converts an image in which a line segment based on the correction data 36C is drawn in an overlapping manner. A process (hereinafter, referred to as a transmission process) is performed so that the image data is displayed in a transparent manner, and the processed image data is passed to the display unit 2 and displayed on the display unit 2. The input data 36A and the correction data 36C are input data 36AA in the following embodiments.
And correction data 36CC.

As the image input unit 1, here, the OCR
(Optical character reading device), a scanner 1A which is an image reading device such as a copying machine or an electronic camera is shown, but is not limited thereto. That is, a recording medium reading device such as a CD-ROM, a hard disk, a flexible disk, a magneto-optical disk, or a semiconductor memory in which an image obtained by reading a document in advance is stored may be used. The input of the document image data may be replaced with the image input unit 1 and may be a method of receiving the document image data via the communication network 33.

The display unit 2 includes a display control unit and an output unit (not shown). The output unit may be an image display device such as a CRT (cathode ray tube) or an LCD (liquid crystal display). In operation, the display unit 2 first displays the image input unit 1
Is displayed, and then a free curve based on the input data 36A provided from the line input unit 3 is displayed over the document image. Then, after the free curve correction processing, the display of the free curve by the input data 36A is deleted, and the line segment according to the corrected input data 36A given from the curve correction unit 5, ie, the line segment according to the correction data 36C is displayed on the display unit 2. Will be displayed on top.

Here, the line input unit 3 is not limited to the tablet 3A. For example, a pointing device such as a mouse, a cursor, and a joystick may be used. The input data 36A of the free curve input by the line input unit 3 is given to the display unit 2, the row extraction unit 4, and the curve correction unit 5, respectively.

The line extracting unit 4 and the curve correcting unit 5
And a storage unit 31.

Next, the overall processing flow of the document image processing apparatus according to the present invention will be described with reference to FIG. Step S
1 (abbreviated as S1 in FIG. 2), the image input unit 1
Input image data 36D. In the following description, the horizontal direction of the area of the image data 36D expanded into the bitmap is the X direction, the vertical direction is the Y direction, and the upper left end is the origin.
In step S <b> 2, the user inputs a free curve over the desired character string by handwriting from the line input unit 3 while viewing the image data 36 </ b> D displayed on the display unit 2. The input data 36A of the free curve is defined by a set of n time-series coordinate values such as (x (i), y (i)) (i = 1 to n) on the main surface of the tablet 3. Is done. Here, the input components of the free curve are set in the input data 36A in ascending order. In step S3, a processing area in which various processing to be described later is to be executed is specified for the area of the image data 36D, and the subsequent processing is performed in this processing area. It is more desirable to designate a partial area in this way and to process only this partial area, rather than to target the entire area of the image data 36D, because the processing speed is improved.

Next, in step S4, the following processing is performed on either the row (X direction) when the document of the image data 36D is written vertically or the column (Y direction) when the document is written horizontally. It is determined whether a free curve has been input in the horizontal direction (row direction, X direction) in order to determine.
In step S5, when it is determined that the free curve has been input in the horizontal direction, the processing after step S6 is executed, but when it is determined that the free curve has not been input in the horizontal direction,
The processing of step S5 is executed, and the rows and columns are exchanged in the processing after step S6.

In step S6, character area information is detected in the processing area as described later. In step S7, the area information of the detected character and the input data 3
The area information of the corresponding row is extracted based on the direction of the free curve indicated by 6A. In step S8, the free curve of the input data 36A is calculated based on the region information of the extracted row.
It is determined which line has been drawn. In step S9, the input data 36A is corrected as described later so that a free curve is drawn over the determined row, and corrected data 36C is obtained. In step S10, the desired character string in the document of the image data 36D is displayed on the display unit 2.
The transparent processing is performed such that the line segment based on the correction data 36C obtained by the correction is displayed transparently.
In S11, the data after the transmission processing and the image data 36D are displayed on the display unit 2. Here, the image input unit 1 of FIG. 1A is step S1, the display unit 2 is step S10, the line input unit 3 is step S2, and the line extraction unit 4 is steps S3 to S
8, the curve correction unit 5 has the processing function of step 9, and the transmission processing unit 71 has the processing function of step S10.

Next, the row extracting section 4 shown in FIG. 1A will be described. FIG. 3 shows the configuration of the row extracting unit 4. 3, the line extracting unit 4 includes a processing area specifying unit 6, a direction determining unit 7, a character region detecting unit 8, a line region extracting unit 9, and a line determining unit 10. Processing region designating unit 6, direction determining unit 7, and line determining unit 1
0 is a processing area designation unit 6A in an embodiment described later,
It is referred to as the direction determination unit 7A and the row determination unit 10A.

The processing area designating section 6 prevents the area of the image data 36D that is apart from the free curve indicated by the input data 36A from being affected by the later-described character area detection and line extraction processing. In other words, the processing area where the processing of character area detection and line area extraction is performed is specified within a certain range from the position indicated by the free curve input data 36A. This will be specifically described below with reference to FIG.

FIG. 4A shows a state in which a document corresponding to the image data 36D obtained from the image input unit 1 is displayed on the display unit 2. FIG. 4A shows the width WD of the image data 36D in the X direction and the height HT of the image data 36D in the Y direction. In FIG. 4B, the free curve data (input data 36A) is input to the displayed image data 36D, and the processing area PE (the portion surrounded by the dotted line in the figure) is specified. Is shown. Free curve input data 36A (x (i), y (i)) (i = 1
To n), the processing area PE is defined by rectangular area data (x (1) based on the coordinate data (x (1), y (1)) of the start point and the coordinate data (x (n), y (n)) of the end point. ) -W, y (1)
−H), (x (n) + W1, y (n) + H1). Here, (x (1) -W, y (1) -H) indicates the coordinate value of the upper left corner of the rectangular area (x (n) + W1, y (n)
+ H1) indicates the coordinate value of the lower right corner of the rectangular area. Here, the variables W, H, W1, and H1 are arbitrary constants, and if they are set to have a constant multiple of the line spacing or character spacing in the image data 36D of the document, the input image data 36D A corresponding processing area PE can be designated. In the present embodiment, the processing area PE is automatically set based on the position information indicated by the input data 36A of the free curve, but is not limited to this. That is, the processing area PE may be specified by the tablet 3 or the like as necessary.

The direction determining unit 7 determines whether the free curve is input in the X direction or the Y direction. This direction determination processing will be described with reference to the flowchart in FIG. First, in step S21, the start point coordinates (x (1), y (1)) and the end point coordinates (x (x)) of the free curve input data 36A (x (i), y (i)) (i = 1 to n).
(N), y (n)) and the absolute value | x of the difference in the X direction
(1) −x (n) | and the absolute value of the difference in the Y direction | y (1) −
y (n) |, and if the former is larger, it is determined in step S22 that a free curve has been drawn in the X direction. Conversely, if the latter is large, step S23
In, it is determined that the free curve is drawn in the Y direction.

FIG. 6 shows a specific example of the processing of FIG.
The absolute value of the difference in the X direction between the coordinates (23, 34) of one end point of the free curve L and the coordinates (31, 35) of the other end point is 8 (=
| 23−31 |), the absolute value of the difference in the Y direction is 1 (= | 34)
−35 |), and the former value is large. Accordingly, it is determined that the input data 36A of the free curve L drawn by hand with respect to the image data 36D of the document is drawn in the X direction.

The character area detecting section 8 calculates the free curve input data 36A for the image data 36D in the processing area PE.
Character area is detected based on the direction of. As an example of a method for detecting a character area, a projection method in each of the X and Y directions is adopted. This projection method refers to image data 36D
Has gray level data of a character portion and a blank portion, and projects the gray level data in a designated direction. That is, a projection of the image data 36D in the Y direction (or the X direction) is taken, and the processing area PE includes one or more candidate row areas (or one or more candidate column areas). , And areas that are not. Then, a projection in the X direction (or Y direction) is similarly made for one or more row regions (or column regions) as candidates, so that the processing region PE becomes a character candidate. It is divided into one or more areas and the other area. Further, a projection in the Y direction (or the X direction) is similarly taken for each of the character candidate regions, and one or more character regions are detected. This will be described with reference to FIGS.

For example, when the free curve L is input in the X direction (see FIG. 7A), first, for the image data 36D before the free curve L is input, first, the Y direction of the processing area PE (in the figure, A projection YSD (in the direction of the dotted arrow) is taken (see FIG. 7B). The processing area PE is a plurality of candidate row areas LE and non-row areas LE (shaded portions in the figure).
(See FIG. 7C). Next, when a projection XSD in the X direction (the direction of the dotted arrow in the figure) is taken for each of the candidate row areas LE (see FIG. 7D), the processing area PE includes one or more candidate rows. Character area C
It is divided into E and non-E. In FIG. 7 (D), although the characters “kawa” and “i” are one character,
These character areas are divided into a plurality of candidate character areas CE. Therefore, in order to correctly detect the character area CE even for such characters, if the distance in the X direction of each character area CE is equal to or less than a certain threshold V, the character area CE A method of integrating CE into one character area CE is employed (FIG. 7).
(E)). However, the threshold value V is an arbitrary constant. If the threshold value V is set to a value that is a constant multiple of the character spacing in the document of the image data 36D, processing according to the image data 36D can be performed. Furthermore, each character area CE
When a projection YSD in the Y direction is taken, a character area CE1 for each character is finally detected (FIG. 7).
(F)).

The row area extraction unit 9 inputs the free curve L input data 36 to the image data 36D in the processing area PE.
A row region is extracted based on the direction of A. As an example of the line area extraction method, there is a method using projections XSD and YSD of the respective character areas CE1 in the X and Y directions. In this method, a row area (or a column area) is extracted by taking a projection of information on each character area CE1 in a direction orthogonal to the direction of the input data 36A.
Specifically, for example, it is assumed that information on a plurality of character areas CE1 has been detected in advance as shown in FIG. Since it is determined that the free curve L is drawn in the X direction, a projection YSD in the Y direction is taken (FIG. 8B).
8), and the line area LE1 can be extracted (FIG. 8).
(C)). In addition, there is a method of extracting the line area LE1 without using information on the plurality of character areas CE1. For example,
In the projection XSD in the X direction in FIG. 7D, if information at both ends is cut off, one or more row regions LE1 are extracted.

Each of the extracted row areas LE1 has rectangular area data ((s (j), t (j)), (u
(J), v (j))) (j = 1 to m). Here, the variable m indicates the total number of the extracted row areas LE1, the variables (s (j), t (j)) indicate the coordinate values of the upper left corner of the rectangular area, and the variables (u (j), v ( j)) indicates the coordinate value of the lower right corner of the rectangular area. The information of one or more row areas LE1 extracted in this way is temporarily stored in an internal memory (not shown) of the CPU 30.

The line determining unit 10 determines which line in the document of the image data 36D the free curve L has been drawn. To determine this row, one or more row areas LE
1 ((S (j), t (j)), (u (j), v
(J))) (j = 1 to m) and input data 3 of the free curve L
6A (x (i), y (i)) (i = 1 to n). Specifically, when the free curve L is input in the X direction, the average value MY (= (y (1) + y (2) +... + Y (n)) of the y coordinate of the input data 36A of the free curve L ) /
n) and the average value MY in the Y direction in the information of the row area LE1
(J) (= (t (j) + v (j)) / 2) (j = 1 to
m) to determine the line on which the free curve L is drawn.

With reference to the flowchart of FIG. 9, the processing by the line discriminating unit 10 will be described for the case where the free curve L is input in the X direction. In step S31, the row area LE
1 ((s (j), t (j)), (u (j), v
(J))) (j = 1 to m) and input data 3 of the free curve L
6A (x (i), y (i)) (i = 1 to n) is the CPU
30 not shown internal memory and data storage unit 31
Read from D. In step S32, it is determined whether one or more line regions LE1 have been extracted by the line region extraction unit 9. If the line area LE1 has not been extracted, it is assumed that no character exists near the free curve L. Therefore, all the processing ends, and the curve correction unit 5 for the input data 36A of the free curve L is completed. No corrections are made.

On the other hand, the row area extracting unit 9 previously determines the row area L
If E1 has been extracted, in step S33, the average value MY of the y coordinate values of the input data 36A of the free curve L is obtained. In step S34, 1 is set to the variable j, and the data of the first row area LE1 is specified. In step S35, the average value MY is the minimum value t in the Y direction of the row area.
(J) and whether or not v (j)>MY> t (j) is satisfied, in order to determine whether or not v (j) is included, and if so, step The process proceeds to S38, and if not satisfied, proceeds to step S39.

In step S38, the variable LNE is set to "j".
Is set, and in a step S39, the next line area LE1 is set.
After that, the process proceeds to step S40. Note that the variable LNE is a line area L on which the free curve L is drawn.
This is a variable in which a value for specifying E1 is set.

In step S40, it is determined whether or not the processing has been performed on the data of all the row areas LE1 read in step S31. If the processing has not been performed, the process returns to step S35, but if it is determined that the processing has been performed, a series of processing ends, and the line area LE1 indicated by the value of the variable LNE becomes the line area LE on which the free curve L is drawn. Is determined. Hereinafter, this is referred to as a determination line.

The discrimination line is a rectangular area data ((s,
t), (u, v)) are stored in the data storage unit 31D as discrimination row area data 36B. Here, the variable (s, t) indicates the coordinate value of the upper left corner of the rectangular area, and the variable (u, v) indicates the coordinate value of the lower right corner of the rectangular area. When the free curve L is input in the Y direction, MY is set to MX in steps S33 and S35 in the processing of FIG.
What is necessary is just to replace t with s and v with u.

Here, a specific process when the free curve L is input in the X direction will be described with reference to FIGS. 10 (A) to 10 (D). FIG. 10A shows a state where the free curve L is input by handwriting to the image of the document on the display unit 2. FIG. 10B shows the result of the detection of a character area by the character area detection section 8 in the processing area PE specified by the processing area specification section 6. In FIG. 10C, information about each detected character area and the direction discriminating unit 7 are shown.
The result obtained by extracting one or more row areas LE1 by the row area extraction unit 9 from the direction indicated by the input data 36A of the free curve L determined by the above is shown. FIG.
(D) shows the result of the line discriminating unit 10 discriminating which of the one or more detected line regions LE1 has been drawn to the line of the one or more detected line regions LE1. . The results of these processes need not be displayed sequentially on the screen, but are displayed here so that the user can easily determine that the processes are being executed appropriately.

Next, the curve correcting section 5 of FIG. 1 will be described. The curve correction unit 5 determines how to correct the free curve L with respect to the discrimination row obtained by the row extraction unit 4 and to which position to draw. In the correction processing of the free curve L, the area data ((s, t), (u, v)) of the discrimination row and the input data 36A of the free curve L (x (i), y (i)) (i = 1 ~ N)
The information about the position of is used. Usually, the free curve L is corrected to a straight line having a predetermined line width. here,
The case where the input data 36A is corrected so that when the free curve L is input in the X direction and a straight line based on the free curve L is drawn over the discrimination row area of the discrimination row area data 36B.

Referring to FIG. 11, a description will be given of a curve correction process when a free curve L is input in the X direction. In step S41, the discrimination row area data 36B ((s, t), (u, v)) and the input data 36A of the free curve L (x (i), y (i)) (i = 1 to n)
Are read. In step S42, 1 is set to the variable i, and the data of the first coordinate in the input data 36A is designated. In the step S43, the input data 3
The data of the x coordinate of 6A is left as it is, and the data of the y coordinate is the intermediate position ((t + v)
/ 2). As a result, the input data 36A is converted into correction data 36C (xc (i), yc (i)). In step S44, data of the coordinates following the input data 36A is specified. In step S45, it is determined whether the processing has been performed on the data of all the coordinates of the input data 36A. If not, step S4
Returning to step 3, if it has been performed, the width data th is obtained from the width setting unit 71 in step S46. After that, the curve correction processing ends, and the correction data 36C (xc (i), yc
(I)) (i = 1 to n) and the width data th are obtained and stored in the data storage unit 31D.

The correction data 36C has the coordinates (xc
(1), yc (1)) and coordinates (xc (n), yc
(N)) may be defined as a straight line having the width of the width data th. The rectangular area may be such that the upper left coordinate is (x (1), t) and the lower right coordinate is (x (n), v). FIGS. 13 (A) and 13 (B) show the specific correction processing when the above-mentioned free curve L is input in the X direction.

Here, the width setting processing included in the curve correction processing when the input data of the free curve is input in the X direction will be described with reference to the flowchart of FIG.

In step S51, the discrimination line area data 36
Read the information of B ((s, t), (u, v)). In step S52, it is determined whether or not the line width is specified by the user. If the line width is specified, in step S53,
The value of the line width specified by the user is set in the width data th. If the user has not specified the line width, the width (v−t) +1 of the discrimination line area in the Y direction is set as the width data th in step S54, and the width of the discrimination line area is determined in accordance with the Y direction size. A width line can be drawn over the string.
When the width setting processing is completed, the width data th is given to the above-described curve correction processing.

When the free curve L is input by handwriting in the Y direction, step S43 in the flowchart of FIG.
Xc (i) = (s + u) / 2, yc
(I) = y (i), and in step S54 in the flowchart of FIG. 12, Y may be changed to X, t may be changed to s, and v may be changed to u. As a result, the input data 36A of the free curve L is corrected so that a straight line having the width of the width data th is drawn at the middle position of the discrimination line area, and the correction data 36
C is obtained.

The coordinate value of the end point of the line segment indicated by the correction data 36C is corrected by the adjusting unit 53 by using the information of each character area CE1 so that the line segment indicated by the correction data 36C can be easily viewed. The length of the line segment indicated by the data 36C may be modified. For example, FIG.
As shown in (A), in each of the character areas CE1 of the characters "tree" and "kawa", a free curve L is drawn from the middle of the character area CE1. In such a case, the free curve L is adjusted by the adjustment unit 53 to each of the character areas CE1 of the discrimination line.
Is determined in the direction of the free curve L to be longer than the predetermined threshold value T, and the correction data 36C is corrected. However, the threshold value T is an arbitrary constant, and when a value that is a constant multiple of the character spacing of the text of the image data 36D is set, the correction processing of the correction data 36C according to the image data 36D can be performed. When the free curve L is drawn with a length equal to or longer than the threshold value T in the direction of the free curve L for each character area CE1,
It is determined that a free curve has been drawn for the character in the character area CE1, and the correction data 36C is corrected so that a straight line is drawn for the entire range of the character area CE1 in the X direction (FIG. 14B). reference). Also, a certain character area C
The correction data 36C is corrected so that the end point of the straight line is located between E1 and the character area CE1 adjacent to E1 (see FIG. 14C). As described above, the length of the straight line indicated by the correction data 36C of the free curve L can be corrected using the information of the character area CE1 of the discrimination line corresponding to the free curve L.

The curve correcting section 5 has a ruby line recognizing section 50. The ruby row recognition section 50 compares the width of the row area LE1 (or column area) determined by the row determination section 4 in the Y direction (or X direction) with the width of another row area LE1 (or column area). If the value is equal to or less than the predetermined threshold value U, it is recognized that the line is a line of ruby touched by kanji or the like. However, the threshold value U is an arbitrary constant, and the image data 36
By setting a value that is a constant multiple of the character spacing or the line spacing in the document D, it is possible to perform ruby line recognition processing according to the image data 36D.

FIG. 15 shows a curve correction process when a line segment is drawn in a ruby line for a character string including a character with ruby. Here, one of the following processes (1) to (3) is selected, and the free curve L is corrected.

(1) A process of drawing a line with a line width including a line of ruby (see arrow (1) in FIG. 15).

(2) The line of ruby is ignored, the line determination process is performed again, a new determination line is determined, and a line is drawn based on the determined determination line (arrow (2) in FIG. 15). )reference). More specifically, the newly determined discrimination line is a line consisting of only characters with ruby. Then, a line segment having a line width is drawn for a new determination line.

(3) In the same manner as the above-mentioned process (2), the line discrimination process is performed again, a new discrimination line is determined, and a predetermined width is determined based on the determined discrimination line. Processing for drawing a line (see arrow (3) in FIG. 15). Specifically, in the newly determined discrimination line, a line is drawn with a line width including the ruby line and the new discrimination line for a character with ruby, and a new line is drawn for a character without ruby. Draw a line having a line width that includes only the appropriate determination line.

Next, the transmission processing section 71 will be described.
FIG. 16 is a processing flowchart of the transmission processing unit 71 of FIG. The transmission processing unit 71 performs transmission processing so that image data corresponding to a position where a line segment according to the correction data 36C and the width data th is drawn is transmitted from the line segment and displayed. In this transmission processing, the image data 36D (Ix (w) (j), Iy (h) (j)) acquired by the image input unit 1
(W = 1 to WD, h = 1 to HT, j = 1 to p) and the correction data 36C (xc (i), yc (i)) (i = 1 to
n), color data c (j) of free curve data (j = 1 to
p), transmittance α. Here, p is the number of planes constituting the image data 36D, and is often 1 for a monochrome image and 3 for a color image. Usually, c (j) and α are specified in advance by the user. c (j) is 0 ≦ c (j) ≦ 1 for a binary image,
0 ≦ c (j) ≦ 2 for gray or color images
Is often an integer of 55, α is an arbitrary constant of 0 ≦ α ≦ 1, and the closer to 1, the thinner the display of the line segment by the correction data 36C, the more the image of the image data 36D is transmitted through the line segment. Will be displayed. Further, it is assumed that the display of the line segment by the correction data 36C is darker as the value approaches 0, and the image by the image data 36D is displayed so as to be harder to transmit through the line segment.

Next, the transmission processing will be described with reference to the flowchart of FIG. In step S61, the image data 36D (Ix (w) (j), Iy (h) (j))
(W = 1 to WD, h = 1 to HT, j = 1 to p), correction data 36C (xc (i), yc (i)) (i = 1 to
n), color data c (j) of free curve data (j = 1 to
p), the transmittance α is read from the data storage unit 31D.

In the next step S62, the correction data 36C
Is designated as the first coordinate data (i = 1). In step S63, the image data 36D is displayed such that the image data 36D is displayed through the line segment of the correction data 36C.
Is processed. Specifically, a value obtained by multiplying each value of the image data 36D by the transmittance α and a value obtained by multiplying the color data c (j) of the free curve data by 1−transmittance α are used. , And treats them as added.

In the next step S64, the correction data 36C
Next coordinate data is specified. In step S65, it is determined whether or not processing has been performed on all coordinate data of the free curve data. If not, step S6
Returning to step 3, if the processing has been performed, a series of transmission processing ends.

Here, the image data 36D after the transmission processing is added to the original image data 36D, but another area where the original image data 36D is copied is prepared in the data storage unit 31D. Alternatively, processing may be performed so that the image data 36D after the transmission processing is added thereto.

(Second Embodiment) Next, a second embodiment will be described. In the above-described first embodiment, a sequential process is adopted in which the process after step S3 in FIG. 2 is performed as soon as one free curve L is completely handwritten. The completion of the input of the free curve L is detected by a known technique. In other words, after the free curve L is input, the elapse of a predetermined time is measured using a timer (not shown). After the elapse of the predetermined time, the data of the next free curve L is not obtained. Input completion is detected.

In the second embodiment, the correction data 36C and the image data 36D obtained by the method shown in the first embodiment are already stored in the data storage section 31D and displayed on the display section 2. In this state, a process when a new free curve L is input for the same image data 36D is shown.

The device configuration of this embodiment is the same as that of the first embodiment, but differs in the functional configuration as follows. That is, as shown in FIG. 1A, the curve correction unit 5 includes the correction data presence / absence determination unit 54 and the position correction unit 55 according to the second embodiment.

When the free curve L is input for the discrimination row indicated by the discrimination row area data 36B ((s, t), (u, v)), the correction data presence / absence determination section 54 It is determined whether or not the corrected free curve has already been drawn. If the corrected data presence / absence determination unit 54 determines that the corrected free curve has already been drawn for the discrimination line, the position correction unit 55 determines that the currently input free curve L is The position is adjusted to be continuous with the modified free curve already drawn on the line.

At least one correction data 36C corresponding to the discrimination row into which the free curve L has been input is stored in the data storage unit 3.
The processing by the correction data presence / absence determining unit 54 for determining whether or not the correction data has already been registered in 1D will be described with reference to the flowchart of FIG. 17 in the case where the free curve L is input in the X direction for the determination line. Here, it is assumed that one or more correction data 36C has already been registered in the data storage unit 31D.

In step S71 of FIG. 17, the input data 36A (x
(I), y (i)) (i = 1 to n) and one or more correction data 36C (x (i) (j), y (k) (j)) (k
= 1 to r) (j = 1 to n (k)) (where the variable n
(K) is the total number of coordinate data constituting the correction data 36C) is read from the data storage unit 31D.

In step S72, 1 is set to a variable k, and one or more of the read out correction data 36C is set.
The first correction data 36C is designated. Step S7
In step 3, the average value MY of the y coordinate of the input data is obtained. In step S74, if | MY-y (k) (1) | is smaller than a certain threshold value R, the process proceeds to step S77, where the free curve L is placed on the same line as the line segment indicated by the specified correction data 36C. It is determined that it has been pulled, and the process ends. Otherwise, in step S75, the variable k is incremented by one, and the next correction data 36C is designated.

However, the threshold value R is an arbitrary constant, and if a value that is a constant multiple of the character spacing or the line spacing in the document indicated by the image data 36D is set, processing according to the image data 36D can be performed. it can.

In step S76, in order to detect the correction data 36C that satisfies the condition of step S74,
All correction data 36C read in step S71
Is determined, that is, whether k> r is satisfied. If the correction data 36C satisfying the condition of step S74 is not detected,
In step S78, it is determined that there is no line segment already drawn on the same line as the free curve L, and the process ends.

When the free curve L is input in the Y direction, the variable y is replaced with the variable x and the variable Y is replaced with the variable X in steps S73 and S74 in FIG.
What is necessary is just to process similarly.

Next, a display process in the case where a free curve L is drawn on the same line as a line on which a line segment has already been drawn based on the correction data 36C in the data storage unit 31D will be described with reference to FIG.
This will be described with reference to (A) to (E).

The image data 36 previously displayed on the display unit 2
When the user inputs a free curve L by hand from the line input unit 3 to the document D (see FIG. 18A), the processing area PE is designated, and one or more character areas CE1 are detected. (See FIG. 18B), the row area LE1 is extracted (see FIG. 18C). A straight line having a width th1 is drawn so as to overlap the discrimination row indicated by the extracted row area LE1,
A curve correction process is performed on the free curve L (FIG. 18).
(D)). At this time, since the correction data presence / absence determination unit 54 determines that the line segment L2 based on the correction data 36C has already been drawn on the discrimination line, the line segments L1 and L2 become one continuous line segment L3. As described above, the correction data 36C of the free curve L is corrected by the position correction unit 55 (FIG. 18E). Here, the line segments L1 and L2 correspond to the width data t.
Assume that each has h0 and th1. Specifically, in order to be drawn over the character string, the minimum value Y0-th0 / in the Y direction is obtained from the y-coordinates Y0 and Y1 and the width data th0 and th1 of each correction data 36C.
2 and Y1-th1 / 2 and the maximum values Y0 + th0 / 2 and Y1 + th1 / 2, and using these maximum and minimum values, the line segments L1 and L2 are centered at (maximum value + minimum value) / 2. A line segment L3 having a maximum value−minimum value + 1 as a line width.
Is corrected to correction data 36C indicating As a result, the position of the different line segments L1 and L2 drawn on the discrimination line is corrected, and the line segment L3 is drawn (FIG. 18).
(E)).

(Third Embodiment) In the first and second embodiments described above, the case where the free curve L is sequentially handwritten and processed is described. However, in the third embodiment, When a plurality of free curves L are input, a procedure for processing these at once will be described. In the third embodiment, when the user selects collective processing as the input mode of the free curve L, if a plurality of free curves L are input by hand, the collective processing (not shown) of the line input unit 3 is performed. The following processing is performed by operating buttons and the like.

The configuration of the document image processing apparatus according to the third embodiment is the same as that of the first embodiment, but has the following new functions in the functional configuration. That is, when q free curves L are input from the line input unit 3, the input data 36A for each free curve L is (x (k) (i),
y (k) (i)) (k = 1 to q) (i = 1 to n (k))
(However, the variable n (k) is defined as the total number of coordinates constituting the k-th free curve L). Then, it is assumed that the components of the corresponding free curve L in the input direction are set in ascending order in the input data 36A.

Since the row extracting unit 4 performs processing on each of the q free curves L, the discrimination row area data 36B ((s (k),
t (k)), (u (k), v (k))) (k = 1 to q)
Are obtained and stored in the data storage unit 31D. FIG. 21 shows an example of the contents of such a data storage unit 31D.
However, the variables (s (k), t (k)) represent the coordinate value of the upper left corner of the corresponding discrimination line area (rectangular area) by the variable (u
(K), v (k)) are the corresponding discrimination row areas (rectangular areas)
Indicates the coordinate value of the lower right corner of. As to the discrimination row area data 36B corresponding to each of the discrimination rows obtained in the row discrimination section 10, as shown in FIG.
In the label value L (k) =
k (k = 1 to q) is assigned.

The curve correcting section 5 includes a plural free curve presence / absence determining section 56 and a plural line position correcting section 57 in the third embodiment. The plurality of free curve presence / absence determination units 56 determine the same discrimination row, that is, the same discrimination row area data 36B.
((S (k), t (k)), (u (k), v (k)))
It is determined whether a plurality of free curves L are drawn for (k = 1 to q). When the plurality of free curve presence / absence determining units 56 determines that the plurality of free curves L are drawn on the same determination line, the plurality of line position correction units 57 determines the plurality of free lines L drawn on the same determination line. The plurality of correction data 36C corresponding to the free curve L are corrected so as to have continuity.

Here, the case where the plurality of free curves L are input by handwriting in the X direction will be described with reference to the flowchart of FIG.

First, in step S81, a plurality of input data 36A (x (k)
(I), y (k) (i)) and label value L (k) (k
= 1 to q) (i = 1 to n (k)) are stored in the data storage unit 31.
Read from D. In step S82, 1 is set to the variable k, and the first input data 36A of the plurality of read input data 36A is designated. In the next step S83, k + 1 is set to the variable j, and the next input data 36A is specified. In step S84, k
Average value Y (k) of y coordinate values of the second input data 36A
(= (Y (k) (1) + y (k) (2) +... + Y (k)
(N (k)) / n (k)) and j-th input data 36
Average value y (j) of y coordinate values of A (= (y (j) (1) +
y (j) (2) + ... + y (j) (n (j)) / n
(J)).

In step S85, | Y (k) -Y
If it is determined that the value of (j) | is smaller than the predetermined threshold value R, in step S86, the label value L (k) and the label value L
Is compared with (j), and based on the comparison result, in each of steps S87 and S88, processing is performed so that the larger label value is changed to the smaller label value.

However, the threshold value R is an arbitrary constant, and if a value that is a constant multiple of the character spacing or line spacing in the document of the image data 36D is set, processing corresponding to the image data 36D is performed. Can be.

In step S89, the variable j is incremented by one, and the next input data 36A is designated. In step S90, it is determined whether j> q is satisfied, and the processing in steps S84 to S89 is repeated until j> q is satisfied. On the other hand, if it is established, in step S91, the variable k
Is incremented by one, and the next input data 36A is designated. Then, the processing of steps S83 to S91 is repeated until it is determined that k> q in step S92.

When the processing is performed on all the combinations of the q free curves L in this manner, the discrimination row area data 36B for all the free curves L corresponding to the same discrimination row have the same value. It will have a label value L (k).

When the free curve L is input in the Y direction, the variable y in S84 and S85 in FIG.
Then, the variable Y may be replaced with the variable X, and the same processing may be performed.

As a result of the above processing, the data storage unit 3 shown in FIG.
In 1D, when a plurality of discrimination row area data 36B having the same label value L (k) is registered, that is, when it is determined that a plurality of free curves L are drawn on the same discrimination row, As shown in FIGS. 20A to 20E, the correction data 36C of the plurality of free curves L is corrected by the multiple line position correction unit 57, and displayed as follows.

That is, two free curves L from the line input unit 3
When (1) and L (2) are input by handwriting (FIG. 20)
(A)), processing areas PE (1) and PE (2) are designated, and one or more character areas CE1 are provided for each processing area.
Is detected (see FIG. 20B), and as a result,
The row areas LE1 (1) and LE1 (2) are extracted as discrimination rows (see FIG. 20C).

Then, the straight line L having the line widths of the width data th0 and th1 is superimposed on each discrimination line by the curve correcting unit 5.
Free curves L (1) and L (1) are drawn so that 1 and L2 are drawn.
The process of correcting the correction data 36C of (2) is performed (see FIG. 20D). At this time, the free curve L for the same discrimination line by the plural
Since it is determined that (1) and L (2) have been subtracted,
The corresponding two correction data 36C are set by the multiple line position correction unit 57 so that these two free curves have continuity.
Is corrected.

That is, in order to draw a line segment on the desired character string, y
From coordinates Y0 and Y1 and width data th0 and th1, Y
The minimum values Y0-th0 / 2 and Y1-th1 / 2 and the maximum values Y0 + th0 / 2 and Y1 + th1 / 2 in the direction are obtained, and (maximum value + minimum value) / 2 is calculated using the maximum value and the minimum value. The center is corrected to correction data 36C indicating a straight line having a line width of maximum value−minimum value + 1 (see FIG. 20E).

In each of the above embodiments,
The data based on the line segment input by hand on the document displayed on the display unit 2 is stored as another data corresponding to the image data 36D as shown in the data storage unit 31D.

(Fourth Embodiment) In each of the embodiments described above, a case has been described in which a handwritten line segment is corrected to a straight line superimposed on a discrimination line. May be corrected to a line segment surrounding. In the present embodiment, a handwritten line segment input so as to surround the character string of the sentence image is corrected to a rectangle formed by a straight line surrounding the character string.

The difference between the apparatus configuration of the present embodiment and that of FIG. 1 is that a row extracting section 4A and a curve correcting section 5A are provided instead of the row extracting section 4 and the curve correcting section 5 in FIG. It is in the point given. Other components are the same as those in FIG. 1 and the description is omitted. The difference between the row extracting unit 4A and the row extracting unit 4 in FIG. 3 is that the processing area specifying unit 6 in FIG.
The direction determining unit 7 and the row determining unit 10
A, in that it is replaced by each of the direction determination unit 7A and the row determination unit 10A. The other parts of the row extracting unit 4A are shown in FIG.
Therefore, the description is omitted.

First, the processing area designating section 6A will be described. FIGS. 22A and 22B are diagrams illustrating a processing procedure of the processing area specifying unit 6A according to the fourth embodiment. The processing area designating unit 6A determines that the area of the image data 36D
An area away from the free curve indicated by the input data 36AA is not affected by character area detection and line extraction processing described later. That is, the processing area where the processing of character area detection and line area extraction is performed is the free curve input data 36A.
It is designated within a certain range from the position indicated by A. FIG.
A specific description will be given below with reference to (A) and (B).

FIG. 22A shows a state in which a text corresponding to the image data 36 D obtained from the image input unit 1 is displayed on the display unit 2. FIG. 22A shows the width WD of the image data 36D in the X direction and the height HT of the image data 36D in the Y direction. In FIG. 22 (B), data of the free curve LL (input data 36AA) is input to the displayed image data 36D, and the processing area PE
(Portion surrounded by a dotted line in the figure) is shown. Input data 36A of the free curve LL (x (i),
y (i)) (i = 1 to n) is the minimum coordinate data m in the X direction
in (x (i)) and maximum coordinate data max (x (i))
In addition, the processing area PE is defined as rectangular area data (min (x (i))-W, m) based on the minimum coordinate data min (y (i)) and the maximum coordinate data max (y (i)) in the Y direction.
in (y (i))-H), (max (x (i)) + W
1, max (y (i)) + H1). Where (min (x (i))-W, min (y (i))-H)
Indicates the coordinate value of the upper left corner of the rectangular area, and (max (x
(I)) + W1, max (y (i)) + H1) indicates the coordinate value of the lower right corner of the rectangular area.

Here, variables W, H, W1 and H1
Is an arbitrary constant, and the image data 36D
Is set to have a value that is a constant multiple of the line spacing and character spacing in, the processing area PE corresponding to the input image data 36D can be designated. In the present embodiment, the processing area PE is automatically set based on the position information indicated by the input data 36AA of the free curve, but is not limited to this. That is, the processing area PE may be specified by the tablet 3 or the like as necessary.

Next, the direction determining unit 7A will be described.
The direction determining unit 7A determines whether the free curve input by handwriting is input in the X direction or the Y direction so as to surround the character string along the periphery. This determination processing will be described with reference to the flowchart in FIG.

First, in step S101, the absolute value | max (x (i))-min (x) of the difference between the maximum value and the minimum value in the X direction for the input data 36AA of the free curve.
(I)) The absolute value of the difference between | and the maximum value and the minimum value in the Y direction | m
ax (y (i))-min (y) (i)) |, and if the former is larger, it is determined in step S102 that the free curve LL has been drawn in the X direction. Conversely, if the latter is large, it is determined in step S103 that the free curve LL has been drawn in the Y direction.

FIG. 24 shows a specific example of the processing of FIG. The minimum and maximum coordinate values of the free curve LL in the X direction are 2
1 and 51, the minimum and maximum coordinate values in the Y direction are 23 and 43, and the absolute value of the difference in the X direction is 30 (= |
51-21 |), the absolute value of the difference in the Y direction is 20 (= | 43)
−23 |), and the former value is large. Therefore, it is determined that the input data 36AA of the free curve LL drawn by hand with respect to the text image data 36D is drawn in the X direction.

Next, the row discriminating section 10A will be described.
FIG. 25 is a processing flowchart of the row determining unit 10A according to the fourth embodiment. The line discriminating unit 10A calculates the free curve L
It is determined to which line in the document of the image data 36D L has been drawn. In order to determine this row, the data ((s (j), t (j)) of one or more row areas LE1,
(U (j), v (j)) (j = 1 to m) and free curve LL
Input data 36AA (x (i), y (i)) (i = 1
To n) are used. Specifically, when the free curve LL is input, the input data 36AA of the free curve LL is the minimum value min (y (i)) and the maximum value max (y) of the y coordinate.
(I)), the minimum value t in the Y direction of the row area LE1
A comparison is made to determine whether both (j) and the maximum value v (j) fall within each other, and the row on which the free curve LL is drawn is determined.

Referring to the flowchart of FIG. 25, the processing performed by the row determining unit 10A will be described for the case where the free curve LL is input in the x direction. In step S111, the data ((s (j), t (j)), (u
(J), v (j))) (j = 1 to m), input data 36AA of free curve LL (x (i), y (i)) (i = 1 to
n) are read from the internal memory (not shown) of the CPU 30 and the data storage unit 31D. In step S112, it is determined whether one or more row regions LE1 have been extracted by the row region extraction unit 9. If the line area LE1 has not been extracted, it is assumed that no character exists near the free curve LL. Therefore, all the processing ends, and the curve correction unit 5A for the input data 36AA of the free curve LL ends. No processing is performed.

On the other hand, the row area extracting unit 9 determines in advance the row area L
If E1 has been extracted, in step S113, the minimum value min (y (i)) and the maximum value (max (y (i))) of the y coordinate value of the input data 36AA of the free curve LL are obtained. In S114, 1 is set to the variable j, and the data of the first row area LE1 is specified.

In step S115, the target row area LE
1 is the input data 36 of the free curve LL.
Max (y (i))> t (j)> min (y (i)) to determine whether it is between the minimum value and the maximum value of AA
Is determined, the process proceeds to step S116 if the condition is satisfied, and to step S118 if the condition is not satisfied, the process proceeds to the next line area LE.
1 is specified.

In step S116, step S115
Of the target row area LE1 among the row areas LE1 satisfying the condition
Is the maximum value v (j) of the input data 36A of the free curve LL.
Max (y (i))> v (j)> min (y to determine if A is between the minimum and maximum values
It is determined whether or not (i)) is established. If it is established, the process proceeds to step S117. If not, step S118 is performed.
Proceed to.

In step S117, "j" is set in the variable LNE, and in step S118, data of the next row area LE1 is specified. Then, step S11
Go to 9. The variable LNE is a variable in which a value for specifying the line area LE1 on which the free curve LL is drawn is set.

In step S119, step S111
It is determined whether or not the processing has been performed on the data of all the row areas LE1 read in. If the process has not been performed, the process returns to step S115. However, if it is determined that the process has been performed, a series of processes ends, and the line area LE1 indicated by the value of the variable LNE becomes a line region in which the free curve LL is drawn. Is determined. Hereinafter, this is referred to as a determination line.

The discrimination line is a rectangular area data ((s,
t), (u, v)) are stored in the data storage unit 31D as discrimination row area data 36B. Here, the variable (s, t) indicates the coordinate value of the upper left corner of the rectangular area, and the variable (u, v) indicates the coordinate value of the lower right corner of the rectangular area. When the free curve LL is input in the Y direction, FIG.
In the process of 5, the variable y is changed to the variable x and the variable t is changed to the variable s
And the variable v may be replaced with the variable u.

Here, specific processing when the free curve LL is input in the X direction will be described with reference to FIGS. 26 (A) to 26 (D). FIG. 26A shows a state in which the free curve LL is input by handwriting to the image of the sentence on the display unit 2. In FIG. 26 (B), the character area detection unit 8 within the processing area PE specified by the processing area specification unit 6A
The result of the detection of the character area is shown. The rectangular character area detected here is used at the time of the curve correction processing, and the rectangles (e (z), f (z)), (g (z),
h (z)) (z = 1 to w). Here, w is the number of detected rectangles.

In FIG. 26 (C), one or more lines are extracted by the line region extraction unit 9 from the information on each detected character region and the direction indicated by the input data 36AA of the free curve LL determined by the direction determination unit 7A. The result of extraction of the row region LE1 is shown. In FIG. 26 (D), the result of the line discriminating unit 10A discriminating whether the free curve LL has been drawn in any one of the detected one or more line regions LE1 is determined. Is shown. The results of these processes do not have to be sequentially displayed on the screen, but are displayed here so that the user can easily grasp that the processes are being executed appropriately.

Next, the curve correcting section 5A will be described. FIG. 27 is a processing flowchart of the curve correction unit 5A according to the fourth embodiment. The curve correcting unit 5A includes a line extracting unit 4A.
Then, it is determined how the free curve LL is corrected for the discrimination row obtained in the step and the position to be drawn. In the process of correcting the free curve LL, ((s, t),
(U, v)) and input data 36AA (x
Information on the position of (i), y (i)) (i = 1 to n) is used. Usually, the free curve LL is corrected to a straight line having a predetermined line width. Here, the free curve LL is X
A case in which the input data 36AA is modified so that, when the input is made in the direction, a straight line rectangle based on the free curve LL is drawn in the discrimination row area of the discrimination row area data 36B.

Referring to the flowchart of FIG. 27, the curve correction processing when the input data 36AA of the free curve LL is input in the X direction will be described.

At step S121, the data storage unit 31D
From the discrimination row area data 36B ((s, t), (u,
v)) and character area data ((e (z), f (z)),
(G (z), h (z))) (z = 1 to w), free curve L
L input data 36AA (x (i), y (i)) (i =
1 to n) are read.

In step S122, width data th is obtained from the width setting unit 61. In step S123, the free curve L
The first coordinate data of the input data 36AA of L (i =
1) is designated. In step S124, the free curve L
The minimum value min (x) of the input data 36AA of L in the X direction
(I)), a maximum value max (x (i)), a minimum value min (y (i)) in the Y direction, and a rectangle ((min (x (i))) having the maximum value max (y (i)) as an end point. ), Min (y
(I)), max (x (i)), max (y (i)))
And this is called the free curve maximum area.

The x coordinate of the input data 36AA of the free curve LL at the minimum position in the X direction among the one or more character areas CE1 which fits completely in the free curve maximum area is x0,
Input data 3 of the free curve LL at the maximum position in the X direction
The x coordinate of 6AA is x1.

In step S125, the x coordinate of the input data 36AA of the free curve LL is processed as follows. That is, (1) subtract th / 2 from x0 when smaller than x0, (2) add th / 2 to x1 when larger than x1, and (3) leave it as it is when it is between x0 and x1. . Further, the y coordinate is processed as follows.
That is, (1) subtracting th / 2 from the minimum value t when it is smaller than the minimum value t of the discrimination row area LE1, and (2) when the maximum value v of the discrimination row area LE1 is larger than th / 2.
(3) When it is between the minimum value t and the maximum value v of the discrimination row area, the value is kept as it is. The data obtained in this manner is converted into corrected data 36CC (xcc (i), y
cc (i)).

In step S126, the next coordinate point of the input data 36AA of the free curve LL is designated. In step S127, it is determined whether or not processing has been performed on all coordinate points of the input data 36AA of the free curve LL. If not, the process returns to step S125. If it is determined that the processing has been performed on all the points, the curve correction processing ends, and the correction data 36CC (xcc (i), ycc
(I)) (i = 1 to n) and width data th are obtained.

In the correction data 36CC, the upper left coordinate is (xcc (1), ycc (1)) and the lower right coordinate is (xcc (1), ycc (1)).
cc (n), ycc (n)) and data indicating a rectangle having a line width indicated by the width data th. FIG. 29A shows a specific curve correction process when the input data 36AA of the free curve LL is input in the X direction.
And (B).

Here, the input data 36A of the free curve LL
The width setting processing included in the curve correction processing when A is input in the X direction will be described with reference to the flowchart in FIG.

In the step S131, the discrimination row area LE1
((S, t), (u, v)) is read out. In step S132, it is determined whether the user has designated the width data th. If the user has specified the width data th, the width data th specified by the user is fetched in step S133. If the user has not specified the width data th, an arbitrary constant is set in the width data th in step S134. In this case, if the width data th is set so as to have a value that is a constant multiple of the line spacing or character spacing of the text according to the image data 36D, the width of the line segment corresponding to the image data 36D can be specified. When the width setting processing is completed, the width data th is given to the curve correction processing.

When the input data 36AA of the free curve LL is input in the Y direction, the minimum value y0 in the Y direction instead of the X direction is determined in step S124 of the flowchart in FIG.
And the maximum value y1 are obtained, and the x coordinate of the input data 36AA of the free curve LL may be corrected as follows in the correction formula of step S125.

That is, (1) When the value is smaller than the minimum value s of the discrimination line area LE1, th / 2 is subtracted from the minimum value s (xc
c (i)) = (s-th / 2), (2) determination line area LE
If it is larger than the maximum value u of 1, 1 is added to the maximum value u (xcc (i)) = u + th / 2). (3) If it is between the minimum value s and the maximum value u of the discrimination row area LE1 The value is used as it is (xcc (i) = x (i)). Also, y
The coordinates may be corrected as follows. (1) y
If it is smaller than 0, subtract th / 2 from y0 (ycc
(I) = y0−th / 2), (2) When y1 is larger than y1, th / 2 is added to y1 (ycc (i) = y1 + th /
2), (3) When it is between y0 and y1, the value is left as it is (ycc (i) = y (i)).

Further, the value of the end point of the correction data 36CC obtained in this manner may be corrected to make it easier to see using the information of the line area LE1 and the information of the character area CE1. For example, as shown in FIGS. 30A and 30B, between a row area LE1 and an adjacent row area LE1,
The data of the free curve LL can be corrected using the information of the character area CE1, such as drawing from between the character area CE1 and the adjacent character area CE1.

Here, before the free curve LL is input, whether the curve correction unit to be applied is the one described in the first embodiment or the one described in the present embodiment. Is set before inputting the free curve LL. However, if a function for automatically determining whether or not the input data of the free curve LL is input so as to surround the free curve LL is prepared, the input free curve LL can be obtained. From the input data of
It is possible to automatically determine whether to perform the curve correction according to the present embodiment or to perform the curve correction according to the present embodiment, and perform the correction.

Further, in order to easily explain the operation by the user, the present embodiment has been described on the assumption that the input data 36AA of the free curve LL is inputted so as to surround the character string. It is apparent that the same processing as in the present embodiment is performed not on the character string but in the same manner as in the third embodiment, but also on a line segment input overlapping the character string.

(Fifth Embodiment) In the various processes including the processes according to the flowcharts described in the above embodiments, the CPU 30 reads out a document image processing program stored in the program storage section 31P in advance and executes it. It is realized by doing. These programs are stored in a recording medium readable by the CPU 30, such as the program storage unit 31P. As this recording medium,
An external storage medium 35 may be used. In that case, by setting the external storage medium 35 in the external storage medium driver 34, the program read from the external storage medium 35 is loaded into a predetermined area including the internal program storage unit 31P, and thereafter, , May be read and executed by the CPU 30. It is assumed that the loading program is stored in advance in a storage area (not shown) of the apparatus main body.

Note that the program storage medium may be a recording medium configured to be separable from the apparatus main body. That is,
Tape systems such as magnetic tapes and cassette tapes, magnetic disks such as flexible disks and hard disks, and disk systems including optical disks such as CD-ROMs, MOs, MDs and DVDs; cards such as IC cards (including memory cards) and optical cards System or mask RO
M, EPROM, EEPROM and flash ROM
For example, the recording medium may include a semiconductor memory such as a semiconductor memory.

Further, as shown in FIG. 1B, the document image processing apparatus has a configuration connectable to a communication network 33 including the Internet, so that a document image processing program is downloaded from the communication network 33 to the image processing apparatus. For this purpose, the medium may carry an image processing program.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[Brief description of the drawings]

FIGS. 1A and 1B are block diagrams showing a configuration of a document image processing apparatus according to the present invention.

FIG. 2 is a schematic processing flowchart of the document image processing apparatus according to the present invention.

FIG. 3 is a block diagram illustrating a configuration of a row extracting unit in FIG. 1;

FIGS. 4A and 4B are diagrams for explaining a processing area designating unit in FIG. 3;

FIG. 5 is a flowchart of a free curve direction determination process performed by the direction determination unit in FIG. 3;

FIG. 6 is a diagram for explaining a direction discriminating unit in FIG. 3;

FIGS. 7A to 7F are diagrams for describing a character area detection unit in FIG. 3;

FIGS. 8A to 8C are diagrams for explaining a row area extracting unit in FIG. 3;

FIG. 9 is a processing flowchart of a row discriminating unit in FIG. 3;

FIGS. 10A to 10D are diagrams for explaining specific processing according to the first embodiment of the present invention;

FIG. 11 is a processing flowchart for correcting a free curve by the curve correcting unit of FIG. 1;

FIG. 12 is a processing flowchart of a width setting unit in FIG. 1;

FIGS. 13A and 13B are diagrams for explaining processing according to the processing flowchart of FIG. 12;

FIGS. 14A to 14C are diagrams illustrating a process of correcting correction data using a character area.

FIG. 15 is a diagram for describing processing when a free curve is input to a document image including a character with ruby in the curve correction unit in FIG. 1;

FIG. 16 is a flowchart of a process performed by the transmission processing unit of FIG. 1;

FIG. 17 is a flowchart of a process performed by a correction data presence / absence determination unit in FIG. 1;

FIGS. 18A to 18E are diagrams illustrating an example of a process performed by the multiple line position correction unit in FIG. 1;

FIG. 19 is a processing flowchart of the multiple free curve presence / absence determination unit in FIG. 1;

FIGS. 20A to 20E are diagrams illustrating another example of the processing performed by the multiple line position correction unit in FIG. 1;

FIG. 21 is a diagram illustrating a content example of a data storage unit 31D applied to the third embodiment of the present invention.

FIGS. 22A and 22B are diagrams illustrating a processing procedure of a processing area specifying unit according to a fourth embodiment.

FIG. 23 is a processing flowchart of a direction determining unit according to a fourth embodiment.

FIG. 24 is a diagram illustrating a specific example of the processing in FIG. 23;

FIG. 25 is a processing flowchart of a row determining unit according to the fourth embodiment.

FIGS. 26A to 26D are diagrams illustrating a specific processing procedure when a free curve according to the fourth embodiment is input in the X direction.

FIG. 27 is a processing flowchart of a curve correcting unit according to the fourth embodiment.

FIG. 28 is a flowchart of a width setting process according to the fourth embodiment.

FIGS. 29A and 29B are diagrams illustrating a specific procedure of a free curve correction process according to the fourth embodiment.

FIGS. 30A and 30B are diagrams illustrating another procedure of the free curve correction processing according to the fourth embodiment.

[Description of Signs] 1 image input unit, 2 display unit, 3 line input unit, 4, 4A
Row extraction unit, 5, 5A curve correction unit, 61 width setting unit, 7
1 Transmission part, L, LL free curve.

Claims (11)

[Claims] A display unit for displaying a document image; a line segment input unit for inputting a handwritten line segment corresponding to an image of an arbitrary character string in the document image displayed by the display unit; Processing means for processing a handwritten line segment input by the line segment input means, the processing means comprising: a direction discriminating means for discriminating a direction of the handwritten line segment; and a direction discriminated by the direction discriminating means. A line extraction unit configured to extract a region of a line including the image of the arbitrary character string in a region of the document image based on one or more character images in the document image; A line segment correcting unit for correcting the line extracted by the unit into a line drawn so as to overlap, and an image of the arbitrary character string in which the line obtained by the correction by the line segment correcting unit is drawn so as to overlap. , And a transmission means for processing to be displayed on the display unit passes through the straight line, the document image processing apparatus. 2. The processing unit corrects the handwritten line segment to a rectangular straight line surrounding the image of the arbitrary character string in the line area extracted by the line extracting unit and displays the straight line on the display unit. The document image processing apparatus according to claim 1, further comprising a rectangle correction unit that performs the correction. 3. The document image processing apparatus according to claim 1, wherein the straight line has a arbitrarily set constant line width. 4. The document image processing apparatus according to claim 3, wherein the constant line width is set according to the size of the image of the arbitrary character string in the line area. 5. The line segment correcting unit has an adjusting unit that optimally adjusts the length of the straight line based on one or more character regions in the line region extracted by the line extracting unit. The document image processing apparatus according to any one of claims 1 to 4. 6. The straight line corresponding to the handwritten line segment when one or more straight lines are already drawn in the line area extracted by the line extracting unit. 2. A linear position correcting means for correcting the position of the line so as to be continuously drawn with one or more of the lines already drawn in the region of the row.
6. The document image processing apparatus according to any one of claims 5 to 5. 7. The one or more handwritten line segments when one or more of the handwritten line segments are input for the line region extracted by the line extracting unit. 7. The document image processing apparatus according to claim 1, further comprising a handwritten line segment position correcting unit that adjusts the position of each of the corresponding straight lines to correct the straight line into one continuous straight line. 8. The ruby line identifying unit includes a ruby line identifying unit that identifies that an area of the line extracted by the line extracting unit corresponds to a ruby line. 2. The method according to claim 1, wherein when the region of the line is identified as corresponding to the line of the ruby by the identification unit, the handwritten line segment is corrected to the straight line drawn along the region of the ruby line. 8. The document image processing apparatus according to any one of claims 7 to 7. 9. A display unit for displaying an image of a document, a line segment input unit for inputting a line segment handwritten so as to surround an image of an arbitrary character string in the document image displayed on the display unit, Processing means for processing a handwritten line segment input by a line segment input means, the processing means comprising: a direction discriminating means for discriminating a direction of the handwritten line segment; and a direction discriminated by the direction discriminating means. A line extracting unit that extracts a line region including the image of the arbitrary character string in the document image region based on one or more character images in the document image; Corrected to a straight line surrounding the image of the arbitrary character string in the area of the line extracted in,
A document image processing apparatus, comprising: a line segment correction display unit that displays on the display unit. 10. A display step of displaying a document image, a line segment input step of inputting a handwritten line segment corresponding to an image of an arbitrary character string in the document image displayed by the display step, A processing step of processing a handwritten line segment input in the line segment input step, wherein the processing step includes: a direction determining step of determining a direction of the handwritten line segment; and a direction determined by the direction determining step. Extracting a line area including the arbitrary character string in an area of the document image based on one or more character images in the document image; and extracting the handwritten line segment by the line extraction. A line segment correcting step of correcting a line drawn over the region of the row extracted in the step, and a correction obtained by the line segment correcting step. An image of said arbitrary character string whose serial straight line is drawn on top, and a transmission step of processing to be displayed through the straight line, the document image processing method. 11. A computer-readable recording medium storing a document image processing program for causing a computer to execute the document image processing method, wherein the document image processing method comprises: a display step of displaying a document image; A line input step of inputting a handwritten line segment corresponding to an image of an arbitrary character string in the document image displayed by the display step; and processing a handwritten line segment input by the line input step A direction determining step of determining a direction of the handwritten line segment, a direction determined by the direction determining step, and an image of one or more characters in the document image. A line extraction step of extracting a line region including the arbitrary character string in the document image region based on the A line segment correcting step of correcting a line segment into a straight line drawn over the region of the row extracted in the line extracting step; and the straight line obtained by the correction by the line segment correcting step is drawn over the line. And a transmission step of processing an image of an arbitrary character string so that the image is displayed while passing through the straight line.