JPH0993427A - Image processing unit and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the image processing unit and its method by which an image area attribute is discriminated more accurately and efficiently. SOLUTION: A character image object area possibly being a character image area, a non-character image area and a character image is classified from an AC component of compressed image data received by a character image extract section 4 and an attribute storage section 5 stores the attribute. As to the character image object area, an average color extract section 6 extracts its average color and a representative color extract section 8 extracts two representative colors from an area discriminated to be the character image area in the vicinity. A color synthesis discrimination section 7 discriminates whether or not the average color of a notice image area is synthesized, based on the extracted representative color and discriminates it to be a character image area when the synthesis is possible and to be a non-character image area when the synthesis is impossible.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method for determining the attributes of individual areas of color image data.

[0002]

2. Description of the Related Art As a conventional image compression apparatus, for example, a JPEG algorithm is used for still image compression, and an M is used for moving image compression.
Those using the PEG algorithm are known. These algorithms use DCT (discrete cosine transform) to map an image in the spatial frequency domain, concentrate the amount of data on low frequency components, and cut high frequency components to reduce the amount of data.

A block diagram of an image compression apparatus using the JPEG algorithm is shown in FIG.

The image data input from the image input section 91 is input to the DCT section 92. The DCT unit 92 further performs a discrete cosine transform for each component of the luminance Y and the color differences Cb, Cr for each block of a predetermined size, for example, 8 × 8 pixels, and a spatial frequency called a DCT coefficient as shown in FIG. Get the ingredients.

The quantizer 93 quantizes the value of the spatial frequency component by dividing it by the value of the quantization table and the quantization coefficient. For example, the quantization table is shown in FIG.
As shown in, the value corresponding to the spatial frequency in the block is determined, and generally, the larger the value corresponding to the high frequency component, the larger the value. Since it is a large value, the obtained numerical value is small, and as a result, it is represented by a small number of bits, so that high frequency components are cut. Further, one quantized coefficient for controlling the compression rate is defined for each block.

The variable-length coding unit 94 encodes the quantized value by entropy coding while performing zigzag scanning from the low frequency component to the high frequency component, and outputs the quantized value to the compressed image storage unit 95.

FIG. 10 is a block diagram of an apparatus for extracting a character image area from compressed image data obtained by the above apparatus.
Shown in

The compressed image data input by the compressed image input unit 1 is decoded by the variable length decoding unit 2 in the reverse procedure of the variable length coding unit 94.

In the inverse quantizer 3, inverse quantization is performed by the inverse procedure of the quantizer 93 to obtain D for each block.
Obtain the CT coefficient.

In the character image extraction unit 11, each block is
It classifies into character image blocks and non-character image blocks, and outputs the classification result to the attribute storage unit 5. In general, a character image tends to have a larger absolute value of the value of the DCT coefficient in the luminance component Y in the low frequency region than a non-character image. According to "DCT Coding Method Using Adaptive Quantization" of the Institute of Image Electronics Engineers of Japan, Vol. 20, No. 5, the sum of absolute values of specific DCT coefficients in the low frequency region 110 is shown for each block as shown in FIG. Is determined to be a character image block, and if the score is larger than a predetermined threshold value, it is determined to be a character image block, and otherwise it is determined to be a non-character image block.

[0011]

In the above image processing apparatus, it is difficult to set an optimum threshold because the distribution of DCT coefficients varies depending on the size, position, color, etc. of characters. I have a problem. That is, there is a problem that the threshold value that is optimal for one image is not necessarily appropriate for another image.

[0012]

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image processing apparatus and method capable of more accurately and efficiently determining a region attribute of an image.

In order to solve this problem, for example, the image processing apparatus of the present invention has the following configuration. That is, an image processing apparatus that classifies each area in an image compressed in area units into two attributes, and based on the image data of the area in the compressed state, the two attributes and the attributes having both the possibility With respect to the first determination unit for determining which is the region having both the possible attributes, the compressed image of the already determined region having a predetermined attribute near the attention region is restored, The second determination unit determines the attribute of the image area of interest based on the characteristic amount of the restored image area.

According to a preferred embodiment of the present invention,
An image in a compressed state is an image that has been orthogonally transformed,
The restoration process is preferably an inverse orthogonal transform process.

Further, it is desirable that the first judging means judges which of the three attributes of the character image area, the non-character image area and the character image candidate area.

Further, in the second judging means, a plurality of representative colors are extracted from the area already judged to be the character image area,
It is desirable to determine whether or not the target character image candidate area becomes the character image area by determining whether or not the average color of the target character image candidate area can be combined.

Another object of the present invention is to provide an image processing apparatus which can efficiently determine a character area and a non-character area in compressed color image data.

To solve this problem, for example, the following configuration is provided. That is, it is an image processing apparatus for inputting color image data and identifying a character image area and a non-character image area in the color image data, and a feature amount of a spatial frequency coefficient for each predetermined area unit in the input color image data. An extracting means for extracting the character image area, a character image area, a character image candidate area, and a non-character image area based on the extracted feature amount, and An average color extracting means for extracting average color information in the character image candidate area, a representative color extracting means for extracting a plurality of representative color information from the character image area in the vicinity of the target character image candidate area, and the representative color extracting means. Based on a plurality of extracted representative color information, the determination means for determining whether the average color information extracted by the average color extraction means can be combined,
A determination unit that determines whether the target character image candidate region is a character image region or a non-character image region based on the determination result.

[0019]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 2 shows a hardware block diagram of an image processing apparatus embodying the present invention.

In the figure, 21 is an OS, various processing programs in this embodiment, a hard disk for storing compressed data, 22 is a tablet which is a pointing device for an operator to give instructions to the apparatus, 23 is a liquid crystal display for displaying information to the operator from the apparatus, 24 is a computing unit for performing inverse DCT, inverse quantization, and other computations, 25 is an OS and various processing programs in the embodiment from the hard disk 21. A random access memory (RAM) 26 for reading and storing data in the middle of processing, etc. is a control unit (constituted by a CPU or the like) for performing processing based on a program.

FIG. 1 is a block diagram of an image processing apparatus according to the first embodiment of the present invention. As will be apparent from the following description of each component in the illustrated block configuration, the compressed image input unit 1 is the hard disk 2 in the embodiment.
1 corresponds, the variable length decoding unit 2, the dequantization unit 3, and the inverse DCT unit 9 are performed by the calculation unit 24 in FIG. 2, and the attribute storage unit 5 corresponds to the area secured in the RAM 24. Then, other processing, for example, processing of passing data from the image input unit 1 to the variable length decoding unit 2, and the character image search unit 10 and the like are performed by the control unit 21. Although the compressed image input unit 1 corresponds to the hard disk 21 here, the compressed image input unit 1 is not limited to this and may be any other unit as long as the compressed image intended by the embodiment is supplied. Of course, part or all of the processing performed by the control unit 26 may be realized by a dedicated circuit.

Now, the basic operation of this apparatus will be described with reference to the flowchart of FIG. The process based on the flowchart is read out from the hard disk 21 to the RAM 25 and executed as a program.

First, in step S501, the compressed image data input by the compressed image input unit 1 is decoded by the variable length decoding unit 2 in the reverse procedure of the variable length coding unit 94.

In step S502, the inverse quantizer 3
Then, the DCT coefficient for each block is obtained by performing inverse quantization by the inverse procedure of the quantization unit 93.

In step S 503, the character image candidate extraction unit 4 classifies each block into a character image block, a character image candidate block, and a non-character image block, and outputs the classification result to the attribute storage unit 5. Here, the character image candidate block cannot be uniquely classified into a character image block or a non-character image block at this stage,
Say each possible block.

Further, in step S 504, each character image candidate block is classified into a character image block and a non-character image candidate block, and the classification result is output to the attribute storage unit 5.

The above is the outline of the overall processing in this embodiment. The content of step S503 in the above processing will be described in detail with reference to the flowchart of FIG. As a matter of course, the program for performing such processing is also RAM
25 is read from the hard disk 21.

First, in step S601, the character likeness is scored based on the distribution of the DCT coefficients for the luminance component Y. Generally, character images are more DC than non-character images.
The absolute value of the value of the T coefficient in the low frequency region tends to be large.
Therefore, for each block, the specific area 11 shown in FIG.
The sum of absolute values of the DCT coefficient of 0 is calculated and used as the score representing the character.

In step S602, if the score is larger than the predetermined threshold value T1, the process proceeds to step S603,
Outputs the attribute of character image block. If not, the process proceeds to step S604, and the score is further compared with a predetermined threshold value T2 (where threshold value 1> threshold value 2). If the score is larger than the threshold value 2, the process proceeds to step S605, and the attribute of the character image candidate block is output. Otherwise, the attribute of non-character image block is output.

As a general tendency, as shown in the example of FIG. 12, when the pixels constituting the character are dense like the block 121, the pixels constituting the character are coarse like the character image block and the block 122. And character image candidate block,
If no character is included like the block 123, it is determined to be a non-character image block.

As a result of the above, two threshold values T1, T2 (T1
> T2), one input image block can be classified into three, that is, a character image block, a character image candidate block, and a non-character image block. Here, the threshold value T1 is a value sufficient to be recognized as a character image when it is more than that, and the threshold value T2 is a value sufficient to be determined as a non-character image area when it is less than that. Therefore, an image block having a value between the thresholds T1 and T2 may have any possibility, and is therefore determined as a character image candidate block.

Next, the contents of the process of step S504 will be described in detail with reference to the flowchart of FIG.

In step S701, the average color extraction unit 6 obtains the average color Cm of the character image candidate block based on the direct current (DC) component of the DCT coefficient for each of the luminance component Y and the color difference components Cb, Cr. . If the DC component with respect to the luminance component Y is high, it is a bright block, and if it is low, it is a dark block. Also, the color difference component C
If the DC component for b is high and the DC component for the color difference component Cr is low, the bluish block is low. If the DC component for Cb is high, the red block is red, and if the DC components for Cb and Cr are both high, it is purple. Block, and if the DC components for Cb and Cr are both low, it is a green block.

Therefore, the block average color Cm is represented by the vector amount of the luminance component Ym and the color components Cbm, Crm, which are the DC components (average values) of the respective components, as described above.

In step S702, the character image search unit 10 reads the attribute of the block located near the character image candidate block from the attribute storage unit 5 and searches for the character image block.

In step S703, the color combination determination unit 7 checks whether or not the average color Cm of the character image candidate block can be combined from the two colors C1 and C2 representing the searched character image block ( How to obtain C1 and C2 will be described later). C1 is a luminance component Y1 and a color component Cb
1 and Cr1, C2 is a luminance component Y2, and a color component C
It is a vector quantity represented by b2 and Cr2.

If C1 and C2 are combined at a ratio of "x to (1-x)" to obtain Cm, x = (Ym-Y1) / (Y2-Y1) = (Cbm-Cb1) / ( Cb2-Cb1) = (Crm-Cr1) / (C
r2-Cr1) holds.

Therefore, (Ym-Y1) / (Y2-Y1), (Cbm-Cb1)
If the three values of / (Cb2-Cb1) and (Crm-Cr1) / (Cr2-Cr1) are the same and 0 <x <1, synthesis is possible. However, in reality, considering the error, it is considered that the three values can be combined if they are within a predetermined range.

If they can be combined, the process advances to step S704 to output the attribute of a character image block. If not, the process proceeds to step S705 to search for another character image block. If it cannot be searched for, the process advances to step S706 to output the attribute of non-character image block.

Next, the process of obtaining the two representative colors C1 and C2 of the character image block in the vicinity of the focused character image candidate block in step S703 will be described with reference to FIG.
This will be described in detail with reference to the flowchart of.

In step S801, the inverse DCT unit 9
Is the inverse D for the DCT coefficient for each of the luminance component Y and the color difference components Cb, Cr for the character image block.
By performing the CT calculation, the pixel value of each component of each pixel of the block is obtained.

In step S802, similar colors are grouped together based on the value of each component of each pixel.
This is to absorb the error in the color of the image input characters and the color in the middle of processing.

In step S803, it is checked which group has many colors in the block. The representative color C1 is selected from the most frequent group, and the representative color C2 is selected from the next most frequent group color.

The character image block is generally composed of a character color, a background color, and a color in which both are mixed at the boundary,
When the representative colors C1 and C2 are determined by the above method, the character color becomes C1, the background color becomes C2, or the background color becomes C1,
The character color becomes C2. Also, the average color is a color in which the character color and the background color are mixed.

Therefore, in the compressed image, the inverse DCT operation is performed only on a very small number of blocks having particularly high character-likeness to extract the character color and the background color. It can be seen that the peripheral blocks having an average color of belong to the same character string.

By adding this determination condition, it is possible to perform the separation more reliably as compared with the conventional method of separating the character image and the non-character image only by the distribution of the DCT coefficients.

Moreover, only a small number of blocks perform the inverse DCT operation, which operates at high speed.

In the above description, a still image by JPEG compression has been described, but a moving image by MPEG compression can be similarly used.

In this embodiment, DCT, quantization,
Although an example of performing compression by a procedure called entropy coding has been shown, any method may be used as long as it is a compression method that can extract the DC component of the spatial frequency of each block during the expansion of the compressed data. Absent.

<Second Embodiment> In the second embodiment of the present invention, contrary to the first embodiment, only the character image candidate blocks are subjected to the inverse DCT operation to represent the representative colors C1 and C.
Ask for 2.

A character image block is generally composed of a character color, a background color, and a color in which both are mixed at the boundary. For each pixel in the character image candidate block, C1,
If C2 and pixels of a color that can be combined with these two colors are present in a predetermined ratio or more, it is highly likely that it is a character image block. Otherwise, it is likely to be a non-character image block.

Further, if the average color Cm is obtained from the character image blocks near the character image candidate block and C1 and C2 can be combined into Cm, the possibility that the character image candidate block is the character image block becomes higher.

In order to classify the character image and the non-character image more accurately, both the DCT coefficient distribution information and the color information of each pixel may be used, but in terms of speed, the inverse DCT is used.
It is desirable to use as few blocks as possible. In the present embodiment, for a block that is difficult to determine by character or non-character or DCT distribution alone, the determination is performed using both pieces of information, so that there is an effect that more accurate determination can be performed.

<Third Embodiment> The present invention is not limited to the first and second embodiments. For example, a general-purpose device is equipped with a storage medium storing a program that realizes the functions of the first and second embodiments, and stored in the storage medium (for example, a floppy disk, a CD-ROM, a memory card, or the like). The implemented program may be read out to a memory and implemented.

In this case, at least the storage medium shown in FIG.
As shown in FIG. 3, a first determination module that determines which of the two attributes or both of the attributes is possible based on the image data of the compressed area, and the both of the possible attributes The second determination module that restores the compressed image of the already determined area having a predetermined attribute near the attention area and determines the attribute of the attention image area based on the feature amount of the restored image area. Will be equipped with.

As described above, according to this embodiment,
In a compressed image compressed in units of a plurality of areas, a means for obtaining a first feature after restoring a part of the area to individual pixel information, and a second feature while a part of the other area remains compressed. By providing a means for obtaining the pixel information and a means for determining whether or not both areas have the same attribute based on the relationship between both features, both pixel information and compressed information can be used. Will be more accurate.

Moreover, since the calculation for obtaining the pixel information only needs to be performed for a part of the area, the processing can be performed at a higher speed than the method of restoring the entire screen to obtain the pixel information.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Further, it goes without saying that the present invention can be applied to the case where it is implemented by supplying a program to a system or an apparatus. In this case, the storage medium storing the program according to the present invention constitutes the present invention. Then, by reading the program from the storage medium to a system or an apparatus, the system or the apparatus operates in a predetermined manner.

[0060]

As described above, according to the present invention, the area attribute of an image can be determined more accurately and efficiently.

According to another aspect of the present invention, it is possible to efficiently determine the character area and the non-character area in the compressed color image data.

[0062]

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment.

FIG. 2 is a hardware block diagram of the image processing apparatus according to the embodiment.

FIG. 3 is a diagram showing a data structure of DCT coefficients.

FIG. 4 is a diagram showing a quantization table.

FIG. 5 is a flowchart showing a basic operation of the image processing apparatus of the first embodiment.

FIG. 6 is a flowchart showing a process of classifying into a character image, a character image candidate, and a non-character image.

FIG. 7 is a flowchart showing a process of classifying character image candidates into character images or non-character images.

FIG. 8 is a flowchart showing a process of obtaining a representative color of a character image block.

FIG. 9 is a block diagram of an image compression apparatus using a JPEG algorithm.

FIG. 10 is a block diagram of a conventional device for extracting a character image area.

FIG. 11 is a specific D for obtaining a character-likeness score.
It is the figure which showed the position of CT coefficient.

FIG. 12 is a diagram showing an example of classification of character images, character image candidates, and non-character image control charts.

FIG. 13 is a diagram showing modules stored in a storage medium according to a third embodiment.

[Explanation of symbols]

 1 Compressed image input unit 2 Variable length decoding unit 3 Inverse quantization unit 4 Character image candidate extraction unit 5 Attribute storage unit 6 Average color extraction unit 7 Color synthesis determination unit 8 Representative color extraction unit 9 Inverse DCT unit 10 Character image search unit

Claims (13)

[Claims] 1. An image processing apparatus for classifying each area in an image compressed in area units into two attributes, wherein the two attributes and the possibility of both of the two attributes are calculated based on image data of the compressed area. For the first determination unit that determines which one of the attributes is present, and for the area having both the possible attributes, a compressed image of an already determined area having a predetermined attribute in the vicinity of the attention area is displayed. An image processing apparatus, comprising: a second determination unit that restores and determines the attribute of the image area of interest based on the characteristic amount of the restored image area. 2. The image processing apparatus according to claim 1, wherein the compressed image is an image that has been orthogonally transformed. 3. The image processing apparatus according to claim 2, wherein the restoration process is an inverse orthogonal transform process. 4. The first determining means is a character image area,
The image processing apparatus according to claim 1, wherein the image processing apparatus determines which of the three attributes of the non-character image area and the character image candidate area is. 5. The second determination means extracts a plurality of representative colors from an area that has already been determined to be a character image area and determines whether or not the average color of the target character image candidate area can be combined. The image processing apparatus according to claim 1, wherein it is determined whether or not the target character image candidate area is a character image area. 6. An image processing method for classifying each region in an image compressed on a region-by-region basis into two attributes, wherein the two attributes and the possibility of both are classified based on image data of the region in a compressed state. For the first determination step of determining which one of the attributes is present, and for the area of both of the possible attributes, a compressed image of an already determined area having a predetermined attribute near the attention area is obtained. A second determination step of restoring and determining the attribute of the image area of interest based on the characteristic amount of the restored image area. 7. The image processing method according to claim 6, wherein the compressed image is an image that has been orthogonally transformed. 8. The image processing method according to claim 7, wherein the restoration process is an inverse orthogonal transform process. 9. The first determination step includes a character image area,
7. The image processing method according to claim 6, wherein it is determined which of the three attributes the non-character image area and the character image candidate area have. 10. In the second determining step, a plurality of representative colors are extracted from a region that has already been determined to be a character image region, and it is determined whether or not an average color of a target character image candidate region can be combined. The image processing method according to claim 6, wherein it is determined whether or not the target character image candidate area is a character image area. 11. An image processing apparatus for classifying each area of an image compressed in area units into two attributes by reading a predetermined program from the memory medium, wherein the memory medium stores a compressed state. Regarding the procedure code of the first determination step of determining which of the two attributes or both of the possible attributes is based on the image data of the area, and the area of the both possible attributes, , A procedure code of a second determination step of restoring a compressed image of an already determined area having a predetermined attribute near the attention area and determining the attribute of the attention image area based on the feature amount of the restored image area An image processing apparatus comprising: 12. An image processing apparatus for inputting color image data to identify a character image area and a non-character image area in the color image data, wherein a spatial frequency coefficient is set for each predetermined area unit in the input color image data. Extracting means for extracting the feature amount of the character image region based on the extracted feature amount,
Classifying means for classifying into a character image candidate area and a non-character image area, an average color extracting means for extracting average color information in the focused character image candidate area from the classified character image candidate area, and a focused character image candidate area A representative color extracting means for extracting a plurality of representative color information from a character image area in the vicinity of, and an average extracted by the average color extracting means based on the plurality of representative color information extracted by the representative color extracting means. And a determination unit that determines whether the target character image candidate region is a character image region or a non-character image region based on the determination result. An image processing device characterized by: 13. An image processing method for inputting color image data to identify a character image area and a non-character image area in the color image data, wherein a spatial frequency coefficient is set in a unit of a predetermined area in the input color image data. Based on the extracted feature amount, the extraction process for extracting the feature amount of
A classification step of classifying into a character image candidate area and a non-character image area, an average color extraction step of extracting average color information in the concerned character image candidate area from the classified character image candidate area, and a target character image candidate area A representative color extraction step of extracting a plurality of representative color information from a character image area in the vicinity of, and an average extracted by the average color extraction means based on the plurality of representative color information extracted by the representative color extraction means. And a determination unit that determines whether the target character image candidate region is a character image region or a non-character image region based on the determination result. An image processing method characterized by: