CN113255412A

CN113255412A - Document image processing method, device and medium

Info

Publication number: CN113255412A
Application number: CN202010091495.9A
Authority: CN
Inventors: 彭鑫; 刘坚强; 吴鹏杰
Original assignee: Beijing Xiaomi Pinecone Electronic Co Ltd
Current assignee: Beijing Xiaomi Pinecone Electronic Co Ltd
Priority date: 2020-02-13
Filing date: 2020-02-13
Publication date: 2021-08-13
Anticipated expiration: 2040-02-13
Also published as: CN113255412B

Abstract

The disclosure relates to a document image processing method, a document image processing device and a document image processing medium. The processing method comprises the following steps: dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to the brightness value of each pixel point in each block; determining a foreground color area, determining color values of all pixel points corresponding to the foreground color area, and correcting the color values of incident light of corresponding blocks according to the color values of all the pixel points corresponding to the foreground color area; calculating the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region based on the foreground color region, and determining whether the background of the document image to be processed is a pure white background or not based on the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region; and processing the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background. The document image processing method provided by the disclosure can accurately position the foreground color area and effectively avoid the phenomenon of background color loss.

Description

Document image processing method, device and medium

Technical Field

The present disclosure relates to the field of computer vision, and in particular, to a method, an apparatus, and a medium for processing a document image.

Background

In the related art, with the rapid development of portable photographing devices, such as smart phones, people can obtain digitized texts of documents by shooting the documents, and the digitized texts are used for recording and sharing related information. Compared with the traditional scanner, the digital document generated by shooting through a mobile phone is easy to use and simple to operate, but the quality of the generally obtained digital document is not high due to the fact that shooting environments are complex and changeable, and therefore high-quality document image processing becomes an urgent problem to be solved.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a method, an apparatus, and a medium for processing a document image.

According to a first aspect of the embodiments of the present disclosure, there is provided a method for processing a document image, including:

dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to the brightness value of each pixel point in each block;

determining a foreground color area, determining color values of all pixel points corresponding to the foreground color area, and correcting the color values of incident light of corresponding blocks according to the color values of all the pixel points corresponding to the foreground color area;

calculating the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region based on the foreground color region, and determining whether the background of the document image to be processed is a pure white background or not based on the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region;

and processing the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background.

Wherein, the calculating the color value of the incident light of each block according to the color value of each pixel point in the block comprises:

and selecting the average value of the color values of the pixel points with the brightness values in the preset percentage in each block according to descending order, and taking the average value as the color value of the incident light of the block.

Determining a foreground color region, determining color values of pixels corresponding to the foreground color region, and correcting color values of incident light of corresponding blocks according to the color values of the pixels corresponding to the foreground color region include:

carrying out binarization processing on the document image to be processed to generate a binarized image;

obtaining the background color region based on the pixel points corresponding to the white region in the binary image; determining a foreground color area of the document image to be processed based on the background color area; (ii) a

And generating a mask area according to the determined foreground color area, correcting the color value of a pixel point corresponding to the mask area in each block by using a preset algorithm, and recalculating the color value of incident light of each block.

Obtaining the background color region based on the pixel point corresponding to the white region in the binarized image comprises:

processing pixel points corresponding to the white area in the binary image according to the following mode to obtain a background color area:

the first method is as follows: performing plane fitting and straight line fitting on pixel points corresponding to a white area in the binary image in an LAB color space to obtain non-outliers; clustering the non-outliers to obtain a plurality of pixel point clusters; taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area;

the second method comprises the following steps: clustering pixel points corresponding to a white area in a binary image to obtain a plurality of pixel point clusters; performing plane fitting and straight line fitting on the pixel points in the pixel point cluster with the maximum average brightness value in an LAB color space, and taking the region corresponding to the non-outlier pixel points obtained through fitting as a background color region;

the third method comprises the following steps: performing plane fitting and straight line fitting on pixel points corresponding to a white area in a binary image in an LAB color space to obtain non-outlier points, and taking an area corresponding to the obtained non-outlier points as a background color area;

the method is as follows: clustering pixel points corresponding to a white area in a binary image to obtain a plurality of pixel point clusters; and taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area.

Wherein, still include:

before the generating a mask region by using the determined foreground color region, the method further includes:

and performing morphological closed operation processing on the determined foreground color area.

Wherein, the determining whether the background of the document image to be processed is a pure white background based on the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region includes:

and determining whether the background of the document image to be processed is a pure white background or not based on the difference value between the average brightness value of the pixel points corresponding to the background color region and the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region.

Wherein the determining whether the background of the document image to be processed is a pure white background based on the average brightness value of the pixel points corresponding to the background color region and the difference value between the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region comprises:

and when the average brightness value of the pixel points corresponding to the background color region is greater than a first threshold value, and the difference value between the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region is greater than a second threshold value, determining that the background of the document image to be processed is a pure white background.

Wherein, the processing the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background comprises:

when the background of the document image to be processed is a pure white background, performing white balance processing on the document image to be processed based on the color value of each pixel point of the document image to be processed and the corrected color value of the incident light of the corresponding block;

acquiring a gray level histogram of the document image to be processed after white balance processing, and performing brightness contrast equalization processing on the image to be processed after white balance processing based on the brightness value of each pixel point in the gray level histogram;

when the background of the document image to be processed is a non-pure white background, acquiring a gray level histogram of the document image to be processed, and performing brightness contrast equalization processing on the image to be processed based on the brightness value of each pixel point in the gray level histogram.

The white balance processing of the document image to be processed based on the color value of each pixel point of the document image to be processed and the corrected color value of the incident light of the corresponding block comprises the following steps:

when the ratio of the color value of the pixel point to the corrected color value of the incident light of the block where the pixel point is located is greater than 1, 255 is used as the color value of the pixel point after white balance processing; and when the ratio of the color value of the pixel point to the corrected color value of the incident light of the block where the pixel point is located is less than 1, taking the product of the ratio and 255 as the color value of the pixel point after white balance processing.

Wherein, the performing brightness contrast equalization processing on the to-be-processed image after white balance processing based on the brightness value of each pixel point in the gray level histogram or the performing brightness contrast equalization processing on the to-be-processed image based on the brightness value of each pixel point in the gray level histogram comprises:

determining the minimum non-0 brightness value and the maximum non-255 brightness value in the gray level histogram;

and adjusting the color value of each pixel point in the image to be processed based on the adjustment parameter according to the minimum non-0 brightness value and the maximum non-255 brightness value.

Wherein the tuning parameters comprise a first tuning parameter a and a second tuning parameter β, wherein,

β＝-V_min*α

wherein, V_maxRepresenting a maximum non-255 luminance value, V_minRepresents the minimum non-0 luminance value;

the adjusting the color value of each pixel point in the image to be processed based on the adjustment parameter according to the minimum non-0 brightness value and the maximum non-255 brightness value comprises:

adjusting the color value of each pixel point in the image to be processed based on the following algorithm:

C_out＝α*C_input+β

wherein, C_outRepresenting the adjusted color value, C, of each pixel in the image to be processed_inputAnd representing the color value of each pixel point in the image to be processed before adjustment.

Wherein, the processing method further comprises:

and filtering the document image to be processed.

According to a second aspect of the embodiments of the present disclosure, there is provided a processing apparatus of a document image, including:

the calculation module is configured to divide the document image to be processed into a plurality of blocks according to the preset pixel size, and calculate the color value of incident light of each block according to the brightness value of each pixel point in each block;

the correction module is configured to determine a foreground color region, determine color values of all pixel points corresponding to the foreground color region, and correct incident light color values of corresponding blocks according to the color values of all the pixel points corresponding to the foreground color region;

the determining module is configured to calculate an average brightness value of each pixel point corresponding to the foreground color region and an average brightness value of each pixel point corresponding to the background color region based on the foreground color region, and determine whether the background of the document image to be processed is a pure white background based on the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region;

and the processing module is configured to process the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background.

Wherein the computing module is configured to:

Wherein the correction module is configured to:

Wherein the correction module is further configured to:

Wherein the determination module is configured to:

Wherein the processing module is configured to:

β＝-V_min*α

the processing module is configured to:

C_out＝α*C_input+β

Wherein the processing device further comprises:

and the filtering module is configured to filter the document image to be processed.

According to a third aspect of the embodiments of the present disclosure, there is provided a processing apparatus of a document image, including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

dividing a document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to color values of all pixel points in each block;

According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium having instructions therein, which when executed by a processor of a mobile terminal, enable the mobile terminal to perform a method of processing a document image, the method comprising:

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: the document image processing method provided by the disclosure can accurately position the foreground color area and effectively avoid the phenomenon of background color loss.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is an exemplary diagram illustrating a document image in accordance with one illustrative embodiment;

FIG. 2 is a flowchart illustrating a document image processing method according to one exemplary embodiment;

fig. 3 is a flowchart illustrating a method for calculating an incident light color value of each tile according to color values of pixels in the tile in step S11 in fig. 1;

fig. 4 is a flowchart of a method for determining a foreground color region, determining color values of pixels corresponding to the foreground color region, and correcting color values of incident light of corresponding blocks according to the color values of the pixels corresponding to the foreground color region in step S12 shown in fig. 1;

FIG. 5 is a flowchart illustrating a document image processing method according to an exemplary embodiment;

FIG. 6 is a schematic illustration of a document image to be processed, shown in accordance with an exemplary embodiment;

fig. 7 is a flowchart of a method, shown in fig. 1, of calculating an average brightness value of each pixel corresponding to the foreground color region and an average brightness value of each pixel corresponding to the background color region in step S13, and determining whether the background of the document image to be processed is a pure white background based on the average brightness values of the pixels corresponding to the foreground color region and the average brightness values of the pixels corresponding to the background color region;

FIG. 8 is a flowchart of a method for determining whether the background of the document image to be processed is a pure white background in step 131 in FIG. 7 based on a difference between an average brightness value of a pixel corresponding to the background color region and an average brightness value of a pixel corresponding to the foreground color region and an average brightness value of a pixel corresponding to the background color region;

FIG. 9 is a flowchart of a method of processing a document image to be processed in step 14 of FIG. 1 based on a determination of whether the background of the document image to be processed is a pure white background;

FIG. 10 is a flowchart illustrating a method for performing white balance processing on the document image to be processed based on color values of pixels of the document image to be processed and corrected color values of incident light of corresponding blocks when the background is a pure white background in step 141 in FIG. 9;

fig. 11 is a gray level histogram of the document image to be processed after the white balance processing is obtained in step 141 in fig. 9, and the luminance contrast equalization processing is performed on the image to be processed after the white balance processing based on the luminance value of each pixel point in the gray level histogram. The method comprises the steps of carrying out brightness contrast equalization processing on an image to be processed after white balance processing is carried out on the basis of the brightness value of each pixel point in a gray level histogram;

FIG. 12 is a flowchart illustrating a document image processing method according to an exemplary embodiment;

FIG. 13 is a block diagram of a document image processing apparatus shown in accordance with an exemplary embodiment;

fig. 14 is a block diagram showing a document image processing apparatus according to an exemplary embodiment (general structure of a mobile terminal).

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

The document itself includes a background color and a foreground color, both of which have a fixed color. For any color, different color values and different brightness values are presented under the action of different brightness and different color incident lights. Therefore, after the document is shot and a document image is formed, the color values of the background color and the foreground color in the document image are different from the color values of the foreground color and the background color of the document. In the same document image, the same background or foreground will also present different colors, having different color values, due to the influence of illumination or shadow.

When a document image is shot, even if a document with a pure white background is shot, in the shot image, a white background part is not pure white, and a black character part is not pure black, so that the background of the document image is not uniform, even the contrast between the characters serving as a foreground and the background is not high, and the visual effect is influenced. The main reason is the influence of the color value of the incident light, so that the color value of each pixel point in the document image is related to the background color value, the foreground color value and the incident light color value. Fig. 1 is an exemplary diagram of a document image according to an exemplary embodiment, and referring to fig. 1, taking a pixel point in a rectangular frame 101 in the diagram as an example, a color of a paper itself is pure white, and under ideal lighting conditions, a color value of the paper itself should be 255, which is represented by color _ paper, and color _ paper is 255; the color value of the incident light is assumed to be 200, and is represented by color _ light, which is 200; in the rectangular frame 101, the color value of the pixel point [ (color _ paper/255) × (color _ light/255) ] × (255/255) × (200/255) ] × 255 ═ 200; similarly, for a pixel corresponding to the text 102 of the foreground, color _ foregorunt is used to indicate, assuming that the color value of the ink is 40, color _ foregorunt is 40, color _ light is 200, and the color value of the pixel of the foreground of the document image is [ (color _ foregorunt/255) (color _ light/255) ] × (40/255) (200/255) ], 255 is 31.3; by the above calculation, due to the influence of the color value of the incident light, the whole document image becomes dark, and the visual effect is influenced. Meanwhile, it can be known that the color value of the background of the document with a pure white background in the document image can be regarded as the color value of the incident light.

In the same way, in the same document image, the color value of the background can be used as a constant in the document image, so long as the incident light color value corresponding to each pixel point is calculated, and the influence of the incident light is eliminated, so that the enhanced image with uniform background and clear foreground can be obtained. The present disclosure thus provides a method of processing a document image.

FIG. 2 is a flowchart illustrating a document image processing method according to an exemplary embodiment, as shown in FIG. 2, for use in a terminal, including the following steps.

In step S11, dividing the document image to be processed into a plurality of blocks according to a preset pixel size, and calculating an incident light color value of each block according to a color value of each pixel point in the block;

in step S12, determining a foreground region, determining a color value of each pixel point corresponding to the foreground region, and correcting an incident light color value of a corresponding block according to the color value of each pixel point;

in step S13, based on the foreground region, calculating an average brightness value of each pixel point corresponding to the foreground region and an average brightness value of each pixel point corresponding to the background region, and based on the average brightness value of each pixel point corresponding to the foreground region and the average brightness value of each pixel point corresponding to the background region, determining whether the background of the document image to be processed is a pure white background;

in step S14, the document image to be processed is processed based on the determination result of whether the background of the document image to be processed is a pure white background.

According to the document image processing method provided by the disclosure, the foreground color area is accurately positioned by calculating and correcting the incident light color value of the document image, and whether the document image is a pure white background is judged, so that the processing mode of the document image with the non-pure white background is different from the processing mode of the document image with the pure white background, and the phenomenon that the background color is lost can be effectively avoided.

In step S11, the document image is divided into blocks, and the incident light color values of the blocks are used as the incident light color values of the pixels in the blocks, so that the light and shadow in the blocks can be balanced, and the processed image background is more uniform. The size of the block is determined according to a preset pixel size, which may be a fixed pixel size, for example, 50x50 pixels, or may be determined according to the size of the characters in the document image, for example, each character occupies an area of 15x15 pixels, and the document image to be processed may be divided into blocks by using a pixel size of 15x 15. In some cases, one pixel may also be used as one block.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 3, in step S11, the calculating the color value of the incident light of each tile according to the color values of the pixels in the tile includes:

step S111, selecting an average value of color values of pixels having color values in a predetermined percentage of the previous block as an incident light color value of the block.

In one block, all regions may be the background of the document image, all regions may be the foreground of the document image, i.e. the text and the picture in the document image, or one block includes both the foreground of the document image and the background of the document image. In this embodiment, the luminance values of all the pixels in the statistical block are arranged in a descending order, and a predetermined percentage of the luminance values in the front is taken, for example, the average value of the color values of the pixels of which the luminance values are 25% of the luminance values in the front can be taken as the color value of the incident light of the current block. For the blocks of which the whole areas are the background of the document image, the incident light color values of the blocks calculated by the method are close to the incident light color values under the real condition, but for the blocks of which the whole areas are the foreground of the document image or the areas of the background color areas in the blocks are less than 25%, the incident light color values calculated by the method are not the real incident light color values. Therefore, the color value of the incident light of the part of the blocks needs to be corrected.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 4, in step S12, determining a foreground region, determining a color value of each pixel corresponding to the foreground region, and correcting a color value of incident light of a corresponding block according to the color value of each pixel corresponding to the foreground region includes:

and step S121, carrying out binarization processing on the document image to be processed to generate a binarized image.

In the binarized image, the pixels corresponding to the foreground will be displayed as black and the pixels corresponding to the background will be displayed as white. In this step, the foreground color region and the background color region are preliminarily distinguished. To accurately distinguish a foreground color region from a background color region. And further processing the preliminarily distinguished background color area.

Step S122, obtaining a background color region based on pixel points corresponding to the white region in the binary image; and determining a foreground color area of the document image to be processed based on the background color area.

As described above, in order to accurately distinguish the foreground color region and the background color region of the document image to be processed, the preliminarily distinguished background color region is further processed, for example, by fitting and/or clustering, so as to obtain the background color region. In this way, for the document image to be processed, the remaining part is the foreground color region except the obtained background color region, that is, the remaining region plus the part corresponding to the black region in the binarized image is the foreground color region of the image to be processed except the background color region in the white region in the binarized image.

Step S123, generating a mask region according to the determined foreground color region, correcting color values of pixel points corresponding to the mask region in each block by using a preset algorithm, and recalculating color values of incident light of each block.

Through the mask region that generates, the mask region covers whole foreground color region, when doing the pixel colour correction in foreground color region, can only restore pixel in the region that the mask region corresponds, if certain block is the block that background color region corresponds, will not select at this in-process, consequently avoid repetitive work. Here, the preset algorithm may be any algorithm for image correction, and is not limited herein.

In the present disclosure, a pixel point corresponding to a white region in a binarized image may be processed in the following manner to obtain a background color region:

the first method is as follows: performing plane fitting and straight line fitting on pixel points corresponding to a white area in a binary image in an LAB Color Space (LAB Color Space) to obtain non-outliers; clustering the non-outliers to obtain a plurality of pixel point clusters; taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area;

in the disclosure, a given algorithm, for example, a ransac algorithm, is utilized to perform straight line and plane fitting on pixel points corresponding to a white region in a binarized image in an LAB color space, so as to obtain outlier pixel points and non-outlier pixel points. The area corresponding to the outlier pixel point is a foreground color area, and the area corresponding to the non-outlier pixel point is a background color area. In order to further accurately position the background color region, the non-outlier pixel points are further processed, namely the non-outlier pixel points are clustered according to a preset algorithm, such as calculation of Euclidean distance, so that a plurality of pixel point clusters are obtained; and taking the area corresponding to the pixel point in the pixel point cluster with the maximum average brightness value as a background color area. By the method, the background color area of the document image to be processed can be accurately positioned, and the area outside the background color area is the foreground color area of the document image to be processed.

in the disclosure, according to a predetermined algorithm, for example, an Euclidean distance is calculated, clustering is performed on pixel points corresponding to a white area in a binary image, so as to obtain a plurality of pixel point clusters; and the region corresponding to the pixel point in the pixel point cluster with the maximum average brightness value is a background color region. In order to further accurately position the background color region, straight line and plane fitting is performed on the pixel points in the pixel point cluster with the maximum average brightness value by using a given algorithm, such as a ransac algorithm, in an LAB color space, and the region corresponding to the non-outlier pixel points obtained by fitting is used as the background color region. By the method, the background color area of the document image to be processed can be accurately positioned, and the area outside the background color area is the foreground color area of the document image to be processed.

in the disclosure, a given algorithm, for example, a ransac algorithm, is utilized to perform straight line and plane fitting on pixel points corresponding to a white region in a binarized image in an LAB color space, so as to obtain outlier pixel points and non-outlier pixel points. The area corresponding to the outlier pixel point is a foreground color area, and the area corresponding to the non-outlier pixel point is a background color area. In such a way, the background color region of the document image to be processed can be accurately positioned, and the region outside the background color region is the foreground color region of the document image to be processed.

In the disclosure, according to a predetermined algorithm, for example, an Euclidean distance is calculated, clustering is performed on pixel points corresponding to a white area in a binary image, so as to obtain a plurality of pixel point clusters; and the region corresponding to the pixel point in the pixel point cluster with the maximum average brightness value is a background color region. In such a way, the background color region of the document image to be processed can be accurately positioned, and the region outside the background color region is the foreground color region of the document image to be processed.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 5, before generating the mask region by using the determined foreground color region, the method further includes step S125 of performing morphological close operation processing on the determined foreground color region. In practical applications, the foreground color region may include not only a text region but also a picture region, and if the foreground color region includes the picture region, the picture region may include colors with various luminances, and may even include a pure white region. For example, in the document image to be processed shown in fig. 6, although the area enclosed by the rectangular frame 100 in the drawing is determined as the foreground area, some of the small areas 125 in the rectangular frame 100 are not determined as the foreground area, and in order to correctly label these small areas 125 as the foreground area, the processing may be performed by using morphological closing operation, and these small areas 125 may be labeled as the foreground area. The morphological closing operation may be performed by a conventional morphological closing operation image processing method, which is not described herein.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 7, in step S13, calculating an average brightness value of each pixel corresponding to the foreground region and an average brightness value of each pixel corresponding to the background region based on the foreground region, and determining whether the background of the document image to be processed is a pure white background based on the average brightness values of each pixel corresponding to the foreground region and the average brightness values of each pixel corresponding to the background region includes:

step 131, determining whether the background of the document image to be processed is a pure white background based on the difference between the average brightness value of the pixel points corresponding to the background color region and the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 8, in step 131, determining whether the background of the document image to be processed is a pure white background based on the difference between the average brightness value of the pixel corresponding to the background color region and the average brightness value of the pixel corresponding to the foreground color region and the average brightness value of the pixel corresponding to the background color region includes:

step 1311, when the average brightness value of the pixel points corresponding to the background color region is greater than the first threshold, and the difference between the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region is greater than the second threshold, determining that the background of the document image to be processed is a pure white background.

For example, in the present embodiment, the first threshold is set to 100, the second threshold is set to 5, and when the above two conditions are satisfied simultaneously, the document image to be processed is determined to be a pure white background.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 9, the processing of the document image to be processed, based on the determination result of whether the background of the document image to be processed is a pure white background in step S14, includes:

step 141, when the background of the document image to be processed is a pure white background, performing white balance processing on the document image to be processed based on the color value of each pixel point of the document image to be processed and the corrected color value of the incident light of the corresponding block;

and 142, when the background of the document image to be processed is a non-pure white background, acquiring a gray level histogram of the document image to be processed, and performing brightness contrast equalization processing on the image to be processed based on the brightness value of each pixel point in the gray level histogram.

The embodiment of the disclosure provides a processing method for determining a document image. As shown in fig. 10, in step 141, when the background is a pure white background, performing white balance processing on the document image to be processed based on the color value of each pixel point of the document image to be processed and the corrected color value of the incident light of the corresponding block includes:

step 1411, when the ratio of the color value of the pixel point to the corrected color value of the incident light of the block where the pixel point is located is greater than 1, taking 255 as the color value of the pixel point after white balance processing; and when the ratio of the color value of the pixel point to the corrected color value of the incident light of the block where the pixel point is located is less than 1, taking the product of the ratio and 255 as the color value of the pixel point after white balance processing.

The following formula can be used for calculation:

wherein, C_outColor value, C, representing a pixel point after white balance processing_inputRepresenting the current colour value of a pixel, C_lightAnd indicating the color value of the incident light of the block where the pixel point is located.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 11, in step 141, a gray level histogram of the document image to be processed after the white balance processing is obtained, and the brightness contrast equalization processing is performed on the image to be processed after the white balance processing based on the brightness value of each pixel point in the gray level histogram. The brightness contrast equalization processing of the to-be-processed image after the white balance processing is carried out based on the brightness value of each pixel point in the gray level histogram comprises the following steps:

step 1412, determining the minimum non-0 brightness value and the maximum non-255 brightness value in the gray level histogram;

and 1413, adjusting the color value of each pixel point in the image to be processed based on the adjustment parameter according to the minimum non-0 brightness value and the maximum non-255 brightness value.

The tuning parameters include a first tuning parameter α and a second tuning parameter β. Wherein,

β＝-V_min*α

according to the minimum non-0 brightness value and the maximum non-255 brightness value, the adjusting the color value of each pixel point in the image to be processed based on the adjusting parameter comprises the following steps:

C_out＝α*C_input+β

The embodiment of the disclosure provides a processing method for determining a document image. In step 142, when the background of the document image to be processed is a non-pure white background, a gray level histogram of the document image to be processed is obtained, and brightness contrast equalization processing is performed on the document image to be processed based on the brightness values of the pixel points in the gray level histogram. The specific processing steps may be performed according to steps 1412 to 1414.

According to the document image processing method provided by the disclosure, whether the document image is a pure white background is judged, so that the processing mode of the non-pure white background document image is different from the processing mode of the pure white background document image, and the phenomenon of background color loss can be effectively avoided.

The embodiment of the disclosure provides a document image processing method. As shown in fig. 12, before step S11, step S10 is further included, and the document image to be processed is filtered. The noise in the document image to be processed can be filtered by filtering the image to be processed, so that the processed document image has a better visual effect. The filtering may be performed in any manner that achieves the above-described effect, and for example, a gaussian filtering method may be used. When filtering is performed in a gaussian filtering manner, a gaussian convolution kernel may be used to filter the document image to be processed.

In order to compare the method with the method of the existing products in the market, the comparison analysis is carried out on the 186 real shot document images. The 186-tap document images include handwritten notes, paper contracts, invoices, business cards, screens, posters, books, and the like. And compared with the existing products in the market from four dimensions, and the comparison result is shown in the table 1. The comparison scores given in table 1 by way of example of a single item full score of 5. As can be seen from Table 1, the document image processed by the document image processing method provided by the disclosure has a strong enhancement effect, so that the visual effect of the enhanced document image is improved, and the user experience is improved.

Table 1: compared with the existing products in the market

	Existing product 1 on the market	The method provided by the present disclosure
			Color preservation	3.92	4.41
Detail preservation	4.18	4.55
			Definition of characters	4.12	4.77
Noise/color block	3.87	4.11

The embodiment of the disclosure provides a document image processing device. FIG. 13 is a block diagram illustrating a document image processing apparatus according to an exemplary embodiment. Referring to fig. 13, the apparatus includes a calculation module 101, a modification module 102, a determination module 103, and a processing module 104.

The calculation module 101 is configured to divide the document image to be processed into a plurality of blocks according to a preset pixel size, and calculate an incident light color value of each block according to a brightness value of each pixel point in each block;

the correction module 102 is configured to determine a foreground color region, determine color values of pixels corresponding to the foreground color region, and correct color values of incident light of a corresponding block according to the color values of the pixels corresponding to the foreground color region;

the determining module 103 is configured to calculate an average brightness value of each pixel point corresponding to the foreground color region and an average brightness value of each pixel point corresponding to the background color region based on the foreground color region, and determine whether the background of the document image to be processed is a pure white background based on the average brightness value of each pixel point corresponding to the foreground color region and the average brightness value of each pixel point corresponding to the background color region;

and the processing module 104 is configured to process the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background.

The calculation module 101 is configured to:

The modification module 102 is configured to:

carrying out binarization processing on a document image to be processed to generate a binarized image;

obtaining a background color region based on pixel points corresponding to a white region in a binary image; determining a foreground color area of the document image to be processed based on the background color area;

The correction module is configured to:

The modification module 102 is further configured to:

before generating a mask region by using the determined foreground color region, the method further includes:

The determination module 103 is configured to:

and when the average brightness value of the pixel points corresponding to the background color region is greater than a first threshold value and the difference value between the average brightness value of the pixel points corresponding to the foreground color region and the average brightness value of the pixel points corresponding to the background color region is greater than a second threshold value, determining that the background of the document image to be processed is a pure white background.

The processing module 104 is configured to:

when the background of the document image to be processed is a non-pure white background, a gray level histogram of the document image to be processed is obtained, and brightness contrast equalization processing is carried out on the document image to be processed based on the brightness value of each pixel point in the gray level histogram.

The processing module 104 is configured to:

The tuning parameters include a first tuning parameter alpha and a second tuning parameter beta, wherein,

β＝-V_min*α

the processing module 104 is configured to:

C_out＝α*C_input+β

As shown in fig. 13, the document image processing apparatus provided by the present disclosure further includes:

a filtering module 105 configured to filter the document image to be processed.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

FIG. 14 is a block diagram illustrating a processing device 1400 for document images, according to an exemplary embodiment. For example, the apparatus 1400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 14, apparatus 1400 may include one or more of the following components: a processing component 1402, a memory 1404, a power component 1406, a multimedia component 1408, an audio component 1410, an input/output (I/O) interface 1412, a sensor component 1414, and a communication component 1416.

The processing component 1402 generally controls the overall operation of the device 1400, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. Processing component 1402 may include one or more processors 1420 to execute instructions to perform all or a portion of the steps of the methods described above. Further, processing component 1402 can include one or more modules that facilitate interaction between processing component 1402 and other components. For example, the processing component 1402 can include a multimedia module to facilitate interaction between the multimedia component 1408 and the processing component 1402.

The memory 1404 is configured to store various types of data to support operation at the device 1400. Examples of such data include instructions for any application or method operating on device 1400, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 1404 may be implemented by any type of volatile or non-volatile storage device or combination of devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power components 1406 provide power to the various components of device 1400. Power components 1406 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for device 1400.

The multimedia component 1408 includes a screen that provides an output interface between the device 1400 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1408 includes a front-facing camera and/or a rear-facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 1400 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 1410 is configured to output and/or input audio signals. For example, the audio component 1410 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 1400 is in operating modes, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 1404 or transmitted via the communication component 1416. In some embodiments, audio component 1410 further includes a speaker for outputting audio signals.

I/O interface 1412 provides an interface between processing component 1402 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor component 1414 includes one or more sensors for providing various aspects of state assessment for the apparatus 1400. For example, the sensor component 1414 may detect an open/closed state of the device 1400, a relative positioning of components, such as a display and keypad of the apparatus 1400, a change in position of the apparatus 1400 or a component of the apparatus 1400, the presence or absence of user contact with the apparatus 1400, an orientation or acceleration/deceleration of the apparatus 1400, and a change in temperature of the apparatus 1400. The sensor assembly 1414 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 1414 may also include a photosensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 1416 is configured to facilitate wired or wireless communication between the apparatus 1400 and other devices. The device 1400 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1416 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 1400 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as the memory 1404 that includes instructions executable by the processor 1420 of the apparatus 1400 to perform the above-described method. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

A non-transitory computer-readable storage medium in which instructions, when executed by a processor of a mobile terminal, enable the mobile terminal to perform a method of processing a document image, the method comprising:

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A processing method of a document image, characterized by comprising:

2. The method of claim 1, wherein the calculating the color value of the incident light of each block according to the color values of the pixels in the block comprises:

3. The method of claim 1, wherein the determining a foreground region and determining a color value of each pixel corresponding to the foreground region, and the correcting an incident light color value of a corresponding block according to the color value of each pixel corresponding to the foreground region comprises:

obtaining the background color region based on the pixel points corresponding to the white region in the binary image; determining a foreground color area of the document image to be processed based on the background color area;

4. The document image processing method according to claim 3, wherein said obtaining the background color region based on the pixel points corresponding to the white region in the binarized image comprises:

5. The method for processing a document image according to claim 3, further comprising:

6. The method for processing the document image according to claim 1, wherein the determining whether the background of the document image to be processed is a pure white background based on the average brightness value of each pixel corresponding to the foreground color region and the average brightness value of each pixel corresponding to the background color region comprises:

7. The method of claim 6, wherein the determining whether the background of the document image to be processed is a pure white background based on the average brightness values of the pixels corresponding to the background color region and the difference between the average brightness values of the pixels corresponding to the foreground color region and the average brightness values of the pixels corresponding to the background color region comprises:

8. The method for processing the document image according to claim 1, wherein the processing the document image to be processed based on the determination result of whether the background of the document image to be processed is a pure white background comprises:

9. The method for processing the document image according to claim 8, wherein the performing the white balance processing on the document image to be processed based on the color value of each pixel point of the document image to be processed and the corrected color value of the incident light of the corresponding block comprises:

10. The method according to claim 8, wherein the performing brightness contrast equalization processing on the to-be-processed image after white balance processing based on the brightness value of each pixel point in the gray histogram or the performing brightness contrast equalization processing on the to-be-processed image based on the brightness value of each pixel point in the gray histogram comprises:

11. The method for processing a document image according to claim 10, wherein the adjustment parameter includes a first adjustment parameter α and a second adjustment parameter β, wherein,

β＝-V_min*α

C_out＝α*C_input+β

12. The method for processing a document image according to claim 1, further comprising:

and filtering the document image to be processed.

13. A processing apparatus of a document image, characterized in that the processing apparatus comprises:

14. The apparatus for processing the document image according to claim 13, wherein the calculation module is configured to:

15. The apparatus for processing the document image according to claim 13, wherein the modification module is configured to:

16. The apparatus for processing the document image according to claim 15, wherein the modification module is configured to:

17. The apparatus for processing the document image according to claim 15, wherein the revision module is further configured to:

18. The apparatus for processing the document image according to claim 13, wherein the determining module is configured to:

19. The document image processing apparatus according to claim 18, wherein the determination module is configured to:

20. The apparatus for processing the document image according to claim 13, wherein the processing module is configured to:

21. The apparatus for processing the document image according to claim 20, wherein the processing module is configured to:

22. The apparatus for processing the document image according to claim 20, wherein the processing module is configured to:

23. The apparatus for processing a document image according to claim 22, wherein the adjustment parameter includes a first adjustment parameter a and a second adjustment parameter β, wherein,

β＝-V_min*α

the processing module is configured to:

C_out＝α*C_input+β

24. The apparatus for processing a document image according to claim 13, further comprising:

25. An apparatus for processing a document image, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

26. A non-transitory computer-readable storage medium in which instructions, when executed by a processor of a mobile terminal, enable the mobile terminal to perform a method of processing a document image, the method comprising: