JP2012048756A - Image processing apparatus, control method thereof, computer program and computer readable storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an expanded image in which noise generation is suppressed, while maintaining the sharpness of a source image, only by setting a simple rate of expansion even without any special knowledge.SOLUTION: An expanded image IA is produced by expanding source image data Ia in accordance with a set rate E of expansion. Then, smoothed image data IB are produced by using a smoothing filter of a size depending on the rate E of expansion for the expanded image data IA. A differential between the expanded image data IA and the smoothed image data IB is calculated and differential image data IC are produced. The produced differential image data are multiplied with an emphasis coefficient and by adding a result of the multiplication and the expanded image data, image data IS on which expansion/unsharp mask processing has been performed are obtained. The emphasis coefficient used at such a time becomes a greater value as a value of a differential image is greater.

Description

  The present invention relates to an image enlargement processing technique.

  One process for emphasizing edge portions of image data is unsharp mask processing. An unsharp mask is called in this way because a process of smoothing image data is involved in the process of enhancing an edge.

In the unsharp mask process, first, smoothing processing is performed on the original image data Ia to be processed to create a smoothed image Ib. The smoothing process is a process for blurring an image by taking an average with surrounding pixels using a smoothing filter 201 as shown in FIG. Then, based on the smoothed image data Ib and original image data Ia, image data Is with improved sharpness is generated. The relationship between these images is as shown in the following equation (1).
Is = Ia + α (Ia−Ib) (1)

  The above formula will be briefly described. By taking the difference between the original image Ia and the smoothed image Ib, an edge component (in other words, a high-frequency component of the image) is extracted. Then, the edge component is emphasized by being multiplied by the enhancement coefficient α. Thereafter, the image data Is with improved sharpness can be obtained by adding the edge component after the enhancement to the original image Ia.

  On the other hand, the printing resolution of recent printing apparatuses is increasing. The printing resolution means the number of pixels per unit length (or unit area). Therefore, when printing with a printing apparatus having a high printing resolution, it is necessary to increase the number of pixels of the image data. That is, since it is necessary to enlarge the image to be printed, it is common to perform enlargement processing as necessary on the image data Is as a result of the expression (1).

  However, if the image data subjected to the unsharp mask processing is enlarged, the emphasized edge portion becomes smooth, and the effect of the unsharp mask is diminished.

  For this reason, a technique is known in which the unsharp mask process is executed by setting the enhancement coefficient α in Expression (1) according to the enlargement ratio of the image, and then the image enlargement process is performed (for example, Patent Documents). 1). As another implementation method, Patent Document 1 also discloses a method of first performing an image enlargement process and then performing an edge enhancement process with a large enhancement coefficient α.

  However, in the case of the above technique, in the image enlargement process after the edge enhancement process, the edge becomes smooth and the deterioration of the image quality becomes conspicuous. Even when edge enhancement is performed after enlargement, since the edge region of the enlarged image is widened, the effect of the smoothing process is reduced. Therefore, no matter how large the enhancement coefficient α is, it is difficult to emphasize the edge portion. Conversely, since the portion other than the edge portion is unnecessarily emphasized, noise is increased in the image.

  As a countermeasure to the problem that the edge region becomes wide, although it is not a technique related to unsharp mask processing, a method of increasing the edge detection filter size for an enlarged image is known (for example, Patent Document 2). . According to Patent Document 2, even in an unsharp mask, the enlargement of the edge region can be dealt with by increasing the size of the smoothing filter. However, in this case, since the smoothing process becomes strong, the region other than the edge region is also emphasized. As a result, noise increases in the image.

JP 2003-141532 A JP 2003-274157 A

  The present invention has been made in view of the above problems. When enlarging an image, for example, the image after edge enhancement by unsharp mask processing is not enlarged, but unsharp mask processing is performed on the image after enlargement as much as possible. Is what you do. Accordingly, it is an object of the present invention to provide a technique for generating an enlarged image that suppresses the occurrence of noise while maintaining the sharpness of the original image with a simple operation without any special knowledge.

In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is,
An enlargement ratio setting means for setting an enlargement ratio;
An image enlarging means for enlarging the original image data to generate enlarged image data according to the enlargement ratio;
Smoothing means for smoothing the enlarged image data and generating smoothed image data using a smoothing filter of a size corresponding to the enlargement ratio;
A difference image data generating means for calculating a difference between the smoothed image data and the enlarged image data and generating difference image data;
Multiplication means for multiplying the value of each difference pixel by an enhancement coefficient corresponding to the value of each difference pixel of the difference image data;
Adding means for adding the multiplication result by the multiplication means and the corresponding pixel value of the enlarged image data to generate an output image,
The multiplication means multiplies each difference pixel by a larger enhancement coefficient as the value of the difference pixel is larger.

  According to the present invention, it is possible to obtain an enlarged image that suppresses the occurrence of noise while maintaining the sharpness of the original image by setting a simple enlargement ratio without special knowledge.

It is a block block diagram of the system in this invention. It is a figure which shows the example of a smoothing filter. It is a figure which shows the relationship between the table of a difference value, an emphasis coefficient, and an expansion rate. It is a figure which shows the table for calculating a filter size and an emphasis coefficient table from an expansion rate. It is a flowchart which shows the process sequence of the expansion / unsharp mask process in 1st Embodiment. It is a figure which shows the correspondence of the scanning order of image data, and an address. It is a figure which shows transition of GUI of the application program in embodiment. It is a flowchart which shows the main process of the application program in embodiment. It is a flowchart which shows the process sequence of the expansion / unsharp mask process in 2nd Embodiment. It is a flowchart which shows the process sequence of the expansion / unsharp mask process in 3rd Embodiment. It is a figure for demonstrating the content of the expansion process of the image by a bilinear method and a bicubic method. It is a figure for demonstrating the process outline | summary of the expansion and unsharp mask process in 1st Embodiment. It is a figure for demonstrating the process outline | summary of the expansion and unsharp mask process in 2nd Embodiment. It is a figure for demonstrating the process outline | summary of the expansion and unsharp mask process in 3rd Embodiment. It is a figure which shows the table for the determination of the hardware resource in 4th Embodiment. It is a flowchart which shows the main process in 4th Embodiment. It is a flowchart which shows the GUI of the setting of the optimal expansion in 5th Embodiment.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. First, the premise thereof will be described.

Assuming that the original image is Ia, the image after smoothing processing of the original image Ia is Ib, and the image after unsharp mask processing is Is, it can be expressed as the above-described equation (1). When an image obtained by enlarging (interpolating) the image Is after the unsharp mask processing using the bilinear method or the like according to the enlargement ratio E is IS, the relationship between IS and Is can be expressed as follows. .
IS = E (Is) = E (Ia + α · (Ia−Ib)) (2)
Here, E (x) means increasing the number of pixels in the horizontal and vertical directions of the image x by E times. Further, the image Ib shows an image smoothed by using the smoothing filter f with respect to the original image Ia. Therefore, when “f (Ia)” is shown in order to clarify it, the expression (2) is expressed as follows. It can be expressed as
IS = E (Is) = E (Ia + α · (Ia−f (Ia))) (2) ′
After such image processing, after all, the enlargement process is smoothed, and the effect of the unsharp mask is diminished. In order to deal with this, it is necessary to set α to a larger value assuming enlargement processing. However, if a value larger than necessary is set, noise will occur and image quality degradation will be avoided. I can't. In addition, in the case of an unfamiliar user, in order to determine the optimum α and the size of the filter for smoothing, the operation related to unsharp mask processing and the operation of enlargement processing are repeated many times to obtain the desired image quality. It takes a considerable amount of time to obtain the image.

  On the other hand, according to the present invention, the user can perform enlargement processing and unsharp mask processing with only one operation, and obtain enlarged image data in which image quality deterioration is suppressed.

Although details will be described later, in the first embodiment, the image IS after the enlargement / unsharp mask processing is generated according to the following equation (3).
IS = E (Ia) + α. (E (Ia) −F (E (Ia), E)) (3)
Here, F (x, E) means that the image x is smoothed using the smoothing filter F determined by the magnification E.

  FIG. 12 is a conceptual diagram of the flow and processing of image data according to the above equation (3).

  First, the original image data Ia is enlarged according to the enlargement ratio E to obtain enlarged image data IA. The enlarged image data IA corresponds to E (Ia) in Expression (3). Next, smoothed image data IB is obtained by smoothing the enlarged image data IA using a smoothing filter F (, E) having a size determined by the enlargement ratio E. The smoothed image data IB corresponds to F (E (Ia), E) in Expression (3). Then, the difference image data IC is obtained by calculating the difference between the enlarged image data IA and the smoothed image data IB. This difference image data IC corresponds to “E (Ia) −F (E (Ia), E)” in Expression (3).

  Then, by multiplying the value of each pixel of the difference image data IC by the enhancement coefficient α and adding the multiplication result and the value of the corresponding pixel in the enlarged image data IA, the image after the enlargement / unsharp mask processing Data IS is obtained.

In the second embodiment, the image IS after the enlargement / unsharp mask process is obtained according to the following equation (4).
IS = E (Ia) + α · (E (Ia) −E (f (Ia)) (4)

  FIG. 13 is a conceptual diagram of the flow and processing of image data according to the above equation (3).

  First, smoothed image data Ib is obtained from the original image data Ia using an average smoothing filter f referring to the size of the original image data Ia and the like. This smoothed image data Ib corresponds to f (Ia) in equation (4).

  Then, each of the original image data Ia and the smoothed image data Ib is enlarged according to the enlargement ratio E to obtain enlarged image data IA and enlarged smoothed image data IB. The enlarged image data IA corresponds to E (Ia) in the equation (4), and the enlarged smoothed image data IB corresponds to E (f (Ia)) in the equation (4).

  Then, the difference image data IC is obtained by calculating the difference between the enlarged image data IA and the enlarged smoothed image data IB. This difference image data IC corresponds to “E (Ia) −E (f (Ia))” in Expression (4).

  Then, by multiplying the value of each pixel of the difference image data IC by the enhancement coefficient α and adding the multiplication result and the value of the corresponding pixel in the enlarged image data IA, the image after the enlargement / unsharp mask processing Data IS is obtained.

In the third embodiment, the image IS after the enlargement / unsharp mask process is obtained according to the following equation (5).
IS = IA + α · E (Ia−f (Ia)) (5)
FIG. 14 is a conceptual diagram of the flow and processing of image data according to the above equation (5).

  First, smoothed image data Ib is obtained from the original image data Ia using an average smoothing filter f referring to the size of the original image data Ia and the like. This smoothed image data Ib corresponds to f (Ia) in equation (5).

  Then, difference image data Ic is obtained by calculating a difference between corresponding pixel values of the original image data Ia and the smoothed image data Ib.

  Then, the original image data Ia and the difference image data Ic are respectively enlarged according to the enlargement ratio E to obtain enlarged image data IA and enlarged difference image data IC. The enlarged image data IA corresponds to E (Ia) in Expression (5), and the enlarged difference image data IC corresponds to “E (I (Ia) −f (Ia))” in Expression (4).

  Then, by multiplying the value of each pixel of the difference image data IC by the enhancement coefficient α and adding the multiplication result and the value of the corresponding pixel in the enlarged image data IA, the image after the enlargement / unsharp mask processing Data IS is obtained.

<Description of device configuration>
Here, matters common to the respective embodiments will be described.

  FIG. 1 is a system configuration diagram according to the present invention, such as a personal computer. In the figure, 101 is an input device such as a keyboard and mouse, 102 is a display device, and 103 is a large-capacity storage device such as a hard disk device. The storage device 103 stores an OS (operating system), application programs shown in the embodiments, image data files, and the like. A CPU 104 controls the entire apparatus, 105 is a ROM storing a BIOS and a boot program, and 106 is a RAM used as a work area for the CPU 104.

  When the power of the apparatus is turned on, the CPU 104 starts processing according to the boot program in the ROM 105, loads the OS in the storage apparatus 103 into the RAM 106, and executes it to function as an information processing apparatus. Thereafter, when the user operates the input device 101 to give an instruction to start an application for image processing, the CPU 104 loads the application program from the storage device 103 to the RAM 106 and executes it. As a result, this apparatus functions as an image processing apparatus.

  When the application is activated, the user reads the desired image file from the image file stored in the storage device 103 and displays it on the display device 102.

  FIG. 7 shows a GUI that reads an image file and displays it on the display device 102. As shown in the figure, the application displays a window 701, which includes an image display area 702 and an optimum enlargement button 705. There are several types of image editing other than the enlargement process, but they are omitted in the figure.

  In the illustrated case, an image 703 to be enlarged is displayed in the image display area 702. Here, when the user operates the input device 101 to instruct the optimum enlargement button 705, a dialog box 707 for setting the enlargement ratio shown is displayed. This dialog box includes a slider bar 707a, an OK button 707b, and a cancel button 707c for setting an enlargement ratio. The slider bar 707a can be set in the range of 1 to 10 times in the illustrated case. The user operates the slider bar 707a to set a desired magnification. Thereafter, when the user instructs the OK button 707b, the dialog box 707 is deleted, and enlargement / unsharp mask processing described later is executed. When the cancel button 707c is instructed, the dialog set top box 707 is simply deleted, and the state before the instruction for the optimum enlargement button 705 is restored. In the above example, the slider bar is used to set the enlargement ratio. However, the user may input a numerical value that is an enlargement ratio by operating the keyboard. That is, the configuration for setting the enlargement ratio is not limited.

  FIG. 8 is a flowchart showing the main processing procedure of the application program. Each step is executed by the CPU 104.

  First, in step S1, the window shown in FIG. 7 is displayed, and it waits for the user to select an image file to be edited in step S2. The image file selection is determined by displaying a file selection dialog box or the like and instructing the target file from the input device 101 by the user. When a file is selected, the image is displayed in the image display area 702. In step S3, the user waits for an editing instruction. If it is determined that the user's editing instruction is other than the optimum enlargement button 705, the process proceeds to step S4 to perform a corresponding process.

  On the other hand, if it is determined that the button is the optimum enlargement button 705, the process proceeds to step S5 to display a magnification setting dialog box 707 for setting the magnification. When the setting of the magnification is completed and the OK button in the dialog box 707 is designated, the process proceeds to step S6 and the automatic optimum enlargement process is executed. The image after the automatic optimum enlargement process is displayed in the image display area 702.

  Here, a supplementary explanation will be given of the enlargement process used in the embodiment. In the embodiment, a bilinear method or a bicubic method is used as the enlargement process.

FIG. 11A is a diagram illustrating a linear interpolation method (enlargement method) used in image enlargement processing. When an image including two pixels Xa and Xb is enlarged n times, 2n−2 interpolation pixels are generated between the two pixels Xa and Xb. The value of an arbitrary pixel X among them can be obtained by the following equation, where s is the distance from the pixel Xa and t is the distance from the pixel Xb.
X = (s × Xa + t × Xb) / 2n

  Thus, the linear method is a process of linearly interpolating between two pixels. In the case of image data, since it is interpolated two-dimensionally, linear interpolation is performed from surrounding four pixels. The bilinear method is a method in which this linear interpolation method is expanded to two-dimensional 2 × 2 pixels. As for the characteristic of the bilinear method, the larger the enlargement ratio, the smoother the entire image. Therefore, after the enlargement, it is possible to obtain a smoothed effect.

FIG. 11B is a diagram for explaining the bicubic method, and shows an example in which an image including four pixels Xa, Xb, Xc, and Xd is enlarged n times. In the bicubic method, a distance between two pixels adjacent in the actual horizontal (or vertical) direction is defined as “1”. Then, when obtaining the value of the pixel X to be interpolated, assuming that the distance between the pixel X and a neighboring actual pixel is d, first, Cubic (d) is calculated according to the following equation.
If 0 <| d | <1:
Cubic (d) = 1−2 | d | ² + | d | ³
When 1 ≦ | d | <2:
Cubic (d) = 4-8 | d | +5 | d | ² − | d | ³
When 2 ≦ | d |
Cubic (d) = 0

Then, the Cubic is obtained for the distance between the interpolation pixel X and one of the pixels {Xa, Xb, Xc, Xd}. Then, the sum is set as an interpolation pixel X. That is, in the case of FIG. 11B, the interpolation pixel X is calculated according to the following equation.
X = Cubic (n + s) × Xa + Cubic (s) × Xb + Cubic (t) × Xc + Cubic (n + t) × Xd
Thus, the bicubic method interpolates pixels in a wider range than linear interpolation. In the case of image data, since interpolation is performed two-dimensionally, interpolation is performed from surrounding 16 pixels. Compared with the bilinear method, the bicubic method tends to leave an edge portion of an image and can obtain a relatively sharp impression even after enlargement.

  Hereinafter, three embodiments according to the present invention will be described. Each of the embodiments has characteristics in the processing corresponding to step S6 in FIG. 8, and the rest is common.

<First Embodiment>
In the first embodiment, enlargement / unsharp mask processing is performed in accordance with the equation (3) shown above. That is, the smoothing filter size and enhancement coefficient table are switched in accordance with the enlargement ratio of the image, and an appropriate unsharp mask is executed on the enlarged image. In the following description, it is assumed that data generated during various processes is stored in the RAM 106.

  The sharpness process in the first embodiment is an unsharp mask process as described above. The unsharp mask process creates a smoothed image for the original image. Then, a difference value for each pixel between the smoothed image and the original image data is calculated. Thereafter, it is executed by adding the corresponding pixel value in the original image data.

  The smoothed image is created using the smoothing filter 201 having a size of 3 × 3 in FIG.

Ia (x, y) is a pixel value of coordinates (x, y) when the upper left corner position of the original image data is the origin (0, 0), and each component of RGB (here, 8 bits each) ). Hereinafter, unless otherwise specified, all pixel values are calculated for each element of RGB. When the coordinate of the upper left corner of the image data is (0, 0), the pixel value Ib (x, y) of the coordinate (x, y) of the smoothed image is calculated by the following equation (6).
Ib (x, y) = (Ia (x-1, y-1) + Ia (x, y-1) + Ia (x + 1, y-1) + Ia (x-1, y) + Ia ( x, y) + Ia (x + 1, y)
+ Ia (x-1, y + 1) + Ia (x, y + 1) + Ia (x + 1, y + 1)} / 9 (6)

Next, the difference between the original image and the smoothed image is obtained. If the difference value is Ic (x, y),
Ic (x, y) = Ia (x, y) -Ib (x, y) (7)
If the enhancement coefficient is α, the pixel value Is (x, y) after execution of the unsharp mask is calculated according to the following equation (8).
Is (x, y) = Ia (x, y) + α × Ic (x, y) (8)
By calculating this for all pixels, unsharp mask processing can be executed.

  When creating a smoothed image, the size of the smoothing filter can be switched. This is a method of obtaining an average value using a 5 × 5 pixel size filter, such as the smoothing filter 202 of FIG. Similarly, although not shown, the filter size can be 7 × 7 or 9 × 9. A larger filter size is useful for an enlarged image because a wider edge portion can be smoothed. However, when the filter size is increased, the area other than the edge portion is emphasized accordingly, and there is a problem that noise is conspicuous in the image data.

  FIG. 3 shows an enhancement coefficient table for reducing such noise.

  This is to determine an appropriate enhancement coefficient α according to the value of the difference value Ic (x, y) obtained by the equation (7). In the figure, the horizontal axis represents the difference value and the vertical axis represents the enhancement coefficient α, which has a curve characteristic that allows the enhancement coefficient α to be calculated according to the difference value. As a feature of the difference value, there is a feature that the difference value is large in the edge portion and the difference value is small in the region other than the edge portion. Therefore, when the difference value is small, the emphasis coefficient is reduced to prevent emphasis other than the edge portion and suppress the generation of noise. When the filter size is increased, the difference value at the portion other than the edge portion is increased. Therefore, an appropriate enhancement coefficient table is prepared in accordance with the filter size. In FIG. 3, the filter size is increased as the image enlargement ratio increases, and accordingly, in the embodiment, three enhancement coefficient tables 301 to 303 having different curve characteristics are prepared. As shown in the drawing, as the enlargement ratio increases, the enhancement coefficient becomes smaller in the range where the difference value is small.

  A table 401 in FIG. 4 shows how to change the size of the filter size and the enhancement coefficient table in accordance with the image enlargement ratio E in the first embodiment. The table 401 is also held by the application program, but may be stored in the storage device 103 as a separate environment file that is referred to by the application.

  As shown in the figure, when the image enlargement ratio E is in the range of 1.0 <E <2.0, the filter size is 3 × 3, and the enhancement coefficient table is the table 301 of FIG.

  In the range of 2.0 ≦ E <5.0, the filter size is 5 × 5, and the enhancement coefficient table is the table 302 of FIG. If 5.0 ≦ E, the filter size is 7 × 7, and the enhancement coefficient table is the table 303 in FIG. Even when the enlargement ratio is 5 times or more, it is possible to subdivide the filter size and enhancement coefficient table. However, if the filter size is increased further, the processing time of the smoothing process becomes very long. If it is more than double, uniform treatment is performed.

  As described above, in the first embodiment, the enlargement process and the unsharp mask process are executed according to the equation (3) described above.

  FIG. 5 is a flowchart showing details of the processing in step S6 of FIG. 8 described above in the first embodiment. As described above, it should be noted that the enlargement ratio E is already set when the process proceeds to step S6.

  In step S501, the original image data Ia (image data selected in step S2 in FIG. 8) is read. In step S502, the read image data Ia is enlarged using the bilinear method in accordance with the set enlargement ratio E to generate image data IA. The generated image data IA is temporarily stored in the RAM 106. The generated enlarged image data IA corresponds to E (Ia) in the above-described equation (3). The image data enlargement process is not limited to the bilinear method, and a bicubic method or the like may be used.

  Next, in step S503, a smoothing filter F having a size according to the enlargement ratio E is determined. In step S504, the generated enlarged image data IA is smoothed by using the smoothing filter F to obtain smoothed image data IB. The smoothed image data IB is also temporarily stored in the RAM 106. Further, the generated smoothed image data IB corresponds to F (E (Ia), E)) in Expression (3) described above. In step S505, one enhancement coefficient table is determined according to the enlargement ratio E. Further, an area for storing the image data IS after execution of the unsharp mask process is secured in the RAM 106.

  In step S506, the variable i representing the processing target pixel position is initialized with “0”. Here, the i-th pixel of the image data is an address value at the time of raster scanning with the upper left pixel of the image data being 0th as shown in FIG. If the number of pixels in the horizontal direction of the image data is W and the number of pixels in the vertical direction is H, the total number of pixels N of the image data is W × H. Further, if the coordinates of the pixel of interest are represented by (x, y), i = x + y × W. Since the variable i starts from 0, the unsharp mask process may be executed until the variable i becomes equal to N-1.

In step S507, a difference value between the i-th pixel values of the image data IA and IB is calculated. If the i-th pixel value of the image data IA is IA (i) and the i-th pixel value of the image data IB is IB (i), the difference image data IC (i) can be calculated according to the following equation.
IC (i) = IA (i) −IB (i)
Each pixel in the image data IA and IB has only a positive value, but the difference pixel value of the difference image data IC can take a negative value. If each pixel of the original image data is represented by 8 bits, each difference pixel of the difference image data has 9 bits or more so that a negative value can be expressed.

  In step S508, the enhancement coefficient α is calculated from the enhancement coefficient table (any of 301 to 303 in FIG. 3) determined in the previous step S503 and the difference image data IC (i).

Next, in step S509, the i-th pixel value of the image IS after the unsharp mask process is calculated according to the following equation, and the calculated pixel value IS (i) is stored at the i-th address position of the area stored in the RAM 106. To do.
IS (i) = IA (i) + α × IC (i)

  In step S510, the variable i is compared with N−1 to determine whether or not unsharp mask processing has been performed for all pixels. If not, the process proceeds to step S511, the variable i is incremented by “1”, and the processes after step S507 are repeated.

  If it is determined that the unsharp mask process has been completed for all pixels, the process proceeds to step S512, and the image data IS after the unsharp mask process generated in the RAM 106 is displayed and output in the image display area 702 of FIG. Will do. At this time, the area in the RAM 106 that stores other intermediately generated image data except for the processed image data IS is released so that the OS can use it. Also. When a save instruction (not shown) is instructed on the application, the image data IS generated in the RAM 106 is saved in the storage device 103 as a file.

  As described above, according to the first embodiment, even a user who does not have exceptional knowledge can use an optimum smoothing filter and enhancement coefficient table simply by performing an operation for instructing an enlargement ratio. The image data after the unsharp mask processing can be obtained. In the unsharp mask process, a smoothing filter and an enhancement coefficient table suitable for the image data IA (= E (Ia)) after the enlargement process of the original image Ia is used. Therefore, the generated image data after the unsharp mask process has a low noise and can be a high quality image.

<Second Embodiment>
In the second embodiment, the unsharp mask process is executed according to the equation (4) shown above. Since the apparatus configuration is common and the UI (user interface) is also common, details of step S6 in FIG. 8 will be described here.

  FIG. 9 is a flowchart showing details of the processing in step S6 of FIG. 8 described above in the second embodiment. As described above, it should be noted that the enlargement ratio E is already set when the process proceeds to step S6.

  In step S901, the original image data Ia (image data selected in step S2 in FIG. 8) is read. In step S902, smoothed image data Ib is obtained using a smoothing filter f according to the resolution or size of the read image data Ia. The generated smoothed image data Ib is temporarily stored in the RAM 106. The generated smoothed image data Ib corresponds to f (Ia) in the above-described equation (4).

  In step S903, the original image Ia is enlarged according to the enlargement ratio E to obtain enlarged image data IA. The enlargement process in step S903 may be either a bicubic method or a bilinear method. However, here, since it is desired to leave the edge of the original image Ia, in this embodiment, enlargement processing is performed according to the bicubic method.

  Next, in step S904, the smoothed image data Ib is enlarged according to the enlargement ratio E to obtain enlarged smoothed image data IB. The enlargement process in step S904 may be either the bicubic method or the bilinear method. However, since the enlargement process target is a smoothed image, the enlargement process is performed according to the bilinear method in this embodiment.

  Further, the enlarged image data IA and the enlarged smoothed image data IB are temporarily stored in the RAM 106. Incidentally, the image data IA corresponds to E (Ia) in the equation (4), and the enlarged smoothed image data IB corresponds to E (f (Ia)).

  Next, the process proceeds to step S905, and a variable i representing the processing target pixel position is initialized with “0”. The meaning of the variable i is the same as in the first embodiment.

In step S906, as in step S507 of the first embodiment, a difference value IC between the i-th pixel values of the image data IA and IB is calculated.
IC (i) = IA (i) −IB (i)

In step S907, the i-th pixel value of the image IS after the unsharp mask process is calculated using the enhancement coefficient α of the enhancement coefficient filter corresponding to the smoothing filter f of the original image Ia. Then, the calculated pixel value is stored as the pixel value IS (i) at the i-th address position of the area stored in the RAM 106.
IS (i) = IA (i) + α × IC (i)

  In step S908, the variable i is compared with N−1 to determine whether or not unsharp mask processing has been performed for all pixels. If not, the process proceeds to step S909, where the variable i is incremented by “1”, and the processes after step S906 are repeated.

  If it is determined that the unsharp mask process has been completed for all pixels, the process advances to step S910 to display and output the unsharp mask processed image data IS generated in the RAM 106 in the image display area 702 of FIG. Will do. Note that when a save instruction (not shown) is instructed on the application, the image data IS generated in the RAM 106 is saved as a file in the storage device 103 as in the first embodiment. It is.

  As described above, according to the second embodiment, it is possible to achieve the same operational effects as those of the first embodiment. Further, according to the second embodiment, since the smoothing process using the smoothing filter f is performed on the original image data Ia before the enlargement process, the process is performed at a higher speed by a smaller number of processing target pixels. It becomes possible to make it. In the second embodiment, two kinds of enlargement methods used in the enlargement process, that is, the bilinear method and the bicubic method are adaptively used in accordance with the properties of the image, so that image quality deterioration can be further suppressed. become.

<Third Embodiment>
In the third embodiment, unsharp mask processing is executed in accordance with Equation (5) shown above. Since the apparatus configuration is common and the UI (user interface) is also common, details of step S6 in FIG. 8 will be described here.

  FIG. 10 is a flowchart showing details of the processing in step S6 of FIG. 8 described above in the third embodiment. As described above, it should be noted that the enlargement ratio E is already set when the process proceeds to step S6.

  In step S1001, original image data Ia (image data selected in step S2 in FIG. 8) is read. In step S1002, smoothed image data Ib is obtained using a smoothing filter f according to the resolution or size of the read image data Ia. The generated image data Ib is temporarily stored in the RAM 106. The generated smoothed image data Ib corresponds to f (Ia) in the above-described equation (4).

  In step S1003, difference image data Ic between the original image data Ia and the smoothed image data Ib is calculated. This difference image data Ic corresponds to (Ia−f (Ia)) of the equation (5) described above.

  In step S1004, the differential image Ic is compressed using a lossless compression encoding technique such as ZIP and stored in the storage device 103. At this time, the area of the RAM 106 storing the generated smoothed image data Ib is released.

  In step S1005, the original image data Ia is enlarged according to the enlargement ratio E. The enlargement process is performed according to the bicubic method, as in the second embodiment.

  In step S1006, the stored compressed differential image data Ic is read and decompressed to obtain differential image data Ic. Then, the difference image data Ic is enlarged according to the enlargement ratio E to obtain enlarged difference image data IC. This enlarged difference image data IC corresponds to E (Ia-f (Ia)) in the equation (5). This enlargement process is also performed according to the bicubic method.

  In step S1007, a variable i representing the processing target pixel position is initialized with “0”. The meaning of the variable i is the same as in the first embodiment. Further, an area for storing the image data after the unsharp mask process is secured in the RAM 106.

In step S1008, the i-th pixel value of the image IS after the unsharp mask process is calculated using the enhancement coefficient α of the enhancement coefficient filter corresponding to the smoothing filter f of the original image Ia. Then, the calculated pixel value is stored as the pixel value IS (i) at the i-th address position of the area stored in the RAM 106.
IS (i) = IA (i) + α × IC (i)
In step S1009, the variable i is compared with N−1 to determine whether or not unsharp mask processing has been performed for all pixels. If not, the process proceeds to step S1010, the variable i is increased by “1”, and the processes in and after step S1008 are repeated.

  If it is determined that the unsharp mask process has been completed for all pixels, the process advances to step S1011 to display and output the unsharp mask processed image data IS generated in the RAM 106 in the image display area 702 of FIG. Will do. Note that when a save instruction (not shown) is instructed on the application, the image data IS generated in the RAM 106 is saved as a file in the storage device 103 as in the first embodiment. It is.

  As described above, according to the third embodiment, the same operational effects as those of the second embodiment can be achieved. Further, according to the third embodiment, the loop processing of steps S1008 to S1010 is simplified as compared with the loop processing of steps S906 to S909 of the second embodiment, so that high-speed processing is also possible. Become.

<Fourth Embodiment>
The first to third embodiments described above differ in the number of image data after enlargement. That is, it is very effective not only to speed up the processing as described above, but also to appropriately select an optimal processing method according to the free space of the RAM 106 (in some cases, the storage device 103) that is a work area in the processing device. It is.

Thus, in the fourth embodiment, the unsharp mask processing method using the three types of expressions described above is appropriately selected according to the hardware configuration related to the processing capability of the image processing apparatus, that is, the hardware resources. To implement. Here, regarding the three types of unsharp mask processing methods, the consumption amount, processing time, and image quality of the RAM 106 are divided into three stages as follows.
Formula (3) (first embodiment): Memory usage is large. Processing time is long. The image quality is good.
Formula (4) (second embodiment): Memory usage is medium. Processing time is short. Medium quality.
Formula (5) (Third Embodiment): Memory usage is small. Processing time is medium. The image quality is low.

  As described above, when compared in terms of image quality, the image quality of the finally obtained image after the enlargement / unsharp mask processing is best performed by the expression (3), and then the expressions (4) and (5) In order. The hardware resources referred to here are the memory size that can be used by the RAM 106 and the processing capability of the CPU 104.

  The reason why the memory size is not the total capacity of the RAM 106 but the usable memory size is usually that the PC can execute a plurality of applications, and the free memory depending on the type and number of applications being executed. This is because the size changes. However, if the environment is such that an unspecified number of applications cannot be executed, the entire capacity of the RAM 106 may be used.

  The processing capability of the CPU 104 is determined by the CPU type and its operation clock.

  Since information regarding the empty area of the RAM 106 and the processing capability of the CPU 104 can be easily obtained by using an OS (operating system) function call or the like, details thereof are omitted here.

  In the fourth embodiment, the available memory size of the RAM 106 is compared with two preset threshold values, and it is determined whether the available memory size is large, medium, or small.

  Further, the processing capacity of the CPU 104 is compared with two preset threshold values, and it is determined whether the processing capacity of the CPU 104 is large, medium, or small.

  Then, based on the usable size of the RAM 106 and the processing capability of the CPU 104, an index value (large, medium, and small) of the hardware resource of the device executed by the application in the fourth embodiment is determined.

  FIG. 15 shows a table for determining hardware resources.

  As shown in the figure, when both the CPU processing capacity and the available memory size are medium or higher, it is determined that the hardware resource is large, and when the available memory size is small, the hardware resource is determined to be small. To do. Otherwise, it is determined that the hardware resource is medium.

  If the hardware resource is large, the enlargement / unsharp mask process is executed according to Expression (3), that is, the first embodiment. If the hardware resource is medium, the enlargement / unsharp mask process is executed according to the equation (4), that is, the second embodiment. If the hardware resource is small, the enlargement / unsharp mask process is executed according to the equation (5), that is, the third embodiment.

  An example of the main process in the fourth embodiment is shown in FIG. The process of FIG. 16 is also a main process that replaces FIG.

  First, in step S1601, the window shown in FIG. 7 is displayed, and it waits for the user to select an image file to be edited in step S16022. The image file selection is determined by displaying a file selection dialog box or the like and instructing the target file from the input device 101 by the user. When a file is selected, the image is displayed in the image display area 702. In step S1603, it waits for an editing instruction from the user. If it is determined that the editing instruction by the user is other than the optimum enlargement button 705, the process proceeds to step S1604 and the corresponding processing is performed.

  On the other hand, if it is determined that the button is the optimum enlargement button 705, the process advances to step S1605 to display a magnification setting dialog box 707 for setting the magnification. When the magnification setting is completed and the OK button in the dialog box 707 is designated, the process proceeds to step S1606. In step S1606, the available memory size in the RAM 106 and the processing capability of the CPU 104 itself are analyzed, and the hardware resource index value is obtained by referring to the table of FIG.

  If the hardware resource is large, the process advances to step S1608 to execute the process according to the equation (3), that is, the process according to the flowchart of FIG. If the hardware resource is medium, the process advances to step S1609 to execute processing according to equation (4), that is, processing according to the flowchart of FIG. If the hardware resource is small, the process advances to step S1610 to execute the process according to the equation (5), that is, the process according to the flowchart of FIG.

  As described above, according to the fourth embodiment, any one of formulas (3) to (5) is adaptively based on the hardware resource of the apparatus that executes the image editing application of the fourth embodiment. Enlarging / unsharp masking can be performed according to one.

<Fifth Embodiment>
In the fourth embodiment, the enlargement / unsharp mask processing is adaptively performed according to any one of the equations (3) to (5) in accordance with the hardware resources of the apparatus.

  However, a user who owns a device with few hardware resources may take time, and may desire enlargement / unsharp mask processing with the highest image quality.

  Therefore, as shown in FIG. 17, a setting menu for optimum enlargement processing may be displayed so that either automatic or manual setting can be set. As shown in the figure, a radio button is prepared, and the user explicitly selects either “automatic” or “manual”. If “automatic” is set, the process of the fourth embodiment is performed. When “manual” is selected, the slider bar is operated to explicitly set which of the first to third embodiments is used.

  If this setting is made, the setting contents are saved in an environment file or the like so as to be reflected in the next editing.

  By doing as described above, the user can set which of the first to fourth embodiments performs processing, and the convenience of the user can be increased.

[Other Embodiments]
The four embodiments according to the present invention have been described above. As described above, since the present invention can be realized by a computer program as described above, such a computer program naturally falls within the scope of the present invention.

  Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.

  Further, the computer program is usually stored in a computer-readable storage medium such as a CD-ROM, and can be executed by setting it in a computer reader (CD-ROM drive) and copying or installing it in the system. Become. Therefore, it is obvious that such a computer-readable storage medium falls within the scope of the present invention.

Claims (10)

An enlargement ratio setting means for setting an enlargement ratio;
An image enlarging means for enlarging the original image data to generate enlarged image data according to the enlargement ratio;
Smoothing means for smoothing the enlarged image data and generating smoothed image data using a smoothing filter of a size corresponding to the enlargement ratio;
A difference image data generating means for calculating a difference between the smoothed image data and the enlarged image data and generating difference image data;
Multiplication means for multiplying the value of each difference pixel by an enhancement coefficient corresponding to the value of each difference pixel of the difference image data;
Adding means for adding the multiplication result by the multiplication means and the corresponding pixel value of the enlarged image data to generate an output image,
The image processing apparatus according to claim 1, wherein the multiplication unit multiplies each difference pixel by a larger enhancement coefficient as the value of the difference pixel is larger. An enlargement ratio setting means for setting an enlargement ratio;
An image enlarging means for enlarging the original image data to generate enlarged image data according to the enlargement ratio;
Smoothing means for smoothing the enlarged image data and generating smoothed image data using a smoothing filter of a size corresponding to the enlargement ratio;
A difference image data generating means for calculating a difference between the smoothed image data and the enlarged image data and generating difference image data;
Multiplication means for multiplying the value of each difference pixel by an enhancement coefficient corresponding to the value of each difference pixel of the difference image data;
Adding means for adding the multiplication result by the multiplication means and the corresponding pixel value of the enlarged image data to generate an output image,
As for the enhancement coefficient, the first value corresponding to the first difference value has a higher degree of enhancement than the second value corresponding to the second difference value having a smaller value than the first difference value. An image processing apparatus characterized by being large. A plurality of enhancement factors are prepared according to the enlargement ratio,
The image processing apparatus according to claim 1, wherein the multiplication unit selects the enhancement coefficient according to an enlargement ratio of the enlarged image data used to generate the difference image data.   The enhancement factor is a second value in which the first value corresponding to the first enlargement ratio according to the first difference value is larger than the first enlargement ratio according to the first difference value. The image processing apparatus according to claim 3, wherein the degree of emphasis is greater than the second value corresponding to the enlargement ratio. A method of controlling an image processing apparatus that performs unsharp mask processing by enlarging original image data,
An enlargement ratio setting step for setting an enlargement ratio;
In accordance with the enlargement ratio, an image enlargement step of enlarging the original image data to generate enlarged image data;
A smoothing step of smoothing the enlarged image data to generate smoothed image data using a smoothing filter having a size corresponding to the enlargement ratio;
A difference image data generation step of calculating a difference between the smoothed image data and the enlarged image data and generating difference image data;
A multiplication step of multiplying the value of each difference pixel by an enhancement coefficient corresponding to the value of each difference pixel of the difference image data;
An addition step of adding the multiplication result of the multiplication step and the corresponding pixel value of the enlarged image data to generate an output image, and
In the multiplication step, the larger the value of the difference pixel is, the larger the enhancement coefficient is multiplied by the difference pixel. A method of controlling an image processing apparatus that performs unsharp mask processing by enlarging original image data,
An enlargement ratio setting step for setting an enlargement ratio;
In accordance with the enlargement ratio, an image enlargement step of enlarging the original image data to generate enlarged image data;
A smoothing step of smoothing the enlarged image data to generate smoothed image data using a smoothing filter having a size corresponding to the enlargement ratio;
A difference image data generation step of calculating a difference between the smoothed image data and the enlarged image data and generating difference image data;
A multiplication step of multiplying the value of each difference pixel by an enhancement coefficient corresponding to the value of each difference pixel of the difference image data;
An addition step of adding the multiplication result of the multiplication step and the corresponding pixel value of the enlarged image data to generate an output image, and
As for the enhancement coefficient, the first value corresponding to the first difference value has a higher degree of enhancement than the second value corresponding to the second difference value having a smaller value than the first difference value. A control method for an image processing apparatus, characterized by being large. A plurality of enhancement factors are prepared according to the enlargement ratio,
7. The image processing apparatus according to claim 5, wherein in the multiplication step, the enhancement coefficient is selected according to an enlargement ratio of the enlarged image data used to generate the difference image data. Control method.   The enhancement factor is a second value in which the first value corresponding to the first enlargement ratio according to the first difference value is larger than the first enlargement ratio according to the first difference value. 8. The method of controlling an image processing apparatus according to claim 7, wherein the degree of emphasis is greater than the second value corresponding to the enlargement ratio.   A computer program that causes a computer to function as the image processing apparatus according to claim 1 by being read and executed by a computer.   A computer-readable storage medium storing the computer program according to claim 9.

Images