JP2024080466A - Image processing device, image inspection device, image processing method, and program - Google Patents

Abstract

To enhance inspection accuracy of a recording medium printed using a transparent color material.SOLUTION: A recording medium kind acquisition part, a glossy photographic image acquisition part, and a glossy reference image acquisition part acquire the kind of a recording medium, a glossy photographic image and a glossy reference image, respectively. A glossy photographic image correction part determines whether or not a surface characteristic value of a recording medium associated with the kind of the recording medium is equal to or more than a threshold. A surface characteristic value of the recording medium is represented by surface roughness which is a parameter having correlation with an uneven shape of the surface of the recording medium. When the surface roughness is equal to or more a threshold, the glossy photographic image correction part corrects a glossy photographic image, and generates a glossy photographic correction image. A glossy image inspection part inspects glossy distribution of the recording medium on the basis of the glossy photographic correction image and the glossy reference image.SELECTED DRAWING: Figure 5

Description

The present invention relates to image processing technology for inspecting printed materials.

An inspection system has been developed that automatically inspects printed matter output by a printing device for defects such as stains and color loss. The inspection system judges whether defects exist based on the difference between a reference image, which serves as the inspection standard for printed matter registered in advance, and an inspection target image obtained by scanning the printed matter to be inspected. In printed matter, not only color is expressed using colored colorants such as color toners, but gloss control is also now performed using transparent colorants such as clear toners, making it necessary to inspect gloss. Transparent colorants are transparent colorants such as varnish, and play a role in enhancing the design of printed matter by controlling the glossiness of the printed matter. In a general inspection method, the gloss formed by transparent colorants is evaluated as uniform if there is little variation in the reflection direction of the irradiated light, and is judged to be defective if the uniformity is below a certain threshold.

Patent Document 1 discloses a method for inspecting glossy prints by reading both a print made only with colored colorants and a print made by overlaying a transparent colorant on top of the colored print, and using reference data calculated from the two sets of data read.

JP 2016-86271 A

However, Patent Document 1 does not take into consideration the effect that the surface characteristics of the recording medium used for printing have on the uniformity of the gloss of the printed matter. Therefore, when the gloss of the printed matter is judged to be uneven, the inspection method described in Patent Document 1 has the problem that it is not possible to distinguish whether the judgment result is due to the roughness of the surface shape of the recording medium or due to poor print quality, resulting in a decrease in the inspection accuracy of print quality.

Therefore, the present invention aims to improve the inspection accuracy of recording media printed using transparent color materials.

The image processing device according to the present invention includes an image acquisition means for acquiring a captured image obtained by photographing a recording medium printed with a transparent coloring material, and an inspection means for inspecting the printed recording medium using the captured image acquired by the image acquisition means, and is characterized in that, when inspecting the gloss distribution on the printed recording medium, the inspection means corrects the captured image in accordance with the surface characteristics of the recording medium before the transparent coloring material is printed, and inspects the gloss distribution on the printed recording medium using the captured image after the correction.

The present invention makes it possible to improve the inspection accuracy of recording media printed using transparent color materials.

FIG. 1 is a schematic diagram showing a configuration of a printing system including an image inspection device. Image reading unit configuration diagram Block diagram showing the hardware configuration of an image inspection device Block diagram showing the functional configuration of an image inspection device 1 is a flowchart showing a process executed by an image inspection device. Schematic diagram showing the surface roughness of a recording medium FIG. 13 is a diagram showing an example of a UI that displays the test results. Block diagram showing the functional configuration of an image inspection device 1 is a flowchart showing a process executed by an image inspection device. A diagram showing the change in surface roughness of a recording medium due to overlapping of color toners. Block diagram showing the functional configuration of an image inspection device 1 is a flowchart showing a process executed by an image inspection device. FIG. 1 is a schematic diagram showing a modified example of the configuration of a printing system including an image inspection device. FIG. 1 is a schematic diagram showing a modified example of the configuration of a printing system including an image inspection device.

The following describes an embodiment of the present invention with reference to the drawings. Note that the following embodiment does not necessarily limit the present invention. Also, not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention. Note that the same components are described with the same reference numerals.

[First embodiment]
In the first embodiment, an image inspection device incorporated in a printing system will be described. The image inspection device switches between correcting and not correcting a glossy photographed image to be inspected based on surface characteristics corresponding to the type of recording medium.

<Printing System Configuration>
1 is a schematic diagram showing an example of a printing system 1. As shown in FIG. 1, the printing system 1 includes an image forming apparatus 200, an image inspection apparatus 300, and a stacker 400.

Image forming device 200 includes an operation panel 201, which is an operation display unit that allows various operation inputs to image forming device 200 and displays various screens. Image forming device 200 includes photoconductor drums 202Y, 202M, 202C, 202K, and 202CL, a transfer belt 203, a secondary transfer roller 204, a paper feed unit 205, a pair of conveying rollers 206, a fixing roller 207, a reversing path 208, and a conveying path 209.

On photoconductor drums 202Y, 202M, 202C, 202K, and 202CL, a toner image is formed by charging, exposing, developing, transferring, and cleaning in the electrophotographic image formation process, and the formed toner image is transferred to transfer belt 203. In this embodiment, a yellow toner image is formed on photoconductor drum 202Y, a magenta toner image is formed on photoconductor drum 202M, and a cyan toner image is formed on photoconductor drum 202C. Also, a black toner image is formed on photoconductor drum 202K, and a clear toner image is formed on photoconductor drum 202CL.

The transfer belt 203 transports the toner images transferred in order from the photoconductor drums 202Y, 202M, 202C, 202K, and 202CL to the secondary transfer position of the secondary transfer roller 204.

The paper feed unit 205 stores recording media stacked on top of each other and feeds the recording media. Specifically, the recording media is paper, but it can be any medium on which an image can be formed, such as film.

The pair of transport rollers 206 transports the recording medium fed by the paper feed unit 205 in the direction indicated by the arrow on the transport path 209.

The secondary transfer roller 204 transfers the toner image conveyed by the transfer belt 203 all at once at the secondary transfer position onto the recording medium conveyed by the conveying roller pair 206.

The fixing roller 207 fixes the toner image to the recording medium by applying heat and pressure to the recording medium to which the toner image has been transferred.

In the case of single-sided printing, the image forming device 200 outputs the recording medium with the fixed toner image to the image inspection device 300. On the other hand, in the case of double-sided printing, the image forming device 200 sends the recording medium with the fixed toner image to the reversal path 208.

The reversing path 208 reverses the recording medium by switching it back and transports it in the direction of the arrow. The recording medium transported by the reversing path 208 is transported again by the transport roller pair 206, fixed by the secondary transfer roller 204, and discharged to the image inspection device 300.

The image inspection device 300 includes an image reading device 301 and an operation panel 302.

The operation panel 302 is an operation display unit that inputs various operations and displays screens for the image inspection device 300. The operation panel 302 may be omitted, in which case the operation panel 201 of the image forming device 200 may be used instead, or an externally connected computer may be used instead.

The image reading device 301 optically reads one side of a printed matter discharged from the image forming device 200. FIG. 2 is a configuration diagram showing an example of the image reading device 301. The image reading device 301 is equipped with a gloss image sensor 311, a color image sensor 312, and an illumination device 313 for the recording medium 314 to be read.

The gloss image sensor 311 and the color image sensor 312 are, for example, line sensors in which multiple light receiving elements are arranged in the main scanning direction of the image forming device 200 (a direction perpendicular to the conveying direction of the recording medium 314). The lighting device 313 is, for example, a line light in which multiple light emitting elements are arranged in the main scanning direction.

The gloss image sensor 311 has an optical axis set so as to receive light reflected at a reflection angle _θa when illumination from the illumination device 313 is incident on the recording medium 314 at a predetermined incidence angle _θc . Here, _θa is set to the same angle as _θc , so that the gloss image sensor 311 acquires specularly reflected light. Specularly reflected light is reflected light when the incidence angle and reflection angle of the illumination are equal with respect to the normal line of the recording medium 314. Moreover, the gloss image sensor 311 acquires the specularly reflected light every time the recording medium 314 is transported by one line, thereby reading a gloss image showing the gloss distribution of the recording medium 314.

The color image sensor 312 has an optical axis set so as to receive diffuse reflected light reflected at a reflection angle _θb when illumination from the illumination device 313 is incident on the recording medium 314 at a predetermined incident angle _θc . Here, _θb may be any angle different from _θc , but is set to 90 degrees, for example. The diffuse reflected light is reflected light when the incident angle and reflection angle of the illumination differ with respect to the normal to the recording medium 314. Moreover, the color image sensor 312 acquires the diffuse reflected light each time the recording medium 314 is transported by one line, thereby reading a color captured image showing the color distribution of the recording medium 314.

The stacker 400 includes a tray 401. The stacker 400 stacks recording media discharged from the image inspection device 300 onto the tray 401.

<Hardware configuration of image inspection device 300>
FIG. 3 is a block diagram showing the hardware configuration of the image inspection device 300.

The image inspection device 300 includes a CPU 311, a ROM 312, a RAM 313, a HDD 314, an image reading device 301, an image forming device interface (I/F) 316, a general-purpose I/F 315, and a system bus 317.

The CPU 311 uses the RAM 313 as a work memory to execute the OS (operating system) and various programs stored in the ROM 312 and HDD 314. The CPU 311 also controls each component via the system bus 317. Note that the processing according to the flowcharts described below is executed by the CPU 311 after program code stored in the ROM 312, HDD 314, etc. is expanded into the RAM 313. The CPU 311 also uses the HDD 314 as a storage location for various data.

An input/output device such as an operation panel 302 is connected to the general-purpose I/F 315 via a serial bus 318. The operation panel may have an input device such as a touch panel or buttons. Note that an input device for the image inspection device 300 may be provided separately from the operation panel 302, such as a mouse or keyboard.

<Functional configuration of image inspection device 300>
Next, the functional configuration of the image inspection device 300 will be described. Fig. 4 is a block diagram showing the functional configuration of the image inspection device 300 in this embodiment. The functional configuration of the image inspection device 300 is roughly divided into a gloss inspection unit 320 and a color inspection unit 330. Note that the image inspection device 300 in this embodiment functions as an image processing device that performs inspection processing, but a separate computer connected to the image inspection device 300 may also function as the image processing device. In this case, the image inspection device 300 only reads the printed matter using the image reading device 301, and the separate computer that functions as the image processing device performs the inspection processing using the information obtained by reading.

The gloss inspection unit 320 inspects the uniformity of the gloss distribution using the gloss image captured by the image reading device 301. The gloss image used during the inspection is a gloss image captured by the gloss image sensor 311 that has been corrected according to the surface characteristics of the recording medium 314.

To achieve the above functions, the gloss inspection unit 320 has a recording medium type acquisition unit 321, a gloss capture image acquisition unit 322, a gloss reference image acquisition unit 323, a gloss capture image correction unit 324, and a gloss image inspection unit 325.

The recording medium type acquisition unit 321 acquires type information of the recording medium transported to the image inspection device 300 and identifies the type of recording medium. The recording medium type information is, for example, a category such as plain paper, high-quality paper, glossy paper, film, etc., or the product name of the recording medium itself. In this embodiment, the recording medium type information is acquired by user input via the operation panel 302 of the image inspection device 300 or the operation panel 201 of the image forming device 200.

The glossy photographed image acquisition unit 322 acquires a glossy photographed image _Ics (x,y) captured by the image reading device 301. The glossy photographed image _Ics (x,y) is, for example, image data in 8-bit RGB format.

The gloss reference image acquisition unit 323 acquires a gloss reference image _Ibs (x,y) used to inspect the captured gloss image _Ics (x,y). The gloss reference image _Ibs (x,y) is, for example, 8-bit RGB format image data. The gloss reference image _Ibs (x,y) is generated in advance based on a printed image used for printing by the image forming device 200 or a read image obtained by reading a printed matter, which is a recording medium printed by the image forming device 200, and is stored in the RAM 313 or the HDD 314.

The glossy captured image correction unit 324 controls correction of the glossy captured image I _cs (x, y) based on the surface characteristics of each type of recording medium, and generates a corrected glossy captured image I' _cs (x, y). The glossy captured image correction unit 324 has surface characteristic values of the recording medium linked to the type of recording medium, and determines whether or not to correct the glossy captured image I _cs (x, y) depending on the surface characteristic value corresponding to the type of recording medium.

The gloss image inspection unit 325 inspects the gloss distribution using the corrected gloss image _I'cs (x,y) and the reference gloss image _Ibs (x,y).

The color inspection unit 330 inspects the color distribution using the color captured image taken by the image reading device 301.

To achieve the above functions, the color inspection unit 330 has a color captured image acquisition unit 331, a color reference image acquisition unit 332, and a color image inspection unit 333.

The color captured image acquisition unit 331 acquires a color captured image I _cd (x, y) captured by the image reading device 301. The color captured image I _cd (x, y) is, for example, image data in 8-bit RGB format.

The color reference image acquisition unit 332 acquires a color reference image I _bd (x, y) used to inspect the captured color image I _cd (x, y). The color reference image I _bd (x, y) is, for example, 8-bit RGB format image data. The color reference image I _bd (x, y) is generated in advance based on a printed image used by the image forming apparatus 200 to print on a recording medium or a read image obtained by reading a printed matter that is a recording medium printed by the image forming apparatus 200, and is stored in the RAM 313 or the HDD 314.

The color image inspection unit 333 inspects the color distribution of the recording medium using the captured color image I _cd (x, y) and the color reference image I _bd (x, y).

<Processing Executed by Image Inspection Device 300>
Fig. 5 is a flowchart showing the processing executed by the image inspection device 300 according to this embodiment. The processing executed by the image inspection device 300 will be described in detail below with reference to Fig. 5. Note that the processing below is started after a recording medium on which various toners have been fixed is conveyed to the image inspection device 300 and the image is read by the image reading device 301. Below, each step (process) is represented by adding an S before the reference number.

In S501, the recording medium type acquisition unit 321, the gloss captured image acquisition unit 322, and the gloss reference image acquisition unit 323 acquire the recording medium type, the gloss captured image _Ics (x,y), and the gloss reference image _Ibs (x,y), respectively. The color captured image acquisition unit 331 and the color reference image acquisition unit 332 acquire the color captured image _Icd (x,y) and the color reference image _Ibd (x,y), respectively.

In S502, the glossy photographed image correction unit 324 judges whether the surface characteristic value of the recording medium associated with the type of recording medium is equal to or greater than a threshold value. The surface characteristic value of the recording medium is expressed by a surface roughness θ _var , which is a parameter correlated with the uneven shape of the surface of the recording medium. This surface roughness θ _var represents the variation in the normal direction of the surface in a unit area of the recording medium (for example, a square area of 5 mm sides). FIG. 6 is a schematic diagram showing the surface shape of the recording medium. In this embodiment, the surface roughness θ _var is the variance of the zenith angle θ in the normal direction of the surface at a position (u, v) in the unit area, and is calculated by the following formula.

Here, N is the total number of sampled positions (u, v) within the unit area, and _{θ ave} (u, v) is the average value of the zenith angle θ(u, v) and is calculated by the following formula.

The surface roughness θ _var of the recording medium is calculated based on the normal direction measured for each recording medium by photometric stereo or the like, and is stored in advance in the RAM 313 or the HDD 314 in association with the type of recording medium. If the surface roughness θ _var is equal to or greater than the threshold value θ _th , the process proceeds to S503, and if it is less than the threshold value σ _th, the process proceeds to S504. θ _th is, for example, 10, and may be separately specified by the user via the operation panel 302 or the like.

In S503, the glossy captured image correction unit 324 corrects the glossy captured image I _cs (x, y) to generate a corrected glossy captured image I′ _cs (x, y). Specifically, the glossy captured image is generated by smoothing the glossy captured image using a bilateral filter based on the following formula:

Here, w is the filter size used for smoothing, and is set, for example, to w=20. σ ² _rough is the strength of smoothing, and is set, for example, to σ ² _rough =20. σ ² _color is a parameter for adjusting the degree of edge preservation, and is set, for example, to σ ² _color =0.01. w, σ ² _rough , and σ ² _color may be separately specified by the user via the operation panel 302 or the like, or may be set based on the surface roughness θ _var .

In S504, the gloss image inspection unit 325 inspects the gloss distribution of the recording medium based on the gloss photographed corrected image _I'cs (x, y) and the gloss reference image _Ibs (x, y). The color image inspection unit 333 inspects the color distribution based on the color photographed image _Icd (x, y) and the color reference image _Ibd (x, y). Specifically, after aligning each photographed image with the reference image, the absolute value of the difference in pixel values between the aligned images is obtained, and inspection is performed based on the absolute value of the difference. The alignment of images is performed by determining deformations such as rotation, enlargement, reduction, and shearing of the images from the corresponding relationship at the feature points (SIFT, SURF, ORB, etc.). Other known alignment methods, such as alignment using deep learning and alignment using mutual information, may also be used. The difference between the images is calculated using the adjusted gloss photographed image _I'cs (x,y), gloss reference image _Ibs (x,y), color photographed image _Icd (x,y), and color reference image _Ibd (x,y) after alignment according to the following formula:

_Irs (x,y) is a gloss difference image obtained by calculating the absolute value of the difference between the gloss photographed corrected image _I'cs (x,y) and the gloss reference image _Ibs (x,y). _Ird (x,y) is a color difference image obtained by calculating the absolute value of the difference between the color photographed image _Icd (x,y) and the color reference image _Ibd (x,y). In the gloss inspection, a gloss image inspection _{result I'rs} (x,y) is output in which the position (x,y) where the value is equal to or greater than a predetermined threshold in the gloss difference image _Irs (x,y) is regarded as a defective position. In the color inspection, a color image inspection result _I'rd (x,y) is output in which the position (x,y) where the value is equal to or greater than a predetermined threshold in the color difference image _Ird (x,y) is regarded as a defective position. Specifically, binary image data is generated by setting 255 to positions in the gloss differential image _Irs (x,y) and color differential image _Ird (x,y) where the value is equal to or greater than a threshold (e.g., 25, which is 10% of the maximum value of 8 bits) and 0 to positions where the value is less than the threshold. This binary image data is designated as the gloss image inspection result _I'rs (x,y) and color image inspection result _I'rd (x,y).

In S505, the gloss inspection unit 320 and the color inspection unit 330 perform display control to display the gloss image inspection result I' _rs (x, y) and the color image inspection result I' _rd (x, y) generated in S504 on the operation panel 302. Fig. 7 shows an example of a UI for displaying the inspection results, which displays the images used in the inspection side by side.

Effects of the First Embodiment
As described above, in the image inspection device of this embodiment, when inspecting the gloss distribution of a printed matter, if the surface roughness of the recording medium is equal to or greater than a predetermined surface roughness, the image inspection device performs the inspection based on the corrected gloss photographed image instead of the gloss photographed image. This reduces the effect of the surface roughness of the recording medium before printing on the gloss inspection, and makes it possible to improve the inspection accuracy of the print quality.

[Second embodiment]
In the first embodiment, when a surface characteristic value defined for each type of recording medium exceeds a predetermined threshold value, a gloss inspection is performed based on a corrected gloss image obtained by correcting a gloss image. In contrast, in this embodiment, a gloss inspection is performed based on a corrected gloss image obtained by changing the amount of correction for the gloss image according to the surface roughness of the recording medium. The following mainly describes the differences between this embodiment and the first embodiment.

<Functional configuration of image inspection device 300>
The functional configuration of the image inspection device 300 in this embodiment will be described. Fig. 8 is a block diagram showing the functional configuration of the image inspection device 300 in this embodiment. The image inspection device 300 has a different configuration from the image inspection device 300 in the first embodiment in that a gloss inspection unit 820 is provided.

The gloss inspection unit 820 inspects the gloss distribution based on the gloss image measured by the image reading device 301, which is a corrected gloss image in which the amount of correction varies according to the surface characteristics of the recording medium.

To achieve the above functions, the gloss inspection unit 820 differs from the gloss inspection unit 320 of the first embodiment in that the gloss image correction unit 824 is configured differently. The recording medium type acquisition unit 321, gloss image acquisition unit 322, gloss reference image acquisition unit 323, and gloss image inspection unit 325 are the same as those of the first embodiment, and therefore descriptions thereof will be omitted.

The glossy photographed image correction section 824 corrects the glossy photographed image I _cs (x, y) with a correction amount that varies according to the surface characteristics corresponding to the type of recording medium, to generate a corrected glossy photographed image I′ _cs (x, y).

The color inspection unit 330, and the color captured image acquisition unit 331, color reference image acquisition unit 332, and color image inspection unit 333 that the color inspection unit 330 possesses are the same as those in the first embodiment, so a description thereof will be omitted.

<Processing Executed by Image Inspection Device 300>
Fig. 9 is a flowchart showing the processing executed by the image inspection device 300. The processing executed by the image inspection device 300 will be described in detail below with reference to Fig. 9.

Since S501 is the same process as in the first embodiment, the explanation will be omitted.

In S803, the glossy captured image correction unit 824 corrects the glossy captured image I _cs (x, y) based on the surface roughness θ _var of the recording medium to generate a corrected glossy captured image I' _cs (x, y). Specifically, the glossy captured image I' cs (x, y) is smoothed using a bilateral filter based on the following formula to generate the corrected glossy captured image I _{' cs} (x, y).

Equation 6 is obtained by replacing ^σ2rough in Equation 3 used in the first embodiment with _αθvar . α is _a parameter for adjusting the variance value of the filter due to the surface roughness _θvar , and is set to, for example, α=1 (no adjustment). It may be separately specified by the user via the operation panel 302 or the like.

Steps S504 and S505 are the same as those in the first embodiment, so their explanation will be omitted.

Effects of the Second Embodiment
In the image inspection device of this embodiment, in the gloss inspection of a printed matter, the inspection is performed using a gloss-corrected photographed image corrected by a correction amount determined according to the surface characteristics of the recording medium. This makes it possible to stably inspect even for gloss distribution that becomes uneven according to the surface roughness, thereby improving the inspection accuracy of print quality.

[Third embodiment]
In the first and second embodiments, the glossy photographed image was corrected based on the surface roughness of the recording medium. On the other hand, since colored coloring materials are placed on the recording medium during the image formation process, the roughness of the surface on which the transparent coloring material is placed varies not only with the surface characteristics of the recording medium but also with the distribution of the colored coloring materials. FIG. 10 is a schematic diagram showing the change in surface roughness depending on the amount of colored coloring materials placed. As shown in FIG. 10(a), when there is no colored coloring material, the surface roughness of the recording medium greatly affects the surface shape of the transparent coloring material, so that the light incident from the direction indicated by the solid arrow is reflected in a scattered manner in the direction indicated by the dashed arrow, resulting in an uneven gloss. On the other hand, as shown in FIG. 10(b), when the colored coloring material is placed on the surface of the recording medium, the colored coloring material smooths the surface shape on which the transparent coloring material is placed. As a result, the light incident from the direction indicated by the solid arrow is reflected uniformly in the direction indicated by the dashed arrow, resulting in a gloss that is more uniform than when only the transparent coloring material is placed on the recording medium as shown in FIG. 10(a). In this embodiment, an image inspection device that corrects a glossy photographed image by taking into consideration not only the surface roughness of a recording medium but also the change in surface roughness caused by a colored coloring material will be described. The following mainly describes the differences between this embodiment and the first and second embodiments.

<Functional configuration of image inspection device 300>
The functional configuration of the image inspection device 300 in this embodiment will be described below. Fig. 11 is a block diagram showing the functional configuration of the image inspection device 300 in this embodiment. The image inspection device 300 is different from the image inspection device 300 in the first embodiment in that a gloss inspection unit 1120 is provided.

The gloss inspection unit 1120 inspects the gloss distribution based on the gloss image captured by the image reading device 301, which is a corrected gloss image in which the amount of correction varies depending on the surface roughness of the recording medium and the amount of colored colorant applied.

To achieve the above functions, the gloss inspection unit 1120 has a different configuration from the gloss inspection unit 320 of the first embodiment in that it has a color information acquisition unit 1126 and a gloss captured image correction unit 1124.

The recording medium type acquisition unit 321, gloss photographed image acquisition unit 322, gloss reference image acquisition unit 323, and gloss image inspection unit 325 are the same as those in the first embodiment, so their explanations are omitted.

The color information acquisition unit 1126 acquires color information, which is print data that indicates the color distribution when printing is performed on a recording medium using colored colorants. The color information is, for example, an 8-bit CMYK format print image after RIP (Raster Image Processor) processing used in the image forming device 200.

The glossy photographed image correction unit 1124 corrects the glossy photographed image I _cs (x, y) based on the surface characteristics corresponding to the type of recording medium and the color information acquired by the color information acquisition unit 1126, and generates a glossy photographed corrected image I' _cs (x, y).

The color inspection unit 330, and the color captured image acquisition unit 331, color reference image acquisition unit 332, and color image inspection unit 333 that the color inspection unit 330 possesses are the same as those in the first embodiment, so a description thereof will be omitted.

<Processing Executed by Image Inspection Device 300>
Fig. 12 is a flowchart showing the processing executed by the image inspection device 300. The processing executed by the image inspection device 300 will be described in detail below with reference to Fig. 12.

In S1201, the recording medium type acquisition unit 321, the gloss captured image acquisition unit 322, and the gloss reference image acquisition unit 323 acquire the type of recording medium, the gloss captured image _Ics (x,y), and the gloss reference image _Ibs (x,y), respectively. The color information acquisition unit 1126 acquires color information _Iod (x,y). The color captured image acquisition unit 331 and the color reference image acquisition unit 332 acquire the color captured image _Icd (x,y) and the color reference image _Ibd (x,y), respectively.

In S1203, the glossy captured image correction unit 1124 corrects the glossy captured image _Ics (x,y) based on the surface roughness _θvar of the recording medium and the color information _Iod (x,y) to generate a corrected glossy captured image _I'cs (x,y). Specifically, a bilateral filter is used that takes into consideration that the larger the pixel value of the color information _Iod (x,y), the greater the amount of colored colorant applied on the recording medium, resulting in a smaller surface roughness _θvar . The formula for generating the corrected glossy captured image _I'cs (x,y) in this embodiment is shown below.

Here, _θ'var is the surface roughness _θvar when the pixel value of the color information I _od (x, y) is 0, and is ε when the pixel value of the color information I _od (x, y) is the maximum value. This makes it possible to weaken the smoothing effect of the bilateral filter when the pixel value of the color information I _od (x, y) is large. ε is a constant for preventing division by zero in Equation 7, and specifically, ε is set to 0.01 or the like. _I'od (x, y) is normalized so that the sum of each channel of the color information I _od (x, y) falls within the range of 0 to 1. Z is a normalization term, and in the case of an 8-bit CMYK format, Z = 255 x 4 = 1020.

Steps S504 and S505 are the same as those in the first embodiment, so their explanation will be omitted.

<Effects of the Third Embodiment>
The image inspection device in this embodiment inspects the gloss distribution using a gloss-corrected image that has been corrected with a correction amount determined by taking into account the surface roughness of the recording medium as well as the smoothing of the surface shape caused by colored colorants. This makes it possible to inspect the gloss distribution with more appropriate standards for areas where the surface shape is smoothed by the colored colorants, thereby improving the inspection accuracy of print quality.

[Modification]
In the above-described embodiment, the variance in the normal direction of the surface of the recording medium is used as an index of surface characteristics used to correct the glossy photographed image, but other indices such as the glossiness of the recording medium or the variance of pixel values may also be used.

In the above-described embodiment, the type of recording medium is identified from the category or product name input by the user, but the surface of the recording medium may be measured using a separately installed media sensor, and the type of recording medium may be identified based on the obtained measurement value. In addition, the measurement value of the recording medium surface obtained by the media sensor may be directly used as an index of surface characteristics in correcting the glossy photographed image.

In the above-described embodiment, both color and glossy images are inspected, but it is also possible to only inspect the glossy images. It is also possible to allow the user to select whether or not to inspect the color images.

In the above-described embodiment, the color inspection and gloss inspection were performed using a captured image of a printout on which the colored and transparent color materials were fixed in the image forming device, but different captured images may be used for the color inspection and the gloss inspection. An image captured after the colored color materials are fixed and before the transparent color materials are fixed may be used for the color inspection, and an image captured after both the colored and transparent color materials are fixed may be used for the gloss inspection. In this case, as shown in FIG. 13, a first inspection device 350 for color inspection may be disposed between the first image forming device 210 that fixes the colored color materials and the second image forming device 220 that fixes the transparent color materials, and a second inspection device 360 for gloss inspection may be disposed after the second image forming device 220.

In the above-described embodiment, the image reading device 301 is disposed within the image inspection device 300, but it may be disposed as an in-line sensor of the image forming device 200. Also, as shown in FIG. 14, an in-line sensor may be disposed in each of the first image forming device 210 that fixes the colored colorant and the second image forming device 220 that fixes the transparent colorant.

In the image forming device configuration, an electrophotographic printer is assumed as the image forming device, but image forming devices with other mechanisms, such as inkjet, may also be used.

In the above-described embodiment, the image forming apparatus uses four types of colored materials, CMYK, but other types of color materials may be added, or the number of types of color materials may be reduced. The order in which the color materials are transferred is not limited to the order shown in the embodiment.

In the above-described embodiment, one lighting device 313 with a fixed irradiation direction and two image sensors with different optical axis directions are used as the image reading device 301, but one image sensor with a fixed optical axis direction and two lighting devices with different irradiation directions may also be used.

In the above-described embodiment, an image captured by an image reading device of an image inspection device is used as the image of the inspection target, but an image captured by a different measurement system may also be used. In that case, the captured image is stored in an external recording device connected to HDD 314 or general-purpose I/F 315.

In the above-described embodiment, 8-bit RGB or CMYK format was used as the data format for the image used in the inspection, but other image formats (such as 16-bit image formats) or other formats of information (such as binary files listing data, text data, etc.) may also be used.

In the above embodiment, w=20 was used as the filter size of the bilateral filter, but other values may be set.

In the above-described embodiment, the gloss distribution and color distribution were inspected based on the difference between the corrected captured image and the reference image, but instead of inspection using the difference, defect detection using correlation values between pixels or defect detection using deep learning may be performed.

In the above-described embodiment, a bilateral filter is used to correct the glossy captured image, but other image processing methods that perform smoothing while preserving the texture edges may also be used.

In the third embodiment described above, an 8-bit CMYK format was used for the color image, but an RGB format may also be used. In that case, the normalization term Z in Equation 9 is Z = 255 x 3 = 765.

In the third embodiment described above, the amount of blurring of the bilateral filter is adjusted using the pixel values of the color image, but the adjustment may also be performed using the amount of colored colorant applied. The amount of colored colorant applied may be derived, for example, from a lookup table or the like that records the amount of colored colorant applied for the CMYK values held by the image forming device 200.

In the third embodiment described above, the amount of blurring of the bilateral filter was adjusted using pixel values of a color image, but the filter may also use values obtained by directly measuring the surface roughness of a recording medium on which color patches of various densities are printed.

Other Examples
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

This disclosure includes the following configurations and methods:

[Configuration 1]
an image acquisition means for acquiring a captured image obtained by photographing a recording medium printed using a transparent color material;
an inspection means for inspecting the printed recording medium using the photographed image acquired by the image acquisition means;
Equipped with
When inspecting the gloss distribution on the printed recording medium, the inspection means corrects the captured image in accordance with surface characteristics of the recording medium before the transparent color material is printed, and inspects the gloss distribution on the printed recording medium using the corrected captured image.
13. An image processing device comprising:

[Configuration 2]
The correction is image processing for smoothing the captured image.
2. The image processing device according to claim 1,

[Configuration 3]
the inspection means determines a surface characteristic of the recording medium based on the type of the recording medium;
3. The image processing device according to claim 1, wherein:

[Configuration 4]
The inspection means includes a receiving means for receiving a user input;
The inspection means identifies the type of the recording medium based on a user input specifying the type of the recording medium received by the reception means.
4. The image processing device according to claim 3.

[Configuration 5]
the inspection means identifies the type of the recording medium from a measurement value obtained from a sensor that measures the surface of the recording medium;
4. The image processing device according to claim 3.

[Configuration 6]
the inspection means has a surface characteristic value for each type of recording medium, and determines the surface characteristics of the recording medium based on the surface characteristic value;
6. The image processing device according to any one of configurations 3 to 5.

[Configuration 7]
the inspection means changes the amount of correction depending on the surface characteristic value;
7. The image processing device according to configuration 6.

[Configuration 8]
When the surface characteristic value is less than a predetermined threshold value, the inspection means inspects the gloss distribution of the printed recording medium using the captured image without performing the correction.
8. The image processing device according to configuration 7.

[Configuration 9]
the image acquisition means further acquires a reference image that serves as an inspection standard for the gloss distribution of the recording medium;
the inspection means inspects the gloss distribution of the recording medium by using the captured image and the reference image.
9. The image processing device according to any one of configurations 1 to 8.

[Configuration 10]
the inspection means determines surface characteristics of the recording medium in accordance with the type of the recording medium and the amount of the colored color material applied when the recording medium has been printed using a colored color material before the transparent color material is printed;
10. The image processing device according to claim 1, wherein the first and second inputs are input to the image processing apparatus.

[Configuration 11]
the inspection means changes the amount of correction according to the amount of the colored color material applied;
11. The image processing device according to claim 10,

[Configuration 12]
The inspection means derives the amount of the coloring material applied based on a print image used for printing using the coloring material.
12. The image processing device according to claim 10 or 11.

[Configuration 13]
The inspection means derives the amount of the colored color material applied based on the captured image.
12. The image processing device according to claim 10 or 11.

[Configuration 14]
the inspection means inspects the color distribution on the printed recording medium based on the captured image.
14. The image processing device according to any one of configurations 1 to 13.

[Configuration 15]
An image processing device according to any one of configurations 1 to 14,
an image reading device that captures an image of the recording medium and generates the captured image;
An image inspection device comprising:

[Configuration 16]
16. An image inspection apparatus according to configuration 15, further comprising a display control means for causing a display device to display an inspection result of the inspection means including the photographed image after the correction.

[Configuration 17]
an image acquisition step of acquiring a captured image obtained by photographing a recording medium printed using a transparent color material;
an inspection step of inspecting the printed recording medium using the photographed image acquired by the image acquisition step;
Equipped with
In the inspection step, when inspecting the gloss distribution on the printed recording medium, the photographed image is corrected in accordance with surface characteristics of the recording medium before the transparent color material is printed, and the photographed image after the correction is used to inspect the gloss distribution on the printed recording medium.
13. An image processing method comprising:

[Configuration 18]
A program for causing a computer to function as the image processing device according to any one of configurations 1 to 14.

320 gloss inspection unit 322 gloss photographed image acquisition unit 324 gloss photographed image correction unit 325 gloss image inspection unit

Claims (18)

an image acquisition means for acquiring a captured image obtained by photographing a recording medium printed using a transparent color material;
an inspection means for inspecting the printed recording medium using the photographed image acquired by the image acquisition means;
Equipped with
When inspecting the gloss distribution on the printed recording medium, the inspection means corrects the captured image in accordance with surface characteristics of the recording medium before the transparent color material is printed, and inspects the gloss distribution on the printed recording medium using the corrected captured image.
13. An image processing device comprising: The correction is image processing for smoothing the captured image.
2. The image processing device according to claim 1, the inspection means determines a surface characteristic of the recording medium based on the type of the recording medium;
2. The image processing device according to claim 1, The inspection means includes a receiving means for receiving a user input;
The inspection means identifies the type of the recording medium based on a user input specifying the type of the recording medium received by the reception means.
4. The image processing device according to claim 3. the inspection means identifies the type of the recording medium from a measurement value obtained from a sensor that measures the surface of the recording medium;
4. The image processing device according to claim 3. the inspection means has a surface characteristic value for each type of recording medium, and determines the surface characteristics of the recording medium based on the surface characteristic value;
4. The image processing device according to claim 3. the inspection means changes the amount of correction depending on the surface characteristic value;
7. The image processing device according to claim 6, When the surface characteristic value is less than a predetermined threshold value, the inspection means does not perform the correction, and inspects the gloss distribution of the printed recording medium using the captured image.
8. The image processing device according to claim 7, the image acquisition means further acquires a reference image that serves as an inspection standard for the gloss distribution of the recording medium;
the inspection means inspects the gloss distribution of the recording medium by using the captured image and the reference image.
2. The image processing device according to claim 1, the inspection means determines surface characteristics of the recording medium in accordance with the type of the recording medium and the amount of the colored color material applied when the recording medium has been printed using a colored color material before the transparent color material is printed;
2. The image processing device according to claim 1, the inspection means changes the amount of correction according to the amount of the colored color material applied;
The image processing device according to claim 10 . The inspection means derives the amount of the coloring material applied based on a print image used for printing using the coloring material.
The image processing device according to claim 10 . The inspection means derives the amount of the colored color material applied based on the captured image.
The image processing device according to claim 10 . the inspection means inspects the color distribution on the printed recording medium based on the captured image.
2. The image processing device according to claim 1, An image processing device according to any one of claims 1 to 14,
an image reading device that captures an image of the recording medium and generates the captured image;
An image inspection device comprising: The image inspection device according to claim 15, further comprising a display control means for displaying the inspection results of the inspection means, including the captured image after the correction, on a display device. an image acquisition step of acquiring a captured image obtained by photographing a recording medium printed using a transparent color material;
an inspection step of inspecting the printed recording medium using the photographed image acquired by the image acquisition step;
Equipped with
In the inspection step, when inspecting the gloss distribution on the printed recording medium, the photographed image is corrected in accordance with surface characteristics of the recording medium before the transparent color material is printed, and the photographed image after the correction is used to inspect the gloss distribution on the printed recording medium.
13. An image processing method comprising: A program for causing a computer to function as an image processing device according to any one of claims 1 to 14.

Images