JP5541698B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program suitable for controlling a deflection characteristic.

  In recent years, with the evolution of devices that handle images, such as digital cameras, displays, and printers, there has been an increasing demand for reproducing the texture of subjects and stereoscopically viewing subjects. For example, using a lenticular lens, a parallax barrier, or the like, the reproduced image has directivity, and a different reproduced image is observed by changing the observation position, thereby obtaining various effects that cannot be obtained with a normal two-dimensional image. There is technology. In this technique, when observing a subject, two images taken from the right eye position and the left eye position are acquired, and the two images are combined into a stripe shape and output. After that, by observing the output composite image through a lenticular lens, only the right-eye image is observed with the right eye, and only the left-eye image is observed with the left eye, so that the reproduced image can be viewed stereoscopically based on parallax information. .

  For example, Patent Document 1 discloses the following technique. First, a parallax image obtained from an external camera or the like, or a parallax image drawn by a personal computer or the like is divided into stripe images, a composite image is created from each stripe image, and the created composite image is a latent image on a photoconductor. Record as. Next, the latent image is transferred or printed onto the back surface of a transparent plastic lenticular sheet using toner or ink to create a three-dimensional image formed body. On the other hand, as a technique for reproducing the texture, for example, Patent Document 2 discloses a technique for correcting an image of an input video signal based on a reflected light model and generating metallic luster. According to this method, it is described that the metallic luster can be obtained based on the luminance signal of the image.

JP-A-8-22091 JP-A-10-285459

  In recent years, when reproducing an image, there is a growing need to reproduce not only the three-dimensional information of the subject but also the glossiness (angle-changing characteristic) of the subject called texture. According to the technique described in Patent Document 1, the three-dimensional shape of the subject can be reproduced using a special optical system such as a lenticular lens, but the variable angle characteristic cannot be reproduced.

  Further, according to the technique described in Patent Document 2, it is possible to convert an image of an input video signal into an image having a metallic luster. However, in order to express metallic luster, the user must set parameters according to the input image and process the image so that the gloss characteristics have changed. For this reason, it is not always possible to reproduce the gloss (angle changing characteristic) of the subject.

  The present invention has been made in view of the above-described problems, and an object thereof is to make it possible to reproduce an angle change characteristic of an image.

An image processing apparatus according to the present invention includes characteristic data acquisition means for acquiring variable angle characteristic data representing variable angle characteristics at a plurality of angles with respect to an image projection plane, image data for projection from a plurality of projection apparatuses, and the image an image data acquisition means for acquiring the variable angle characteristic data of each pixel of data, the error between the acquired variable angle characteristic data by been a deformation characteristic data acquired by the image data acquiring means and the characteristic data acquiring means Adjusting means for adjusting the light emission intensities of the plurality of projection devices so as to be reduced, and output means for outputting the image data to the plurality of projection devices according to the light emission intensities adjusted by the adjustment means. And
An image processing method according to the present invention includes a characteristic data acquisition step of acquiring variable angle characteristic data representing variable angle characteristics at a plurality of angles with respect to an image projection plane, image data for projection from a plurality of projection devices, and the image an image data acquisition step of acquiring a bending characteristic data of each pixel of data, the error of the obtained bending characteristic data in variable angle characteristic data and said characteristic data acquiring process acquired in the image data acquisition process An adjustment step of adjusting the light emission intensities of the plurality of projection devices so as to decrease; and an output step of outputting the image data to the plurality of projection devices according to the light emission intensity adjusted in the adjustment step. And

  According to the present invention, it is possible to reproduce an angle change characteristic of an image using a plurality of general-purpose projectors without requiring a complicated optical system.

1 is a block diagram illustrating a configuration example of an image processing apparatus according to a first embodiment. It is a flowchart which shows an example of the process sequence which reproduces a variable angle characteristic. It is a figure which shows the example of arrangement | positioning of the projector in embodiment. It is a figure which shows an example of the change angle characteristic data of a some projector and a display image. It is a block diagram which shows the structural example of the image processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows an example of the process sequence which reproduces a variable angle characteristic. It is a figure which shows an example of the angle-change characteristic data of a representative projector. It is a figure which shows an example of the calculated deflection angle characteristic data of the other projector.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus 1 according to the present embodiment.
In FIG. 1, an image projection unit 2 is, for example, a screen that is an image projection plane. For example, as shown in FIG. 3, the image projection device 3 is a device in which a plurality of projectors are arranged so that the image projection unit 2 can project an image by changing the projection angle. The image output unit 4 is for outputting a display image to the image projector 3.

  The variable angle characteristic measuring apparatus 5 is an apparatus for measuring the variable angle characteristic on the image projection unit 2 of an image projected from the image projection apparatus 3 such as a goniophotometer. The variable angle characteristic data acquisition unit 6 is for inputting the variable angle characteristic data measured by the variable angle characteristic measuring apparatus 5 to the image processing apparatus 1. The image data storage unit 7 is a storage unit that stores image data to be projected from the image projection device 3 and displayed on the image projection unit 2. The image data acquisition unit 8 is for reading the display image data stored in the image data storage unit 7 to the image processing apparatus 1. The reproduction variable angle characteristic data optimizing unit 9 functions as an adjusting unit, and the image projection device 3 input from the variable angle characteristic data acquiring unit 6 with respect to the variable angle characteristic of the image data read by the image data acquiring unit 8. This is for optimizing the combination with the angle-changing characteristics. The control unit 10 is for controlling the entire image processing apparatus 1.

FIG. 2 is a flowchart illustrating an example of a processing procedure for reproducing the variable angle characteristic in the present embodiment.
First, in step S <b> 201, the control unit 10 uses the variable angle characteristic measuring device 5 to individually measure the variable angle characteristics when projected from the projectors of the image projection apparatus 3 onto the image projection unit 2. Then, the angle change characteristic data acquisition unit 6 inputs the angle change characteristic data to the image processing apparatus 1.

  Next, in step S <b> 202, the image data acquisition unit 8 reads the display image data stored in the image data storage unit 7 into the image processing apparatus 1. At this time, the image data for display stored in the image data storage unit 7 includes, as shown in FIG. 4 (b), the angle change characteristic data to be reproduced (hereinafter referred to as target angle change characteristic data) for each pixel. ) Is also held. Next, in step S203, the reproduction variable angle characteristic data optimizing unit 9 sets an initial value (for example, the upper left pixel) of the pixel that optimizes the variable angle characteristic of the reproduced image as a processing target pixel.

  Next, in step S204, the reproduction variable angle characteristic data optimization unit 9 optimizes the display pixel value by adjusting the light emission intensity of each projector of the image projection device 3. Specifically, the error between the target deflection characteristics data of the processing target pixel in the display image data and the sum of the deflection angles of the plurality of projectors projected by the image projection device 3 is minimized. The display pixel value of each projector is optimized. Details of the processing in step S204 will be described later.

  Next, in step S205, the reproduction variable angle characteristic data optimization unit 9 determines whether or not the display pixel value of each projector has been optimized for all the pixels in the display image data. If the result of this determination is that there are still unoptimized pixels, the process proceeds to step S206, where the process target pixel is changed to an unprocessed pixel. Then, the process returns to step S204. On the other hand, if all pixels are optimized as a result of the determination in step S205, the process proceeds to step S207. In step S207, based on the display pixel value of each projector optimized in step S204, the image output unit 4 outputs a display image, and the image projection device 3 projects the image on the image projection unit 2. To do.

Next, details of the process of optimizing the display pixel values of the plurality of projectors in step S204 will be described with reference to FIG.
FIG. 4A is a diagram showing an example of the angle change characteristics of 11 projectors installed at different angles with respect to the screen by 10 degrees, and FIG. 4B shows the target held by the display image data. It is a figure which shows an example of a variable angle characteristic. In step S204, the output value of each projector is calculated in order to reproduce the target variable angle characteristic shown in FIG. 4B by the linear sum of the variable angle characteristics of a plurality of projectors as shown in FIG. . The change angle characteristic O of the display image output in step S207 (hereinafter referred to as a composite change angle characteristic) can be expressed by the following equation 1 as a linear sum of the change angle characteristics I in a plurality of projectors. .

Here, the element {Oθ °} of the matrix O represents the reflection intensity in the θ ° direction of the composite variable angle characteristic, and the element {a _n } of the matrix a represents the output intensity of the nth projector. An element {I _n _θ °} of the matrix I represents the reflection intensity in the angle θ ° direction of the nth projector. In step S204, the output value a which is the light emission intensity of the plurality of projectors is determined so that the error between the target conversion characteristic T and the variable angle characteristic O synthesized by the plurality of projectors is minimized.

  FIG. 4C is a diagram showing the composite variable angle characteristic O calculated by each output value a in a plurality of projectors calculated using the variable angle characteristic of FIG. 4B as a target. Here, the variable angle characteristic shown in FIG. 4C is calculated using the following equation (2).

  In the equation, the element {Tθ °} of the matrix T represents the reflection intensity in the θ ° direction of the target deflection characteristic.

  By performing the processing control described above, it is possible to reproduce the angle change characteristic of the input image data. In the present embodiment, as an example of how to arrange a plurality of projectors, a case has been described in which eleven projectors installed at intervals of 10 degrees on a circumference in one direction are used. However, the number of projectors and the geometric conditions to be installed are not limited to this, and the number and geometric conditions are not particularly limited as long as the target variable angle characteristic data can be reproduced with desired accuracy. It is not something.

  Further, in the present embodiment, the explanation has been made on the assumption that the target variable angle characteristic data is stored in advance in the image data storage unit 7 together with the image data. On the other hand, for example, after inputting the shape data of the subject and the angle change characteristic data, rendering processing is performed in the image processing apparatus using a general CG rendering technique to calculate the angle change characteristic data of the display image. Such a configuration may be adopted. Furthermore, when an image is projected by a plurality of projectors, pixel misalignment between the plurality of projectors can be reduced by performing general keystone correction in order to correct a difference in angle with the projection surface.

(Second Embodiment)
In the first embodiment, the variable angle characteristic on the image projection unit 2 is measured for each of a plurality of projectors. In the present embodiment, an example will be described in which the angle change characteristics of other projectors are estimated using measured values of representative projectors.

FIG. 5 is a block diagram illustrating a configuration example of the image processing apparatus 500 according to the present embodiment. The configuration from the image projection unit 2 to the control unit 10 is the same as the configuration described in FIG. 1 in the first embodiment, and thus the description thereof is omitted.
In FIG. 5, the angle variation characteristic data estimation unit 501 is for estimating the angle variation characteristic on the image projection unit 2 of the image projected from the image projection device 3.

FIG. 6 is a flowchart illustrating an example of a processing procedure for reproducing the variable angle characteristic in the present embodiment. In FIG. 6, steps S202 to S207 other than steps S601 and S602 are the same as steps S202 to S207 of FIG.
First, in step S601, the control unit 10 uses the variable angle characteristic measuring device 5 to measure the variable angle characteristic when the representative projector represented by a representative of the plurality of projectors is projected onto the image projection unit 2. Then, the angle change characteristic data acquisition unit 6 inputs the angle change characteristic data to the image processing apparatus 500.

  Next, in step S602, the angle change characteristic data estimation unit 501 estimates the angle change characteristics of the other projectors using the angle change characteristic data of the representative projector that is the measurement value. Here, in the present embodiment, a projector (a projector whose optical axis coincides with the perpendicular direction of the screen) installed in a direction of 90 degrees with respect to the screen (image projection unit 2) is a representative projector. In the following, an example of estimating the deflection characteristics of a projector whose optical axis is tilted by 10 degrees from the normal of the screen is shown.

  FIG. 7A is a diagram illustrating an example of measured deflection angle characteristic data of the representative projector. First, by fitting the variable angle characteristics using, for example, the Phong model represented by the following equation (3) and calculating coefficients A, B, and n, FIG. And a regular reflection component as shown in FIG. 7C.

  Here, the diffuse reflection component and the regular reflection component are estimated by changing the values of θ and φ in the equation shown in Equation 3 based on the geometric condition in which the projector for estimating the deflection characteristic is arranged. be able to. FIG. 8B shows the estimated diffuse reflection component, and FIG. 8C shows the estimated regular reflection component. As a result, the total angle change characteristic can be estimated as the sum of both, that is, the angle change characteristic shown in FIG.

  As described above, according to the present embodiment, it is not necessary to measure all the deflection characteristics of a plurality of projectors, and the number of measurement of the deflection characteristics can be reduced by estimating from the measurement values of the representative projector. In this embodiment, the description has been made based on an example in which a projector perpendicular to the screen is selected as the representative projector. However, another projector may be used as the representative projector. Further, although the Phong model has been described as the estimation model of the variable angle characteristic data, it is not particularly limited as long as it is a model that can be estimated with desired accuracy.

(Other embodiments)
In the first and second embodiments described above, for the sake of simplicity, a method for reproducing the angle-change characteristics of one-dimensional color information such as luminance information has been described. On the other hand, when reproducing a color image such as a natural image, the angle-changing characteristics are measured independently for each of the three channels R, G, and B, and the measured angle-changing characteristic data is obtained to obtain the angle-changing angle. The characteristics may be optimized. Further, the spectroscopic data may be optimized for each wavelength, and optimized by obtaining the angle change characteristic data.

  In the first and second embodiments, the target deflection characteristic data is stored in advance in the image data storage unit 7 together with the display image data. Therefore, when the target deflection characteristic data is not held, the calculation may be performed internally using, for example, a general CG (computer graphics) rendering technique. That is, the angle change characteristic of each pixel in the image to be reproduced is calculated by calculating and rendering the angle between the normal of the object surface and the light source / viewpoint from the three-dimensional shape data and the angle change characteristic data of the object. It may be.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

2 image projection unit, 3 image projection device, 4 image output unit, 6 variable angle characteristic data acquisition unit, 8 image data acquisition unit, 9 reproduction variable angle characteristic data optimization unit, 10 control unit

Claims (7)

Characteristic data acquisition means for acquiring variable angle characteristic data representing variable angle characteristics at a plurality of angles with respect to the image projection plane;
Image data acquisition means for acquiring image data to be projected from a plurality of projection devices, and angle variation characteristic data in each pixel of the image data;
Adjusting means for adjusting the light emission intensities of the plurality of projection devices so as to reduce an error between the variable angle characteristic data acquired by the image data acquisition unit and the variable angle characteristic data acquired by the characteristic data acquisition unit;
An image processing apparatus comprising: output means for outputting the image data to the plurality of projection apparatuses based on the light emission intensity adjusted by the adjustment means.   The characteristic data acquisition means acquires angle-changing characteristic data by a representative projection apparatus from the plurality of projection apparatuses, and estimates angle-changing characteristics by another projection apparatus based on the acquired angle-change characteristic data. The image processing apparatus according to claim 1. The characteristic data acquisition means acquires the variable angle characteristic data for each of R, G, and B channels,
The image processing apparatus according to claim 2, wherein the adjustment unit adjusts light emission intensities of the plurality of projection apparatuses for each of the channels. The characteristic data acquisition means acquires spectral data of the variable angle characteristic data for each wavelength,
The image processing apparatus according to claim 1, wherein the adjustment unit adjusts light emission intensities of the plurality of projection apparatuses using the acquired spectral data.   The image data acquisition means calculates an angle change characteristic when an image represented by the image data is projected from the plurality of projection devices based on the three-dimensional shape data and the angle change characteristic data of the image indicated by the image data. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. A characteristic data acquisition step of acquiring variable angle characteristic data representing variable angle characteristics at a plurality of angles with respect to the image projection plane;
An image data acquisition step for acquiring image data for projection from a plurality of projection devices, and angle variation characteristic data in each pixel of the image data;
An adjustment step of adjusting the light emission intensities of the plurality of projection devices such that an error between the variable angle characteristic data acquired in the image data acquisition step and the variable angle characteristic data acquired in the characteristic data acquisition step is reduced ;
An image processing method comprising: an output step of outputting the image data to the plurality of projection devices based on the light emission intensity adjusted in the adjustment step.   A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 5.

Images