JP5585266B2 - Image display device, image supply device, and image processing method - Google Patents

Description

  The present invention relates to an image display apparatus having a modulation unit that modulates light emitted from a light source, an image supply apparatus that supplies image data to the image display apparatus, and an image processing method.

Conventionally, for the purpose of expressing an image three-dimensionally, a right-eye image and a left-eye image are alternately projected on a screen based on the input right-eye image data and the left-eye image data. An image display device is known (see, for example, Patent Document 1).
Conventionally, in an image display device that projects an image on a screen, a device that performs a luminance expansion process on the image data in order to expand the dynamic range of the image projected on the screen and improve the contrast is known. (For example, refer to Patent Document 2).

JP 2009-232308 A JP 2002-31846 A

Here, in an image display device that alternately projects an image for the right eye and an image for the left eye, such as the image display device according to Patent Document 1, the image data for right eye and the image data for left eye are disclosed in Patent Document 2. Assume that the luminance expansion process is executed.
In this case, the image data for the right eye and the image data for the left eye corresponding to the image data for the right eye are data generated by reflecting the parallax between the right eye and the left eye in order to express the image in three dimensions. Based on the above characteristics, there is a need for effective luminance extension processing.
The present invention has been made in view of the above-described circumstances, and an object thereof is to effectively perform luminance expansion processing on image data based on parallax between right-eye image data and left-eye image data. To do.

In order to achieve the above object, according to the present invention, in an image display device, an image feature amount related to luminance of image data is calculated based on right-eye image data and left-eye image data constituting stereoscopic image data. A feature amount calculating section; and an expansion coefficient calculating section for calculating an expansion coefficient related to a luminance expansion process applied to the right-eye image data and the left-eye image data based on the image feature amount calculated by the feature amount calculating section. The right-eye image data and the left-eye image data based on the parallax between the right-eye image data and the left-eye image data and the expansion coefficient calculated by the expansion coefficient calculation unit. A luminance expansion processing unit that performs luminance expansion processing; and an image display unit that displays an image based on the right-eye image data and the left-eye image data subjected to the luminance expansion processing; Characterized in that it obtain.
According to this configuration, the luminance expansion processing unit performs luminance expansion processing on each image data based on the parallax between the right-eye image data and the left-eye image data and the calculated expansion coefficient. The processing unit can perform effective luminance expansion processing based on parallax.

Here, in the image display device according to the invention, the luminance expansion processing unit forms image data to be subjected to luminance expansion processing from the right-eye image data and the left-eye image data. For each of the reference areas, the image data corresponding to each reference area is subjected to luminance expansion processing based on the expansion coefficient calculated by the expansion coefficient calculation unit using the parallax in each reference area. Also good.
According to this configuration, the luminance expansion processing unit performs luminance expansion processing on the entire image data based on the uniform expansion coefficient, as in the past, for the right-eye image data and the left-eye image data. Instead of reflecting the parallax in each of the reference areas formed by dividing the image data, and applying the luminance expansion processing to the image data corresponding to each reference area, the luminance expansion processing unit takes into account the parallax It is possible to perform a typical luminance expansion process.

Further, in the image display device according to the above invention, the luminance expansion processing unit is configured to each of the reference regions of one image data to be subjected to luminance expansion processing among the right-eye image data and the left-eye image data. For the other image data, a separation amount with each of the regions corresponding to the reference region is detected, a disparity value having a positive correlation with the detected separation amount is calculated, and the calculated disparity value is used. The image data corresponding to each reference area may be subjected to luminance expansion processing based on the expansion coefficient calculated by the expansion coefficient calculation unit.
Here, the “amount of separation between one reference area in one image data and an area corresponding to the one reference area in the other image data” refers to right-eye image data, left-eye image data, and Is a value corresponding to a coordinate distance between one reference area and an area corresponding to the one reference area.
In addition, the “region corresponding to the one reference region in the other image data” is a region in which the image in which the parallax of the image indicated by the reference region of one image data is reflected is located in the other image data. That is.
One reference area of the reference areas formed by dividing one image data, and the distance between areas corresponding to the one reference area in the other image data, and the one reference area The magnitude of parallax has the following relationship. That is, in the image data for the right eye and the image data for the left eye, for the image related to one object included in the stereoscopic image to be expressed by these image data, the one object is in the virtual space corresponding to the stereoscopic image. The closer to the front side (= the larger the parallax), the larger the distance between the region indicating the object in the right-eye image data and the region indicating the object in the left-eye image data. Therefore, the amount of separation between one reference area of one image data and the area corresponding to the one reference area in the other image data and the magnitude of the parallax in the one reference area are positively correlated. There is a relationship, and the larger the separation amount, the larger the parallax. In the above description, the object is in front of the reference plane virtually set as the reference for perspective determination in the virtual space in the stereoscopic image to be represented by the right-eye image data and the left-eye image data. It is assumed to exist on the side.
Based on this, according to the above-described configuration, the luminance expansion processing unit calculates a parallax value having a positive correlation with the above-described separation amount, and uses the calculated parallax value to generate an image corresponding to each reference region. Since the luminance expansion processing is performed on the data, the luminance expansion processing unit can perform effective luminance expansion processing based on parallax.

Further, in the image display device according to the invention, the luminance expansion processing unit includes the reference region in which the calculated parallax value exceeds a predetermined threshold among the reference regions of the image data to be subjected to luminance expansion processing. Luminance expansion processing may be performed on the image data corresponding to.
Here, the stereoscopic image represented by the right-eye image data and the left-eye image data is composed of a background image indicating the background and an image expressed with a certain amount of stereoscopic effect on the background image. There is a case. In this case, there is a tendency that an image expressed with a certain degree of stereoscopic effect is more important than a background image, and when the image data is subjected to luminance expansion processing, the image expressed with the corresponding stereoscopic effect. There is a need to perform luminance expansion processing on image data related to the above.
Based on this, according to the above configuration, the luminance expansion processing unit has a probability that the parallax value is higher than a predetermined threshold and is an area related to an image that should be expressed with a certain degree of stereoscopic effect rather than an area related to the background image. In order to perform the luminance expansion processing on the image data related to the reference area having a high level, the luminance expansion processing unit emphasizes the luminance on the image data on the image that should be expressed with a certain degree of stereoscopic effect, not on the background image. An expansion process can be performed.

Further, in the image display device according to the above invention, the luminance expansion processing unit increases the luminance of the reference region having a larger calculated parallax value for each of the reference regions of the image data to be subjected to the luminance expansion processing. After correcting the expansion coefficient so as to increase the expansion rate, the image data corresponding to each reference area may be subjected to luminance expansion processing.
Here, in the stereoscopic image expressed by the image data for the right eye and the image data for the left eye, the image related to the object expressed so as to exist closer to the front is an important image in which the stereoscopic effect and power are emphasized. There is a tendency, and when performing the luminance expansion processing, there is a need to perform the luminance expansion processing on the image data of the image related to such an object based on the expansion coefficient having a higher luminance expansion ratio. .
Based on this, according to the above configuration, the luminance expansion processing unit corrects the expansion coefficient so that the luminance expansion rate becomes higher in the reference region having a larger parallax value, and then the image data corresponding to each reference region. Therefore, the image data of the image related to the object expressed so as to exist in the foreground can be subjected to the luminance expansion processing based on the expansion coefficient having a higher luminance expansion rate.

Further, in the image display device according to the above invention, the luminance expansion processing unit detects the frequency of the reference region having a similar parallax value for each of the reference regions of the image data to be subjected to luminance expansion processing. Then, the luminance expansion processing may be performed on the image data corresponding to each reference region after correcting the expansion coefficient so that the higher the frequency of the reference region, the higher the luminance expansion rate.
Here, in the stereoscopic image expressed by the image data for the right eye and the image data for the left eye, the image related to the object in which there are more objects expressed so that the positions in the depth direction are approximately the same. There is a tendency to be an important image in a stereoscopic image, and when performing luminance expansion processing, luminance expansion processing is performed on image data of an image related to such an object based on an expansion coefficient having a higher luminance expansion rate. There is a need to apply.
Based on this, according to the above configuration, the luminance expansion processing unit detects the frequency of the reference area having the same degree of parallax value, and sets the expansion coefficient so that the higher the frequency of the reference area, the higher the luminance expansion rate. In order to perform luminance expansion processing on the image data corresponding to each reference area after correction, the image data of an image related to an object in which there are more objects expressed so that the positions in the depth direction are approximately the same. Therefore, it is possible to perform the luminance expansion processing based on the expansion coefficient having a higher luminance expansion rate.

The image display device according to the invention includes a modulation unit that modulates light emitted from a light source, and the luminance expansion processing unit includes the right-eye image data and the left-eye image subjected to the luminance expansion processing. A light control unit may be provided that outputs data to the modulation unit and controls light emitted from the light source in accordance with the luminance expansion processing performed by the luminance expansion processing unit.
According to this configuration, with respect to the image projected via the modulation unit, it is possible to increase the dynamic range of the image and improve the contrast while maintaining the apparent brightness of the image.

In order to achieve the above object, the present invention provides an image supply device that supplies image data to an image display device, and is based on right-eye image data and left-eye image data constituting stereoscopic image data. A feature amount calculation unit that calculates an image feature amount related to the luminance of the image data, and luminance expansion applied to the right-eye image data and the left-eye image data based on the image feature amount calculated by the feature amount calculation unit A parallax between the right-eye image data and the left-eye image data with respect to the expansion coefficient calculation unit that calculates an expansion coefficient for processing, the right-eye image data, and the left-eye image data; and A luminance expansion processing unit that performs luminance expansion processing based on the expansion coefficient calculated by the expansion coefficient calculation unit.
According to this configuration, the luminance expansion processing unit performs luminance expansion processing on each image data based on the parallax between the right-eye image data and the left-eye image data and the calculated expansion coefficient. The processing unit can perform effective luminance expansion processing based on parallax.

In order to achieve the above object, the present invention is an image processing method, which is an image feature related to brightness of image data based on right-eye image data and left-eye image data constituting stereoscopic image data. A right-eye image data, and a left-eye image; a right-eye image data; and a left-eye image; a right-eye image data; and a left-eye image; A luminance expansion process is performed on the data based on a parallax between the right-eye image data and the left-eye image data and a calculated expansion coefficient.
According to this image processing method, luminance expansion processing is performed on each image data based on the parallax between the right-eye image data and the left-eye image data and the calculated expansion coefficient. Thus, it is possible to perform a luminance expansion process.

  According to the present invention, it is possible to effectively perform luminance expansion processing on image data based on parallax between the image data for the right eye and the image for the left eye.

It is a block diagram which shows the functional structure of the image display apparatus which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the image display apparatus in an adaptive light control process. It is a figure which shows typically the image data in which the pixel block was formed. It is a figure which shows typically the LUT for calculating | requiring a luminance expansion rate. The figure which looked at the virtual space corresponding to a synthetic | combination stereo image from the top in order to demonstrate parallax. It is a figure which shows typically the image data for right eyes, and the image data for left eyes. It is a block diagram which shows the functional structure of a brightness expansion process part. It is a flowchart which shows operation | movement of a reference | standard area | region coordinate calculation part. It is a flowchart which shows operation | movement of a parallax value calculation part. It is a figure which shows the image data for left eyes typically. It is a flowchart which shows operation | movement of a brightness expansion process execution part.

<First Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a functional configuration of an image display apparatus 1 according to the first embodiment to which the present invention is applied.
The image display device 1 shown in FIG. 1 is a projector that projects a 3D (stereoscopic) image on a screen 5, and includes a light source device 110, a light valve 130 as a modulation unit that modulates light emitted from the light source device 110, A projection optical system 140 that condenses and diffuses the light modulated by the light valve 130 and projects the light onto the screen 5.
The image display device 1 projects a three-dimensional image on the screen 5 in a time division manner by alternately projecting a right-eye image and a left-eye image onto the screen 5. A person who sees this projection image can see a three-dimensional image by wearing a glasses-type filter equipped with a liquid crystal shutter, for example.

The light source device 110 includes a light source such as a xenon lamp, an ultra-high pressure mercury lamp, or an LED (Light Emitting Diode) . The light source device 110 may include a reflector and an auxiliary reflector that guide light emitted from the light source to the light valve 130, and includes a lens group (not shown), a polarizing plate, and the like for enhancing the optical characteristics of the light. It may be a thing.
The light source device 110 includes a light control element 120 that reduces the amount of light on a path where light emitted from the light source reaches the light valve 130. The light control element 120 includes, for example, a light reduction plate that blocks light emitted from the light source device 110, and a drive circuit that adjusts the position or angle of the light reduction plate according to a predetermined light reduction rate. It is dimmed by advancing the light-reducing plate to the blocking position. The light control element 120 can also be constituted by a liquid crystal shutter instead of the light reducing plate, and in this case, the light is reduced by adjusting the transmittance of the whole or a part of the liquid crystal shutter.

The light valve 130 is composed of three transmissive liquid crystal panels corresponding to each color of RGB, and synchronizes the image data subjected to the luminance expansion processing by the luminance expansion processing unit 50 described later with a vertical synchronization signal (Vsync). Then draw on the transmissive LCD panel.
The projection optical system 140 includes a prism that synthesizes RGB three-color modulated light modulated by the light valve 130, a lens group that forms a projection image synthesized by the prism on the screen 5, and the like.
Each component related to image display including the light source device 110, the light valve 130, and the projection optical system 140 corresponds to the image display unit of the present invention as a whole, but if an image can be displayed, Some or all of them can be replaced by various functional units as described above.

The image display device 1 is also based on a video source (not shown) stored in a built-in storage device or a stereoscopic video signal input from an external image supply device (not shown) such as a personal computer or various video players. Project an image.
The image display apparatus 1 uses the control unit 10 that controls the entire image display apparatus 1, and the right-eye image data and the left-eye image data based on a stereoscopic video signal input from the video source or an external image supply apparatus. An image input unit 20 that outputs alternately, a feature amount calculation unit 30 that calculates an image feature amount based on the image data for the right eye and the image data for the left eye that are input from the image input unit 20, and the image that the feature amount calculation unit 30 determines A luminance expansion rate calculation unit 40 (expansion coefficient calculation unit) that calculates a luminance expansion rate (expansion coefficient) based on the feature amount, and a luminance expansion processing unit that performs luminance expansion processing according to the luminance expansion rate calculated by the luminance expansion rate calculation unit 40 50, a dimming rate calculating unit 60 that calculates the dimming rate from the image feature amount obtained by the feature amount calculating unit 30, and the dimming element 120 is driven based on the dimming rate calculated by the dimming rate calculating unit 60. Then dimmed And a light reduction processing unit 70.
The image display device 1 performs adaptive dimming processing of the image to be projected by each of the above function units. In other words, the light emitted from the light source device 110 is reduced, and the gradation of the image drawn by the light valve 130 is extended, thereby increasing the dynamic range and improving the contrast.

  FIG. 2 is a flowchart showing the operation of the image display device 1 and shows a processing procedure in the adaptive light control processing executed by each unit of the image display device 1 described above. The details of the adaptive dimming process will be described below with reference to the flowchart of FIG. 2 and FIG.

The image input unit 20 supports various 3D video formats such as a frame packing method, a side-by-side method, and a top-and-bottom method. When a stereoscopic video signal is input (step S11), the image input unit 20 generates right-eye image data and left-eye image data from the input stereoscopic video signal, and the feature amount calculation unit 30 in the input order. And output to the luminance expansion processing unit 50 (step S12).
In the present embodiment, each of the right-eye image data and the left-eye image data generated by the image input unit 20 is bitmap format data, and for each pixel arranged in a dot matrix on the data, For each pixel, RGB color components are held as gradation values (for example, gradation values of 0 to 255 levels).
When the format of the input stereoscopic video signal is the side-by-side format or the top-and-bottom format, the image input unit 20 cuts out the right-eye image data and the left-eye image data from the input signal, and outputs the cut-out image data as a light valve. The image data is decompressed in accordance with the display resolution of 130, and the decompressed image data is output.
Here, the image input unit 20 outputs the right eye image data and the left eye image data alternately to the feature amount calculation unit 30 and the luminance expansion processing unit 50 so that the right eye image data comes first. . The image input unit 20 also performs vertical synchronization of the RL identification signal indicating whether the image data being output is right-eye image data or left-eye image data, and right-eye image data and left-eye image data. The signal VSync is output. When the format of the input stereoscopic video signal is a side-by-side format or a top-and-bottom format, one vertical synchronization signal is included in this input signal. In this case, the image input unit 20 cuts out the right-eye image data and the left-eye image data from the input signal, and the vertical synchronization signal indicating the drawing start timing of the cut-out right-eye image data and left-eye image data. Generate and output VSync.

The control unit 10 controls each unit of the image display device 1 based on the RL identification signal and the vertical synchronization signal VSync input from the image input unit 20.
The feature amount calculation unit 30 receives the right-eye image data and the left-eye image data output from the image input unit 20, the RL identification signal, and the vertical synchronization signal VSync. Based on the RL identification signal and the vertical synchronization signal VSync, the feature amount calculation unit 30 identifies whether the image data being input from the image input unit 20 is right-eye image data or left-eye image data, and right-eye image data and left-eye image data. Image data is obtained. Then, the feature amount calculation unit 30 calculates an image feature amount for each of both acquired image data (step S13). The image feature amount calculated by the feature amount calculation unit 30 is, for example, the maximum luminance value (white peak value WP) of the entire image data , APL (Average Picture Level) that is an average value of luminance values, and minimum luminance value (black peak value). BP), a luminance histogram. The feature amount calculation unit 30 outputs the calculated image feature amount to the luminance expansion rate calculation unit 40 and the dimming rate calculation unit 60.

FIG. 3 is a diagram schematically showing a configuration of image data to be processed in the present embodiment. In the present embodiment, since the size and resolution of the right-eye image data and the left-eye image data are the same, both the right-eye image data and the left-eye image data are processed in the same manner.
For example, the feature amount calculation unit 30 processes image data of 1920 pixels × 1080 pixels (either right-eye image data or left-eye image data) as shown in FIG. The pixel blocks are divided into 200-1 to 200-144. The size of each pixel block 200-1 to 200-144 is 120 pixels vertically and 120 pixels horizontally.
The feature amount calculation unit 30 obtains a value obtained by averaging the luminance values of the pixels constituting the pixel block as a representative luminance value of the pixel block, and stores the representative luminance value in the RAM or the like for each pixel block. Here, as described above, in the right-eye image data, the RGB color components are held as gradation values for each pixel, but the luminance of a certain pixel is, for example, the maximum value of the RGB gradation values. Or a total value of 0.299 × R signal value, 0.587 × G signal value, and 0.144 × B signal value may be employed. In addition, the representative luminance value is not limited to the average luminance value, and for example, the luminance value (representative value) of a pixel near the center of each pixel block may be adopted. The feature amount calculation unit 30 sets the maximum value of the representative luminance value of each pixel block constituting one image data as the white peak value WP of the image data, sets the minimum value as the black peak value BP, and averages the representative luminance values. The value is APL. Also, a luminance histogram is generated from the distribution of representative luminance values of each pixel block of the image data.

The feature amount calculation unit 30 performs the white peak value WP, the black peak value BP, APL, and the white peak value BP of the image data as described above for each of a set of right-eye image data and left-eye image data constituting one stereoscopic image. Then, a luminance histogram is obtained, and then representative values of the right-eye image data and the left-eye image data are obtained (step S14). A method for determining the representative value is predetermined for each image feature amount.
(A) White peak value The representative value of the white peak value WP is the larger of the white peak value WP of the right eye image data and the white peak value WP of the left eye image data, that is, the brighter value. To do. This can be expressed by the following equation (1).
WP ₀ = Max (WP _R , WP _L ) (1)
Here, WP ₀ is the representative value of the white peak value, WP _R is the white peak value of the right-eye image data, and WP _L is the white peak value of the left-eye image data.
This is because, in the light control processing, it is suitable to use the luminance of the highest luminance portion in the image data as a reference. For example, when the representative value of the white peak value WP is set to the darker side, in the right-eye image data or the left-eye image data, there is a possibility that the highest luminance portion will be blown out by luminance expansion.

(B) In the case of APL The representative value of APL is the average value of the APL of the image data for the right eye and the APL of the image data for the left eye. This is expressed by the following formula (2).
APL ₀ = Avg (APL _R , APL _L ) (2)
Here, APL ₀ is the representative value of APL, APL _R is the APL of the image data for the right eye, and APL _L is the APL of the image data for the left eye.
Since APL is essentially an average value of luminance, it is suitable that the APL of two image data also obtains an average value.
(C) In the case of the black peak value The representative value of the black peak value BP is the smaller of the black peak value BP of the right-eye image data and the black peak value BP of the left-eye image data, that is, the darker value. To do. This is expressed by the following formula (3).
BP ₀ = Min (BP _R , BP _L ) (3)
Here, BP ₀ is the representative value of the black peak value, BP _R is the black peak value of the right-eye image data, and BP _L is the black peak value of the left-eye image data.
Since the black peak value BP is the luminance of the portion with the lowest luminance in the image data, if there are two target image data, the luminance of the darkest portion of these two images may be adopted as a representative value. This is preferable because luminance expansion processing suitable for the contrast of image data can be performed.
(D) In the case of the luminance histogram The representative value of the luminance histogram is the average of the luminance histogram of the right-eye image data and the luminance histogram of the left-eye image data. This is expressed by the following equation (4).
Hist ₀ (X) = {Hist _R (X) + Hist _L (X)} / 2 (4)
Here, Hist ₀ (X) represents the representative value of the luminance histogram, Hist _R (X) represents the luminance histogram of the right-eye image data, and Hist _L (X) represents the luminance histogram of the left-eye image data. In the case of image data having a gradation value of 10 bits, X = 0 to 1023.
According to this equation (4), the average value of each class of luminance is the representative value.

  As described above, the feature amount calculation unit 30 obtains various image feature amounts for each of the right-eye image data and the left-eye image data, and uses the obtained representative value of the image feature amount as the type or attribute of the image feature amount. Since it is obtained by a corresponding method, an optimum value for the luminance expansion processing can be obtained from the image feature amounts of the two image data. Then, the representative value of the image feature amount is output to the luminance expansion rate calculation unit 40 and the dimming rate calculation unit 60.

Note that the feature quantity calculation unit 30 may obtain not only the image feature quantity for the entire image data as shown in FIG. 3 but also the image feature quantity for only a partial region of the image data. For example, when a black region occurs in the peripheral portion of the image data due to the relationship between the input image data and the display resolution of the light valve 130, an image feature amount is obtained in the region excluding the peripheral portion of the image data, Weighting may be performed so that the central pixel block is more important than the peripheral pixel block of the image data, and the image feature amount may be obtained in consideration of the weight.
In the above-described example, a plurality of image feature amounts are obtained for each of the right-eye image data and the left-eye image data, and the representative values of the obtained image feature amounts are obtained by a method corresponding to the type or attribute of the image feature amount. The configuration is such that the obtained representative value is obtained, but the image feature amount of any one of the image data for the right eye and the image data for the left eye is obtained, and the obtained image feature amount is output. It may be.
Further, without dividing the image data into pixel blocks, the maximum value among the luminance values of each pixel of the entire image data is the white peak value WP of the image data, the minimum value is the black peak value BP, The average value may be APL, and a luminance histogram may be generated from the distribution of luminance values of each pixel.

The luminance expansion rate calculation unit 40 calculates the luminance expansion rate based on the representative value of the image feature amount input from the feature amount calculation unit 30 (step S15).
FIG. 4 is a diagram schematically showing an LUT 210 for obtaining the luminance expansion rate. In the LUT 210 illustrated in FIG. 4, the luminance expansion rate is defined corresponding to the white peak values WP and APL.
The feature amount calculation unit 30 refers to the LUT 210 and acquires the luminance expansion rate defined in the LUT 210 corresponding to the white peak value WP and the APL value input from the image input unit 20, thereby obtaining the luminance expansion rate. Ask for. When the white peak value WP and the APL value are out of the lattice points where the luminance expansion rate is defined, the image input unit 20 determines the three or four lattice points around the white peak value WP and the APL value. The luminance expansion rate is calculated by performing an interpolation operation based on the luminance expansion rate defined in (1). In this way, the feature amount calculation unit 30 calculates the luminance expansion rate, and outputs the calculated luminance expansion rate to the luminance expansion processing unit 50.
The feature quantity calculation unit 30 is not limited to the LUT 210 shown in FIG. 4, and may use a three-dimensional LUT that defines a luminance expansion rate corresponding to the white peak value WP, the black peak value BP, and the APL. A two-dimensional LUT using the peak value BP and the white peak value WP or the black peak value BP and APL may be used, and the calculation is based on one or more of the white peak value WP, the black peak value BP, APL, and the luminance histogram. The luminance expansion rate may be obtained by processing.

The luminance expansion processing unit 50 expands the gradation of the right-eye image data and the left-eye image data input from the image input unit 20 based on the luminance expansion rate obtained by the luminance expansion rate calculating unit 40 (step S16).
The operation of the luminance expansion processing unit 50 will be described in detail later.

On the other hand, the light attenuation rate calculation unit 60 calculates the light attenuation rate based on the representative value of the image feature amount input from the feature amount calculation unit 30 (step S17). Calculation of extinction ratio, for example, similarly to the luminance expansion rate as described with reference, white peak value WP, APL, the extinction ratio in response to two or more of the black peak value BP is defined to 4 The light attenuation rate can be calculated by using the LUT (not shown) and referring to this LUT. That is, the light attenuation rate calculation unit 60 acquires the light attenuation rate defined in the LUT corresponding to the white peak value WP, APL, or black peak value BP input from the image input unit 20. Further, when the white peak value WP, APL, or black peak value BP input from the image input unit 20 is out of the lattice point where the light attenuation rate is defined, the light attenuation rate calculation unit 60 The light attenuation rate is calculated by performing an interpolation operation based on the light attenuation rate defined for the point or the four lattice points. In this way, the light attenuation rate calculation unit 60 obtains the light attenuation rate and outputs the obtained light attenuation rate to the light attenuation processing unit 70. The light attenuation rate calculation unit 60 is not limited to a two-dimensional LUT, and may use a three-dimensional LUT, or may be based on one or more of a white peak value WP, a black peak value BP, an APL, and a luminance histogram. The dimming rate may be obtained by arithmetic processing.
Then, the dimming rate calculation unit 60 generates a drive signal for driving the dimming element 120 so that the calculated dimming rate ka is obtained, and outputs the drive signal to the dimming processing unit 70 (step S18).

Here, under the control of the control unit 10, the image data subjected to the luminance expansion processing by the luminance expansion processing unit 50 is input to the light valve 130, drawn in synchronization with the vertical synchronization signal VSync, and at this timing. In synchronization, the dimming processing unit 70 controls the dimming element 120 according to the drive signal input from the dimming rate calculating unit 60, and dimming is performed (step S19).
When the stereoscopic video signal input to the image input unit 20 is 60 frames / second, the image input unit 20 alternately outputs right-eye image data and left-eye image data at 120 frames / second. These right-eye image data and left-eye image data are paired to form one frame of stereoscopic image data. When an image is projected at such a high speed, the calculation of the light valve 130 is not delayed by the calculation associated with the dimming process, so the calculation of the luminance expansion rate and the dimming rate and the dimming process may shift. That is, for the right-eye image data and the left-eye image data constituting the n-th frame stereoscopic image data, the luminance expansion rate is calculated by the luminance expansion rate calculation unit 40, and the dimming rate is calculated by the dimming rate calculation unit 60. In this case, the dimming process based on the luminance expansion rate and the dimming rate is applied from the (n + 1) th frame. In this case, the target image data for which the luminance expansion rate and the light reduction rate are calculated are different from the image data subjected to the light control processing based on the luminance expansion rate and the light attenuation rate. Since the shift is limited to one frame, the possibility of an uncomfortable feeling due to this shift is extremely low, and an effect of improving the contrast feeling by the dimming process and improving the quality by expanding the dynamic range can be expected.

Next, the concept of parallax in the present embodiment will be described.
In the following description, “the image data for the right eye and the image data for the left eye correspond” means that these image data reflect the parallax of the right eye and the left eye so as to represent one image in three dimensions. That is, the data is generated. More specifically, when the format of the input stereoscopic video signal is a side-by-side format or a top-and-bottom format, a pair of right-eye image data and left-eye image data generated by being cut out from the stereoscopic video signal are: In the case of a system that corresponds to corresponding image data and that sequentially inputs a stereoscopic video signal related to right-eye image data and a stereoscopic video signal related to left-eye image data, in the present embodiment, one right-eye image It is assumed that the data and the left-eye image data input next to the one right-eye image data correspond to the corresponding image data.

FIG. 5 is a diagram for explaining parallax in the present embodiment.
In FIG. 5, a virtual space (a virtual with a depth expressed by a composite stereoscopic image) related to an image (hereinafter referred to as a “composite stereoscopic image”) that is stereoscopically represented by the right-eye image data and the left-eye image data. A typical view of a typical space) is shown from above.
For objects that are represented as being closer to the front side in the composite stereoscopic image, for objects that are placed closer to the front side in the virtual space, while being represented to be present in the deeper side in the composite stereoscopic image In the virtual space, it is arranged at the back side. For example, in the virtual space of FIG. 5, the object M1 is arranged in front of the object M2, so that in the synthesized stereoscopic image, the image related to the object M1 is closer to the front than the image related to the object M2. It is expressed to exist on the side. In the present embodiment, it is assumed that all objects are arranged in front of a reference plane that is a perspective reference in the virtual space.
In FIG. 5, the symbol P <b> 1 indicates the left eye point corresponding to the position of the left eye that visually recognizes the virtual space, and the symbol P <b> 2 indicates the right eye point corresponding to the position of the right eye that visually recognizes the virtual space. As is well known, right-eye image data and left-eye image data are generated using the parallax between the left eye located at the left eye point P1 and the right eye located at the right eye point P2.

As shown in FIG. 5, a virtual straight line SL1 connecting the left eye point P1 and the object M1 and a virtual straight line SR1 connecting the right eye point P2 and the object M1 intersect with each other at a parallax corresponding angle θ1 of the angle α in the object M1. A gap T1 is formed between the intersection KL1 between the virtual straight line SL1 and the reference plane and the intersection KR1 between the virtual straight line SR1 and the reference plane.
Similarly, a virtual straight line SL2 connecting the left eye point P1 and the object M2 and a virtual straight line SR2 connecting the right eye point P2 and the object M2 intersect with each other at the parallax corresponding angle θ2 of the angle β in the object M2, and the virtual straight line SL2 A gap T2 is formed between the intersection KL2 between the reference plane and the intersection KR2 between the virtual straight line SR2 and the reference plane.
The parallax correspondence angles θ1 and θ2 and the gaps T1 and T2 are values that appear due to the positional difference between the left eye point P1 and the right eye point P2, and the position of the object in the virtual space is closer to the front. The closer to the side, the larger the parallax-corresponding angle θ 1 and the gap T related to the object, and conversely, the closer the position of the object in the virtual space is to the far side, the disparity related to the object The corresponding angle θ 1 and the gap T are smaller values.

In the present embodiment, the parallax-corresponding angle θ and the conceptual representation of the gap T correspond to “parallax”. That is, the parallax in the present embodiment is relatively larger as the object located in the front in the virtual space due to the positional difference between the left eye point P1 and the right eye point P2, while the object located in the back is relatively larger. It is a value that conceptually indicates a value that becomes smaller.
Therefore, in the following description, for example, for an image related to one object and an image related to another object included in the right-eye image data, “the image related to one object is more than the image related to the other object. When the expression “parallax is large”, it means that one object is arranged in front of other objects in the virtual space in the composite stereoscopic image, and in the composite stereoscopic image, 1 This means that the image related to the object is expressed so as to exist in front of the image related to the other object.
The magnitude of the parallax is reflected in the right-eye image data and the left-eye image data as follows.

FIG. 6 is a diagram schematically illustrating the right-eye image data and the left-eye image data.
In FIG. 6, it is assumed that the image data for the right eye and the image data for the left eye are developed so as to correspond to each other in the same coordinate system. That is, in the coordinate system, the four corners of the right-eye image data and the four corners of the left-eye image data are expanded so as to overlap, and the coordinates defined for one pixel included in the right-eye image data and the left-eye image data The coordinates defined in other pixels arranged at the same position as the one pixel in the right-eye image data are the same.
As described above, since the image data for the right eye and the image data for the left eye are data in which each pixel is arranged in a dot matrix on the data, by developing these image data in the coordinate system, the image data The coordinates of each pixel of data are uniquely defined by the relative position from the position defined as the origin in the coordinate system.
In the image data for right eye and the image data for left eye in FIG. 6, the image data M′1 is image data corresponding to the object M1 in FIG. 4, and the image data M′2 corresponds to the object M2 in FIG. Image data to be processed.
As shown in FIG. 6, the image data M′1 in the right-eye image data and the image data M′1 in the left-eye image data are arranged so as to be shifted by a separation amount R1. Similarly, in these image data, the image data M′2 is shifted and arranged by the separation amount R2.
The separation amounts R1 and R2 are values corresponding to the parallax of the objects M1 and M2, and become larger as the object is placed closer to the front in the virtual space (= the larger the parallax), and vice versa. In addition, the smaller the object is located in the virtual space (= the smaller the parallax), the smaller the value.
That is, an object that is arranged closer to the front in the virtual space and should be expressed to be present on the nearer side in the composite stereoscopic image, in other words, an object with a larger parallax, the image data for the right eye and the left eye In the image data for use, the separation amount R corresponding to the image related to the object is large, and conversely, it should be expressed so that it is arranged deeper in the virtual space and is present in the deeper side in the composite stereoscopic image. In other words, in other words, in the right-eye image data and the left-eye image data, the separation amount R corresponding to the image related to the object becomes smaller as the object has a smaller parallax.

Next, the luminance expansion processing unit 50 will be described in detail.
FIG. 7 is a block diagram illustrating a functional configuration of the luminance expansion processing unit 50.
The luminance expansion processing unit 50 expands the gradations of the right-eye image data and the left-eye image data input from the image input unit 20 based on the luminance expansion rate obtained by the luminance expansion rate calculation unit 40. Yes, as shown in FIG. 7, a reference area coordinate calculation unit 51, a parallax value calculation unit 52, and a luminance expansion process execution unit 53 are provided.

FIG. 8 is a flowchart showing the operation of the reference area coordinate calculation unit 51.
First, the reference area coordinate calculation unit 51 is inputting from the image input unit 20 based on the right-eye image data and the left-eye image data input from the image input unit 20, the RL identification signal, and the vertical synchronization signal VSync. Are identified as right-eye image data or left-eye image data, and each of right-eye image data and left-eye image data is acquired (step S21).
Next, the reference area coordinate calculation unit 51 develops the acquired right-eye image data and left-eye image data into a predetermined coordinate system, and classifies each of these image data according to the reference area 300 (step). S22).
Referring to FIG. 3, the reference area coordinate calculation unit 51 converts, for example, 1920 × 1080 pixel right-eye image data and left-eye image data to 16 × It is divided into ninety-four vertical reference regions 300-1 to 300-144. In this case, the sizes of the reference areas 300-1 to 300-144 are 120 pixels vertically and 120 pixels horizontally.
Next, the reference area coordinate calculation unit 51 obtains the right eye image data, the left eye image data, and information related to coordinates defining each of the reference areas 300 in the image data, the parallax value calculation unit 52, and the luminance extension. It outputs to the process execution part 53 (step S23).

FIG. 9 is a flowchart showing the operation of the parallax value calculation unit 52.
The parallax value calculation unit 52 expands the right-eye image data and the left-eye image data input from the reference region coordinate calculation unit 51 into a predetermined coordinate system (step S31).
Next, the parallax value calculation unit 52 selects one of the reference regions 300 based on information on the coordinates of the reference region 300 formed by dividing the right-eye image data input from the reference region coordinate calculation unit 51. The reference area 300 is specified as the reference area to be processed, and image data corresponding to the reference area 300 specified as the reference area to be processed (hereinafter referred to as “reference area image data” as appropriate) is acquired (step S32). For example, in step S <b> 32, the parallax value calculation unit 52 acquires image data (reference area image data) in an area defined by the reference area 300-1 when the reference area 300-1 is a reference area to be processed. .
Next, the parallax value calculation unit 52 performs template matching processing on the left-eye image data using the reference region image data acquired in step S32 as a template image, so that the left-eye image data is converted into the reference region image data. The coordinates of the area where the corresponding image data is located are acquired (step S33).
Hereinafter, the operation in step S33 will be described in detail.

FIG. 10 is a diagram used for explaining the operation in step S33.
In step S33, first, the parallax value calculation unit 52 arranges the reference area image data acquired in step S32 in a predetermined coordinate system in which the left-eye image data is developed. At that time, the parallax value calculation unit 52 arranges the reference area image data at the arrangement position of the reference area image data in the right-eye image data in a predetermined coordinate system. For example, referring to FIG. 10, when the reference area image data acquired in step S32 is the reference area image data corresponding to the pixel block 200-1 (see FIG. 3), the predetermined left image data is developed. In the coordinate system, the reference area image data is arranged at the position of the pixel block 200-1 in the right-eye image data (position corresponding to the area X-0 in FIG. 10).
Next, the parallax value calculation unit 52 calculates the degree of similarity between the reference area image data and the image data of the left-eye image data in the area where the reference area image data is arranged. In this embodiment, the similarity is a value in a range of −1 to 1 calculated using a normalized mutual function, and the similarity is higher as the value is closer to 1.
Next, the parallax value calculation unit 52 shifts the reference area image data, which is a template image, in the coordinate system in which the left-eye image data is developed, by one pixel in the right direction (the direction indicated by the arrow Y1 in FIG. 10). Similar to the method described above, the similarity between the reference area image data and the image data of the image data for the left eye in the area where the reference area image data is arranged is calculated.

In this way, the parallax value calculation unit 52 calculates the similarity between the reference region image data and the left-eye image data after shifting the reference region image data to the right by one pixel. Is repeated until the reference area image data reaches the right end of the left-eye image data. For example, referring to FIG. 10, when the reference area image data acquired in step S32 is reference area image data corresponding to the area X-0, the parallax value calculation unit 52 determines that the reference area image data is the area X The above-described operation is repeated until -1.
Next, the parallax value calculation unit 52 compares the calculated similarities, identifies a region having the highest similarity among the regions for which the similarity is calculated in the image data for the left eye, Get the coordinates that define. For example, referring to FIG. 10, when the reference region image data acquired in step S32 is reference region image data corresponding to region X-0, the reference region image data and region X-2 are associated. When the degree of similarity with the left-eye image data is the highest, the parallax value calculation unit 52 acquires coordinates that define the region X-2.
As described above, in step S33, the parallax value calculation unit 52 acquires the coordinates of the region where the image data corresponding to the reference region image data acquired in step S32 is located in the left-eye image data.

Here, as described with reference to FIGS. 5 and 6, regarding the image data of an image related to a certain object, the position where the image data of the image is arranged in the image data for the right eye, and the image for the left eye In the data, there is a deviation of the separation amount R reflecting the parallax from the position where the image data of the image is arranged.
Each of the reference area image data and the image data corresponding to the area of the image data for the left eye specified in step S33 has the highest similarity, that is, an image indicating the most “similar” image. It is data, and these are each image data formed by shifting the separation amount R by reflecting the parallax with respect to the image relating to the same object.

In this embodiment, a similarity using a normalized mutual function is adopted as the similarity. This reflects the parallax between the right eye and the left eye in the image data for the right eye and the image data for the left eye even when the image data is related to the same object (how the object looks when viewed with the right eye, Reflecting the difference in appearance when the object is viewed with the left eye), the content of the data is different, and considering this, the reference area image in the image data for the left eye is appropriately This is because an area corresponding to the data is detected. The detection of the region where the image data corresponding to the reference region image data is located is not limited to a method using the similarity using a normalized mutual function. For example, the reference region image data is binarized and the left eye The image data may be binarized and the template matching process may be performed using these image data.
In the present embodiment, the operation of calculating the similarity between the reference area image data and the left-eye image data after shifting the reference area image data by “one pixel” in the “right direction” By repeatedly performing the region image data until it reaches the right end of the image data for the left eye, the region corresponding to the reference region is detected. With reference to FIG. 5, when it is assumed that all objects are arranged in front of the reference plane in the virtual space as in this embodiment, as shown in FIG. The position in the image data for the right eye of the image data of the image relating to one object is always the horizontal direction (the straight line connecting the right eye and the left eye) of the position in the image data for the left eye of the image data of the image relating to the one object. This is because it is “left” in the direction corresponding to the extending direction), and by detecting the region corresponding to the reference region by the above-described operation, it is possible to prevent unnecessary calculation of the similarity and to improve the processing efficiency. This is because improvement can be achieved.

Returning to FIG. 9, the parallax value calculation unit 52 determines the center coordinates of the area where the reference area image data is arranged in the right-eye image data and the center of the area where the similarity is the highest in the left-eye image data. A separation amount R, which is a distance from the coordinates, is calculated (step S34). Referring to FIG. 10, when the region X-2 has the highest similarity, the parallax value calculation unit 32 sets the distance between the center of the region X-2 and the center of the region X-0 as the separation amount R. calculate.
Next, the parallax value calculation unit 52 calculates the calculated separation amount R as it is as the parallax value of the reference region 300 to be processed in step S32 (step S35). In the present embodiment, the separation amount R is used as the parallax value as it is, but the parallax value may be a value having a positive correlation with the separation amount R. That is, the parallax value may be a value having a positive correlation with the magnitude of the parallax by having a positive correlation with the separation amount R.

The parallax value calculation unit 52 performs the process according to steps S32 to S35 on all the reference areas 300 formed in the right-eye image data, and calculates the parallax values of all the reference areas 300.
Further, the parallax value calculation unit 52 performs the processing corresponding to step S32 to step S35 on all the reference regions 300 formed in the left-eye image data, so that all the reference regions 300 in the left-eye image data are processed. A parallax value is calculated. At that time, in executing the processing corresponding to step S33, the parallax value calculation unit 52 uses the reference area image data related to the reference area 300 formed in the left-eye image data in the coordinate system in which the right-eye image data is developed. The operation of calculating the degree of similarity between the reference area image data and the left eye image data after shifting by one pixel “leftward” is performed on the left end of the right eye image data. By repeatedly performing the steps up to, detection of a region corresponding to the reference region is executed. As a result, the processing efficiency is improved.
Thus, after calculating the parallax values of all the reference areas 300 formed in the right-eye image data and the left-eye image data, the parallax value calculation unit 52 indicates the parallax values of all the reference areas 300. Information is output to the luminance expansion processing execution unit 53 (step S36).

FIG. 11 is a flowchart showing the operation of the luminance expansion processing execution unit 53.
Based on the information input from the parallax value calculation unit 52, the luminance expansion processing execution unit 53 selects a reference region 300 in which the parallax value exceeds a predetermined threshold among the reference regions 300 formed in the right-eye image data. Identify. Similarly, the luminance expansion process execution unit 53 is based on the information input from the parallax value calculation unit 52, and the reference in which the parallax value exceeds a predetermined threshold in the reference region 300 formed in the left-eye image data. The area 300 is specified (step S41).
Next, the luminance expansion processing execution unit 53 corresponds to (includes) the image data corresponding to (included in) the reference region 300 (the reference region 300 in which the parallax value exceeds a predetermined threshold value) identified in step S41 in the right-eye image data. Then, a luminance expansion process is performed using the luminance expansion ratio obtained by the luminance expansion ratio calculation unit 40. Similarly, in the left-eye image data, the reference region 300 specified in step S41 (with a parallax value of a predetermined value) is executed. A luminance expansion process using the luminance expansion ratio obtained by the luminance expansion ratio calculation unit 40 is executed on the image data corresponding to the reference area 300) exceeding the threshold (step S42). Here, the color information of the image data input from the feature amount calculation unit 30 to the luminance expansion processing unit 50 is R, G, B, the color information after luminance expansion is R ′, B ′, G ′, and the luminance expansion rate is Assuming that kg, the luminance expansion processing is executed so that R ′ = kg × R, G ′ = kg × G, and B ′ = kg × B.
That is, in step S42, for each of the right-eye image data and the left-eye image data, although the luminance expansion is performed on the reference region 300 in which the parallax value exceeds a predetermined threshold, the parallax value is predetermined. The luminance expansion is not performed on the reference region 300 that is below the threshold value.
Next, the luminance expansion processing execution unit 53 alternately outputs the right-eye image data and the left-eye image data after the expansion processing (Step S43).

As described above, in the present embodiment, although the luminance expansion is performed on the reference area 300 in which the parallax value exceeds the predetermined threshold for each of the right-eye image data and the left-eye image data, the parallax is performed. The luminance expansion is not performed on the reference region 300 whose value is below a predetermined threshold value. This is due to the following reason.
That is, the composite stereoscopic image represented by the right-eye image data and the left-eye image data includes a background image indicating a background as infinity, and an image expressed with a certain degree of stereoscopic effect on the background image. May be configured. In this case, there is a tendency that an image expressed with a certain degree of stereoscopic effect is more important than a background image, and when the image data is subjected to luminance expansion processing, the image expressed with the corresponding stereoscopic effect. There is a need to perform a luminance expansion process on image data related to the above.
In view of this, in the present embodiment, the parallax value exceeds a predetermined threshold value for each of the right-eye image data and the left-eye image data, and is expressed with a certain degree of three-dimensionality rather than a region related to the background image. For the reference region 300 having a high probability that is a region related to the power image, the luminance extension is performed. On the other hand, for the reference region 300 having a high probability of being a region related to the background image, the parallax value is below a predetermined threshold value. It does not perform luminance expansion and responds appropriately to the needs.
Note that the predetermined threshold value is appropriately determined through prior simulation and testing from the viewpoint that it is a reference for discriminating between a background image and an image expressed with a certain degree of stereoscopic effect.

As described above, the image display device 1 according to the present embodiment calculates the image feature amount related to the brightness of the image data based on the right-eye image data and the left-eye image data that constitute the stereoscopic image data. A feature amount calculation unit 30 and a luminance expansion rate calculation unit 40 that calculates a luminance expansion rate related to the luminance expansion processing applied to the right-eye image data and the left-eye image data based on the image feature amount calculated by the feature amount calculation unit. Based on the parallax between the (expansion coefficient calculating unit), the right-eye image data and the left-eye image data, and the luminance expansion rate calculated by the luminance expansion rate calculating unit 40, luminance expansion is performed on the image data. And a processing unit 50.
More specifically, the luminance expansion processing unit 50 performs each of the reference regions 300 formed by dividing the image data to be subjected to the luminance expansion processing among the right-eye image data and the left-eye image data. Reflecting the parallax in each reference region 300, the image data corresponding to each region is subjected to luminance expansion processing based on the luminance expansion rate calculated by the luminance expansion rate calculating unit 40.
According to this, the luminance expansion processing unit 50 performs the luminance expansion processing on the entire image data based on the uniform expansion coefficient as in the past with respect to the right-eye image data and the left-eye image data. Instead of reflecting the parallax in each of the reference areas 300 formed by dividing the image data, the luminance expansion processing unit 50 applies the parallax to the image data corresponding to each reference area 300. It is possible to perform effective luminance expansion processing based on this.

Further, in the present embodiment, the luminance expansion processing unit 50 performs the other image for each of the reference regions 300 of one image data to be subjected to the luminance expansion processing among the right-eye image data and the left-eye image data. A separation amount R with each of the regions corresponding to the reference region 300 in the data is detected, a disparity value having a positive correlation with the detected separation amount R is calculated, and luminance expansion is performed using the calculated disparity value Based on the luminance expansion rate calculated by the rate calculation unit 40, the image data corresponding to each reference area 300 is subjected to luminance expansion processing.
According to this, the luminance expansion processing unit 50 calculates a parallax value having a positive correlation with the separation amount R, and performs luminance expansion processing using the calculated parallax value. Since the parallax value used when executing the luminance expansion processing has a positive correlation with the separation amount R and has a positive correlation with the magnitude of the parallax in one reference region 300, the parallax value By performing the luminance expansion processing using the value, it is possible to execute the luminance expansion processing while appropriately reflecting the parallax (parallax magnitude).

In the present embodiment, the luminance expansion processing unit 50 performs luminance expansion processing on the reference region 300 in which the calculated parallax value exceeds a predetermined threshold among the reference regions 300 of the image data to be subjected to the luminance expansion processing. Apply.
Here, the composite stereoscopic image expressed by the image data for the right eye and the image data for the left eye is a background image indicating the background as infinity, and an image expressed with a certain degree of stereoscopic effect on the background image. May be configured. In this case, there is a tendency that an image expressed with a certain degree of stereoscopic effect is more important than a background image, and when the image data is subjected to luminance expansion processing, the image expressed with the corresponding stereoscopic effect. There is a need to perform a luminance expansion process on image data related to the above.
In view of this, in the present embodiment, the parallax value exceeds a predetermined threshold value for each of the right-eye image data and the left-eye image data, and is expressed with a certain degree of three-dimensionality rather than a region related to the background image. For the reference region 300 having a high probability that is a region related to the power image, the luminance extension is performed. On the other hand, for the reference region 300 having a high probability of being a region related to the background image, the parallax value is below a predetermined threshold value. It does not perform luminance expansion and responds appropriately to the needs.

Further, in the present embodiment, a dimming element 120 for dimming light emitted from the light source device 110 is provided corresponding to the luminance expansion processing by the luminance expansion processing unit 50.
According to this, with respect to the image projected on the screen 5, it is possible to increase the dynamic range of the image and improve the contrast while maintaining the apparent brightness of the image.

Second Embodiment
Next, a second embodiment will be described.
In the present embodiment, the operation of the luminance expansion processing execution unit 53 is different from that in the first embodiment described above.
The luminance expansion processing execution unit 53 according to the present embodiment has a higher parallax value for each of the reference regions 300 of the image data targeted for luminance expansion processing among the right-eye image data and the left-eye image data. The luminance expansion rate is corrected so that the larger the reference region, the higher the luminance expansion rate, and the luminance expansion processing is performed on the image data corresponding to the reference region 300.
Specifically, the luminance expansion processing execution unit 53 performs the processing for each of the reference regions 300 formed in the image data to be subjected to the luminance expansion processing among the right-eye image data and the left-eye image data. A luminance expansion rate (hereinafter referred to as “corrected luminance expansion rate”) when performing luminance expansion processing on image data corresponding to the reference region 300 is calculated by the following equation (5).
T = kg × D / Dmax (5)
Here, T represents the corrected luminance expansion rate, kg represents the luminance expansion rate input from the luminance expansion rate calculation unit 40 to the luminance expansion processing unit 50, and D represents the calculated corrected luminance expansion rate. Dmax is a parallax value having the maximum value among the parallax values of the reference area 300 formed in the image data to be subjected to luminance expansion processing (hereinafter, “ The maximum parallax value).
Then, the luminance expansion processing execution unit 53 calculates the corrected luminance expansion ratio using the above-described equation (5) for all the reference regions 300 formed in the image data that is the target of the luminance expansion processing. The luminance expansion processing is performed on each of the reference areas 300 based on the corrected luminance expansion ratio applied to each reference area 300.

Here, consider a case where the corrected luminance expansion rate applied to a certain reference region 300 is calculated using equation (5).
In this case, “D / Dmax” in Expression (5) is the maximum parallax value (the value of the parallax values of the reference area 300 formed in the image data to be subjected to the luminance expansion processing is the maximum). Is the ratio of the parallax value of the one reference area 300 to the (parallax value). Therefore, when the parallax value of the one reference region 300 is the maximum parallax value, the value of “D / Dmax” is “1”, and the parallax value of the one reference region 300 is compared with the maximum parallax value. The lower the value, the smaller the value of “D / Dmax” accordingly.
In equation (5), the corrected luminance expansion rate (T) is calculated by multiplying the luminance expansion rate (kg) calculated by the luminance expansion rate calculating unit 40 by “D / Dmax”. Therefore, when the parallax value of the one reference area 300 is the maximum parallax value, the corrected luminance expansion rate (T) = luminance expansion rate (kg). In this case, the luminance expansion process is performed on the image data corresponding to the one reference region 300 using the luminance expansion rate (kg) calculated by the luminance expansion rate calculation unit 40 as it is.
On the other hand, as the parallax value of the one reference region 300 is relatively lower than the maximum parallax value, the corrected luminance expansion rate (T) is increased according to the ratio of the parallax value to the maximum parallax value. The value is relatively low compared to the luminance expansion rate (kg). In this case, for the image data corresponding to the one reference region 300, the luminance expansion rate (kg) calculated by the luminance expansion rate calculation unit 40 is determined according to the ratio of the parallax value to the maximum parallax value. The luminance expansion process is performed after the value is corrected to be low.
In other words, in the present embodiment, the luminance expansion rate calculated by the luminance expansion rate calculation unit 40 is used as it is for the reference region 300 having the largest parallax value, while the parallax value is The luminance expansion process is performed after the luminance expansion rate is corrected so as to decrease as the value decreases.

In the present embodiment, the luminance expansion processing is performed after correcting the luminance expansion rate as described above for the following reason.
That is, in the composite stereoscopic image expressed by the image data for the right eye and the image data for the left eye, the image related to the object expressed so as to exist in the foreground is an important image in which stereoscopic effect and power are emphasized. There is a tendency, and when performing the luminance expansion processing, there is a need to perform the luminance expansion processing on the image data of the image related to such an object based on the expansion coefficient having a higher luminance expansion ratio. . As described above, for the reference region 300 having the largest parallax value, the luminance expansion rate calculated by the luminance expansion rate calculation unit 40 is used as it is, and the luminance expansion processing is performed. The image data of the image related to the object expressed so as to exist on the nearer side by adopting a configuration in which the luminance expansion process is performed after the luminance expansion rate is corrected so as to become smaller. On the other hand, it is possible to perform luminance expansion processing based on an expansion coefficient having a higher luminance expansion rate.

As described above, in the present embodiment, the luminance expansion processing unit 50 increases the luminance expansion ratio of the reference region 300 having a larger parallax value for each of the reference regions 300 of the image data to be subjected to the luminance expansion processing. After correcting the luminance expansion rate so as to increase, the luminance expansion processing is performed on the image data corresponding to each reference region 300.
As a result, as described above, it is possible to perform luminance expansion processing based on an expansion coefficient having a higher luminance expansion ratio on the image data of an image related to an object that is expressed so as to exist on the near side. Thus, effective luminance expansion processing based on parallax by the luminance expansion processing unit can be realized.

<Third Embodiment>
Next, a third embodiment will be described.
In the present embodiment, the operation of the luminance expansion processing execution unit 53 is different from that in the first embodiment described above.
The luminance expansion processing execution unit 53 according to the present embodiment detects the frequency of the reference region 300 having the same parallax value for each of the reference regions 300 of the image data to be subjected to the luminance expansion processing, and the frequency is higher. The luminance expansion rate is corrected so that the luminance expansion rate of the reference area 300 becomes higher, and the luminance expansion processing is performed on the image data corresponding to each reference area 300.
Specifically, first, the luminance expansion processing execution unit 53 acquires the upper limit value and the lower limit value of the parallax value based on the parallax value of the reference region 300 of the right-eye image data input from the parallax value calculation unit 52. The upper limit value and the lower limit value are divided for each predetermined range. Next, the luminance expansion processing execution unit 53 detects the number of reference regions 300 having a parallax value in each range for each divided range. For example, the luminance expansion processing execution unit 53 indicates that the number of reference areas 300 with value A1 ≦ parallax value <value A2 is 20, and the number of reference areas 300 with value A2 ≦ parallax value <value A3 is 15. Detect that.
Then, based on the detection result, the luminance expansion processing execution unit 53 generates a histogram in which the parallax values divided into a predetermined range are classified into classes, and the number (frequency) of the reference regions 300 having the parallax values in the range is represented as the frequency. To do. By referring to the histogram generated here, with respect to a certain reference region 300, there are a number of reference regions 300 having disparity values that belong to the same range as the disparity value of the reference region 300 in the right-eye image data. It is possible to detect whether or not That is, by referring to the histogram, it is possible to detect the frequency in the right eye image data of the reference region 300 having the same degree of parallax value for each of the reference regions 300.
Similarly, the luminance expansion processing execution unit 53 classifies the parallax values divided into a predetermined range based on the parallax value of the reference region 300 of the image data for left eye input from the parallax value calculation unit 52, and the range. A histogram is generated with the number (frequency) of the reference areas 300 having the parallax values as the frequency.
In the following description, “frequency of the parallax value of the reference area 300” refers to the number of reference areas 300 having a parallax value belonging to the same range as the parallax value of the reference area 300.

Next, the luminance expansion processing execution unit 53 performs each reference region 300 for each of the reference regions 300 formed in the image data to be subjected to the luminance expansion processing among the right-eye image data and the left-eye image data. The luminance expansion rate (hereinafter referred to as “corrected luminance expansion rate”) when performing the luminance expansion processing on the image data corresponding to is calculated by the following equation (6).
T = kg × F / Fmax (6)
Here, T represents the corrected luminance expansion rate, kg represents the luminance expansion rate input from the luminance expansion rate calculation unit 40 to the luminance expansion processing unit 50, and F represents the calculated corrected luminance expansion rate. Fmax represents the maximum frequency of the parallax values of the reference area 300 formed in the image data that is the target of the luminance expansion processing (hereinafter referred to as “maximum frequency”). Represents.
Then, the luminance expansion processing execution unit 53 calculates the corrected luminance expansion ratio using the above-described equation (6) for all the reference regions 300 formed in the image data that is the target of the luminance expansion processing. The luminance expansion processing is performed on each of the reference areas 300 based on the corrected luminance expansion ratio applied to each reference area 300.

Here, consider a case where the corrected luminance expansion rate applied to a certain reference region 300 is calculated using Expression (6).
In this case, “F / Fmax” in Expression (6) is the 1 for the maximum frequency (the maximum frequency of the parallax values of the reference region 300 formed in the image data that is the target of the luminance expansion processing). The frequency ratio of the parallax values of the reference area 300 of FIG. Therefore, when the frequency of the parallax value of the one reference region 300 is the maximum frequency, the value of “F / Fmax” is “1”, and the frequency of the parallax value of the one reference region 300 is the maximum frequency. The lower the value, the smaller the value of “F / Fmax” accordingly.
In Expression (6), the corrected luminance expansion rate (T) is calculated by multiplying the luminance expansion rate (kg) calculated by the luminance expansion rate calculating unit 40 by “F / Fmax”. For this reason, when the frequency of the parallax value of the one reference region 300 is the maximum frequency, the corrected luminance expansion rate (T) = the luminance expansion rate (kg). In this case, the luminance expansion process is performed on the image data corresponding to the one reference region 300 using the luminance expansion rate (kg) calculated by the luminance expansion rate calculation unit 40 as it is.
On the other hand, the lower the parallax value frequency of the one reference region 300 is, the lower the value of the parallax value frequency relative to the maximum frequency, the more the corrected luminance expansion rate (T ) Is a relatively low value compared to the luminance expansion rate (kg). In this case, for the image data corresponding to the one reference region 300, the luminance expansion rate (kg) calculated by the luminance expansion rate calculation unit 40 depends on the ratio of the frequency of the parallax value to the maximum frequency. The luminance expansion processing is performed after the value is corrected to be low.
In other words, in the present embodiment, the luminance expansion rate calculated by the luminance expansion rate calculation unit 40 is used as it is for the reference region 300 having the highest frequency of parallax values, and the luminance expansion processing is performed. As the frequency of the value decreases, the luminance expansion rate is corrected, and the luminance expansion processing is performed.

In the present embodiment, the luminance expansion processing is performed after correcting the luminance expansion rate as described above for the following reason.
That is, in the composite stereoscopic image expressed by the right-eye image data and the left-eye image data, the image related to the object in which there are more objects expressed so as to have the same position in the depth direction, There is a tendency to be an important image in a stereoscopic image, and when performing luminance expansion processing, luminance expansion processing is performed on image data of an image related to such an object based on an expansion coefficient having a higher luminance expansion rate. There is a need to apply. In this embodiment, the luminance expansion rate calculated by the luminance expansion rate calculation unit 40 is used as it is for the reference region 300 having the highest frequency of the parallax value, and the luminance expansion processing is performed. As the frequency of the value decreases, the luminance expansion rate is corrected so that the luminance expansion rate is reduced, and then the luminance expansion processing is performed. The image data of the image can be subjected to the luminance expansion processing based on the expansion coefficient having a higher luminance expansion rate, and appropriately meets the needs.

As described above, in this embodiment, the luminance expansion processing unit 50 detects the frequency of the reference region 300 having the same degree of parallax value for each of the reference regions 300 of the image data to be subjected to the luminance expansion processing. The luminance expansion rate is corrected so that the higher the frequency of the reference region 300 is, the higher the value is, and the luminance expansion processing is performed on the image data corresponding to each reference region 300.
According to this, as described above, the luminance expansion processing is performed on the image data of the image related to the object expressed so as to exist on the near side based on the expansion coefficient having a higher luminance expansion ratio. Thus, the luminance expansion processing unit can perform effective luminance expansion processing based on parallax.

The above-described embodiment is merely an aspect of the present invention, and can be arbitrarily modified and applied within the scope of the present invention.
For example, the image input unit 20 generates right-eye image data and left-eye image data from the input stereoscopic video signal, and outputs the right-eye image data and the left-eye image data alternately. However, the present invention is not limited to this. For example, the luminance expansion processing unit 50 may output the right-eye image data and the left-eye image data to the light valve 130 alternately.
Moreover, each function part mentioned above shows the functional structure of the image display apparatus 1, Comprising: A specific mounting form is not restrict | limited in particular. That is, it is not always necessary to mount hardware corresponding to each function unit individually, and it is of course possible to adopt a configuration in which the functions of a plurality of function units are realized by one processor executing a program.

  In the present embodiment, the image display apparatus 1 includes various functional units such as the feature amount calculation unit 30, the luminance expansion rate calculation unit 40, and the luminance expansion processing unit 50, and these functional units calculate image feature amounts. In addition, the configuration for executing the calculation of the luminance expansion ratio, the luminance expansion processing, and the like has been described as an example. However, the image supply device configured separately from the image display device 1 such as a personal computer has various functional units. The image processing apparatus may be configured to perform luminance expansion processing on the right-eye image data and the left-eye image data and supply the image data from the image processing apparatus to the projector. That is, the functions of various functional units such as the feature amount calculation unit 30, the luminance expansion rate calculation unit 40, and the luminance expansion processing unit 50 may be provided in the image display device 1 itself. An image supply device that supplies image data may be provided.

  In addition, the image display device 1 described above is a type of device that projects an image onto the screen 5 using the transmissive light valve 130, but may be a projector that uses a reflective liquid crystal panel, or a digital device. A DMD projector using a mirror device may be used. The projector is not limited to a 3LCD projector that projects a color image by three light valves, and includes a projector and a color wheel that project a color image by displaying time-division images corresponding to RGB using a single liquid crystal light valve. The present invention can be applied to both a single-plate DMD projector and a 3DMD projector. As the light source, as described above, various light sources such as an ultra-high pressure mercury lamp and an LED lamp can be used in addition to the xenon lamp. Moreover, the image display apparatus 1 mentioned above may be the image display apparatus 1 of the type arrange | positioned on the front side of a projection surface, and projects a projection light on the front surface of a projection surface, and is arrange | positioned on the back side of a projection surface. Then, the image display device 1 of the type that projects the projection light onto the back surface of the projection surface may be used. That is, the present invention is widely applicable to image display devices having a function of modulating light emitted from a light source.

  Furthermore, the image display device 1 of the present invention is not limited to the projector that projects a stereoscopic (3D) image on the screen 5 as described above, but a liquid crystal monitor that displays a stereoscopic (3D) image / video on a liquid crystal display panel or Liquid crystal television, or a monitor device or television receiver that displays a three-dimensional (3D) image / video on a PDP (plasma display panel), organic light-emitting diode (OLED), organic electro-luminescence (OEL), etc. Various display devices such as a monitor device for displaying a stereoscopic (3D) image / video on an EL display panel or a self-luminous display device such as a television receiver are also included in the image display device of the present invention. In this case, a liquid crystal display panel, a plasma display panel, and an organic EL display panel correspond to the image display unit.

  DESCRIPTION OF SYMBOLS 10 ... Control part, 20 ... Image input part, 30 ... Feature-value calculation part, 32 ... Parallax value calculation part, 40 ... Luminance expansion rate calculation part (expansion coefficient calculation part), 50 ... Luminance expansion process part, 51 ... Reference | standard area | region Coordinate calculation unit, 52 ... parallax value calculation unit, 53 ... luminance extension processing execution unit, 60 ... dimming rate calculation unit, 70 ... dimming processing unit, 110 ... light source device (light source, image display unit), 120 ... dimming Element (light control unit), 130... Light valve (modulation unit, image display unit), 140... Projection optical system (image display unit), 300.

Claims (8)

A feature amount calculation unit that calculates an image feature amount related to the brightness of the image data based on the image data for the right eye and the image data for the left eye constituting the stereoscopic image data;
Based on the image feature amount calculated by the feature amount calculation unit, an expansion coefficient calculation unit that calculates an expansion coefficient related to a luminance expansion process applied to the right-eye image data and the left-eye image data;
Luminance based on the parallax between the right-eye image data and the left-eye image data and the expansion coefficient calculated by the expansion coefficient calculation unit with respect to the right-eye image data and the left-eye image data A luminance expansion processing unit for performing expansion processing;
An image display unit that displays an image based on the right-eye image data and the left-eye image data subjected to the luminance expansion processing ,
The luminance expansion processing unit is configured to perform the reference in the other image data for each of a plurality of reference regions in one image data to be subjected to luminance expansion processing among the right-eye image data and the left-eye image data. Detecting a separation amount with each of the regions corresponding to the region, and using a disparity value having a positive correlation with the detected separation amount, based on the expansion coefficient calculated by the expansion coefficient calculation unit, An image display device that performs luminance expansion processing on image data corresponding to the reference region . The reference area is formed by dividing image data to be subjected to luminance expansion processing among the right-eye image data and the left-eye image data ,
The image display device according to claim 1, wherein the luminance expansion processing unit detects the separation amount for each of the reference regions. The luminance expansion processing unit
Of the reference area of the image data to be subjected to the luminance expansion process claims visual difference values for the image data corresponding to the reference area exceeds the predetermined threshold value, and characterized by applying the luminance expansion process Item 3. The image display device according to Item 1 or 2 . The luminance expansion processing unit
For each of the reference region of the image data to be subjected to the luminance expansion process, the more visual difference value is greater than the reference area, the expansion coefficient as more luminance expansion rate is higher in terms of corrected, to each of said reference region the image display apparatus according to claim 1 or 2, characterized by applying the luminance expansion process on corresponding image data. The luminance expansion processing unit
The frequency of the reference area having the same parallax value is detected for each of the reference areas of the image data to be subjected to the luminance expansion processing, and the luminance expansion ratio is higher for the reference area having a higher frequency. in the expansion coefficient on corrected, the image display apparatus according to claim 1 or 2, characterized by applying the luminance expansion processing on the image data corresponding to each of said reference region. A modulation unit for modulating the light emitted from the light source;
The luminance expansion processing unit outputs the right-eye image data and the left-eye image data subjected to the luminance expansion processing to the modulation unit,
Corresponding to the luminance expansion process by the luminance expansion processing unit, an image display apparatus according to any one of claims 1 to 5 wherein the light source may comprise a dimmer for dimming the light emitted, characterized by. An image supply device for supplying image data to an image display device,
A feature amount calculation unit that calculates an image feature amount related to the brightness of the image data based on the image data for the right eye and the image data for the left eye constituting the stereoscopic image data;
Based on the image feature amount calculated by the feature amount calculation unit, an expansion coefficient calculation unit that calculates an expansion coefficient related to a luminance expansion process applied to the right-eye image data and the left-eye image data;
Luminance based on the parallax between the right-eye image data and the left-eye image data and the expansion coefficient calculated by the expansion coefficient calculation unit with respect to the right-eye image data and the left-eye image data A luminance expansion processing unit for performing expansion processing ,
The luminance expansion processing unit is configured to perform the reference in the other image data for each of a plurality of reference regions in one image data to be subjected to luminance expansion processing among the right-eye image data and the left-eye image data. Detecting a separation amount with each of the regions corresponding to the region, and using a disparity value having a positive correlation with the detected separation amount, based on the expansion coefficient calculated by the expansion coefficient calculation unit, An image supply apparatus that performs luminance expansion processing on image data corresponding to the reference area . Based on the image data for the right eye and the image data for the left eye constituting the stereoscopic image data, an image feature amount related to the luminance of the image data is calculated,
Based on the calculated image feature amount, calculates an expansion coefficient related to the luminance expansion processing applied to the right-eye image data and the left-eye image data,
Of each of the image data for the right eye and the image data for the left eye , each of a plurality of reference areas in one image data to be subjected to luminance expansion processing, each of the areas corresponding to the reference area in the other image data And a luminance expansion process on the image data corresponding to each of the reference regions based on the calculated expansion coefficient using a parallax value having a positive correlation with the detected separation amount. An image processing method characterized by applying the image processing method.

Images