JP6849696B2 - A low-cost color extension module that extends the color of an image - Google Patents

Description

This principle is generally relevant in the areas of high dynamic range imaging and dynamic range extension of low dynamic range content to prepare low dynamic range content, especially for display devices with high peak brightness.

Recent advances in display technology have begun to enable extended ranges of displayed chrominance, brightness and contrast. A technology that allows an extension of the brightness or range of brightness in image content is often known as high dynamic range imaging, abbreviated as HDR. HDR technology focuses on capturing, processing, and displaying content with a wider dynamic range.

A large number of HDR display devices have emerged and cameras have been developed that are capable of capturing images with increased dynamic range, but the available HDR content remains very limited. While recent developments promise an innate capture of HDR content in the near future, they do not address existing content.

Traditional (referred to in this application as LDR for low dynamic range) content is prepared for HDR display devices by an inverse tone mapping operator (ITMO) or color enhancement. Operators can be used. Such a method processes the luminance information of the colored region in the image content, in particular, for the purpose of restoring or reproducing the appearance of the original scene. Normally, ITMO takes a conventional (LDR) image as an input, expands the brightness range of the colored region of the image in a global manner, and then locally processes the highlight or bright region in the image. Strengthen the HDR appearance of the color.

There are several ITMO solutions, but they focus on perceptually reproducing the appearance of the original scene and rely on strict content requirements. In addition, most of the expansion methods proposed in the literature have been optimized for extreme increases in dynamic range.

HDR imaging is typically defined by an extension of the dynamic range between dark and light values of luminance in the tinted region, combined with an increase in the number of quantization steps. To achieve a more extreme increase in dynamic range, many methods combine local processing steps with global expansion to enhance the appearance of image highlights and other bright areas. The known global expansion steps proposed in the literature range from inverse sigmoids to linear or piecewise linear.

It is known that each pixel of an image produces a brightness extension map associated with an extension value to be applied to the brightness of this pixel so as to enhance the bright local features in the image. In the simplest case, a clipped area in the image can be detected and then extended with a steeper extension curve, but such a solution does not provide sufficient control over the appearance of the image.

Those and other determinations and disadvantages of the prior art are methods and devices that extend the dynamic range of low dynamic range content to prepare such content, especially for display devices with high peak brightness. It is addressed by various described embodiments that cover.

The object of the present invention is a color expansion module that expands the color of an image, wherein the color is represented in a color space that separates a luminance component from a chrominance component.
A low-pass filter that applies a low-pass filter to the luminance component,
A first look-up table that stores the pre-computed values of the first expansion function with only low-pass filtered luminance values,
A second look-up table that stores pre-computed values for the second expansion function with only luminance values,
The output of the second look-up table corresponding to the input luminance value of the expanding color corresponds to the input low-pass filtered luminance value associated with the input luminance value. It is a color expansion module having a processor configured to multiply the output of the first look-up table.

Such a color expansion module is particularly advantageous and inexpensive because it uses only two look-up tables of small size.

Desirably, this module will be incorporated into electronic devices such as television receivers, mobile devices, communication devices, game consoles, tablets, smartphones, laptops, cameras, coding chips and servers.

According to one general aspect, there is also provided a method of extending the dynamic range of low dynamic range content to prepare such content especially for display devices having high peak brightness. The color expansion method for expanding the color of an image is the color expansion method in which the color is represented in a color space in which the luminance component is separated from the luminance component, and the luminance component is subjected to a low-pass filter. It includes expanding the value range of the luminance component by scaling the low-pass filtered luminance component exponentially using an exponential map. The extended value of brightness is then asked to represent the extended color of the image in the new HDR version. Such enhanced colors may be used to reproduce images, especially on display devices that have higher peak brightness.

Desirably, the method
Applying a high-pass filter to the luminance component
Exponentially scaling the high-pass filtered luminance component using a detail map,
It further comprises weighting the scaled low-pass filtered luminance component with the scaled high-pass filtered luminance component.

The extended value of the brightness is then asked to represent the enhanced color of the image in the new HDR version, where the details have been enhanced to compensate for the loss of detail due to the low-pass filtering of the brightness component. ..

Desirably, the high-pass filter is applied by dividing the luminance component by the low-pass filtered luminance component.

Desirably, the exponential map is calculated according to the expansion function of the low-pass filtered luminance component, and the weighted low-pass filtered luminance component is
The low-pass filtered luminance scaled exponentially using the expansion function and further using the first detail function of the low-pass filtered luminance component only. It is the product of the luminance components, which are exponentially scaled using the second detail function of the luminance components only.

Such a definition of this weighted, scaled, low-pass filtered luminance component corresponds to equation (15) in the following main embodiments:

Advantageously, the first term of this product is only related to the low-pass filtered brightness and is not directly related to the brightness, and the second term of this product is It is then related to brightness and is not directly related to low-pass filtered brightness. Such an advantage makes it possible to calculate the weighted scaled low-pass filtered brightness component (ie, extended brightness) using only two separate one-dimensional LUTs. To do.

Desirably, the method
It further has to extend the value range of at least one chrominance component of the image by the chrominance extension factor.

In the first modification, the chrominance extension factor corresponds to the extended luminance component normalized by the luminance component. See equations (6) and (7) in the following main embodiments.

In the second variant, the chrominance extension factor is a function of the exponential map scaled by the extended luminance component normalized by the luminance component. See equations (8) and (9) in the main embodiments below.

In a third preferred variant, the chrominance extension factor is a function of the luminance component of the exponential map or the low-pass filtered, and the function is the low-pass filtered. It is scaled by the extended brightness component normalized by the brightness component. See equations (10) and (11) in the following main embodiments.

The object of the present invention is also a non-temporary storage medium carrying program code instructions that perform the steps of the color enhancement method described above when executed on a computing device.

The object of the present invention is also a color expansion method for expanding the color of an image, wherein the color is represented in a color space that separates a luminance component from a chrominance component.
Applying a low-pass filter to this luminance component
The luminance component was extended by calculating the product of the first function f (Ylow) of the low-pass filtered luminance component only and the second function g (Y) of the luminance only. It is a color expansion method that has to obtain a value.

According to other general embodiments, devices are provided that extend the dynamic range of low dynamic range content to prepare such content, especially for display devices that have high peak brightness. The device has a low-pass filter acting on at least one component in the image and a higher peak luminance value than the at least one image component prior to the low-pass filtering. With a display device that extends the value range of the at least one component in the image having a lower peak brightness.

According to other general embodiments, a second method of extending the value range of at least one component in the image is provided in order to reproduce the image on a display device having higher peak brightness. .. The method exponentially scales the low-pass filtered luminance component using an exponential map, and the scaled low-pass filtered luminance component of the luminance component. It includes weighting by the luminance component of an image that is exponentially scaled by a function.

According to other general embodiments, a third method of extending the value range of at least one component in the image is provided in order to reproduce the image on a display device having higher peak brightness. .. The method comprises the features of the first method, further comprising generating an exponential map calculated for each single pixel and calculating an extended map, wherein the extended map comprises said luminance. It is a function of the component and the low-pass filtered luminance component. The above-mentioned extension applies the exponential map exponentially to the low-pass filtered luminance component, and scales the extended value of the at least one component by the extension map. Have that.

According to other general embodiments, a second device is provided that extends the value range of at least one component in the image in order to reproduce the image on a display device having higher peak brightness. .. The device has a first look-up table that stores the values of the low-pass filtered luminance components that are exponentially scaled using an exponential map. The device further has a second look-up table that stores the values of the luminance components exponentially scaled by the scaling parameters. The device provides a first multiplier that combines the outputs of the first look-up table and the second look-up table, and an exponential map applied to the low-pass filtered luminance component. It further has a third look-up table to store and a second multiplier that combines the chromanance component with the output of the third look-up table. The device further comprises a third multiplier that combines the luminance component with the output of the first multiplier to produce an extended luminance component. The device further comprises fourth and fifth multipliers that act on each of the chrominance components and combine the output of the first multiplier with the output of the second multiplier to produce an extended chrominance component.

According to other general embodiments, a third device is provided that extends the value range of at least one component in the image in order to reproduce the image on a display device having higher peak brightness. .. The device has a first processor configured to generate an exponential map calculated for each single pixel. The apparatus further comprises a second processor configured to compute an extended map that is a function of the luminance component and the low-pass filtered luminance component, and uses the exponential map to perform an exponential function. It further comprises a third processor configured to extend the low-pass filtered luminance component and scale the result by the extended map.

Details of one or more practices will be described in the accompanying drawings and in the description below. Of course, implementations can be configured or embodied in various ways, even if they are described in one particular way. For example, the implementation may be performed as a method, or embodied as, for example, a device configured to perform a set of actions or a device such as a device that stores instructions to perform a set of actions. Alternatively, it can be embodied in a signal. Other aspects and features will become apparent from the accompanying drawings and the following detailed description examined in conjunction with the claims.

This principle can be better understood by following the example diagram below.

An embodiment of the embodiment of this principle is shown. An embodiment of the treatment of the luminance component and the two chrominance components is shown. Other embodiments of the treatment of the luminance component and the two chrominance components are shown. This is a modified version of the embodiment of FIG. This is a modified version of the embodiment of FIG. An embodiment of the method under this principle is shown. A second embodiment of the method under this principle is shown. An embodiment of a system including a color expansion device under this principle, including display of an expanded image, is shown. An embodiment of a color enhancer under this principle is shown with a focus on LUTs and multipliers.

As stated, it is known that each pixel of an image produces a brightness extension map associated with an extension value to be applied to the brightness of this pixel so as to enhance the bright local features in the image. In the simplest case, a clipped area in the image can be detected and then extended with a steeper extension curve, but such a solution does not provide sufficient control over the appearance of the image.

A more controllable luminance extension solution is provided in WO 2015/096955, which patent document discloses an image luminance extension based on the application of an exponent to image luminance, where the exponent is the image luminance. Obtained from the low-pass filtered version of. However, in this case, the exponential expansion function of luminance applies not to the image luminance itself, but to the low-pass filtered version of the luminance. The exponent applied to the brightness depends on the low-pass filtered version of the image brightness, so that the extension then has two variables: the brightness itself Y and the low-pass of the brightness. It is not a filtered version Y _low , but a function of only one variable (ie, a low pass filtered version of image brightness such that _{Yexp = f (Y low)).} In particular, because of this difference, the invention can advantageously be implemented with a small size look-up table, which makes it more suitable for hardware applications.

The embodiments described are intended for methods and devices that extend the dynamic range of low dynamic range content and, in particular, prepare such content for display devices that have high peak brightness.

An object of the present extension method is to extend the luminance channels Y of those colors in order to obtain the _{extended luminance Y exp for the colors of the image I.}

The extending (and weighting (see below)) luminance channel Y is such that by transforming the data representing the colors of the image into a color space that separates the chrominance from the luminance, such as the YUV color space. It can be obtained in a way that is known in itself.

A low-pass filtered version of luminance Y is first calculated to obtain an extended luminance Y _exp , which is expressed as _{Y low.} The pixel-by-pixel exponential map E can be calculated as a function of this low-pass filtered version of luminance Y. Moreover, the enhancement map _Yance can be sought as a function of a high-pass filtered version of luminance Y. This enhancement map _Yance is used as a weight applied to the intermediate extended brightness. Such a high-pass filtered version of the brightness Y preferably obtains this brightness Y in particular by comparing _{this brightness Y with the low-pass filtered version Y low.} It is determined by dividing by its low-pass filtered version Y _low. This enhancement map _{Yance encodes} grains and other details in the image and recovers the details lost by low path filtering to provide visibility of such details after intermediate expansion. Can be used to enhance.

The final extended brightness Y _exp applies the pixel-by-pixel exponential E to the low-pass filtered version Y _low to get an intermediate extended brightness, and finally E is the exponential map. by multiplying the _{Y enchance} the ^{_{Y exp = (Y low) E}} × Y the intermediate expanded luminance such that _enchance as being obtained from obtained.

The chromatic channels of the color of image I (eg, U and V) preferentially relate between _{Y and Y exp} so as to reconstruct all the color components of the expanded color image I _exp. According to, in particular, according to the ratio Y _exp / Y, U _exp and V _exp are obtained accordingly.

_{An inverse color space transformation is applied to inversely transform the luminance-chrominance representations Y exp} , U _exp and V _exp to the original color space of the image (eg RGB) so as to obtain the final expanded image.

As described below in relation to formula (15), in particular, when the index map E is calculated as an exponential function E (Y _low) of Y _low, and, in particular, be used to enhance the details If a high-pass filtered version of the brightness Y is _{obtained by dividing the brightness Y by Y low} , the extension is approximated through only two LUTs that simplify the process in the hardware application. obtain. This is described below as an alternative embodiment with reference to LUT1 and LUT2.

From this, further details are given with respect to the above-mentioned luminance expansion method.

To calculate the low-pass filter is multiplied by the version of the luminance Y _low, in one embodiment, a two-dimensional Gaussian filter is represented as Y _low, subjected to low-pass filter for luminance Used to calculate the version. The standard deviation σ of the Gaussian filter can be set to a value of 2, for example, while the kernel size is set to a small value, for example, 3x3, 3x5, 5x5, and so on.

In other embodiments, the low-pass filtered version of luminance may be calculated using a divisible Gaussian filter, whereby for example, the horizontal dimension of luminance Y is initially 1D Gaussian. -Filtered by a filter, then its output is filtered in the vertical dimension by another 1D Gaussian filter. Alternatively, the order may be rearranged so that vertical filtering is performed first, followed by horizontal filtering.

In one specific but unrestricted embodiment, the following process is used to calculate the exponential map E as _{an exponential function of Y low.} For a given pixel p, the exponent E is given by the following equation:

E = a (Y _low ) ² + bY _low + c = E (Y _low ) (1)

Here, the parameters a, b and c are calculated according to the maximum desired output luminance value of the expanded image _{we represent by L max.} For example, L _max can be set to a value between 500 nits and 2000 nits.

All pixel expansion index values E (Y _low ) are obtained from the pixel extension index map E (p) when they are combined with a pixel-by-pixel map with a _low -pass luminance value Y low.

The following formula is used to calculate the enhancement map _Yenchange:

Y _chance = (Y / Y _low ) ^d (2)

Here, the detailed parameter d can be set to, for example, d = 1.25. Y _chance is, in this case, a function of both the image luminance Y and the low pass version Y _low. The ratio (Y / Y _low ) corresponds to one method of applying a high-pass filter to the luminance Y, and high-pass filtering divides the luminance component by the low-pass filtered luminance component. Please note that it is acquired by. Other known methods of applying a high pass filter to the luminance Y may be used instead.

Note that the detail parameter d is pixel mappable to a so-called detail map and is therefore used to exponentially scale the high pass filtered luminance component.

Finally, the final extended brightness Y _exp is calculated as follows:

Y _exp = (Y _low ) ^{E (Ylow)} × Y _chance (3)

_{By substituting Change} in the above equation with the right side of equation (2), we can replace equation (3) with:

Y _exp = (Y _low ) ^{E (Ylow)} × (Y / Y _low ) ^d (4)

It can also be reformulated as. In this equation (4), the scaled low-pass filtered luminance component (Y _low ^{) E (Ylow)} is the exponentially scaled high-pass filtered luminance component (Y low). Y / Y _low ) weighted by ^d. Equation (4) can be written as:

Y _exp = (Y _low ) ^{(E (Ylow) -d)} x Y ^(d-1) x Y (5)

Note that equations (3), (4) and (5) are mathematically equivalent.

FIG. 1 shows an embodiment of an embodiment of this principle.

_{After obtaining the final extended luminance Y exp} according to equation (3-4-5), the color channels of the colors in the image are preferably expanded accordingly. In one embodiment where the luminance-chrominance representation used for a color is the YUV color space, the input color channels are given by U and V. In specific, non-restrictive embodiments, _{their scaled counterparts, represented by U exp} and V _exp , can be determined as follows:

U _exp = U (Y _exp / Y) (6)
V _exp = V (Y _exp / Y) (7)

In formulas (6) and (7), the chrominance component is extended using the chrominance extension coefficient corresponding to the extended luminance component normalized by the luminance component.

To increase image saturation, the color channels can be further scaled according to parameter _{E'depending on exponential map E and / or Y low.} In this case, the extended color channels U _exp and V _exp are calculated as follows, instead of using equation (6-7):

U _exp = U (Y _exp / Y) E'(E) or U _exp = U (Y _exp / Y) E'(Y _low ) (8)

V _exp = V (V _exp / Y) E'(E) or V _exp = V (Y _exp / Y) E'(Y _low ) (9)

The color channels U _exp and V _exp are finally recombinated with _{the extended luminance channel Y exp} to form the colors of the final HDR image.

FIG. 2A shows an embodiment of the above processing of the luminance component and the two chrominance components, which requires six multipliers. In FIG. 2 (b), a single multiplier is _{used to form (Y exp} / Y) E', then (Y _exp / Y) E'is used for each color channel with two additional multipliers. Since it is multiplied, it shows how reversing the order of the multipliers in the color channel can actually reserve one multiplier.

In other practices, the extended color channels U _exp and V _exp are calculated as follows, instead of using equation (8-9):

U _exp = U (Y _exp / Y _low ) E'(E), or U _exp = U (Y _exp / Y _low ) E'(Y _low ) (10)

V _exp = V (V _exp / Y _low ) E'(E), or V _exp = V (Y _exp / Y _low ) E'(Y _low ) (11)

In equations (6) and (7), the chrominance component is a function of the low-pass filtered brightness component or exponential map normalized by the extended brightness component normalized by the brightness component. It is extended using the chrominance extension factor.

FIG. 3 (a) is an inverted version of FIG. 2 (a) updated to comply with this practice. Similar to FIG. 2 (b), FIG. 3 (b) shows that a single multiplier is _{used to form (Y exp} / Y) E, followed by two additions of _{(Y exp / Y) E.} Since each color channel is multiplied by the multipliers, we show how reversing the order of the multipliers in the color channels can actually secure one multiplier.

This describes a specific embodiment, which advantageously allows the extended luminance Y _exp to be calculated from only two LUTs, especially when the detailed parameters change. To.

In equation (5), _{the detailed parameter d that exponentially scales Y low} on the one hand and Y on the other can be set to a constant, for example 1.25, as in equation (2), or It can vary with and / or with Y _low.

For example, especially _{when scaling Y low} exponentially, the detailed parameters _{can change according to the value of Y low} according to the first detailed function d1 as follows:

_{d1 = d1 max - (d1 max} -d1 min) × ((255-Y low) / 255)
= D1 (Y _low ) (12)

Here, in the first implementation, d1 _min = 1.3 and d1 _max = 1.5, and in the second implementation, d1 _min = 1.4 and d1 _max = 1.8.

Other implementations in which the function d1 is a Y _low only function can be envisioned without departing from the present invention.

It, in Formula (5), the details to be particularly applicable to _{Y low} is that depends on _{Y low.}

In equation (5), especially when Y is exponentially scaled, the detailed parameters can change according to the second detailed function d2 depending on the value of Y, for example:

d2 = d2 _max- (d2 _{max -d2 min} ) x ((255-Y) / 255)
= D2 (Y) (13)

Here, in one implementation, d2 _min = 1.3 and d2 _max = 1.7. That is, in equation (4), the details that apply specifically to Y depend on Y.

Other implementations in which d2 is a Y-only function can be envisioned without departing from the present invention.

Using the detail function d2 that depends on Y when the detail function d2 scales Y exponentially, and the detail function d1 that depends on Y _row when the detail function d1 _{scales Y low exponentially,} We formulate equations (1), (2) and (3):

Y _exp = (Y _low ) ^{E (Ylow)} × Y ^{d2 (Y)} × Y _{low −} ^{d1 (Ylow)} (14)

Can also be reformulated. is this:

Y _exp = (Y _low ) ^{(E (Ylow) -d1 (Ylow))} × Y ^{d2 (Y)} (15)

Or

Y _exp = (Y _low ) ^{(E (Ylow) -d1 (Ylow))} x Y ^{(d2 (Y) -1)} x Y (15')

Is the same as. Here, the function d1 () _{can follow equation (12) when applied to Y low} , and the function d2 () can follow equation (13) when applied to Y. Which in formula _(15), details that apply to _{Y low} is dependent only on the _{Y low,} is that details that apply to Y is dependent only on the Y.

In all the above equations, the detailed functions d1 and d2 do not change linearly as seen in the examples given earlier in equations (12) and (13), but in the Y or Y _low function. Note that it can change non-linearly.

As described below, according to the first detail function d1 () that depends on Y _low when applied to exponential acting on Y _low, and, depending on the Y when applied to exponential acting on Y Such implementation of the luminance enhancement method by the second detailed function d2 () (see equations (15) and (15') above) is to minimize the memory size required for the hardware. Is particularly advantageous for. In the implementations represented by FIGS. 2 (a) and 2 (b), the following functions of unique variables are used:
a) uniquely first expansion function _f which depends on the ^{_{Y low (Y low) = (}} Y low) (E (Ylow) -d1 (Ylow))
b) Second expansion function that uniquely depends on Y g (Y) = Y ^{(d2 (Y) -1)}

In other practices, such as those shown in FIGS. 3 (a) and 3 (b), the following functions of unique variables are used:
a) First expansion function that _{uniquely depends on Y low f'(Y low} ) = (Y _low ) ^{(E (Ylow) -d1 (Ylow) -1)}
b) Second expansion function g'(Y) = Y ^{d2 (Y) that is uniquely dependent on Y}

FIG. 5 shows an embodiment of Method 500 for expanding the luminance component of an image. The method begins with a start block 501 that supplies the luminance component Y and proceeds to block 510 that applies a low pass filter to the luminance component. The method also proceeds from the low-pass filtered luminance values supplied by block 510 to block 520, which produces an exponential map calculated pixel by pixel. Control proceeds from the luminance-supplying block 501 and the low-pass filtered luminance-supplying block 510 to the enhancement map-generating block 530. The enhancement map is a function of the luminance component and the low-pass filtered luminance component (see equation (2) above). The control also applies the exponential map from block 520 exponentially to the low-pass filtered luminance components from block 510 to extend the image component value range to block 510. And proceed from block 520. From block 540, control proceeds to block 550, which weights the extended image component value range with the enhancement map generated from block 530.

We will now detail how such implementation of the above luminance enhancement method can be advantageous in terms of hardware requirements.

In low-cost hardware applications, complex calculations of functions (such as exponential functions) are often replaced by look-up tables (LUTs). _{Here, thanks to the new formulation of the extended luminance Y exp} according to equation (15) or (15'), a simpler LUT architecture, in favor of the first expansion function, which depends only on _{Y low.} An extension method is implemented using a first LUT having an output value corresponding to a pre-calculated value and a second LUT having an output value corresponding to a pre-calculated value of a second expansion function that depends only on Y. Can be used for.

In such a hardware implementation detailed below, the first LUT is addressed by a _{Y low entry and the Y low is 8 so that each value of the Y low} (eg, a LUT with a depth size of 256 data) is obtained. Incorporate, for example, the calculation result of the first expansion function for 0 to 255) when encoded in bits:

[0. .. 255] Depending on the x integer in the range, f (x) = x ^{(E (x) -d1 (x))}

The second LUT, on the other hand, is addressed by the Y entry to each value of Y (eg, 0 to 255 when Y is encoded with 8 bits to result in a LUT with a depth size of 256 data). On the other hand, for example, the calculation result of the second expansion function is incorporated:

[0. .. 255] Depending on the x real number in the range, g (x) = x ^{(d2 (x) -1)}

Alternatively, note that both LUTs can be addressed with data larger than 8 bits (eg, n = 10 bits) (in which case Y and Y _low are [0.1203]. ] An integer in the range.). In this case, the input data Y and Y _low are used in the embedding function [0. .. 255] Scaled to fit range:

x ′ = x × 255 / (2 ⁿ -1)
[0. .. 1023] Depending on the x integer in the range, f (x) = x ^{(E (x') -d1 (x'))}
[0. .. 1023] Depending on the x integer in the range, g (x) = x ^{(d2 (x) -1)}

_{In the first embodiment of the present invention dedicated to hardware, two function blocks (Y low} ) ^E-d1 and Y ^d2-1 dedicated to luminance expansion described in FIGS. 2 (a) and 2 (b). , And the function block E'dedicated to the chromanance extension is replaced by the lookup tables LUT1, LUT2 and LUT3 in FIG. 7 (see below). In the second embodiment of the present invention dedicated to hardware, the three function blocks (Y _low ) ^E-d1-1 , Y ^d2 and E'described in FIGS. 3 (a) or 3 (b) are Similarly, it is replaced by three lookup tables. Since memory size is also an important criterion in choosing a hardware architecture, there is _{only one variable (Y low} and Y, respectively) compared to one function of the two _{variables Y low and Y in the prior art.} The above-mentioned brightness enhancement method using two separate functions f (Y _low ) and g (Y) of is particularly well adapted to hardware applications in which this function is replaced by a LUT. In particular, this requires _{a LUT as compared to a single LUT that has a Y low} entry and a Y entry ( _{64k entries in the LUT if both Y low} and Y are 8-bit data). This is because the memory size to be generated can be dramatically reduced ( _{256 entries in the LUT1 when Y low} is 8 bits + 256 entries in the LUT2 when Y is an 8-bit entry).

Color enhancement and detailed recovery / enhancement at the time of low-pass filtering, as shown in FIGS. 2 (a), 2 (b), 3 (a) and 3 (b) and especially in FIG. Only 3 LUTs and 6 or 7 multipliers are needed to implement. The low-pass filtering step itself is known in hardware implementation and can be implemented in a manner known in itself. With reference to those figures, the three LUTs are, for example:
a) The first LUT, i.e., LUT1, supplies the value for the first expansion function with an entry based on only _{one variable, Y low.}
b) The second LUT, i.e., LUT2, supplies the value for the second expansion function with an entry based on only one variable Y.
c) The third LUT, ie LUT3, is used for chrominance expansion and supplies E'when addressed by _{Y low.}

In common with the advantages of this approach over previous methods, the invention has been optimized to facilitate implementation in the HW and to use three independent small LUTs. It is a 2D separable symmetric filter with reasonable apertures, some (only 3 in Figures 2 and 3) look-up tables, and some (only 5 in Figures 2 (b) and 3 (b)). ) Multiplier can be used.

As before, the main realization factor that facilitates HW implementation is that the dynamic expansion _{retains the same extension factor E and the same low-pass filtered luminance signal Y low} , International Publication No. 2015/096955. It is a point that applies to "filtered brightness", not to the brightness itself, as seen in the issue. Thus, instead of having a function of two different variables (luminance and filtered brightness) for brightness extension, the present invention has one LUT with a small size by having only one variable entry. Uses two functions with only one variable, each implemented by.

To maintain the same quality after expansion, the sharpening of details has also been modified to restore detail to the expanded filtered brightness. This change also made it possible to reduce the number of operations to calculate the final extended brightness.

FIG. 4 shows an embodiment of Method 400 that implements this principle. The method starts at the start block 401 and proceeds to block 410 for low-pass filtering of the luminance component of the image. The control reproduces the image from block 410 on a display device that has a higher peak brightness by using a low pass filtered brightness to produce the extended color of the HDR image. To block 420, which extends the color of at least one image component.

FIG. 6 shows an embodiment of a system 600 that reproduces an image on a display that has a higher peak luminance value than the image. The system has a low pass filter 610 that acts on the image luminance component. The output of the low-pass filter 610 is signal-connected to the input of the processor 620. Processor 620 extends the colors of the image. The output of processor 620 is signal-connected to the input of display 630. The display 630 displays the extended colors of the image.

FIG. 7 shows an embodiment of a color expansion device 700 that expands the color of an image in order to reproduce the expanded image on a display device having higher peak brightness. The device stores a low-pass filter (not shown, but similar to reference numeral 610 in FIG. 6) and a first look that stores the pre-computed values of the first expansion function as described above. A low-pass filter is applied to the up table (LUT1) 710 and the second look-up table (LUT2) 720 that stores the precomputed values of the second expansion function as described above. A third look-up table (LUT3) 730 that stores a pre-computed value of the chrominance extension coefficient defined as a function E'(Y _{low) of the determined brightness value, and in particular as defined below.} It has a processor (not shown, but similar to reference numeral 620 in FIG. 6) configured to implement the multipliers 740, 750, 760, 770 and 780.

The input to the LUT 710 is a low-pass filtered luminance component from the output of a low-pass filter (not shown). The LUT 710 has a function f (Y _low ) = (Y _low ) ^(E (Y low) −d1 (Y low)), that is, an exponential function E (Y _low ) and a low-pass filtered luminance component. Stores the values of the low-pass filtered luminance components that are exponentially scaled using the first detail function d1 that depends only on them. The output of the LUT 710 is signal-connected to one input of the first multiplier 740. Other inputs to the multiplier 740 come from the output of the second look-up table (LUT2) 720. The input to the LUT 720 is the luminance component of the image. The LUT 720 stores the value of the function g (Y) = Y ^{(d2 (Y) -1)} , i.e., the luminance component exponentially scaled by the second detail function d2, which depends only on the luminance component. .. The output of the multiplier 740 is sent to the first input of the other multiplier 760. The second input of the multiplier 760 is the luminance component of the image. The output of the multiplier 760 is an extended value Y _exp of the luminance component of the image.

A third look-up table (LUT3) 730 further possessed by the apparatus 700 takes a low-pass filtered luminance component as its input. The LUT 730 stores an E'(E) value, i.e., a value corresponding to another exponential function E'(E) adapted to scale the chromatic component of the color. The output of the LUT 730 is sent to one input of the multiplier 750. The second input of the multiplier 750 is the output of the multiplier 740. The output of multiplier 750 is then sent to multiplier 770 and multiplier 780. The respective chrominance components U and V are sent to the multiplier 770 and the multiplier 780, respectively. The chrominance component can be multiplexed over time so that only one multiplier is needed to apply the exponential map to both chrominance components. The outputs of the multipliers 770 and 780 are, for example, the extended chrominance components U _exp and V _exp .

This specification describes this principle. Thus, as will be apparent, one of ordinary skill in the art can embody this Principle, even if not explicitly stated or illustrated herein, thereby devising various arrangements contained in this Principle. Let's do it.

As will be apparent to those skilled in the art, the block diagram presented herein represents a conceptual diagram of a circuit configuration that is an example of embodying this principle. Similarly, as is apparent, any flow chart, flow diagram, state transition diagram, pseudo-code, and the like are substantially represented in a computer-readable medium and are therefore such by a computer or processor. Represents various processes that can be performed, whether the computer or processor is explicitly indicated.

The functionality of the various elements shown in the figure may be provided through the use of dedicated hardware and hardware that can run the software in connection with the appropriate software. When provided by a processor, functionality can be provided by a single dedicated processor, by a single shared processor, or by multiple individual processors that may be partially shared. Moreover, the explicit use of the term "processor" or "controller" should not be understood as referring exclusively to the hardware capable of running the software, and without limitation, the digital signal processor ("DSP"). Read-only memory (“ROM”) for storing hardware and software, random access memory (“RAM”), and non-volatile storage can be implicitly included.

Other conventional and / or custom hardware may also be included. Similarly, any switch shown in the figure is merely conceptual. These functions can be executed through the operation of the program logic, through the dedicated logic, through the interaction between the program control and the dedicated logic, or manually, so that the specific technique can be understood more concretely from the context. It can be selected by the practitioner.

Within the scope of the patent claims of the present application, any element represented as a means for performing a particular function, eg, a) a combination of circuit elements performing that function, or b) in any form and thus firmware. , Microcode or the like, and is intended to include any method of performing a function, including said software combined with an appropriate circuit configuration that performs the software to perform the function. The principle defined by such claims is that the functionality provided by various listed means is combined and united in the manner required by the claims. It exists in fact. Therefore, any means capable of providing those functionalitys is considered equivalent to that set forth herein.

References herein to "one embodiment" or "embodiment" and other variations of this Principle include specific features, structures, properties, etc. described in connection with the embodiment. It means that it is included in at least one embodiment of the principle. Thus, the appearance of the phrase "in one embodiment" or "in an embodiment" and some other variation appearing in various places throughout the specification does not necessarily all refer to the same embodiment. is not it.

Those and other features and advantages of this principle can be readily identified by those skilled in the art based on the teachings herein. It should be understood that the teachings of this principle can be implemented in various forms of hardware, software, firmware, dedicated processors, or combinations thereof.

Most preferably, the teachings of this principle are implemented as a combination of hardware and software. In addition, the software can be implemented as a tangibly embodied application program in the program storage unit. The application program can be uploaded to and executed by a machine with any suitable architecture. Desirably, the machine is a computer with hardware such as one or more central processing units (“CPU”), random access memory (“RAM”), and input / output (“I / O”) interfaces. Implemented on the platform. The computer platform can further include an operating system and microinstructions. The various processes and functions described herein can be either parts of the microinstruction code, parts of the application program, or any combination thereof, and can be performed by the CPU. Moreover, various other peripherals, such as additional data storage and printing devices, can be connected to the computer platform.

Since some of the component system components and methods shown in the accompanying drawings are preferably implemented in software, the actual connection between system components or process functional blocks is based on this principle. It should be further understood that it can vary depending on how it is programmed. In view of the teachings herein, one of ordinary skill in the art will be able to anticipate those and similar practices or structures of this principle.

Although exemplary embodiments have been described herein with reference to the accompanying drawings, this principle is not limited to those exact embodiments, and various modifications and improvements are within the scope of this principle. It should be understood that what can be achieved in them by those skilled in the art without departing from. All such changes and improvements are intended to be included within the scope of this Principle set forth in the appended claims.
Some or all of the above embodiments may be described, but not limited to:
(Appendix 1)
In a color expansion module (700) that expands the colors of an image, the colors are represented in a color space that separates the luminance component from the chrominance component.
A low-pass filter that applies a low-pass filter to the luminance component,
A first look-up table (710) that stores the pre-computed values of the first expansion function with only low-pass filtered luminance values, and
A second look-up table (720) that stores pre-computed values of the second expansion function with only luminance values, and
The output of the second look-up table corresponding to the input luminance value of the expanding color is the first corresponding to the input low-pass filtered luminance value associated with the input luminance value. With processors (740,760) configured to multiply the output of the look-up table
Color expansion module with.
(Appendix 2)
The first expansion function is defined as a low-pass filtered brightness component exponentially scaled using an exponential function and a first detail function d1 (Y _low ), and is defined as an exponential function and a first detail function. Both are themselves functions of low-pass filtered brightness components only,
The color expansion module according to Appendix 1.
(Appendix 3)
The second expansion function is defined as an exponentially scaled luminance component using the second detail function d2 (Y), which itself is a function of the luminance component only.
The color expansion module according to Appendix 1 or 2.
(Appendix 4)
An electronic device incorporating the color expansion module according to any one of Supplementary note 1 to 3.
(Appendix 5)
Select from a group of TV receivers, mobile devices, communication devices, game consoles, tablets, smartphones, laptops, cameras, coding chips, and servers.
The electronic device according to Appendix 4.
(Appendix 6)
In a color expansion method that expands the color of an image, the color is represented in a color space that separates the luminance component from the chrominance component.
Applying a low-pass filter to the luminance component and
Expanding the range of luminance components by scaling the low-pass filtered luminance component exponentially using an exponential map.
Color expansion method with.
(Appendix 7)
Applying a high-pass filter to the luminance component and
Exponentially scaling the high-pass filtered luminance component using a detail map,
Weighting the scaled low-pass filtered luminance component with the scaled high-pass filtered luminance component
Further have,
The color expansion method according to Appendix 6.
(Appendix 8)
High-pass filtering is obtained by dividing the luminance component by the low-pass filtered luminance component.
The color expansion method described in Appendix 7.
(Appendix 9)
The exponential map is calculated according to the low-pass filtered luminance component expansion function.
The weighted low-pass filtered luminance component (Y _exp ) is
_{The low emissivity} scaled exponentially using the expansion function E (Y low) and further using the first detail function d1 (Y _{low) containing only the low-pass filtered luminance components.} Path-filtered luminance components (Y _low ^{(E (Ylow) -d (Ylow))} ) and
^{Of the luminance component (Y d (Y)} ) exponentially scaled using the second detailed function d2 (Y) of the luminance component only
Product,
The color expansion method according to any one of Appendix 6 to 8.
(Appendix 10)
Extending the value range of at least one chrominance component of the image by the chrominance extension factor.
Have more
The color expansion method according to any one of Appendix 6 to 9.
(Appendix 11)
The chrominance extension factor corresponds to the extended brightness component normalized by the brightness component.
The color expansion method according to Appendix 10.
(Appendix 12)
The chrominance extension factor is a function of the exponential map scaled by the extended brightness component normalized by the brightness component.
The color expansion method according to Appendix 10.
(Appendix 13)
The chrominance extension factor is a function of the exponential map or low-pass filtered luminance component, and the function is by the extended luminance component normalized by the low-pass filtered luminance component. Scaled,
The color expansion method according to Appendix 10.
(Appendix 14)
A non-temporary storage medium that carries program code instructions that perform the steps of the color expansion method according to any one of Appendix 6 to 13 when executed on a computing device.

Claims (9)

A color dynamic range extension modules that extend the color dynamic range of the image, the color is represented in a color space which separates a luminance component from the chrominance components, in the color dynamic range extension,
A low-pass filter that applies a low-pass filter to the luminance component,
A first look-up table that stores the pre-computed values of the first expansion function for only low-pass filtered luminance values.
A second look-up table that stores the pre-computed values of the second expansion function for only the luminance values,
A third lookup table that stores the precomputed values of the third expansion function for only the chrominance values, and
It ’s a processor,
The low-pass filtered input luminance value is fed to the first look-up table, the input luminance value is fed to the second look-up table, and the input chrominance value is fed to the third look-up table. Supply to the up table,
The extended chrominance is obtained by multiplying the output of the first look-up table by the output of the second look-up table and multiplying the result of this multiplication by the luminance component.
The output of the first look-up table is multiplied by the output of the second look-up table, the result of this multiplication is multiplied by the output of the third look-up table, and the result of this multiplication is multiplied by the chrominance component. With a processor that is configured to get extended chrominance by multiplying by
Color dynamic range expansion module with . The first expansion function is defined as the low-pass filtered brightness component exponentially scaled using the exponential function and the first detail function, and is defined as the exponential function and the first detail function. The color dynamic range expansion module according to claim 1, wherein both are themselves functions of only the low-pass filtered brightness component. The color dynamic according to claim 1 or 2, wherein the second expansion function is defined as the luminance component that is exponentially scaled using a second detail function that is itself a function of only the luminance component. Range expansion module. An electronic device incorporating the color dynamic range expansion module according to any one of claims 1 to 3. The electronic device according to claim 4, which is selected from a group consisting of a television receiver, a mobile device, a communication device, a game machine, a tablet, a smartphone, a laptop, a camera, a coding chip, and a server. A color expansion method for expanding the dynamic range of colors in an image, wherein the color is represented in a color space that separates a luminance component from a chrominance component.
The low-pass filtered input luminance value is fed to the first look-up table, the input luminance value is fed to the second look-up table, and the input chrominance value is fed to the third look-up table. To supply to
By multiplying the output of the first look-up table by the output of the second look-up table and multiplying the result of this multiplication by the luminance component, the extended chrominance can be obtained.
Multiply the output of the first look-up table by the output of the second look-up table, multiply the result of this multiplication by the output of the third look-up table, and multiply the result of this multiplication by the chrominance component. To get extended chrominance by multiplying, including,
The first look-up table stores the pre-computed values of the first expansion function for only the low-pass filtered luminance values, and the second look-up table contains only the luminance values. A color expansion method that stores the pre-computed values of the second expansion function and the third look-up table stores the pre-computed values of the third expansion function only for the chromanance values . The first expansion function is defined as a low-pass filtered, exponentially scaled brightness component using an exponential function and a first detail function, the exponential function and the first detail function. Is the color expansion method according to claim 6, wherein is itself a function of only the low-pass filtered brightness components. The second expansion function is defined as an exponentially scaled luminance component using a second detail function, and the second detail function is itself a function of only the luminance component. The color expansion method according to 6 or 7. A non-temporary storage medium that carries program code instructions that perform the steps of the color expansion method according to any one of claims 6 to 8 when executed on a computing device.

Images