JP3356337B2 - Image processing apparatus and image processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[Table of Contents] The present invention will be described in the following order. Industrial Application Conventional Technology (FIGS. 12 and 13) Problems to be Solved by the Invention Means for Solving the Problems (FIGS. 1 to 11) Operation (FIGS. 1 and 11) Examples (FIGS. 1 to 11) (FIG. 11) (1) Principle of hierarchical encoding (FIGS. 1 to 3) (2) Overall configuration (FIG. 4) (3) Hierarchical encoding encoder unit 40A (FIGS. 5 and 6) (3-1) Block configuration (FIGS. 5 and 6) (3-2) Processing (4) Generated information amount control unit 40B (FIGS. 7 to 9) (4-1) Block configuration (FIG. 7) (4-2) Frequency distribution table (FIG. 8) (4-3) Processing (FIG. 9) (5) Data Configuration of Transmission Block (FIG. 10) (6) Another Embodiment (FIG. 11) Effect of the Invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for dividing predetermined image data into a plurality of image data having different resolutions. It is something.

[0003]

2. Description of the Related Art Conventionally, as this type of image coding apparatus,
There is a method in which input image data is hierarchically encoded using a hierarchical encoding technique such as pyramid encoding. In this hierarchical encoding device, high-resolution input image data
Are sequentially and recursively formed as second hierarchical data having lower resolution than the first hierarchical data, third hierarchical data having lower resolution than the second resolution data, and so on. The plurality of hierarchical data are transmitted through a transmission path including a communication path and a recording / reproducing path.

At this time, in an image decoding apparatus for decoding a plurality of hierarchical data, all the plurality of hierarchical data may be decoded, or any one of the hierarchical data may be decoded according to the resolution of a television monitor corresponding to each of the hierarchical data. May be selected and decoded. By decoding only desired hierarchical data from a plurality of hierarchical data hierarchized in this way, desired image data can be obtained with a minimum necessary transmission data amount.

[0005] As shown in FIG. 12, in the image coding apparatus 1 that realizes, for example, four-layer coding as the layer coding, three-stage thinning filters 2, 3, and 4 are respectively provided.
And interpolation filters 5, 6, 7 and input image data D
1, the reduced image data D2, D3, and D4 having lower resolution are sequentially formed by the thinning filters 2, 3, and 4 at each stage, and the reduced image data D2, D3, and D4 are reduced by the interpolation filters 5, 6, and 7 before the reduction. To the resolution of.

The outputs D2 to D4 of the respective thinning filters 2 to 4
The outputs D5 to D7 of the interpolation filters 5 to 7 are input to difference circuits 8, 9, and 10, respectively, whereby difference data D8, D9, and D10 are generated. As a result, in the image encoding device 1, the amount of hierarchical data is reduced and the signal power is reduced. Here, this difference data D
8 to D10 and reduced image data D4 have sizes of 1, 1/4, 1/16, and 1/64, respectively, with respect to the input image data D1.

The difference data D8 to D10 obtained from the respective difference circuits 8 to 10 and the reduced image data D4 obtained from the thinning-out filter 4 are output to the encoders 11, 12, 13,
14, and as a result, each encoder 11,
First, second, and third different resolutions from 12, 13, and 14
And fourth hierarchical data D11, D12, D13 and D1
4 are transmitted to the transmission line in a predetermined order.

[0008] The first to fourth hierarchical data D11 to D14 transmitted in this manner are decoded by the image decoding apparatus shown in FIG. That is, the first to fourth hierarchical data D11 to D14 are respectively supplied to the decoders 21, 22, 2 and
3 and 24. As a result, first, the decoder 24
Outputs the fourth hierarchical data D24.

The output of the decoder 23 is supplied to a fourth hierarchical data D obtained from the interpolation filter 26 in an adder circuit 29.
Thus, the third hierarchical data D23 is restored. Similarly, the output of the decoder 22 is added to the interpolation data of the third hierarchical data D23 obtained from the interpolation filter 27 in the adding circuit 30, whereby the second hierarchical data D22 is restored. Further, the output of the decoder 21 is added to the interpolation filter 2 in the addition circuit 31.
8 is added to the interpolated data of the second hierarchical data D22, whereby the first hierarchical data D21 is restored.

[0010]

However, in an image coding apparatus for realizing such a hierarchical coding method, since input image data is divided into a plurality of hierarchical data and coded, the data amount is inevitably limited to hierarchical components. Therefore, there is a problem that the compression ratio is reduced as compared with a high efficiency coding method that does not use hierarchical coding. Further, when an attempt is made to improve the compression efficiency, there is a problem that the image quality deteriorates due to the quantizer applied between the respective hierarchical data. As an even more important problem, there is a problem that the data structure when transmitting each layer of data obtained by such a layer encoding method is not considered at all for transmission errors.

The present invention has been made in view of the above points, and proposes an image processing apparatus and an image processing method that can provide robustness against transmission errors while solving the conventional problems at once. What you want to do.

[0012]

According to the present invention, in order to solve the above-mentioned problems, a plurality of image data consisting of the highest hierarchical information D35 having the lowest resolution and the lowest hierarchical information D31 having the highest resolution are transmitted. Processing apparatus for generating image transmission data for use in an image processing apparatus, as the layer information D31 to D34 excluding the highest layer information D35, the inter-layer difference data D41 to D44, which are the differences from the layer information of the adjacent upper layer, are obtained. Difference data generation means 4
1, 43, 45, 47, and layer information D41 of a predetermined layer
Determination means 51 for determining the block activity P for each block comprising a plurality of pixels
When the block activity P is less than a predetermined threshold value TH1 to TH4 for each block of the hierarchy information D41 to D44 of the predetermined hierarchy except the top hierarchy information D35, the transmission of the hierarchy information D41 to D44 of the block is performed. Setting means 51 for setting a flag indicating that no transmission is performed, and control means 51 for controlling the presence or absence of transmission or non-transmission for each block of each layer information D41 to D44 based on the flag.
And the uppermost layer information D35 and the flag
An image transmission data generating means 80 for generating image transmission data that transmits the inter-layer difference data D41 to D44 as variable length data D51 to D54 after being transmitted as 1, D55 and C2.

According to the present invention, there is provided an image processing method for generating image transmission data for transmitting a plurality of image data consisting of the lowest hierarchical information D31 having the highest resolution and the lowest hierarchical information D31 having the highest resolution. , Hierarchical information D31 to D3 excluding the highest hierarchical information D35
4, the first step of obtaining inter-layer difference data D41 to D44, which is a difference from the layer information of the adjacent upper layer, and the layer information D41 to D44 of a predetermined layer,
A second step of determining the block activity P for each block composed of a plurality of pixels, and the block activity P is less than a predetermined threshold value TH1 for each block of the hierarchy information D41 to D44 of a predetermined hierarchy except the top hierarchy information D35. In the case of .about.TH4, a third step for setting a flag indicating that the layer information D41 to D44 of the block is not transmitted, and transmission or non-transmission for each block of the layer information D41 to D44 based on the flag. A fourth step for controlling the presence or absence of transmission;
35 and a fifth step of generating image transmission data such that the inter-layer difference data D41 to D44 are transmitted as variable length data D51 to D54 after transmitting the flags as fixed length data C1, D55 and C2. did.

Further, according to the present invention, an image for generating image transmission data for transmitting image data composed of a plurality of pieces of hierarchical information including the highest hierarchical information D35 having the lowest resolution and the lowest hierarchical information D31 having the highest resolution is provided. In the processing device, the hierarchy information D3 excluding the highest hierarchy information D35
Inter-layer difference data generating means 41, 43, 45, and 47 for obtaining inter-layer difference data D41 to D44, which are differences from adjacent upper layer information, as layer information 1 to D34, and predetermined layer information D41 to D44. , The block activity P for each block consisting of a plurality of pixels.
Means 51 to 54 for determining the highest hierarchical information D3
For each block of the hierarchy information D41 to D44 of the predetermined hierarchy except for the block 5, when the block activity P is equal to or more than the predetermined threshold value TH1 to TH4, the block hierarchy information D41 is obtained.
To D44, and when the block activity P is less than the thresholds TH1 to TH4, the block hierarchy information D
Control means 5 for controlling not to transmit 41 to D44
1 and the uppermost hierarchical information D35 are fixed length data C1, D5.
5, after being transmitted as C2,
An image transmission data generating means 80 for generating image transmission data for transmitting D44 as variable length data D51 to D54 is provided.

Further, in the present invention, an image for generating image transmission data for transmitting image data consisting of a plurality of pieces of hierarchical information including the highest hierarchical information D35 having the lowest resolution to the lowest hierarchical information D31 having the highest resolution is provided. In the processing method, the hierarchy information D3 excluding the highest hierarchy information D35
First to obtain inter-layer difference data D41 to D44, which are differences from the layer information of the adjacent upper layer, as 1 to D34.
, The second step of determining the block activity P for each block composed of a plurality of pixels with respect to the predetermined layer information D41 to D44, and the predetermined layer information other than the highest layer information D35. D41-
For each block of D44, the block hierarchy information D41 to D44 is transmitted when the block activity P is equal to or more than the predetermined threshold value TH1 to TH4, and when the block activity P is less than the threshold value TH1 to TH4, the block hierarchy information D41 is transmitted. To D44, and transmission of the uppermost layer information D35 as fixed-length data C1, D55, and C2, and then inter-layer difference data D41 to D44 to variable-length data D51 to D44.
A fourth step for generating image transmission data to be transmitted as 54 is provided.

[0016]

According to the first and second aspects of the present invention, when the block activity P is less than a predetermined threshold value TH1 to TH4 for each block of hierarchy information D41 to D44 of a predetermined hierarchy except for the highest hierarchy information D35, A flag indicating that the block hierarchy information D41 to D44 is not transmitted is set, and each layer information D41 to D44 is set based on the flag.
By controlling whether transmission or non-transmission is performed for each block of D44, the amount of transmission information can be reduced.

After transmitting the uppermost layer information D35 and the flag as fixed-length data C1, D55 and C2, the inter-layer difference data D41 to D44 are converted to variable-length data D51 to D44.
By generating image transmission data to be transmitted as data 54, when decoding, the variable length data D54
~ D51, even if an error occurs, the fixed length data C
1, D55 and C2 can be accurately detected. Further, the accessibility to desired image data can be improved based on the fixed length data C1, D55, and C2.

On the other hand, in the third and fourth inventions, the hierarchy information D41 of a predetermined hierarchy other than the highest hierarchy information D35
The block hierarchy information D41 to D44 is transmitted when the block activity P is equal to or greater than the predetermined threshold value TH1 to TH4, and the block hierarchy information is set when the block activity P is less than the threshold value TH1 to TH4. By controlling not to transmit D41 to D44, the amount of transmission information can be reduced. Similarly to the first and second inventions, an image in which the highest-layer information D35 is transmitted as fixed-length data C1, D55, and C2, and then the difference data D41 to D44 between layers is transmitted as variable-length data D51 to D54. By generating transmission data, when decoding
Even when an error occurs in the variable length data D54 to D51, the contents of the fixed length data C1, D55, and C2 can be accurately detected. Further, fixed length data C1, D5
5. Accessibility to desired image data can be improved based on C2.

[0019]

BRIEF DESCRIPTION OF THE DRAWINGS FIG.

(1) Principle of Hierarchical Coding FIG. 1 shows, as a whole, the principle of hierarchical coding according to the present invention, in which a still image such as a high-definition television signal is hierarchically coded and compressed. In this hierarchical coding, upper hierarchical data is formed by a simple arithmetic average of lower hierarchical data, and lower hierarchical data to be transmitted is reduced, thereby realizing a hierarchical structure without increasing the amount of information. For decoding from the upper layer to the lower layer, the division is adaptively controlled based on the activity for each block, thereby reducing the amount of information in the flat portion. Further, in the encoding of the differential signal performed for the lower layer, the quantization characteristic is switched for each block without an additional code based on the activity of the upper layer, thereby realizing high efficiency.

That is, in the hierarchical structure of the hierarchical coding,
First, the input high-definition television signal is defined as a lower layer. For four pixels X1 to X4 in a small block of 2 lines × 2 pixels of the lower layer, the following equation is used.

[Mathematical formula-see original document] m = (X1 + X2 + X3 + X4) / 4 An arithmetic average represented by the following equation (1) is obtained, and the value m is set as the value of the upper hierarchy. In this lower hierarchy,

.Times..times..times..times..times..times..times..times..times..times..tim- es..times..times..times..times..times..times..times..times..times..tim- es..times..times..times..times..times. A hierarchical structure is constructed by the amount of information.

On the other hand, when decoding the lower layer, three pixels X1
~ X3 is the following formula

[Equation 3] E [Xi] = ΔXi + m (where i = 1 to 3) As represented by (3), each difference value Δ
Xi is added to obtain a decoded value E [Xi], and the remaining one pixel is expressed by the following equation.

[Equation 4] E [X4] = m × 4-E [X1] -E [X2] -E [X3]... As shown by (4), three decodings of the lower layer from the average value m of the upper layer The decoded value E [X4] is determined by subtracting the value. Here, E [] means a decoded value.

Here, in the hierarchical coding, the resolution is quadrupled from the upper layer to the lower layer for each layer, but in a flat portion, the division is prohibited to reduce the redundancy. A flag for instructing the presence / absence of the division is prepared in units of 1 bit and block. The determination of the necessity of division in the lower hierarchy is determined as local activity, for example, by the maximum value of difference data.

Here, as an example of hierarchical coding, HD of ITE
FIG. 2 shows a result of adaptive division in the case of performing 5-layer encoding using a standard image (Y signal). The ratio of the number of pixels in each layer when the threshold value for the maximum difference data is changed to the original number of pixels is shown. It can be seen that redundancy is reduced based on spatial correlation. The reduction efficiency varies depending on the image, but when the threshold value for the maximum difference data is changed from 1 to 6, the average reduction rate becomes 28 to 69%.

In practice, the resolution of the upper layer is increased by a factor of 4 to form the upper layer. At that time, by encoding the difference data from the upper layer data in the lower layer, the signal level width can be effectively reduced. FIG. 3 shows the case of five layers by the layer coding described above with reference to FIG. 2. Here, the layers are named as the first to fifth layers counting from the lower layer.

Compared to the original 8-bit PCM data,
The signal level width is reduced. Especially the first with a large number of pixels
Since the fourth to fourth layers are differential signals, a significant reduction can be achieved, and the subsequent quantization improves the efficiency. As can be seen from the table of FIG. 3, the dependence of the reduction efficiency on the pattern is small and is effective for all pictures.

Further, by forming the upper layer by the average value of the lower layer, the lower layer is converted into a difference from the average value of the upper layer while the error propagation is stopped in the block, thereby achieving high efficiency. Can be. In practice, in hierarchical coding, there is a correlation between the activities between layers at the same spatial position, and by determining the quantization characteristics of the lower layer from the quantization result of the upper layer, the receiving side performs inverse quantization for inverse quantization. An adaptive quantizer that does not need to transmit quantization information (except for the initial value) can be realized.

In practice, an image is hierarchically coded based on the above-described five-stage hierarchical structure, is represented by multi-resolution, and is subjected to adaptive division and adaptive quantization using the hierarchical structure.
Approximately 1 HD standard image (8-bit Y / PB / PR)
/ 8. The additional code for each block prepared for adaptive division is subjected to run-length encoding in each layer in order to improve compression efficiency. In this way, an image with sufficient image quality can be obtained at each layer, and a good image with no visual deterioration can be obtained at the final lowest layer.

(2) Overall Configuration In FIG. 4, reference numeral 40 denotes an image coding apparatus according to the present invention. The layer coding encoder section 40A and the layer coding encoder section 4A which layer-code the input image data D1 and output the same.
A generated information amount control unit 40B controls the generated information amount at 0A to achieve a target value.

The image encoding device 40 can set a threshold value for each layer so that encoding can be performed in consideration of human visual characteristics. As a result, the block division processing is selected for the image data of the upper layer that affects the large area, while the division processing of the block is not selected for the image data of the lower layer that affects only the small area. Division processing according to the hierarchy can be realized.

Here, the hierarchical coding encoder section 40A is constituted by a memory for data delay (not shown) and an encoder. The memory is provided in the input stage so that the data can be delayed so that the encoding process is not executed until the generated information amount control unit 40B determines the optimum control value.

On the other hand, the generated information amount control unit 40B receives the input image data and determines a threshold value TH suitable for the data to be processed, and the hierarchical encoding encoder unit 40A efficiently codes the input image data. The optimal control value determined to be converted is transmitted to the encoder. This is a so-called feedforward type buffering configuration. With this configuration, it is possible to accurately control the amount of generated information and to eliminate the time delay caused by the feedforward type buffering.

(3) Hierarchical encoding encoder section 40A (3-1) Block configuration The hierarchical encoding encoder section 40A has the configuration shown in FIG. 5. In this example, processing is performed in five layers. First, the input image data D31 is input to the first difference circuit 41 and the first averaging circuit 42. The first averaging circuit 42 calculates the second layer data D3 by averaging four pixels of the input image data D31 (that is, the first layer data (lowest layer data)).
Generate 2. In the case of this embodiment, the first averaging circuit 4
2 generates a pixel X1 (2) of the second layer data D2 from the four pixels X1 (1) to X4 (1) of the input image data D31 as shown in FIGS. 6 (D) and 6 (E).

Similarly, the pixels X2 (2) to X4 (2) adjacent to the pixel X1 (2) of the second hierarchy data D32 also have the first hierarchy data D3.
It is generated by an average of 31 four pixels. Second layer data D
32 is input to a second difference circuit 43 and a second averaging circuit 44, and the second averaging circuit 44
Third-layer data D33 is generated by averaging 32 4-pixels. For example, pixels X1 (2) to X4 (2) of the second layer data D32 shown in FIGS.
In addition to the pixel X1 (3) of the third layer, the pixels X2 (3) to X4 (3) adjacent to the pixel X1 (3) are similarly stored in the second hierarchical data D32.
Is generated by averaging four pixels.

The third hierarchical data D33 is stored in the third difference circuit 4
5 and the third averaging circuit 46. The third averaging circuit 46 averages four pixels of the third hierarchical data D33 as shown in FIGS. 6B and 6C in the same manner as described above. ,
The fourth hierarchical data D34 including the pixels X1 (4) to X4 (4) is generated. The fourth hierarchical data D44 is input to the fourth difference circuit 47 and the fourth averaging circuit 48,
8 generates fifth layer data D35 which is the highest layer by averaging four pixels of the fourth layer data D34. That is, as shown in FIGS. 6A and 6B, the fourth hierarchical data D
The pixel X1 (5) of the fifth hierarchical data D35 is generated by averaging the 34 four pixels X1 (4) to X4 (4).

Here, the first to fifth hierarchical data D31 to D3
Assuming that the block size of the first hierarchy data D31, which is the lowest hierarchy, is 1 line × 1 pixel, the second hierarchy data D32 is 1/2 line × 1/2 pixel, and the third hierarchy data D33 is 1/4 line x 1/4 pixel, fourth hierarchical data D34 is 1/8 line x 1/8 pixel, and fifth hierarchical data D35 which is the highest hierarchical data is 1
/ 16 line × 1/16 pixel.

The hierarchical coding encoder unit 40A repeats the recursive processing in order from the highest hierarchical data (ie, the fifth hierarchical data D35) among the first to fifth hierarchical data D31 to D35, and repeats the recursive processing. Differences between the two hierarchical data are obtained in difference circuits 41, 43, 45, and 47, and only the difference data is compression-encoded by encoders 51 to 55. Thus, the hierarchical coding encoder unit 40A compresses the amount of information transmitted to the transmission path. Further, as described above with respect to Expression (2), the hierarchical encoding encoder unit 40A reduces the transmission data amount by reducing one pixel of the lower hierarchical four pixels corresponding to the upper hierarchical one pixel by the encoders 51 to 54. Reduce.

In order to keep such a compression condition optimal, the hierarchical encoding encoder 40A transmits the second to fourth hierarchical differential encoded data D5 as transmission data obtained for each hierarchical layer.
1 to D54 and the highest-layer encoded data D55 are decoded by decoders 56 to 59. Among them, the decoder 59 corresponding to the highest layer outputs the decoded data D corresponding to the fifth layer data D35 compressed and encoded by the encoder 55.
48 is decoded from the highest-layer encoded data D55, and the decoded data is supplied to the fourth-layer difference circuit 47.

On the other hand, each of the other decoders 51 to 54 switches the decoding operation based on a flag indicating the presence / absence of division / non-division processing. That is, when the division processing is performed, the second to fourth hierarchical difference value encoded data D52 to
The upper layer data (ie, the fourth, third, and second layer data) is decoded from the difference data transmitted as D54 by a decoding process, and the third-layer difference circuit 45 and the second-layer difference circuit 43 are decoded. , And the first hierarchical data 41. Thereby, each difference circuit 41, 4
From 3, 45, and 47, difference data D41, D42, D43, and D44 for adjacent layers are obtained.

The encoders 51 to 55 corresponding to the respective layers are provided with the difference circuits 41, 43, 45, 47 and the averaging circuit 4 respectively.
8, the difference data D41, D42, D4
3, D44 or the fifth hierarchical data D35 is inputted, and a threshold value judgment and a division selection process for the activity obtained for each block are executed. At this time, the encoders 51 to
When the processing target is a divided block, the differential data obtained between the hierarchies is compression-encoded as it is, and at the same time, is transmitted with a division judgment flag for each block.

On the other hand, when the processing target is a non-divided block, the encoders 51 to 55 exclude the block from the coding target on the receiving side as it is decoded from the upper layer data. Incidentally, in this case as well, the division determination flag for each block is transmitted. The first to fourth layer difference value encoded data D51 to D54 and the highest layer encoded data D55 output from these five sets of encoders 51 to 55 are transmitted to a predetermined transmission path.

(3-2) Processing Next, specific signal processing by the hierarchical coding encoder unit 40A will be described. First, a case is considered in which processing for an inter-layer difference value is selected based on block activity based on the inter-layer difference value. Each block is composed of 2 lines × 2 pixels.

Here, the data value of each pixel is X, and the hierarchy of the data value X is represented by suffix. That is, when the upper hierarchical data is Xi + 1 (0), the adjacent lower hierarchical data is Xi (j) (j = 0 to 3). The difference code value between layers is ΔXi (j) (j = 0 to 3), and the layer coding encoder unit 40A compression-codes the difference code value.

In the compression encoding processing by the encoders 51 to 55 in each layer, the block activity P obtained for each block is compared with the threshold data D57, and the processing is selected based on the comparison result. That is, when the block activity P is equal to or larger than the threshold value TH, the division process is sequentially performed on the lower hierarchy, whereas when the block activity P is smaller than the threshold value TH, the division process on the lower hierarchy is stopped.

As a result, in the area where the block activity P is low, only the upper hierarchical data need be sent.
The amount of transmitted information can be reduced. Further, the image data decoding apparatus which receives these data via the transmission path, restores the lower hierarchical data with the upper hierarchical data in an area having a low block activity P by using the upper hierarchical data among the transmission data sequentially transmitted. . On the other hand, in an area where the block activity P is high, the data is restored by adding the difference decoding value between layers and the upper layer data.

A one-bit decision flag is introduced for the result of the division or non-partition decision. With this flag, it is possible to instruct the judgment result for each block. This determination flag is set to 1 for each block of each layer.
Although it is necessary to use each bit, it is effective when image quality is considered. By the way, in the hierarchical coding method in this embodiment, it is assumed that this determination flag is not reflected in the determination in the lower layers thereafter. This determination flag is encoded by run-length encoding or the like, and transmitted together with the encoded code.

(4) Generated Information Amount Control Unit 40B (4-1) Block Configuration On the other hand, the generated information amount control unit 40B is configured as shown in FIG. The generated information amount control unit 40B is configured to enable the hierarchical encoding encoder unit 40A to efficiently encode image data without deteriorating image quality.
Threshold T for each layer as a selection criterion for division / non-division
A combination of H1 to TH4 is set, and this is output to the hierarchical encoding encoder unit 40A as threshold data D57.

The generated information amount control unit 40B averages the input image data D31 by 1/4 through averaging circuits 42, 44, 46 and 48 in order to generate image data of five layers with different resolutions. Subsequently, in order to obtain the amount of generated information for each layer of the image data transmitted as the difference data, the layer image data D32, D33, and D34 in the next higher layer
And D35 and image data D31, D32, and D3 of each layer
3 and D34 are obtained in the respective difference circuits 61, 62, 63 and 64.

Each of these difference circuits 61, 62, 63 and 6
4 can be regarded as difference data of each layer obtained by the layer processing in the layer coding encoder unit 40A. The activity detection circuits 65, 66, 67 and 68 correspond to the image data of the first to fourth layers, respectively, and determine the block activities for each block of each layer and register them in the corresponding frequency distribution tables 69 to 72. It has been made like that. Here, in the generation process of the frequency distribution table, among the four pixels in the lower layer corresponding to one pixel in the upper layer, three pixels to be actually transmitted by the encoder are used in order to accurately grasp the transmission data amount of the encoder unit. Has been made. The fifth layer image data is the highest layer data,
The dynamic range of each block is directly registered in the frequency distribution table 73 because it is transmitted directly, not as difference data.

The control unit 74 controls these five sets of frequency distribution tables 69.
, And a combination of the threshold values TH1 to TH4 of the block activity P, which is used as a criterion for determining whether the lower layer is divided or not.
Is stored in The control unit 74 gives these sets to the frequency distribution tables 69 to 73, reads out the amount of generated information that will occur with respect to the threshold for each layer, and based on all these generated information amounts, the total amount of information as a whole. Find the amount of generated information. Then, an optimum threshold value is obtained until the total amount of generated information reaches the target value, and the obtained threshold value is given to the hierarchical encoding encoder unit 40A as control data.

The control unit 74 adjusts control data to be applied to the hierarchical encoding encoder unit 40A in consideration of the properties of image signal data and human visual characteristics for each hierarchy, so that an optimum threshold value can be given. ing. As a result, subjective improvement in the image quality reproduced on the receiving side is realized.

(4-2) Frequency Distribution Table Here, frequency distribution tables 69 to 73 for information amount control will be described. FIGS. 8A to 8E show frequency distribution tables of block activities obtained from the highest hierarchical data (fifth hierarchical data) to the lowest hierarchical data (first hierarchical data), respectively. . Here, as for the frequency distribution table for the fifth hierarchy shown in FIG. 8A, a frequency distribution table based on the dynamic range is generated because the target data is not difference data. For example, when PCM coding is applied, the dynamic range of the coded block is registered as data, and ADRC (adaptive dynamic range coding (USP-4703) is used as a compression processing method.
When 352)) is applied, the DR of the ADRC block is registered.

On the other hand, in the other frequency distribution tables 69 to 72, the target data is difference data, and blocks having block activities equal to or higher than the threshold values TH1, TH2, TH3, and TH4 given for each frequency distribution table are divided target blocks. Becomes Therefore, the amount of generated information can be calculated by calculating the number of blocks having a block activity equal to or greater than the threshold value in each layer.

Next, an example of calculating the amount of generated information will be described. Here, assuming that the number of blocks in the first layer is N1, the number of blocks to be divided whose block activity is greater than the threshold value TH1 is N1 ', and the number of quantization bits in that case is Q1, the amount of generated information I1 in the first layer is Next formula

I1 = 4 · Q1 · N1 ′ · (3/4) + N1 (5)

The reason why the number of bits is quadrupled in the first term in the equation (5) is that in this example, each block is divided into 2 lines × 2 pixels. In the first term, the factor of 3/4 is that in the structure in which the upper hierarchical value is generated from the average value of the lower hierarchical values,
This is because it reflects the property that the fourth non-transmission pixel value of the lower layer can be restored by an arithmetic expression using the upper layer value and the transmitted lower layer value of 3 pixels. By the way, in the second term, the addition of N1 to the number of blocks in the first layer indicates that one bit is added to each block as a division determination flag and transmitted.

Similarly, for the second, third, and fourth layers, the number of blocks in each layer is N2, N3, and N4, and the block activities are threshold values TH2 and TH.
3, the number of blocks to be divided larger than TH4 is N2 ', N
3 ′ and N4 ′, the number of quantization bits at that time is Q
2, Q3 and Q4, the amount of generated information I in each layer
k (k = 2, 3, 4) is given by

Ik = 4 · Qk · Nk ′ · (3/4) + Nk (6)

By using the generated information amounts I1 to I4 for the first to fourth layers and the generated information amount I5 for the fifth layer, the total generated information amount generated by the coding process of the layer coding encoder unit 40A. I is

(7) I = I1 + I2 + I3 + I4 + I5 (7) It can be obtained as the sum of the generated information amount for each layer.

(4-3) Processing The generated information amount control unit 40 B
As in the case of 0A, the input image data D31 is input, and the average value is obtained for each 2 lines × 2 pixels by the averaging circuit 42, and the number of pixels is reduced to ４ to lower the resolution. Subsequently, the resolution of the hierarchical data D32 is similarly reduced by reducing the number of pixels to 1/4 by sequentially passing through the averaging circuits 43, 46, and 48.

The generated information amount control section 40B gives the highest-order (that is, the lowest resolution) hierarchical data D35 of the image data of a plurality of resolutions to the frequency distribution table 73, and outputs the fifth hierarchical data D35. The frequency of the activity P of each block is registered. This is a measurement of the frequency of data corresponding to the compression process performed by the above-described hierarchical coding encoder unit 40A. For example, the fifth hierarchical data D3
5 is subjected to compression processing by PCM coding, a dynamic range given for each block is registered as data, and ADRC is used as a compression processing method.
In case of applying, the DR of the ADRC block is registered.

Next, difference data D64 is obtained from the difference between the fourth hierarchical data D34 and the fifth hierarchical data D35. The activity detection circuit 68 calculates the difference data D6
The activity is detected for 4 and registered in the frequency distribution table 72 as activity data D68. Similarly, the activity P of each block obtained for each of the lower hierarchical data D33, D32, and D31 is sequentially registered in the frequency distribution tables 71, 70, and 69 as activity data D67, D66, and D65.

The control unit 74 reads combinations of the threshold values TH1, TH2,... TH4 set for each layer from the ROM table shown in FIG. 9 in ascending order of the number (QNO1) having the smallest number. Subsequently, block frequencies having an activity P having a large value with respect to each of the thresholds TH1, TH2,..., TH4 are read from the frequency distribution tables 69 to 73 for each hierarchy, and the amount of generated information for each threshold is detected for each hierarchy.

The control unit 74 controls the frequency distribution tables 69 to 7 of each hierarchy.
3 is calculated, and the total amount of generated information that will be generated as a result of encoding in the hierarchical encoding encoder 40A is calculated. The control unit 74 compares the amount of generated information with the target value. If the difference from the target value is large, a threshold number TH1, TH2,... TH4 of the next number (QNO2) is obtained in order to obtain a combination of threshold values satisfying the target value. Move to the set. Thereafter, the above processing is repeated until the total amount of generated information reaches the target value, and a set of thresholds TH1, TH2,... TH5 that can obtain the total generated amount closest to the target value is obtained. The output is provided to the encoding encoder unit 40A.

(5) Data Structure of Transmission Block Here, the data generated in the hierarchical coding encoder unit 40A by the generated information amount control method as described above is the highest hierarchical data output from the encoder 55 (FIG. 5). Fixed-length data such as encoded data D55, and first to fourth hierarchical difference value encoded data D51 to D output from encoders 51 to 54
It can be classified into 54 variable length data. For this reason, in the hierarchical coding encoder unit 40A, as shown in FIG. 10, the fixed-length data are fixed together for each frame constituting the image by the transmission block forming unit 80 provided at the output stage. In addition to forming the long data block 100, the variable length data block 101 is formed by putting together the variable length data of the frame, and the variable length data block 101 is divided into the fixed length data block 100.
Are arranged after the unit block 102 for transmission.
(Hereinafter, this will be referred to as a transmission block 102), which is sequentially transmitted to the transmission line.

In practice, in the transmission block 102, a SYNC code for recognizing the head position of the block 102 at the head of the fixed-length data block 100 (that is, the head of the transmission block 102) or an information code indicating the contents of image data (Hereinafter, referred to as an identification information code), etc., are arranged. In the transmission block 102, the highest-layer coded data D55 is arranged immediately after the transmission block identification code C1, so that when searching for a desired image using the identification code, the highest-layer coding data with a small amount of information is used. Data D
With the use of H.55, images can be sequentially restored at high speed. As a result, a high-speed data search (quick look) function at the time of reproduction can be realized.

Further, in the above-mentioned hierarchical coding method,
Discrimination codes for forming an image on the decoding side, which are composed of the above-described division determination flags output from the encoders 51 to 55 of the hierarchical encoding encoder unit 40A (hereinafter, these are collectively referred to as an inter-layer data division determination code and ) Is equal to the total number of blocks in each layer, and therefore becomes fixed-length data. Therefore, in the transmission block 102, as apparent from FIG. 10, the inter-layer data division determination code C2, which is a determination code for forming an image for each layer on the decoding side, is used for the uppermost layer encoded data D55. Even after the above arrangement of the generated information amount is performed, the data existing in the section in which the uppermost layer encoded data D55 and the inter-layer data division determination code C2 are combined can have a fixed length as a whole. It has been made like that.

In the variable length data block 101,
Fourth to first hierarchical difference value encoded data D54, D53, D
52 and D51 are sequentially arranged, and immediately after the variable length data 101, a transmission block end code C3 for recognizing the end of the transmission block 102 is added.

In the above-described configuration, in this image coding apparatus, the transmission block 102 is formed by arranging the variable-length data after the fixed-length data for each frame on the data generated in the hierarchical coding encoder 40A. Then, this is transmitted to the transmission path. Therefore, even if an error occurs in any of the variable length data, the decoding side can decode the data in the fixed length data block 100 without error. As a result, the image coding apparatus can provide transmission data with robustness against errors. Further, in the image encoding device 40, when transmitting the data generated by the generated information amount control method as described above, the uppermost layer encoded data D55 is arranged and output immediately after the transmission block identification code C1. As a result, when the quick-view function is added to the decoding side, after detecting the transmission block identification code C1,
It is possible to make access to the uppermost layer encoded data D55 in a short time.

According to the above configuration, hierarchical coding having a plurality of resolutions can be easily realized. In addition, the total amount of generated information of the transmission image data encoded and output from the hierarchical encoding encoder 40A can be made substantially equal to the target value, and encoding can be realized without lowering the compression efficiency. Furthermore, it is possible to realize hierarchical coding with less deterioration in image quality. Further, the management of the amount of generated information at the time of hierarchical coding can be made much easier than in the past. In addition, by being able to set an optimal threshold value in consideration of the characteristics of image signal data and human visual characteristics for each layer, the subjective image quality on the receiver side can be further improved compared to the case where the threshold value is set uniformly. Can be.

(6) Other Embodiments In the above embodiment, the block activity is determined by the maximum difference value between the decoded data obtained for the upper hierarchical data and the lower hierarchical data for each block. However, the present invention is not limited to this, and the determination may be made based on the average error or the sum of absolute values in the block, the standard deviation or the sum of the n-th power, or the data frequency equal to or higher than the threshold value.

Further, in the above-described embodiment, the case where the frequency distribution table obtained for each layer is used as it is has been described. However, the present invention is not limited to this. It may be created and used for calculating the amount of generated information. That is, after generating a frequency distribution table for each hierarchy, a cumulative addition value is obtained for the block frequencies from the upper value of the block activity to the value of each block activity, and each cumulative addition value is assigned to an address corresponding to the value of each block activity. To create a cumulative frequency distribution. Then, the frequency corresponding to each block activity becomes an integrated value of the block frequency having a value equal to or greater than the block activity.

If the integrated frequency distribution table is generated in advance as described above, it is not necessary to calculate the block frequency integrated value corresponding to each threshold value, and the block frequency integrated value can be calculated simply by reading the threshold address of the memory. Can be made possible, and the time required for the calculation can be greatly reduced.

Further, in the above-described embodiment, a case has been described in which the threshold value set to a different value for each layer is compared with the block activity to determine the division / non-division of the block. However, the present invention is not limited to this. Instead, a threshold value set to a different value for each layer may be compared with a data difference value between the layers, and block division / non-division may be determined based on the comparison result.

Further, in the above-described embodiment, the case where image data is subjected to PCM encoding in the encoder has been described. However, the present invention is not limited to this, and another encoding method, for example, an orthogonal encoding method is applied. Is also good.

Further, in the above-described embodiment, a plurality of combinations of the threshold values of the frequency distribution table obtained for each hierarchical level are stored in the ROM, and a combination of the threshold values in which the amount of generated information is closest to the target value is obtained. However, the present invention is not limited to this, and it may be possible to set each layer independently.

Further, in the above-described embodiment, a case has been described in which the average value of the lowest hierarchical data is obtained by 2 lines × 2 pixels to obtain the image data of the higher hierarchical. However, the present invention is not limited to this. The average value may be obtained by another combination.

[0076]

As described above, according to the present invention, image transmission data for transmitting a plurality of image data consisting of a plurality of pieces of image data consisting of the lowest hierarchical information having the highest resolution from the highest hierarchical information having the lowest resolution is generated. In the image processing apparatus and method, the hierarchical information other than the highest hierarchical information is obtained as inter-layer difference data that is a difference from the hierarchical information of an adjacent higher hierarchical level. The block activity is determined for each block consisting of, and for each block of the layer information of the predetermined layer excluding the highest layer information, when the block activity is smaller than the predetermined threshold, the block information of the block is not transmitted. A flag is set to indicate whether transmission or non-transmission is performed for each block of each layer information based on the flag. And by transmitting the flag as fixed-length data and then generating the image transmission data such that the inter-layer difference data is transmitted as variable-length data, it is possible to reduce the amount of transmission information and at the time of decoding, Even when an error occurs in the variable-length data, the content of the fixed-length data can be accurately detected, and thus robustness against a transmission error can be provided. In addition, it is possible to improve the accessibility to desired image data based on the fixed length data, and to facilitate addition of additional functions such as a quick-view function on the user side. According to the present invention, there is provided an image processing method for generating image transmission data for transmitting image data composed of a plurality of pieces of hierarchical information including the lowest hierarchical information having the highest resolution from the highest hierarchical information having the lowest resolution. Inter-layer difference data, which is a difference from adjacent upper layer information, is obtained as layer information excluding the highest layer information, and block activity is determined for each block composed of a plurality of pixels with respect to the predetermined layer information. For each block of the hierarchy information of the predetermined hierarchy except the highest hierarchy information, the block hierarchy information is transmitted when the block activity is equal to or more than a predetermined threshold, and when the block activity is less than the threshold, the block hierarchy information is transmitted. Is transmitted while the highest level information is transmitted as fixed-length data. By having the over data to generate an image transmission data as transmitted as variable length data,
It is possible to reduce the amount of transmission information and to accurately detect the content of fixed-length data even when an error occurs in variable-length data during decoding, thus providing robustness against transmission errors. it can.
In addition, it is possible to improve the accessibility to desired image data based on the fixed length data, and to facilitate addition of additional functions such as a quick-view function on the user side.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating the principle of an image encoding method according to the present invention.

FIG. 2 is a table showing a processing result of a captured image adaptively divided by an image encoding method according to the present invention.

FIG. 3 is a table showing signal levels for respective layers obtained by an image encoding method according to the present invention.

FIG. 4 is a block diagram showing an embodiment of an image encoding device according to the present invention.

FIG. 5 is a block diagram showing a hierarchical encoding encoder unit.

FIG. 6 is a schematic diagram for explaining a hierarchical structure.

FIG. 7 is a block diagram showing a generated information amount control unit.

FIG. 8 is a characteristic curve diagram showing a frequency distribution table of each hierarchy.

FIG. 9 is a table showing combinations of thresholds obtained for each hierarchy.

FIG. 10 is a conceptual plan view showing a data configuration of a transmission block.

FIG. 11 is a characteristic curve diagram showing an integrated frequency distribution table.

FIG. 12 is a block diagram showing a conventional hierarchical encoding device.

FIG. 13 is a block diagram showing a conventional hierarchical decoding device.

[Explanation of symbols]

40 hierarchical coding device, 40A hierarchical coding encoder unit, 40B generated information amount control unit, 41, 43, 4
5, 47, 61, 62, 63, 64... Difference circuit, 4
2, 44, 46, 46... Averaging circuit, 51, 52, 5
3, 54, 55 ... encoder, 56, 57, 58, 59 ...
... Decoder, 65, 66, 67, 68 ... Activity detection circuit, 69, 70, 71, 72, 73 ... Frequency distribution table, 74 ... Control unit, 100 ... Fixed length data block, 101 ... Variable Long data block, 102 ... transmission block, C1 ... transmission block identification code, C2 ...
Inter-layer data division determination code, D51... First layer difference value encoded data, D52... Second layer difference value encoded data, D53... Third layer difference value encoded data, D54.
... Fourth layer difference value encoded data, D55...

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-5-183870 (JP, A) JP-A-62-84630 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 7/24-7/68 H04N 1/41-1/419

Claims (4)

(57) [Claims] 1. An image processing apparatus for generating image transmission data for transmitting image data including a plurality of pieces of hierarchical information including the lowest hierarchical information having the highest resolution and the lowest hierarchical information having the highest resolution. Inter-layer difference data generating means for obtaining inter-layer difference data which is a difference from the above-mentioned layer information of an adjacent upper layer as the above-mentioned layer information excluding the upper layer information; Judging means for judging block activity for each block of pixels; and for each block of the hierarchical information of the predetermined hierarchy except for the top hierarchy information, when the block activity is smaller than a predetermined threshold value, Setting means for setting a flag indicating that the above-mentioned hierarchy information is not transmitted; and Control means for controlling whether or not transmission or non-transmission is performed for each block of the layer information, and transmitting the uppermost layer information and the flag as fixed length data and then transmitting the inter-layer difference data as variable length data. An image processing apparatus comprising: image transmission data generating means for generating the above image transmission data. 2. An image processing method for generating image transmission data for transmitting image data composed of a plurality of pieces of hierarchical information from the highest hierarchical information having the lowest resolution to the lowest hierarchical information having the highest resolution. A first step of obtaining inter-layer difference data which is a difference from the above-mentioned layer information of an adjacent upper layer as the above-mentioned layer information excluding the upper layer information; A second step of judging the block activity for each of the blocks, and for each of the blocks of the hierarchy information of the predetermined hierarchy except for the top hierarchy information, when the block activity is smaller than a predetermined threshold value. And setting a flag indicating that the above-mentioned hierarchy information is not transmitted.
And a fourth step of controlling the presence or absence of transmission or non-transmission for each block of the layer information based on the flag, and transmitting the uppermost layer information and the flag as fixed length data. Fifth generation of the image transmission data for transmitting the difference data between layers as variable-length data later
An image processing method comprising the steps of: 3. Resolution from the highest hierarchical information having the lowest resolution
From multiple hierarchical information consisting of the lowest hierarchical information with the highest degree
Image transmission data for transmitting different image data
Image processing apparatus,As the above-mentioned hierarchy information excluding the above-mentioned highest hierarchy information,
Difference data that is the difference from the upper layer information
Inter-layer difference data acquisition means, The above-mentioned hierarchy information of a predetermined hierarchy is composed of a plurality of pixels.
Judge block activity for each block
Determining means; The hierarchical information of the predetermined hierarchy excluding the top hierarchy information
For each of the above blocks in the report,
When the tee is equal to or greater than a predetermined threshold, the hierarchy of the block
Information is transmitted, and the block activity is lower than the threshold.
Do not transmit the layer information of the block when full
Control means for controlling After transmitting the above top layer information as fixed length data
The above difference data between layers is transmitted as variable length data.
Transmission data generating means for generating such image transmission data
When An image processing apparatus comprising: 4. Resolution from the highest hierarchical information having the lowest resolution
From multiple hierarchical information consisting of the lowest hierarchical information with the highest degree
Image transmission data for transmitting different image data
Image processing method,As the above-mentioned hierarchy information excluding the above-mentioned highest hierarchy information,
Difference data that is the difference from the upper layer information
A first step, The above-mentioned hierarchy information of a predetermined hierarchy is composed of a plurality of pixels.
About each block Determine block activity
A second step, The hierarchical information of the predetermined hierarchy excluding the top hierarchy information
For each of the above blocks in the report,
When the tee is equal to or greater than a predetermined threshold, the hierarchy of the block
Information is transmitted, and the block activity is lower than the threshold.
Do not transmit the layer information of the block when full
A third step of controlling After transmitting the above top layer information as fixed length data
The above difference data between layers is transmitted as variable length data.
A fourth step of generating such image transmission data With
An image processing method comprising: