CN112040236B

CN112040236B - Image processing method, image processing device, image display method, and image display device

Info

Publication number: CN112040236B
Application number: CN202010923460.7A
Authority: CN
Inventors: 兰永城
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2020-09-04
Filing date: 2020-09-04
Publication date: 2022-02-18
Anticipated expiration: 2040-09-04
Also published as: CN112040236A

Abstract

The invention provides an image processing method, a processing device, an image display method and a display device, wherein the method comprises the following steps: the plurality of line data of the image data is divided into a plurality of line groups according to the number of colors, and the number of colors of the line data in each line group is proportional to the number of line data in the line group. This allows dynamic line splitting of image data according to color complexity, i.e., the number of lines grouped can be increased appropriately for areas of relatively complex image color, and decreased appropriately for areas of relatively simple image color, thereby reducing the impact of compression on image quality of the image. In addition, before image compression, the identification code is added to ensure that image data is accurately compressed and transmitted according to the line grouping divided by the dynamic line division, and provide a basis for the subsequent decoding and displaying.

Description

Image processing method, image processing device, image display method, and image display device

Technical Field

The present invention relates to the field of image display technologies, and in particular, to an image processing method, an image processing apparatus, an image display method, and a display apparatus.

Background

With the development of electronic devices, the resolution and refresh rate of display screens are increasing, and thus the amount of data transmitted by images is also increasing significantly. The bandwidth of the current universal transmission interface D-PHY 4Lane cannot meet the transmission requirement.

To solve the above problems, a standard image Compression protocol (DSC) is proposed in the industry. However, the compression transmission scheme adopting the DSC compression protocol at present has a large influence on the image quality of the decompressed display.

Disclosure of Invention

In view of the above, the present application provides an image processing method, an image processing apparatus, an image display method, and a display apparatus. The technical scheme is as follows:

the application provides an image processing method, which comprises the following steps:

acquiring image data, wherein the image data comprises a plurality of line data;

dividing the plurality of line data into a plurality of line groups according to the color quantity of the image data, wherein the color quantity of the line data in the line groups is in direct proportion to the quantity of the line data in the line groups;

adding an identification code for characterizing the row group in which the row group is positioned to the row data in the row group;

and encoding the row data in the row group based on the identification code and outputting.

The present application also provides an image processing apparatus, the apparatus comprising:

the image acquisition module is used for acquiring image data, and the image data comprises a plurality of line data;

the line grouping and dividing module is used for dividing the plurality of line data into a plurality of line groups according to the color quantity of the image data, and the color quantity of the line data in the line groups is in direct proportion to the quantity of the line data in the line groups;

the identification code adding module is used for adding identification codes for representing the row groups in which the row groups are positioned for the row data in the row groups;

and the coding module is used for coding the line data in the line group based on the identification code and outputting the line data.

The present application also provides an image display method, including:

acquiring an encoding result of image data, wherein the image data comprises a plurality of line data, the encoding result of the image data comprises an encoding result of each line data and an identification code representing a line group where each line data is located, the line group is obtained by dividing the plurality of line data for the color quantity of the image data, and the color quantity of the line data in the line group is in direct proportion to the quantity of the line data in the line group;

determining the coding result of the line data belonging to the same line group based on the identification code;

and decoding and displaying the coded result of the line data belonging to the same line group.

The present application also provides an image display apparatus, the apparatus comprising:

the image coding device comprises a coded image acquisition module, a coding module and a data processing module, wherein the coded image acquisition module is used for acquiring a coding result of image data, the image data comprises a plurality of line data, the coding result of the image data comprises a coding result of each line data and an identification code for representing a line group where each line data is located, the line group is obtained by dividing the plurality of line data according to the color number of the image data, and the color number of the line data in the line group is in direct proportion to the number of the line data in the line group;

the line grouping determination module is used for determining the encoding result of the line data belonging to the same line group based on the identification code;

and the decoding module is used for decoding and displaying the coding result of the line data belonging to the same line group.

In the image processing method provided by the present application, the plurality of line data of the image data are divided into the plurality of line groups according to the number of colors, and the number of colors of the line data in each line group is proportional to the number of line data in the line group. This allows dynamic line splitting of image data according to color complexity, i.e., the number of lines grouped can be increased appropriately for areas of relatively complex image color, and decreased appropriately for areas of relatively simple image color, thereby reducing the impact of compression on image quality of the image.

In addition, before image compression, the identification code is added to ensure that image data is accurately compressed and transmitted according to the line grouping divided by the dynamic line division, and provide a basis for the subsequent decoding and displaying.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a flow chart of the DSC compression protocol applied in the current image display system;

FIG. 2 is a diagram of a conventional row compression scheme;

FIG. 3 is a schematic diagram of the row compression provided herein;

FIG. 4 is a flow chart illustrating the application of DSC compression protocol in the image display system provided by the present application;

fig. 5 is a flowchart of a method of processing an image according to an embodiment of the present application;

fig. 6 is a signaling flowchart of an image processing method according to a second embodiment of the present application;

fig. 7 is a signaling flowchart of an image processing method according to a third embodiment of the present application;

fig. 8 is a signaling flowchart of an image processing method according to a fourth embodiment of the present application;

fig. 9 is a signaling flowchart of an image processing method according to a fifth embodiment of the present application;

fig. 10 is a signaling flowchart of an image processing method according to a fifth embodiment of the present application;

fig. 11 is a signaling flowchart of an image processing method according to a sixth embodiment of the present application;

fig. 12 is a schematic diagram of row grouping division provided in a sixth embodiment of the present application;

fig. 13 is a schematic structural diagram of an image processing apparatus according to a seventh embodiment of the present application;

fig. 14 is a flowchart of a method of displaying an image according to an eighth embodiment of the present application;

fig. 15 is a schematic structural diagram of an image display device according to a ninth embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

For ease of understanding, the DSC compression protocol is described in the following application:

the resolution of electronic device displays has been increasing from original HD (720 x 1280) to FHD (1080 x 1920) to WQHD (1440 x 2560), and so on, while the display refresh rate has been increasing from original 60fps to 90fps to 120 fps. The resolution and the refresh rate are improved, and the image transmission data volume is increased.

At present, the image transmission interface generally adopts D-PHY (DSI) of MIPI alliance. Taking a mobile phone as an example, a D-PHY interface adopted by a mobile phone platform manufacturer is generally 8 sets of lanes, but in order to reduce the use of lanes (the number of lanes affects space and cost), most of them adopt 4 lanes. Due to the influence of the process, the transmission rate of the communication interface at the display module end is about 1.2Gbps/Lane at most, so the image transmission at the FDH &120fps refresh frame rate cannot be met by adopting the bandwidth of the D-PHY 4 Lane.

To address this problem, the industry proposes a standard image compression protocol, namely, dsc (display Stream compression) standard display compression protocol proposed by vesa (video Electronics Standards association) video Electronics Standards association.

Referring to an application flow of the DSC compression protocol shown in fig. 1 in the current image display system, the current image display system includes a processing module (AP) and a display module (IC), and image data in the processing module (AP) is divided into rows and groups by a processor (CPU), compressed by a VESD DSC Encoder, and then transmitted to the display module (IC) through an internal communication interface (MIPI DPHY 4Lane Tx). The display module (IC) receives Line data sent by the processing module (AP) through an internal communication interface (MIPI DPHY Rx), caches the Line data through a Line Buffer, and finally decompresses the Line data through a Decoder (VESD DSC Decoder) and outputs the decompressed Line data to a display screen for display.

See figure 2 for a prior art row compression diagram. The DSC compression protocol divides the line groups into a fixed number of lines, which is generally selected from standard groups of 4, 8, 12, 16, 20, 22, 32 lines, and so on, and divides one frame of image data into a plurality of fixed number of lines for compression. For example, the compression ratio is 1/3, and the compression has an effect on the image quality of an image, and for example, 8 lines are used for line grouping, and 8 continuous lines of image data are compressed as one packet, and the amount of compressed data is only that of the original packet 1/3. However, the way of dividing the lines into groups is relatively simple for image colors, for example, the image quality influence caused by a pure color picture is small, but is relatively complex for image colors, for example, the image quality influence caused by various irregular colors is large.

Since the image is compressed, the final displayed image will be different from the original image to some extent. Therefore, in order to measure the displayed image quality, the unit of Peak Signal to Noise Ratio (PSNR) which is an image quality evaluation index is usually referred to as dB, and a larger value indicates less distortion, i.e. less influence on the image quality.

The method aims at the problem that the image quality of decompressed display is greatly influenced by the existing compression transmission scheme adopting a DSC compression protocol. The application proposes an improvement to divide a plurality of line data of image data into a plurality of line groups according to the number of colors, the number of colors of the line data in each line group being proportional to the number of line data in the line group. Since the color number can represent the color complexity, that is, the color complexity is higher as the color number is larger, the line grouping is divided by adopting the dynamic line grouping mode, the line number of the line grouping can be properly increased for the region with relatively complicated image colors, and the image quality influence is smaller as the PSNR after compression is larger. Fig. 3 is a schematic diagram of the line compression provided in the present application.

With continued reference to fig. 1. The method is limited by the hardware structures of a processing module and a display module in the current image display system, and before processing an image, a Code representing the number of fixed standard grouping lines needs to be set in a Code field of a DSC compression protocol in advance, so that the processing module and the display module respectively compress and decompress the image according to the number of standard grouping lines represented by the Code.

In order to realize the dynamic line division, the image display system provided by the application adds a controller in the display module on the basis of the hardware structure of the current image display system. Refer to the DSC compression protocol shown in fig. 4, which is an application flow in the image display system provided in the present application. Image data in the processing module (AP) is divided into line groups by a processor (CPU) in a dynamic line division mode, compressed by a coder (VESD DSC Encoder), and then sent to a display module (IC) through an internal communication interface (MIPI DPHY 4Lane Tx). After receiving Line data sent by the processing module (AP) through an internal communication interface (MIPI DPHY Rx), the display module (IC) caches the Line data through a Line Buffer (Line Buffer), a Controller (CON) identifies the coding result of the Line data belonging to the same Line group in the Line Buffer (Line Buffer) and controls the Line Buffer (Line Buffer) to output, and finally a Decoder (VESD DSC Decoder) decompresses the output of the Line Buffer (Line Buffer) and outputs the decompressed output to a display screen for display.

The detailed description of the program refining function and the extended function of the processing module (AP) and the display module (IC) are described in detail below.

The image processing method is applied to a processing module. A flowchart of a method of an image processing method disclosed in an embodiment of the present application is shown in fig. 5, and the method includes the following steps:

s101, image data is acquired, and the image data comprises a plurality of line data.

In the embodiment of the application, an image can be divided into a plurality of lines based on the resolution of a display screen, wherein the image data is color data of the image, and one line of color data of the image is one line of data in the image data.

Taking FHD + resolution 2340 × 1080 as an example, an image is divided into 2340 rows, each row has 1080 pixel points, and one row of data in the image data is color data of 1080 pixel points located in the same row.

S102, dividing the plurality of line data into a plurality of line groups according to the color number of the image data, wherein the color number of the line data in the line group is in direct proportion to the number of the line data in the line group.

In the embodiment of the present application, the number of colors is a color type, and the number of colors can represent the color complexity, i.e., the more the number of colors (color type) is, the higher the color complexity is.

The RGB color standard is mostly used for the current display screen, i.e. the display screen generates color by hitting an electron gun on the red, green and blue light emitting electrodes of the screen. The color of each pixel point on the image is formed by mixing red, green and blue color lights according to different proportions, so that the color data of the pixel points can be recorded and expressed by a group of RGB values.

According to the color data of each row of pixel points in the image data, the color quantity of the row of data corresponding to the row can be determined, and the more the color quantity is, the higher the color complexity of the row of data is represented. And then, the line data of the image data are sequentially divided into corresponding line groups based on the color complexity of each line data of the image data, and the higher the color complexity of a plurality of continuous line data in one line group is, the more the number of the line data is.

It should be noted that the line packets need to be divided according to the DSC protocol, that is, the number of line data in the line packets is the standard number of packet lines specified by the DSC protocol.

And S103, adding an identification code for representing the row grouping where the row grouping is located for the row data in the row grouping.

In the embodiment of the present application, the method steps of steps S103 to S104 are performed for each row group.

For each line data of the image data, the identification code characterizing the line group in which the identification code is located can be added to the line data of the image data, and can also be added to other line data.

That is, after the identification codes are added to all the line data of the image data, each line data may carry the identification code, and a part of the line data may carry the identification code.

And S104, coding the row data in the row group based on the identification code and outputting the coded row data.

In the embodiment of the application, the processor sends the image data to the encoder after the identification codes are added to the plurality of line data of the image data. The encoder can determine the row group where each row of data is based on the identification code, further encode and compress the row of data belonging to the same row group as a sub-packet, and output the encoding result of the image data according to the row.

Therefore, the image processing method provided by the embodiment of the application can realize dynamic line division of the image data according to the color complexity degree, and reduce the influence of compression on the image quality of the image. In addition, before image compression, the identification code is added to ensure that image data is accurately compressed and transmitted according to the line grouping divided by the dynamic line division, and provide a basis for the subsequent decoding and displaying.

As an implementation manner of step S20 "dividing a plurality of line data into a plurality of line groups according to the number of colors of image data", an image processing method disclosed in the second embodiment of the present application, a signaling flowchart is shown in fig. 6, and the method includes the following steps:

s201, image data is acquired, and the image data comprises a plurality of line data.

S202, the plurality of line data of the image data are divided into a plurality of grouping units, each of which is a minimum unit for performing line grouping, and each of the grouping units is formed by a predetermined number of continuous line data.

In the embodiment of the application, the grouping unit is the minimum unit of dynamic row division, and the number of row data in each grouping unit is the same.

S203, obtaining the line similarity of the grouping unit, wherein the line similarity of the grouping unit represents the similarity between all the lines of data in the grouping unit.

In the embodiment of the present application, the method steps of steps S203 to S206 are performed for each packet unit.

One grouping unit corresponds to one row similarity, and the description is given in terms of one grouping unit:

the row similarity of the grouping unit is positively correlated with the similarity between two adjacent rows of data in the grouping unit. Specifically, the average of the similarities between all adjacent two rows of data may be used as the row similarity of the grouping unit.

And the similarity between two adjacent rows of data can be calculated according to the color data of two adjacent rows of pixel points of the image, namely the difference of RGB values of two adjacent rows of pixel points is smaller, and the similarity between the rows of data is larger. Specifically, the RGB values may be converted into a binary system, and gray scale levels expressed by logic 0 and logic 1, and the gray scale levels of two adjacent rows of data are subjected to an exclusive or operation, and the proportion of logic 0 in the operation result is used as the similarity between two adjacent rows of data.

And S204, distributing row grouping for the grouping unit based on the row similarity of the grouping unit.

In the embodiment of the present application, the row similarity of the grouping unit is inversely proportional to the color number of the row data in the grouping unit, so when the row grouping is allocated to the grouping unit, the allocation rule according to which the row similarity of the grouping unit is inversely proportional to the number of the grouping unit in the row grouping where the row similarity of the grouping unit is located is adopted.

Based on the above allocation rule, a standard grouping line number matching the line similarity of the grouping unit is selected from among standard grouping line numbers specified by the DSC protocol, and the standard grouping line number can be used as a basis for determining the number of lines in the line group in which the grouping unit is located.

And S205, adding an identification code for characterizing the row data in the row group.

And S206, coding the row data in the row group based on the identification code and outputting the coded row data.

Therefore, the image processing method provided by the embodiment of the application can perform line grouping by taking the grouping unit as the minimum unit, so that the influence of the color complexity of an individual line on the grouping of the divided lines can be avoided, and the efficiency of line grouping and dividing is improved on the basis of reducing the image quality influence.

As an implementation manner of step S204 "allocating row groups to the grouping units based on the row similarity of the grouping units", an image processing method disclosed in the third embodiment of the present application, a signaling flowchart of which is shown in fig. 7, includes the following steps:

s301, image data is acquired, and the image data comprises a plurality of line data.

S302, the plurality of line data of the image data are divided into a plurality of grouping units, each of which is a minimum unit for performing line grouping, and each of the grouping units is formed by a predetermined number of continuous line data.

S303, obtaining the line similarity of the grouping unit, wherein the line similarity of the grouping unit represents the similarity between all the lines of data in the grouping unit.

S304, according to the similarity level of the row similarity of the grouping unit, determining a grouping reference set of the grouping unit, wherein the similarity level is one of a plurality of preset similarity levels, the grouping reference set comprises a plurality of standard grouping row numbers, and the standard grouping row number is more than or equal to the number of row data in the grouping unit.

In the embodiment of the present application, the method steps of steps S303 to S307 are performed for each packet unit.

For a plurality of preset similarity levels, the maximum standard grouping line number in the grouping reference set corresponding to the higher similarity level is smaller than the maximum standard grouping line number in the grouping reference set corresponding to the lower similarity level.

In addition, since one grouping unit corresponds to one grouping reference set, and there are a plurality of standard grouping line numbers in the grouping reference set, there are a plurality of choices for the number of lines in the line grouping of the grouping unit, and correspondingly, there are a plurality of choices for the row grouping of the grouping unit, which requires continuous attempts to choose different standard grouping line numbers for the grouping unit, so as to ensure that the line number of the line data in the image data is the same as the total line number of the obtained plurality of line groupings after the division of the line grouping is completed.

In order to reduce the time for dividing the image data into line groups, in the embodiment of the present application, in a plurality of preset similarity levels, a grouping reference set corresponding to two adjacent similarity levels includes at least one same standard grouping line number. This makes it easier to divide the grouping units with similar similarity levels into the same row group.

The following table is a correspondence example of the similarity level, the line similarity range, and the grouping reference set.

Degree of similarity	Line similarity range	Standard grouping line number (Slice Height)
			A	[90％,100％]	4 rows or 8 rows
B	[80％,90％]	4 rows or 8 rows
			C	[70％,80％]	8 rows or 12 rows
D	[60％,70％]	8 rows or 12 rows or 16 rows
			E	[40％,50％]	12 rows or 16 rows or 22 rows
F	[0％,40％]	12 rows or 16 rows or 22 rows or 32 rows

S305, distributing row grouping for the grouping unit according to a plurality of standard grouping row numbers corresponding to the grouping unit at least.

In the embodiment of the present application, the line groups may be sequentially allocated to each group unit according to the order of processing the line data when the compression transmission and decoding display of the image are performed. When a row group is allocated to each grouping unit, whether the grouping unit can be allocated to the row group of the previous grouping unit (i.e. the closest grouping unit to which the row group has been allocated) can be determined according to the plurality of standard grouping line numbers corresponding to the grouping unit.

Specifically, if the capacity of the line group in which the previous packet unit is located (i.e., the maximum number of line data that can be allocated) belongs to one of the standard packet line numbers corresponding to the packet unit, it indicates that the packet unit can be allocated to the line group in which the previous packet unit is located, and at this time, the packet unit can be allocated to the line group in which the previous packet unit is located; on the contrary, it means that the grouping unit cannot be allocated to the row grouping in which the previous grouping unit is located, at this time, a row grouping may be created, and the grouping unit may be allocated to the created row grouping, where the capacity of the created row grouping is one of the plurality of standard grouping row numbers corresponding to the grouping unit.

It should be noted that the premise of creating a row packet for the packet unit is that the row packet in which the previous packet unit is located is in an unallocated state, that is, the number of row data actually allocated is equal to the capacity of the row packet.

If the previous grouping unit is in an allocable state before the row grouping is created for the grouping unit, that is, the amount of the actually allocated row data is smaller than the capacity of the previous grouping unit, the capacity of the row grouping where the previous grouping unit is located needs to be adjusted, and the adjusted row grouping is ensured to be in an unallowable state.

And S306, adding an identification code for characterizing the row grouping where the row grouping is located for the row data in the row grouping.

And S307, coding the row data in the row group based on the identification code and outputting the coded row data.

Therefore, when the image processing method provided by the embodiment of the application performs row grouping by taking the grouping unit as the minimum unit, the grouping unit with small row similarity difference can be allocated to the same row grouping by matching the similarity level of the row similarity, so that the efficiency of row grouping division can be obviously improved.

As an implementation manner of step S305 "allocating row packets for packet units according to at least a plurality of standard packet line numbers corresponding to the packet units", an image processing method disclosed in the fourth embodiment of the present application is shown in fig. 8 as a signaling flowchart, and the method includes the following steps:

s401, image data is acquired, and the image data comprises a plurality of line data.

S402, dividing a plurality of line data of the image data into a plurality of grouping units, wherein a grouping unit is a minimum unit for performing line grouping, and a grouping unit is formed by a preset number of continuous line data.

S403, obtaining the line similarity of the grouping unit, wherein the line similarity of the grouping unit represents the similarity between all the lines of data in the grouping unit.

S404, determining a grouping reference set of the grouping unit according to the similarity level of the row similarity of the grouping unit, wherein the similarity level is one of a plurality of preset similarity levels, the grouping reference set comprises a plurality of standard grouping row numbers, and the standard grouping row number is larger than or equal to the number of row data in the grouping unit.

S405, whether a line group in an allocable state exists is detected, and the quantity of the allocated line data of the line group in the allocable state is smaller than the capacity of the line group.

In the embodiment of the present application, the method steps of steps S403 to S409 are performed for each packet unit.

For most images, the color data of two adjacent rows of pixel points are generally related, and thus the row similarity of two adjacent grouping units is not different. Therefore, dividing a small number of adjacent two grouping units into the same row group has little influence on dividing the row group as a whole of the image regardless of the standard number of grouping rows.

Therefore, in the embodiment of the present application, when allocating row packets to each packet unit in sequence, it may be determined whether there is a row packet in an allocable state for the packet unit first, that is, whether the row packet in which the previous packet unit is located is in the allocable state.

Of course, in order to avoid dividing two individual adjacent grouping units with greatly different row similarity into the same row grouping, the embodiment of the present application may determine the difference degree between the grouping unit and the previous grouping unit according to the row similarity while detecting the row grouping in the assignable state, where the difference degree is proportional to the difference of the row similarity, that is, the larger the difference of the row similarity between the grouping unit and the previous grouping unit is. The similarity difference and the difference degree may be set specifically based on the actual scene, which is not limited in this embodiment.

Further, if the difference degree between the grouping unit and the previous grouping unit is smaller than a preset difference degree threshold value, determining that the grouping unit can be allocated to the row grouping in which the previous grouping unit is located, and at this time, further allocating the row grouping to the grouping unit according to the detection result of the row grouping in an allocable state; and if the difference degree between the grouping unit and the previous grouping unit is greater than or equal to a preset difference degree threshold value, determining that the grouping unit cannot be allocated to the row grouping in which the previous grouping unit is located, and at this time, ensuring that the row grouping in which the previous grouping unit is located is in an unallocated state.

In addition, since the emphasis of image transmission is on a portion with relatively complicated colors, when a plurality of line data of image data are grouped into lines, the line data can be processed in a hierarchical manner according to the color complexity, that is, the higher the color complexity is, the higher the processing priority is.

Therefore, in some other embodiments, a processing unit in which the grouping unit is located is first determined, where the processing unit is composed of consecutive grouping units with similarity levels at the same processing priority, the processing priority is one of a plurality of preset processing priorities, and the processing priority corresponds to at least one similarity level;

accordingly, step S405 "detecting whether there is a row group in an allocable state", may adopt the following steps:

whether a line group in an allocable state exists in a processing unit in which a group unit is located is detected.

In this embodiment, after dividing a plurality of line data of image data into a plurality of grouping units and determining the similarity level of each grouping unit according to the line similarity, the processing priority can be matched for each grouping unit according to the similarity level. This makes it possible to constitute a processing unit by a group of consecutive packet units having the same processing priority, and to sequentially process the line packets in each processing unit in accordance with the processing priority of each processing unit.

For a processing unit, the row grouping can be allocated to each grouping unit in the processing unit in turn according to the processing order of compression transmission of the image and row in decoding display. Accordingly, the row group in the assignable state is detected in step S405, and is also within the scope of the processing unit. Therefore, the line grouping division of the relatively complicated color part in the image can be preferentially ensured, and the influence of compression on the image quality of the image is greatly reduced.

S406, if no line group in an allocable state exists, creating a line group, wherein the capacity of the created line group is one of a plurality of standard group line numbers corresponding to a group unit.

In the embodiment of the present application, the selection rule for selecting one from a plurality of standard grouping line numbers as the capacity of the created line grouping is not limited. When the row group is allocated to each grouping unit in sequence, the number of rows to which the row group of the grouping unit can be allocated may be adjusted when the row group is allocated to the subsequent grouping unit, so that the standard grouping row number may be selected in a manner from small to large, from large to small, or randomly.

Further, if it is detected that there is a row group in an allocable state and the difference between the capacity thereof and the number of allocated row data is not smaller than the number of row data in the grouping unit, the grouping unit is allocated to the row group in the allocable state. At this time, the grouping unit is allocated to the row grouping in which the previous grouping unit is located.

It should be noted that the assigned row data is the row data already assigned to the row group in the assignable state.

S407, the grouping unit is assigned to the created row group.

And S408, adding an identification code for characterizing the row group in which the row group is positioned for the row data in the row group.

And S409, coding the row data in the row grouping based on the identification code and outputting.

Therefore, the image processing method provided by the embodiment of the application can determine the row grouping of the grouping unit by detecting the row grouping in the distributable state when the row grouping is performed by taking the grouping unit as the minimum unit, so that the efficiency of dividing the row grouping can be rapidly improved.

As an implementation manner of step S305 "allocating row packets for packet units according to at least a plurality of standard packet line numbers corresponding to the packet units", an image processing method disclosed in fifth embodiment of the present application is shown in fig. 9 as a signaling flowchart, and the method includes the following steps:

s501, image data is acquired, and the image data comprises a plurality of line data.

S502 divides a plurality of line data of the image data into a plurality of grouping units, each of which is a minimum unit for performing line grouping, and each of the grouping units is formed by a predetermined number of continuous line data.

S503, obtaining the line similarity of the grouping unit, wherein the line similarity of the grouping unit represents the similarity between all the lines of data in the grouping unit.

S504, according to the similarity level of the row similarity of the grouping unit, determining a grouping reference set of the grouping unit, wherein the similarity level is one of a plurality of preset similarity levels, the grouping reference set comprises a plurality of standard grouping row numbers, and the standard grouping row number is larger than or equal to the number of row data in the grouping unit.

S505, whether a row group in an allocable state exists is detected, and the quantity of the allocated row data of the row group in the allocable state is smaller than the capacity of the row group.

S506, if the line group in the distribution state exists and the difference between the capacity of the line group and the number of the distributed line data is smaller than the number of the line data in the grouping unit, the line similarity of the grouping unit in the line group in the distribution state is acquired.

In the embodiment of the present application, when allocating row packets to each packet unit in turn, if the difference between the capacity of a row packet in an allocable state and the number of row data allocated to the row packet is smaller than the number of row data in the packet unit for the packet unit, this indicates that the packet unit cannot be allocated to the row packet in the allocable state, subject to the number of row data. At this time, the capacity of the line group in the assignable state may be further adjusted based on the line similarity of the group unit.

And S507, comparing the row similarity of the grouping unit with the row similarity of the grouping unit in the row grouping in the distributable state to determine the difference degree between the grouping unit and the row grouping in the distributable state, wherein the difference degree is in direct proportion to the difference of the row similarities.

In the embodiment of the present application, when allocating row groups to each group unit in sequence, for the group unit:

if the grouping unit in the row grouping in the distributable state is one, namely the row similarity of the previous grouping unit is acquired, the difference degree of the grouping unit and the row grouping in the distributable state is determined according to the difference between the row similarity of the previous grouping unit and the row similarity of the grouping unit.

If there are a plurality of grouping units in the row grouping in the assignable state, assuming N, that is, the row similarity of the first N grouping units is obtained, the row similarity of the first N grouping units and the row similarity of the grouping unit can be compared, so as to determine a difference for determining the degree of difference. Some of these implementations are illustrated below:

and calculating the mean value of the line similarity of the first N grouping units, and using the difference between the mean value and the line similarity of the grouping unit to determine the difference of the difference degree.

Of course, a difference between the row similarity of the first N grouping units and the row similarity of the grouping unit may also be selected, which is not limited in the embodiment of the present application.

It should be noted that the difference degree is proportional to the difference of the row similarity, i.e. the larger the difference of the row similarity, the larger the difference of the grouping unit and the detected row grouping is, and the rule of correspondence between the difference of the row similarity and the difference degree can be set based on the actual scene.

The obtained row similarity of the plurality of grouping units may be further processed to obtain a row similarity that can characterize the detected row grouping, such as calculating an average value. This embodiment is not limited to this.

And S508, under the condition that the difference degree is smaller than a preset difference degree threshold value, increasing the capacity of the row grouping in the distributable state to distribute the grouping unit to the row grouping after the capacity is increased.

In the embodiment of the present application, when allocating a row group to each grouping unit in sequence, for the grouping unit, if the degree of difference between the grouping unit and the row group in the allocable state is low, it indicates that the grouping unit can be allocated to the row group in the allocable state, but since the capacity of the row group in the allocable state is small, the capacity of the row group in the allocable state needs to be increased at this time to meet the allocation requirement.

It should be noted that, when increasing the capacity of the line group in the assignable state, the number of standard group lines corresponding to the grouping unit may be determined, and the number of other standard group lines in the DSC protocol may also be determined, which is not limited in this embodiment.

It should be further noted that, the difference degree threshold in the embodiment of the present application may be the same as or different from the difference degree threshold in the fourth embodiment, which is not limited herein.

S509, if the difference degree is equal to or greater than the preset difference degree threshold, the capacity of the line group in the assignable state is reduced, and the process returns to step S505.

In the embodiment of the present application, when allocating a row group to each group unit in turn, for the group unit, if the degree of difference between the group unit and the row group in the assignable state is high, it indicates that the group unit cannot be allocated to the row group in the assignable state, and at this time, there are two solutions:

first, the capacity of the line packet in the allocable state is reduced to contain just the previous packet unit. In this case, returning to step S505 again detects that there is no row packet in the assignable state.

Secondly, the capacity of the row group in the allocable state is reduced to the point that the row group in the allocable state cannot contain the previous group unit, that is, all the group units in the row group in the allocable state are redistributed into a plurality of row groups, and the capacity of one or more row groups in which the other group units are located needs to be ensured to be equal to the number of row data contained in the other group units except the previous group unit.

Therefore, if the line grouping capacity of the previous grouping unit is equal to the sum of the number of the allocated line data, in this case, returning to step S505 again, it is detected that there is no line grouping in the allocable state.

If the line grouping capacity of the previous grouping unit is larger than the sum of the number of the allocated line data, in this case, returning to step S505, it is detected that there is a line grouping in the allocable state.

It should be noted that, when the capacity of the line group in the assignable state is reduced, the number of the standard group lines corresponding to the previous packet unit may be determined, and the number of other standard group lines in the DSC protocol may also be determined, which is not limited in this embodiment.

And S510, adding an identification code for characterizing the row data in the row group.

And S511, coding the row data in the row group based on the identification code and outputting the coded row data.

Therefore, according to the image processing method provided by the embodiment of the application, when the line grouping is performed by taking the grouping unit as the minimum unit, the line grouping in the assignable state is detected, and the line capacity of the detected line grouping is dynamically adjusted, so that the efficiency of dividing the line grouping can be effectively improved.

As an implementation manner of step S103 "adding an identification code representing a row packet in which the row packet is located for the row data", a signaling flowchart of an image processing method disclosed in sixth embodiment of the present application is shown in fig. 10, and the method includes the following steps:

s601, image data is acquired, and the image data comprises a plurality of line data.

S602, dividing the plurality of line data into a plurality of line groups according to the color number of the image data, wherein the color number of the line data in the line group is in direct proportion to the number of the line data in the line group.

S603, adding an identifier for marking the row group in which the row group is positioned to each row of data in the row group.

In the embodiment of the present application, after dividing a plurality of line data of image data into a plurality of line groups, a line group to which each line data belongs may be determined. For each row of data, an identifier for marking the row group where the row is located, i.e. a row group identifier, such as the name of the row group, and further such as the sequence number of the row group, etc., is added to the head byte of the row.

And S604, coding the row data in the row group based on the identification code and outputting the coded row data.

Therefore, the image processing method provided by the embodiment of the application can dynamically divide the line of the image data according to the color complexity, and reduce the influence of compression on the image quality of the image. In addition, before image compression, by adding an identifier for marking line grouping to each line of data, the image data can be ensured to be compressed and transmitted accurately according to the line grouping divided by dynamic line division, and a basis is provided for subsequent decoding and displaying.

As an implementation manner of step S103 "adding an identification code representing a row packet in which the row packet is located for the row data", a signaling flowchart of an image processing method disclosed in sixth embodiment of the present application is shown in fig. 11, and the method includes the following steps:

s701, image data is obtained, and the image data comprises a plurality of line data.

S702, dividing the plurality of line data into a plurality of line groups according to the color number of the image data, wherein the color number of the line data in the line group is in direct proportion to the number of the line data in the line group.

And S703, adding an identifier for representing the capacity of the row group in which the row data is positioned to the row data positioned in the first row in the row group.

In the embodiment of the present application, after dividing a plurality of line data of image data into a plurality of line groups, a line group to which each line data belongs may be determined. And adding an identifier for representing the capacity of the row packet in the head byte of the row data positioned in the first row in the row packet.

See the line grouping division diagram shown in fig. 12. The first to fourth lines (i.e., G1 to G4) of the image data belong to a line group 1, the fifth to twelfth lines (i.e., G5 to G12) belong to a line group 2, the thirteenth to twenty fourth lines (G13 to G24) belong to a line group 3, and the twenty fifth to forty fourth lines (G25 to G40) belong to a line group 4 … …. Therefore, an identifier SH1 representing the capacity of the row group 1 is added to the head byte of the first row (G1), an identifier SH2 representing the capacity of the row group 2 is added to the head byte of the fifth row (G5), an identifier SH3 representing the capacity of the row group 3 is added to the head byte of the tenth row (G13), and an identifier SH4 … … representing the capacity of the row group 4 is added to the head byte of the twenty-fifth row (G25), where SH1 is 4, SH2 is 8, SH3 is 12, and SH4 is 16.

And S704, coding the row data in the row group based on the identification code and outputting the coded row data.

Therefore, the image processing method provided by the embodiment of the application can dynamically divide the line of the image data according to the color complexity, and reduce the influence of compression on the image quality of the image. In addition, before image compression, by adding an identifier for representing the capacity of the line group where the line is located to the line data of the first line in the line group, it can be ensured that image data is accurately compressed and transmitted according to the line group divided by dynamic line division, and a basis is provided for subsequent decoding and displaying.

Corresponding to the image processing method, an image processing apparatus according to a seventh embodiment of the present application is shown in fig. 13, and includes:

an image acquisition module 11, configured to acquire image data, where the image data includes a plurality of line data;

a line grouping and dividing module 12, configured to divide the plurality of line data into a plurality of line groups according to the color number of the image data, where the color number of the line data in a line group is in direct proportion to the number of the line data in the line group;

an identification code adding module 13, configured to add an identification code representing the row packet in which the row packet is located for the row data in the row packet;

and the encoding module 14 is used for encoding the line data in the line group based on the identification code and outputting the line data.

It should be noted that please refer to the disclosure part of the above embodiments of the image processing method for the refining function and the expanding function of each module in the image processing apparatus, which are not described herein again.

Corresponding to the image processing method, the application also discloses a computer readable storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps disclosed in the above image processing method embodiment are executed.

Based on the image processing method disclosed by each embodiment, the application also provides an image display method, and the method is applied to a display module. Fig. 14 shows a flowchart of a method for displaying an image according to an eighth embodiment of the present application, where the method includes the following steps:

s801, acquiring an encoding result of image data, wherein the image data comprises a plurality of line data, the encoding result of the image data comprises an encoding result of each line data and an identification code representing a line group where each line data is located, the line group is obtained by dividing the plurality of line data according to the color number of the image data, and the color number of the line data in the line group is in direct proportion to the number of the line data in the line group.

With continued reference to fig. 4. In the embodiment of the present application, after the display module (IC) receives Line data sent by the processing module (AP) through its internal communication interface (MIPI DPHY Rx), the Line data is buffered by a Line Buffer (Line Buffer).

The Controller (CON) can determine the Line data belonging to the same Line group in the Line Buffer (Line Buffer) by acquiring the identification code carried by part or all of the Line data in the encoding result of the image data, thereby controlling the output of the Line Buffer (Line Buffer), and finally outputting the output of the Line Buffer (Line Buffer) to a display screen for display after the output of the Line Buffer (Line Buffer) is decompressed by a Decoder (VESD DSC Decoder).

S802, determining the coding result of the line data belonging to the same line group based on the identification code.

In the embodiment of the application, when the identification code is the identifier for marking the row packet, the identifier is located in the head byte of each row of data during encoding, so that the encoding result of the row of data of the same identifier can be used as the encoding result of the row of data belonging to the same row packet.

And under the condition that the identification code is the identification for representing the capacity of the row group, the identification is positioned in the row data of the first row of the row group during coding, so that the coding result of the continuous row data with preset quantity starting from the coding result of the row data carrying the identification can be used as the coding result of the row data belonging to the same row group, and the preset quantity is the same as the capacity represented by the identification.

It should be noted that, when the controller controls the Line Buffer (Line Buffer) to output the Line data belonging to the same Line group, since counting is required, the controller may adopt a pointer controller, and the pointer controller outputs a preset number of consecutive Line data starting from the Line data carrying the identifier through the pointer control Line Buffer (Line Buffer).

And S803, decoding and displaying the coded result of the line data belonging to the same line group.

The image display method provided by the embodiment of the application can accurately identify the line grouping of the dynamic line division of the image data by the processing module, thereby realizing the accurate decoding display of the image data.

Corresponding to the image display method, a schematic structural diagram of an image display device disclosed in ninth embodiment of the present application is shown in fig. 15, and the image display device includes:

the coded image acquisition module 21 is configured to acquire a coding result of image data, where the image data includes a plurality of line data, the coding result of the image data includes a coding result of each line data and an identification code representing a line group in which each line data is located, the line group is obtained by dividing the plurality of line data according to the color number of the image data, and the color number of the line data in the line group is in direct proportion to the number of the line data in the line group;

a row grouping determination module 22 for determining an encoding result of the row data belonging to the same row group based on the identification code;

and the decoding module 23 is configured to decode and display an encoding result of the line data belonging to the same line group.

It should be noted that please refer to the disclosure of the above embodiments of the image display method for the refinement function and the expansion function of each module in the image display apparatus, which are not described herein again.

Corresponding to the image display method, the application also discloses a computer readable storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps disclosed in the embodiment of the image display method are executed.

The image processing method, the image display method and the related products provided by the present application are described in detail above, and specific examples are applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include or include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. An image processing method, characterized in that the method comprises:

2. The method of claim 1, wherein the dividing the plurality of line data into a plurality of line groups according to the number of colors of the image data comprises:

dividing a plurality of line data of the image data into a plurality of grouping units, wherein the grouping unit is the minimum unit for grouping lines, and the grouping unit is formed by continuous line data with preset quantity;

acquiring the row similarity of the grouping unit, wherein the row similarity of the grouping unit represents the similarity between all rows of data in the grouping unit;

and allocating the grouping unit to be grouped according to the row similarity of the grouping unit.

3. The method according to claim 2, wherein the obtaining of the row similarity of the grouping unit comprises:

calculating the similarity between two adjacent rows of data in the grouping unit;

and taking the average value of the similarity between all the adjacent two rows of data in the grouping unit as the row similarity of the grouping unit.

4. The method of claim 2, wherein the assigning the grouping units into row groups based on the row similarity of the grouping units comprises:

determining a grouping reference set of the grouping unit according to a similarity level of the row similarity of the grouping unit, wherein the similarity level is one of a plurality of preset similarity levels, the grouping reference set comprises a plurality of standard grouping line numbers, and the standard grouping line numbers are more than or equal to the number of row data in the grouping unit;

and distributing row grouping for the grouping unit at least according to a plurality of standard grouping row numbers corresponding to the grouping unit.

5. The method according to claim 4, wherein the reference grouping set corresponding to two adjacent similarity levels of the preset similarity levels includes at least one same standard grouping row number.

6. The method according to claim 4, wherein said assigning row groups to the grouping units at least according to a plurality of standard grouping row numbers corresponding to the grouping units comprises:

detecting whether a row group in an allocable state exists or not, wherein the quantity of the allocated row data of the row group in the allocable state is less than the capacity of the row group;

if the row grouping in the distributable state does not exist, creating a row grouping, wherein the capacity of the created row grouping is one of a plurality of standard grouping row numbers corresponding to the grouping unit;

and distributing the grouping unit into the created row grouping.

7. The method according to claim 6, wherein said assigning row groups to said grouping units at least according to a plurality of standard grouping row numbers corresponding to said grouping units, further comprises:

and if the row group in the distributable state exists and the difference between the capacity of the row group and the quantity of the distributed row data is not less than the quantity of the row data in the grouping unit, distributing the grouping unit into the row group in the distributable state.

8. The method according to claim 6, wherein said assigning row groups to said grouping units at least according to a plurality of standard grouping row numbers corresponding to said grouping units, further comprises:

if the row grouping in the distributable state exists and the difference between the capacity of the row grouping and the quantity of the distributed row data is smaller than the quantity of the row data in the grouping unit, acquiring the row similarity of the grouping unit in the row grouping in the distributable state;

comparing the row similarity of the grouping unit with the row similarity of the grouping unit in the row grouping in the assignable state to determine a difference degree between the grouping unit and the row grouping in the assignable state, wherein the difference degree is in direct proportion to a difference amount of the row similarity;

under the condition that the difference degree is smaller than a preset difference degree threshold value, increasing the capacity of the row group in the distributable state to distribute the grouping unit to the row group with increased capacity;

and when the difference degree is larger than or equal to a preset difference degree threshold value, reducing the capacity of the row group in the distributable state, and returning to the step of detecting whether the row group in the distributable state exists or not.

9. An image processing apparatus, characterized in that the apparatus comprises:

10. An image display method, characterized in that the method comprises:

acquiring an encoding result of image data, wherein the image data comprises a plurality of line data, the encoding result of the image data comprises an encoding result of each line data and an identification code representing a line group where each line data is located, the line group is obtained by dividing the plurality of line data according to the color number of the image data, and the color number of the line data in the line group is in direct proportion to the number of the line data in the line group;

11. An image display apparatus, characterized in that the apparatus comprises: