KR102726930B1 - Variable bit-precision multiplier-accumulator structure for deep neural network operation - Google Patents

Abstract

The present invention relates to a variable bit-precision multiply-accumulator, which receives first data, second data, and third data of n bits, which is the maximum data bits that can be processed, and adds third data to the product of the first data and the second data, the multiply-accumulator including: an input processing unit which divides the first data, the second data, and the third data according to bit-precision information of the first data, the second data, and the third data; and a multiply-accumulator which determines the number of multiplication operations according to the number of bits of the first data, the second data, and the third data divided by the input processing unit and performs a multiplication operation on the first data and the second data.

Description

{Variable bit-precision multiplier-accumulator structure for deep neural network operation}

The present invention relates to a variable bit-precision multiplier-accumulator structure for deep neural network operations, and more particularly, to a variable bit accumulator structure capable of varying the precision and the accumulated result value.

In general, deep neural networks consist of numerous operations, but each operation has a different effect on the accuracy of the neural network results. Some operations have little effect on accuracy even when performed with low bit precision, while some operations have a significant decrease in accuracy if not performed with high bit precision.

That is, there is a difference in the bit-precision of the operation required for each neural network parameter. In addition, there is a large difference in the range of data to be processed and the bit-precision of the operation required depending on the neural network model such as CNN (Convolutional Neural Network) and NLP (Natural Language Processing), so if the bit-precision can be varied, it is possible to perform neural network operations that are more efficient (reduced amount of operation for the same accuracy) than using the same bit-precision for all operations.

Since deep neural network operations consist of repeated multiply-accumulate operations between input data and neural network weight values, a multiply-accumulator that supports variable bit precision is essential to cope with such various bit precisions. Since most deep neural network operation hardware has the characteristic of processing multiple multiply-accumulators by configuring them in an array such as a systolic array, it is possible to maximize hardware operation efficiency even if the bit precision is varied through a variable multiply-accumulator structure that considers such array operations.

Deep neural network operations have shown high accuracy in various fields of application through numerous operations. However, since they have millions to billions of parameters and computational amounts, there are problems in hardware implementation that consume a lot of power and energy and have long processing times. The most effective way to reduce parameter storage space and computational amounts is to reduce the bit precision of input data. If the bit precision of input data is reduced by half, the power and energy of hardware consumed for the same operation can be reduced by 1/4. In particular, in the case of inference operations, since they often operate on mobile or edge devices and require lower bit precision than learning operations, various methods that can significantly reduce bit precision are becoming popular.

If you lower the bit-precision without significantly reducing the accuracy, the bit-precision of the neural network parameters will vary for each neural network model and each layer within the neural network. For example, in the case of the VGG-16 network with high redundancy, the bit-precision can be lowered further than the MobileNet, which is designed to be relatively compact, and even within the VGG-16 model, there are layers that can further lower the bit-precision and layers that cannot.

Since deep neural network operations consist of repeated multiply-accumulate operations between input data and neural network weight values, a multiplier-accumulator that supports variable bit precision is essential to cope with these various bit precisions.

Although it is possible to change the bit precision by filling the unused bits with 0 in a multiplier-accumulator that supports only a fixed bit precision, in this case, the hardware utilization is very low in low bit precision operations and it is inefficient in terms of power efficiency due to leakage power, etc. In order to have similar hardware efficiency (e.g. processing speed per area, power efficiency) to the maximum bit precision operations even at low bit precision, a new multiplier-accumulator structure that enables efficient change of bit precision is required.

Additionally, most deep neural network computational hardware has the characteristic of processing multiple multipliers-accumulators by configuring them in an array such as a systolic array.

At this time, it is also possible to perform more filter operations at once as the bit precision changes. For example, if the multiplier-accumulator processes multiple multiplication-accumulation operations but accumulates them all and outputs only one multiplication-accumulation operation result, the efficiency in terms of the entire array operation may decrease. Therefore, a new multiplier-accumulator structure is required that can maintain a high hardware utilization rate of each single multiplication-accumulator while increasing the efficiency of the entire deep neural network hardware composed of an array of multiplication-accumulators through parallel processing that increases the number of multiplications-accumulations to be processed as the bit precision decreases by developing a multiplier-accumulator that can adjust the number of operation results considering array processing while varying the bit precision.

The problem that the present invention seeks to solve in consideration of the above-mentioned needs is to provide a variable bit-precision multiplier-accumulator structure that can be commonly applied to various deep neural network operations requiring various bit-precisions.

In particular, another problem that the present invention seeks to solve is to provide a variable bit-precision multiplier-accumulator structure that can be applied to all deep neural network computational processing devices (mobile, edge, server) that optimize bit-precision to improve hardware power efficiency.

In order to solve the technical problem described above, the variable bit-precision multiply-accumulator of the present invention receives first data, second data, and third data of n bits, which is the maximum data bits that can be processed, and adds the third data to the product of the first data and the second data, the multiply-accumulator includes: an input processing unit which divides the first data, the second data, and the third data according to bit-precision information of the first data, the second data, and the third data; and a multiply-accumulator which determines the number of multiplication operations according to the number of bits of the first data, the second data, and the third data divided by the input processing unit and performs a multiplication operation on the first data and the second data.

In an embodiment of the present invention, the divided data may be n-bit, n/2, or n/4-bit data.

In an embodiment of the present invention, the result of the multiplication of the first data and the second data is determined by the number of bits of the divided data, and the third data can be accumulated for each of the multiplication results.

In an embodiment of the present invention, the result of multiplication of the first data and the second data is determined by the number of bits of the divided data, and after adding all the results of multiplication, n-bit third data can be accumulated, or after adding each pair of results of multiplication, n/2-bit third data can be accumulated for each result.

The present invention has the effect of improving versatility by allowing the bit precision and multiplication-accumulation to be varied, thereby enabling application to various deep neural network models.

In addition, the present invention has the effect of improving the efficiency of an array processing method based on a multiplication-accumulation operation method that occurs due to a change in bit precision, thereby improving hardware power efficiency.

FIG. 1 is a block diagram of a variable bit-precision multiplier-accumulator according to a preferred embodiment of the present invention.
Figure 2 is an operational description diagram to explain the change in the number of multiplication operations according to bit precision.
Figure 3 shows an example of a cumulative variable.
FIG. 4 is a configuration diagram of a hardware accelerator that processes actual deep neural network operations using the variable bit-precision multiplier-accumulator of the present invention as a basic operation unit.
FIG. 5 is an example diagram of the number of filters processed by the accelerator according to the variation in the number of operations of the multiplier-accumulator of the present invention.
Figure 6 is a graph comparing the accuracy versus processing time (latency) of a deep neural network model to which a variable bit-precision multiplier-accumulator of the present invention and a conventional multiplier-accumulator are applied.

In order to fully understand the configuration and effect of the present invention, preferred embodiments of the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various forms and can have various changes. However, the description of the embodiments is provided so that the disclosure of the present invention is complete, and so that a person having ordinary knowledge in the technical field to which the present invention belongs can fully understand the scope of the invention. In the accompanying drawings, the components are illustrated with their actual sizes enlarged for convenience of explanation, and the ratio of each component may be exaggerated or reduced.

The terms "first", "second", etc. may be used to describe various components, but the components should not be limited by the terms. The terms may only be used to distinguish one component from another. For example, without departing from the scope of the present invention, the "first component" may be referred to as the "second component," and similarly, the "second component" may also be referred to as the "first component." In addition, singular expressions include plural expressions unless the context clearly indicates otherwise. The terms used in the embodiments of the present invention may be interpreted as having a meaning commonly known to a person of ordinary skill in the art, unless otherwise defined.

Hereinafter, a variable bit-precision multiplier-accumulator structure according to a preferred embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram of a variable bit-precision multiplier-accumulator according to a preferred embodiment of the present invention.

Referring to FIG. 1, the variable bit-precision multiply-accumulator (1) of the present invention is configured to include an input register (10) which receives data (A, B, C) and a mode signal (MODE), an input processing unit (20) which checks the bit information of the data (A, B, C) input through the input register (10), divides and combines the data by bit, and outputs it, a multiplication-accumulation unit (40) which receives the processing result of the input processing unit (20) through the first pipeline register (30) and performs multiplication-accumulation, but performs it variably according to the bit precision, and an output processing unit (60) which receives the execution result of the multiplication-accumulation unit (40) through the second pipeline register (50), converts it into an output data format, and outputs it through the output register (70).

Hereinafter, the configuration and operation of the variable bit-precision multiplier-accumulator of the present invention configured as described above will be described in more detail.

First, the input processing unit (20) receives data (A, B, C) through the input register (10).

In the drawing, data (A, B, C) are all depicted as 16 bits, but this can be changed to 32, 64 bits, etc., as needed. That is, the embodiment of the present invention is described based on 16 bits, but this can be changed as needed.

At this time, 16-bit data (A, B, C) should be understood as the maximum bit. That is, data (A, B, C) is at most 16 bits, and 8-bit or 4-bit data can be input.

This means that the bit precision is variable, and it can be applied to deep neural network models with different precision.

The mode signal (MODE) can contain information about the number of bits that are the mode of the input data.

The input processing unit (20) processes data (A, B, C) into a format required for operation in the multiplication-accumulation unit (40) according to the number of bits of the input data and outputs it.

The processing at this time will be explained in more detail later.

Data (A, B, C) processed in the above input processing unit (20) is provided to the multiplication-accumulation unit (40) through the first pipeline register (30).

The multiplication-accumulation unit (40) performs an operation of accumulating data (C) by multiplying two data (A, B), but the number of multiplication-accumulation operations varies depending on the number of bits of data (A, B, C).

Figure 2 is an operational description diagram to explain the change in the number of multiplication operations according to bit precision.

Figure 2 (a) is an example of a multiplication operation based on input of n (e.g. 16 bit) bit data (A, B), and the operation is completed by performing one multiplication operation (AXB).

Figure 2 (b) is an example of a multiplication operation based on input data (A0, A1, B0, B1) of n/2 (e.g. 8 bits), and performs two multiplication operations (A0XB0, A1XB1).

In this way, for n/4 bits, four multiplication operations must be performed as in (c) of Fig. 2.

In this way, the multiplication-accumulation unit (40) varies the number of multiplication operations depending on the number of bits of the input data.

At this time, the divided data (A0, A1, A2, A3, B0, B1, B2, B3) are provided by dividing and processing the original data (A, B) in the input processing unit (20).

In addition, the multiplication-accumulation unit (40) performs the operation of AXB+C, which can also change the accumulation of C.

Figure 3 shows an example of an accumulation variable of data C.

In (a) of Fig. 3, for example, four multiplication operations are performed on n/4 bit data (A0, A1, A2, A3, B0, B1, B2, B3), and n/4 bit data (C0, C1, C2, C3) is added to each of the four multiplication operation results.

In addition, as shown in (b) of Fig. 3, n/2 bit data (C0, C1) can be accumulated by adding two of the four multiplication operation results to each other.

That is, (b) of Fig. 3 can obtain two multiplication-accumulation results.

As another example, as shown in (c) of Fig. 3, a method can be exemplified in which the results of four multiplication operations on n/4-bit data (A0, A1, A2, A3, B0, B1, B2, B3) are added together and n-bit accumulation target data (C) is added.

In this case, the result of the operation is one.

A transformation of this structure can be implemented through a modification of the carry save adder.

In this way, the result of the multiplication-accumulation operation is converted into a desired data format through the output processing unit (60) and output through the output register (70).

FIG. 4 is a configuration diagram of a hardware accelerator (100) that processes actual deep neural network operations using the variable bit-precision multiplier-accumulator of the present invention as a basic operation unit.

Referring to FIG. 4, a MAC array (130) is implemented that receives data from an activation buffer (120), receives weights from a weight input unit (110), and performs multiplication-accumulation.

The MAC array (130) can two-dimensionally arrange the multiplier-accumulator (1) of the present invention. For example, the MAC array (130) can be implemented by arranging 256 multiplier-accumulators (1) in a horizontal direction and 256 in a vertical direction.

In the accelerator structure, data is accumulated in columns of an array, and each filter is processed by a final accumulation operation in the accumulation unit (140).

As in the example above, when the MAC array (130) has an array structure of 256x256, it has 256 columns, so a maximum of 256 filter operations can be processed.

Figure 5 is an example diagram of the number of filters processed by the accelerator (100) according to the variation in the number of operations of the multiplier-accumulator (1) of the present invention.

Assuming that the above MAC array (130) includes an NxN array of the present invention's multiply-accumulators (1), when a single multiply-accumulator (1) in the accelerator (100) processes n/4 bits, if only one multiply-accumulate operation can be processed, only N filter operations can be processed, whereas if the structure can be changed to process two or four multiply-accumulate operations, the array structure can process 2N or 4N filter operations simultaneously.

It is not always advantageous to process a large number of filters, but there are cases where the number of filters processed is better depending on the characteristics of the layers of the deep neural network. In such cases, if it is advantageous to process many filters simultaneously and process a small number of filter operations quickly, a structure that can be changed to process a small number of filters (instead, process multiple operations of the same filter) is much more efficient than a structure that only performs fixed filter operations.

Figure 5 (a) is an example diagram for processing 4N filters through multiplication-accumulation operations on n/4 bit data, and Figures 5 (b) and 5 (c) are example diagrams for configurations for processing 2N and N filters, respectively, through multiplication-accumulation operations on n/2 and n bit data, respectively.

Figure 6 is a graph comparing the accuracy versus processing time (latency) of a deep neural network model to which a variable bit-precision multiplier-accumulator of the present invention and a conventional multiplier-accumulator are applied.

MobileNet was used as a deep neural network model, and it was confirmed that the same accuracy was achieved and processing time could be shortened when quantized using the variable bit-precision multiplier-accumulator of the present invention.

That is, it shows that applying different bit-precisions to each parameter within a single neural network model is more efficient than applying a fixed bit-precision to all parameters.

In this way, the present invention can process a multiplier-accumulator in a variable manner according to the bit accuracy of data, and therefore can be applied to various deep neural network models, and can also improve the processing speed.

Although the embodiments according to the present invention have been described above, they are merely exemplary, and those with ordinary skill in the art will understand that various modifications and equivalent embodiments are possible. Accordingly, the true technical protection scope of the present invention should be determined by the following claims.

10: Input register 20: Input processing unit
30:1st pipeline register 40:Multiply-accumulate unit
50:2nd pipeline register 60:Output processing unit
70: Output register

Claims (7)

In a multiplier-accumulator that receives first data, second data, and third data of n bits, which is the maximum data bit that can be processed, and adds the third data to the product of the first data and the second data,
An input processing unit that divides the first data, the second data, and the third data according to bit-precision information of the first data, the second data, and the third data;
It includes a multiplication-accumulation unit that determines the number of multiplication operations according to the number of bits of the first data, second data, and third data divided in the above input processing unit and performs a multiplication operation on the first data and the second data.
The result of multiplying the first and second data above is,
It is determined by the number of bits of the above divided data,
After adding all the multiplication results, the n-bit third data is accumulated for each multiplication result, or
A variable bit-precision multiplier-accumulator characterized by adding the multiplication results in pairs and then accumulating n/2 bits of third data for each result. In the first paragraph,
The above divided data is,
A variable bit-precision multiplier-accumulator characterized by n-bit, n/2, or n/4-bit data. delete delete An accelerator comprising a MAC array in which variable bit-precision multiplier-accumulators of the first clause are arranged in a horizontal and vertical matrix. In paragraph 5,
The above MAC array is an accelerator that quantizes and processes different numbers of filters depending on the number of bits of input data. In Article 6,
An accelerator further including an accumulation unit that processes a filter by finally accumulating the operation results accumulated in the vertical direction of the above MAC array.

Images