JP2022183833A - Neural network circuit and neural network operation method - Google Patents

Abstract

To provide a high-performance neural network circuit that can be embedded in an embedded device such as an IoT device.SOLUTION: There is provided a neural network apparatus including operation means for a plurality of layers for performing operations on input data. The plurality of layers include at least a first layer for performing a first operation using at least a first parameter and a second layer for performing a second operation that differs from the first operation. In the second operation, the operation is performed based on the first parameter and a second parameter that differs from the first parameter.SELECTED DRAWING: Figure 10

Description

The present invention relates to a neural network circuit and a neural network operation method.

In recent years, a convolutional neural network (CNN) has been used as a model for image recognition and the like. A convolutional neural network has a multilayer structure having convolution layers and pooling layers, and requires a large number of operations such as convolution operations. Various calculation techniques have been devised for speeding up calculations by convolutional neural networks (Patent Document 1).
In addition, a method using an exclusive OR circuit and a bit shift has been devised as a technique for high-speed processing of a convolution operation with few resources in an embedded device (Patent Document 2).

JP 2018-077829 A Japanese Patent Application Laid-Open No. 2020-060968

On the other hand, it is desired to realize image recognition using a convolutional neural network even in embedded devices such as IoT devices. In an embedded device, it is difficult to incorporate the large-scale dedicated circuit described in Patent Document 1 and the like. Further, in embedded devices with limited hardware resources such as CPUs and memories, it is required to use hardware resources more efficiently to improve computational performance.

In view of the above circumstances, an object of the present invention is to provide a high-performance neural network circuit and a neural network operation method that can be incorporated in an embedded device such as an IoT device.

In order to solve the above problems, the present invention proposes the following means.
A neural network device according to a first aspect of the present invention is a neural network device comprising computing means for a plurality of layers that perform computations on input data, wherein the plurality of layers have at least a first parameter. and a second layer that performs a second operation different from the first operation, wherein the second operation is the first parameter and the first The calculation is performed based on a second parameter different from the parameter of .

The neural network circuit and neural network operation method of the present invention can be incorporated into embedded equipment such as IoT equipment and have high performance.

FIG. 2 illustrates a convolutional neural network; FIG. 4 is a diagram for explaining convolution operations performed by a convolution layer; FIG. 4 is a diagram for explaining expansion of data in a convolution operation; 1 is a diagram showing the overall configuration of a neural network circuit according to a first embodiment; FIG. It is a timing chart which shows the operation example of the same neural network circuit. 4 is a timing chart showing another operation example of the same neural network circuit; 3 is an internal block diagram of the DMAC of the same neural network circuit; FIG. 4 is a state transition diagram of a control circuit of the same DMAC; FIG. 4 is an internal block diagram of a convolution operation circuit of the same neural network circuit; FIG. FIG. 4 is an internal block diagram of a multiplier of the convolution arithmetic circuit; 3 is an internal block diagram of a sum-of-products operation unit of the same multiplier; FIG. FIG. 4 is an internal block diagram of an accumulator circuit of the same convolution arithmetic circuit; It is an internal block diagram of the accumulator unit of the same accumulator circuit. 4 is an internal block diagram of a quantization arithmetic circuit of the same neural network circuit; FIG. 3 is an internal block diagram of a vector operation circuit and a quantization circuit of the same quantization operation circuit; FIG. 3 is a block diagram of an arithmetic unit; FIG. 4 is an internal block diagram of a vector quantization unit of the same quantization circuit; FIG.

(First embodiment)
A first embodiment of the present invention will be described with reference to FIGS. 1 to 17. FIG.
FIG. 1 is a diagram showing a convolutional neural network 200 (hereinafter referred to as "CNN 200"). The computations performed by the neural network circuit 100 (hereinafter referred to as "NN circuit 100") according to the first embodiment are at least part of the learned CNN 200 used during inference.

[CNN200]
The CNN 200 is a multi-layered network including a convolution layer 210 that performs convolution operations, a quantization operation layer 220 that performs quantization operations, and an output layer 230 . In at least part of CNN 200, convolutional layers 210 and quantization operation layers 220 are interleaved. CNN200 is a model widely used for image recognition and moving image recognition. The CNN 200 may further have layers with other functions, such as fully connected layers.

FIG. 2 is a diagram for explaining the convolution operation performed by the convolution layer 210. As shown in FIG.
The convolution layer 210 performs a convolution operation on input data a using weight w. The convolution layer 210 performs a sum-of-products operation with input data a and weight w as inputs.

Input data a (also called activation data or feature map) to the convolutional layer 210 is multidimensional data such as image data. In this embodiment, the input data a is a three-dimensional tensor consisting of elements (x, y, c). The convolution layer 210 of the CNN 200 performs a convolution operation on low-bit input data a. In this embodiment, the elements of the input data a are 2-bit unsigned integers (0, 1, 2, 3). Elements of input data a may be, for example, 4-bit or 8-bit unsigned integers.

If the input data input to the CNN 200 has a different format from the input data a to the convolutional layer 210, such as a 32-bit floating point type, the CNN 200 has an input layer that performs type conversion and quantization before the convolutional layer 210. You may have more.

The weights w (also called filters, kernels) of the convolutional layer 210 are multidimensional data whose elements are learnable parameters. In this embodiment, the weight w is a 4-dimensional tensor consisting of elements (i,j,c,d). The weight w has d three-dimensional tensors (hereinafter referred to as “weight wo”) each having elements (i, j, c). The weight w in the learned CNN 200 is learned data. Convolutional layer 210 of CNN 200 performs a convolution operation using low-bit weights w. In this embodiment, the elements of the weight w are 1-bit signed integers (0,1), where the value '0' represents +1 and the value '1' represents -1.

The convolution layer 210 performs the convolution operation shown in Equation 1 and outputs output data f. In Equation 1, s indicates stride. The area indicated by the dotted line in FIG. 2 indicates one of the areas ao (hereinafter referred to as “applied area ao”) to which the weight wo is applied to the input data a. An element of the application area ao is represented by a(x+i, y+j, c).

The quantization operation layer 220 performs quantization and the like on the convolution operation output from the convolution layer 210 . The quantization operation layer 220 has a pooling layer 221 , a batch normalization layer 222 , an activation function layer 223 and a quantization layer 224 .

The pooling layer 221 performs operations such as average pooling (equation 2) and MAX pooling (equation 3) on the output data f of the convolutional operation output by the convolutional layer 210 to compress the output data f of the convolutional layer 210. do. In Equations 2 and 3, u indicates the input tensor, v indicates the output tensor, and T indicates the size of the pooling region. In Equation 3, max is a function that outputs the maximum value of u for combinations of i and j contained in T.

The Batch Normalization layer 222 normalizes the data distribution of the output data of the quantization operation layer 220 and the pooling layer 221 by, for example, the operation shown in Equation 4. In Equation 4, u denotes the input tensor, v the output tensor, α the scale, and β the bias. In the trained CNN 200, α and β are trained constant vectors.

The activation function layer 223 computes an activation function such as ReLU (Formula 5) on the outputs of the quantization computation layer 220 , the pooling layer 221 and the batch normalization layer 222 . In Equation 5, u is the input tensor and v is the output tensor. In Expression 5, max is a function that outputs the largest numerical value among the arguments.

The quantization layer 224 quantizes the outputs of the pooling layer 221 and the activation function layer 223 based on the quantization parameter, as shown in Equation 6, for example. The quantization shown in Equation 6 reduces the input tensor u to 2 bits. In Equation 6, q(c) is the vector of quantization parameters. In the trained CNN 200, q(c) is a trained constant vector. The inequality “≦” in Equation 6 may be “<”.

The output layer 230 is a layer that outputs the results of the CNN 200 using an identity function, a softmax function, or the like. A layer preceding the output layer 230 may be the convolution layer 210 or the quantization operation layer 220 .

In the CNN 200, the quantized output data of the quantization layer 224 is input to the convolution layer 210, so the convolution operation load of the convolution layer 210 is small compared to other convolutional neural networks that do not perform quantization. .

[Division of convolution operation]
The NN circuit 100 divides the input data for the convolution operation (Equation 1) of the convolution layer 210 into partial tensors and performs the operation. The method of division into partial tensors and the number of divisions are not particularly limited. A partial tensor is formed, for example, by splitting the input data a(x+i, y+j, c) into a(x+i, y+j, co). Note that the NN circuit 100 can also perform computation without dividing the input data for the convolution computation (equation 1) of the convolution layer 210 .

In the input data division of the convolution operation, the variable c in Equation 1 is divided into blocks of size Bc as shown in Equation 7. Also, the variable d in Equation 1 is divided into blocks of size Bd, as shown in Equation 8. In Equation 7, co is the offset and ci is the index from 0 to (Bc-1). In Equation 8, do is the offset and di is the index from 0 to (Bd-1). Note that the size Bc and the size Bd may be the same.

Input data a(x+i, y+j, c) in Equation 1 is divided by size Bc and represented by divided input data a(x+i, y+j, co). In the following description, the divided input data a is also referred to as "divided input data a".

The weight w(i,j,c,d) in Equation 1 is divided by the sizes Bc and Bd and denoted by the divided weight w(i,j,co,do). In the following description, the divided weight w is also referred to as "divided weight w".

The output data f(x, y, do) divided by the size Bd is obtained by Equation (9). By combining the divided output data f(x, y, do), the final output data f(x, y, d) can be calculated.

[Development of convolution operation data]
The NN circuit 100 develops the input data a and the weight w in the convolution operation of the convolution layer 210 and performs the convolution operation.

FIG. 3 is a diagram for explaining expansion of data in a convolution operation.
Divided input data a(x+i, y+j, co) is developed into vector data having Bc elements. Elements of the divided input data a are indexed by ci (0≤ci<Bc). In the following description, divided input data a developed into vector data for each i and j is also referred to as "input vector A". Input vector A has elements from divided input data a(x+i, y+j, co×Bc) to divided input data a(x+i, y+j, co×Bc+(Bc−1)).

The division weight w(i, j, co, do) is developed into matrix data having Bc×Bd elements. The elements of the division weight w developed into matrix data are indexed by ci and di (0≦di<Bd). In the following description, the divided weight w developed into matrix data for each i and j is also referred to as "weight matrix W". The weight matrix W includes division weights w(i, j, co×Bc, do×Bd) to division weights w(i, j, co×Bc+(Bc−1), do×Bd+(Bd−1)). element.

By multiplying the input vector A and the weight matrix W, vector data is calculated. Output data f(x, y, do) can be obtained by shaping the vector data calculated for each of i, j, and co into a three-dimensional tensor. By developing such data, the convolution operation of the convolution layer 210 can be performed by multiplying the vector data and the matrix data.

[NN circuit 100]
FIG. 4 is a diagram showing the overall configuration of the NN circuit 100 according to this embodiment.
The NN circuit 100 includes a first memory 1, a second memory 2, a DMA controller 3 (hereinafter also referred to as "DMAC 3"), a convolution operation circuit 4, a quantization operation circuit 5, and a controller 6. . NN circuit 100 is characterized in that convolution operation circuit 4 and quantization operation circuit 5 are formed in a loop via first memory 1 and second memory 2 .

The first memory 1 is a rewritable memory such as a volatile memory such as an SRAM (Static RAM). Data is written to and read from the first memory 1 via the DMAC 3 and the controller 6 . The first memory 1 is connected to the input port of the convolution operation circuit 4 , and the convolution operation circuit 4 can read data from the first memory 1 . The first memory 1 is also connected to the output port of the quantization arithmetic circuit 5 , and the quantization arithmetic circuit 5 can write data to the first memory 1 . The external host CPU can input/output data to/from the NN circuit 100 by writing data to and reading data from the first memory 1 .

The second memory 2 is, for example, a rewritable memory such as a volatile memory such as an SRAM (Static RAM). Data is written to and read from the second memory 2 via the DMAC 3 and the controller 6 . The second memory 2 is connected to the input port of the quantization arithmetic circuit 5 , and the quantization arithmetic circuit 5 can read data from the second memory 2 . The second memory 2 is also connected to the output port of the convolution circuit 4 , and the convolution circuit 4 can write data to the second memory 2 . The external host CPU can input/output data to/from the NN circuit 100 by writing data to or reading data from the second memory 2 .

The DMAC 3 is connected to the external bus EB and performs data transfer between an external memory such as a DRAM and the first memory 1 . The DMAC 3 also transfers data between an external memory such as a DRAM and the second memory 2 . The DMAC 3 also transfers data between an external memory such as a DRAM and the convolution circuit 4 . The DMAC 3 also transfers data between an external memory such as a DRAM and the quantization arithmetic circuit 5 .

The convolution operation circuit 4 is a circuit that performs convolution operation in the convolution layer 210 of the trained CNN 200 . The convolution operation circuit 4 reads the input data a stored in the first memory 1 and performs a convolution operation on the input data a. The convolution operation circuit 4 writes output data f of the convolution operation (hereinafter also referred to as “convolution operation output data”) to the second memory 2 .

The quantization operation circuit 5 is a circuit that performs at least part of the quantization operation in the quantization operation layer 220 of the trained CNN 200 . The quantization operation circuit 5 reads the output data f of the convolution operation stored in the second memory 2, and performs quantization operations (pooling, batch normalization, activation function, and quantization) on the output data f of the convolution operation. calculation including at least quantization). The quantization operation circuit 5 writes the output data of the quantization operation (hereinafter also referred to as “quantization operation output data”) to the first memory 1 .

The controller 6 is connected to the external bus EB and operates as a slave of an external host CPU. The controller 6 has registers 61 including parameter registers and status registers. A parameter register is a register that controls the operation of the NN circuit 100 . The status register is a register that indicates the status of the NN circuit 100 including the semaphore S. FIG. An external host CPU can access the register 61 via the controller 6 .

Controller 6 is connected to first memory 1, second memory 2, DMAC 3, convolution circuit 4, and quantization circuit 5 via internal bus IB. An external host CPU can access each block via the controller 6 . For example, the external host CPU can issue commands to the DMAC 3, the convolution circuit 4, and the quantization circuit 5 via the controller 6. FIG. Also, the DMAC 3, the convolution operation circuit 4, and the quantization operation circuit 5 can update the status register (including the semaphore S) of the controller 6 via the internal bus IB. The status register (including the semaphore S) may be configured to be updated via dedicated wiring connected to the DMAC 3, the convolution operation circuit 4, and the quantization operation circuit 5. FIG.

Since the NN circuit 100 has the first memory 1, the second memory 2, etc., it is possible to reduce the number of data transfers of overlapping data in data transfer by the DMAC 3 from an external memory such as a DRAM. As a result, power consumption caused by memory access can be greatly reduced.

[Operation example 1 of the NN circuit 100]
FIG. 5 is a timing chart showing an operation example of the NN circuit 100. FIG.
The DMAC 3 stores the layer 1 input data a in the first memory 1 . The DMAC 3 may divide the input data a of the layer 1 according to the order of the convolution operation performed by the convolution operation circuit 4 and transfer the divided data to the first memory 1 .

The convolution operation circuit 4 reads the layer 1 input data a stored in the first memory 1 . The convolution operation circuit 4 performs the layer 1 convolution operation shown in FIG. 1 on the layer 1 input data a. The output data f of the layer 1 convolution operation is stored in the second memory 2 .

The quantization arithmetic circuit 5 reads the layer 1 output data f stored in the second memory 2 . A quantization operation circuit 5 performs a layer 2 quantization operation on layer 1 output data f. The output data of the layer 2 quantization operation is stored in the first memory 1 .

The convolution operation circuit 4 reads the output data of the layer 2 quantization operation stored in the first memory 1 . The convolution operation circuit 4 performs a layer 3 convolution operation using the output data of the layer 2 quantization operation as input data a. The output data f of the layer 3 convolution operation is stored in the second memory 2 .

The convolution operation circuit 4 reads the output data of the quantization operation of the layer 2M-2 (M is a natural number) stored in the first memory 1. FIG. The convolution operation circuit 4 performs the convolution operation of the layer 2M-1 using the output data of the quantization operation of the layer 2M-2 as the input data a. The output data f of the layer 2M-1 convolution operation is stored in the second memory 2. FIG.

The quantization arithmetic circuit 5 reads the layer 2M-1 output data f stored in the second memory 2 . The quantization operation circuit 5 performs a layer 2M quantization operation on the output data f of the 2M-1 layer. The output data of the layer 2M quantization operation are stored in the first memory 1 .

The convolution operation circuit 4 reads the output data of the layer 2M quantization operation stored in the first memory 1 . The convolution operation circuit 4 performs a layer 2M+1 convolution operation using the output data of the layer 2M quantization operation as input data a. The output data f of the layer 2M+1 convolution operation are stored in the second memory 2 .

The convolution calculation circuit 4 and the quantization calculation circuit 5 alternately perform calculations to advance the calculation of the CNN 200 shown in FIG. In the NN circuit 100, the convolution operation circuit 4 performs the convolution operation of the layer 2M-1 and the layer 2M+1 by time division. In the NN circuit 100, the quantization operation circuit 5 performs the layer 2M-2 convolution operation and the layer 2M by time division. Therefore, the NN circuit 100 has a significantly smaller circuit scale than a case where separate convolution operation circuits 4 and quantization operation circuits 5 are implemented for each layer.

The NN circuit 100 performs computation of the CNN 200, which has a multi-layered structure, by means of circuits formed in loops. The NN circuit 100 can efficiently use hardware resources due to its looped circuit configuration. Since the NN circuit 100 forms a circuit in a loop, the parameters in the convolution operation circuit 4 and the quantization operation circuit 5 that change in each layer are updated as appropriate.

If the operations of CNN 200 include operations that cannot be performed by NN circuit 100, NN circuit 100 transfers intermediate data to an external computing device such as an external host CPU. After the external arithmetic device performs arithmetic on the intermediate data, the arithmetic result by the external arithmetic device is input to the first memory 1 and the second memory 2 . The NN circuit 100 restarts the operation on the operation result by the external operation device.

[Operation example 2 of the NN circuit 100]
FIG. 6 is a timing chart showing another operation example of the NN circuit 100. FIG.
The NN circuit 100 may divide the input data a into partial tensors and perform operations on the partial tensors by time division. The method of division into partial tensors and the number of divisions are not particularly limited.

FIG. 6 shows an operation example when the input data a is decomposed into two partial tensors. Let the decomposed partial tensors be “first partial tensor a ₁ ” and “second partial tensor a ₂ ”. For example, the convolution operation of layer 2M-1 is the convolution operation corresponding to the first partial tensor a ₁ (denoted as “layer 2M-1 (a ₁ )” in FIG. 6) and the convolution operation corresponding to the second partial tensor a ₂ Convolution operation (denoted as “Layer 2M-1 (a ₂ )” in FIG. 6).

The convolution and quantization operations corresponding to the first partial tensor a ₁ and the convolution and quantization operations corresponding to the second partial tensor a ₂ can be performed independently, as shown in FIG.

The convolution operation circuit 4 performs a layer 2M-1 convolution operation (operation indicated by layer 2M-1 (a ₁ ) in FIG. 6) corresponding to the first partial tensor a ₁ . After that, the convolution operation circuit 4 performs a layer 2M-1 convolution operation (operation indicated by layer 2M-1 (a ₂ ) in FIG. 6) corresponding to the second partial tensor a ₂ . The quantization operation circuit 5 also performs a layer 2M quantization operation (operation indicated by layer 2M (a ₁ ) in FIG. 6) corresponding to the first partial tensor a ₁ . Thus, the NN circuit 100 can perform the layer 2M-1 convolution operation corresponding to the second partial tensor a ₂ and the layer 2M quantization operation corresponding to the first partial tensor a ₁ in parallel.

Next, the convolution operation circuit 4 performs a layer 2M+1 convolution operation (operation indicated by layer 2M+1 (a ₁ ) in FIG. 6) corresponding to the first partial tensor a ₁ . The quantization operation circuit 5 also performs _{a layer 2M quantization operation (operation indicated by layer 2M (a 2 )} in FIG. 6) corresponding to the second partial tensor a2. Thus, the NN circuit 100 can perform the layer 2M+1 convolution operation corresponding to the first partial tensor a ₁ and the layer 2M quantization operation corresponding to the second partial tensor a ₂ in parallel.

By dividing the input data a into partial tensors, the NN circuit 100 can operate the convolution operation circuit 4 and the quantization operation circuit 5 in parallel. As a result, the waiting time of the convolution operation circuit 4 and the quantization operation circuit 5 is reduced, and the operation processing efficiency of the NN circuit 100 is improved. Although the division number is 2 in the operation example shown in FIG. 6, the NN circuit 100 can operate the convolution operation circuit 4 and the quantization operation circuit 5 in parallel when the division number is greater than 2. can.

As a method for calculating a partial tensor, an example (Method 1) is shown in which a partial tensor in the same layer is calculated by the convolution calculation circuit 4 or the quantization calculation circuit 5, and then a partial tensor in the next layer is calculated. However, the calculation method is not limited to this. The NN circuit 100 may compute the remaining partial tensors after computing some of the partial tensors in multiple layers (Method 2). Also, the NN circuit 100 may combine method 1 and method 2 to compute a partial tensor. In addition, parallel operations are not limited to consecutive layers such as the convolution operation of layer 2M-1 and the quantization operation of layer 2M, but the convolution operation of layer 2M-1 and the quantization operation of layer 2M+2. You may perform between discontinuous layers like this. Further, when the results of a plurality of layers 2M-1 are integrated by performing an operation such as addition, the time required for the convolution operation for the layer 2M-1 and the quantization operation for the layer 2M differ greatly. In this case, since the period during which both operations are performed in parallel is shortened, different operations may be performed in parallel during the remaining period.

Next, each configuration of the NN circuit 100 will be described in detail.

[DMAC3]
FIG. 7 is an internal block diagram of the DMAC3.
The DMAC 3 has a data transfer circuit 31 and a state controller 32 . The DMAC 3 has a dedicated state controller 32 for the data transfer circuit 31, and when an instruction command is input, DMA data transfer can be performed without the need for an external controller.

The data transfer circuit 31 is connected to the external bus EB and performs DMA data transfer between an external memory such as a DRAM and the first memory 1 . The data transfer circuit 31 also performs DMA data transfer between an external memory such as a DRAM and the second memory 2 . A data transfer circuit 31 transfers data between an external memory such as a DRAM and the convolution circuit 4 . A data transfer circuit 31 transfers data between an external memory such as a DRAM and the quantization arithmetic circuit 5 . The number of DMA channels of the data transfer circuit 31 is not limited. For example, each of the first memory 1 and the second memory 2 may have a dedicated DMA channel.

State controller 32 controls the state of data transfer circuit 31 . Also, the state controller 32 is connected to the controller 6 via an internal bus IB. The state controller 32 has an instruction queue 33 and a control circuit 34 .

The instruction queue 33 is a queue in which the instruction command C3 for the DMAC 3 is stored, and is composed of, for example, a FIFO memory. One or more instruction commands C3 are written to the instruction queue 33 via the internal bus IB.

The control circuit 34 is a state machine that decodes the instruction command C3 and sequentially controls the data transfer circuit 31 based on the instruction command C3. The control circuit 34 may be implemented by a logic circuit or by a CPU controlled by software.

FIG. 8 is a state transition diagram of the control circuit 34. As shown in FIG.
When the instruction command C3 is input to the instruction queue 33 (Not empty), the control circuit 34 transitions from the idle state S1 to the decode state S2.

The control circuit 34 decodes the instruction command C3 output from the instruction queue 33 in the decode state S2. Also, the control circuit 34 reads the semaphore S stored in the register 61 of the controller 6 and determines whether the operation of the data transfer circuit 31 instructed by the instruction command C3 can be executed. If it is not ready (Not ready), the control circuit 34 waits until it becomes ready (Wait). If it is ready (ready), the control circuit 34 transitions from the decode state S2 to the run state S3.

In the execution state S3, the control circuit 34 controls the data transfer circuit 31 to cause the data transfer circuit 31 to perform the operation instructed by the instruction command C3. When the operation of the data transfer circuit 31 is completed, the control circuit 34 removes the executed instruction command C3 from the instruction queue 33 and updates the semaphore S stored in the register 61 of the controller 6 . When there is an instruction in the instruction queue 33 (Not empty), the control circuit 34 transitions from the execution state S3 to the decode state S2. When the instruction queue 33 has no instruction (empty), the control circuit 34 transitions from the execution state S3 to the idle state S1.

[Convolution arithmetic circuit 4]
FIG. 9 is an internal block diagram of the convolution arithmetic circuit 4. As shown in FIG.
The convolution circuit 4 has a weight memory 41 , a multiplier 42 , an accumulator circuit 43 and a state controller 44 . The convolution operation circuit 4 has a dedicated state controller 44 for the multiplier 42 and the accumulator circuit 43, and when an instruction command is input, the convolution operation can be performed without the need for an external controller.

The weight memory 41 is a memory that stores the weight w used in the convolution operation, and is a rewritable memory such as a volatile memory such as an SRAM (Static RAM). The DMAC 3 writes the weight w required for the convolution operation into the weight memory 41 by DMA transfer.

FIG. 10 is an internal block diagram of the multiplier 42. As shown in FIG.
The multiplier 42 calculates each element a(x+i, y+j, ci) of the divided input data a(x+i, y+j, co) and each element w(i, j, co, do) of the divided weight w(i, j, co, do). ci, di) and . The multiplier 42 has Bd product-sum operation units 48 connected in a pipeline, and the element a(x+i, y+j, ci) of the divided input data a(x+i, y+j, co) and the division weight w( The multiplication of i,j,co,do) with element w(i,j,ci,di) can be performed in parallel.

In the present embodiment, the element a(x+i, y+j, ci) is quantized to low bits such as 2 bits, so the divided input data a(x+i, y+j, co) is becomes data having a size of 16 bits. Since this is about the same as the bus width of the first memory 1 or the second memory 2, multiple elements of the divided input data a(x+i, y+j, co) can be efficiently read and written in short timing. It becomes possible. Similarly, since the element w(i, j, ci, di) is also quantized to, for example, 1 bit, multiple elements of the division weight w(i, j, co, do) can be read at short timing. or can be written.

The multiplier 42 reads the elements a(x+i, y+j, ci) and the elements w(i, j, ci, di) required for multiplication from the first memory 1 and the weight memory 41 and performs the multiplication. The multiplier 42 outputs Bd sum-of-products operation results O(x+i, y+j, di). Here, in the present embodiment, 32×32 elements a can be stored in the first memory 1 at one time, but the number of elements a is not limited to this.

FIG. 11 is an internal block diagram of the sum-of-products operation unit 48. As shown in FIG.
The sum-of-products operation unit 48 includes a plurality of XOR circuits 47, a plurality of counter circuits 49, and an addition circuit 50, and divides input data a (x+i, y+j, co) and divides weight w (i, j, co, do). Multiply with . In this embodiment, one sum-of-products operation unit 48 corresponds to a circuit that performs a 1×1 size convolution operation.

The sum-of-products operation unit 48 inputs data a[1] consisting of the upper bits of each 2-bit element a contained in the divided input data a(x+i, y+j, co) and data a[0] consisting of the lower bits. It has an input unit for As an example, if Bc is 8, a[1] and a[0] are 8-bit data each consisting of 8 binary values. Further, the sum-of-products operation unit 48 has an input section for inputting the division weight w(i, j, co, do). Note that the division weight w(i, j, co, do) is composed of 1-bit elements, so if Bc is 8 as an example, it becomes an 8-bit value.

The XOR circuit 47 performs an XOR operation of the input data a[1] or data a[0] and the division weight w(i, j, co, do). The result of the XOR operation on data a[1] and data a[0] is data consisting of Bc binary values.

The calculation results of the XOR calculation in the XOR circuit 47 are input to the counter circuit 49 respectively. The counter circuit 49 counts the number of 1s as the binary value included in the result of each XOR operation (popcount operation). The adder circuit 50 adds the outputs of a plurality of counters and outputs the sum-of-products operation result O(x+i, y+j, do). In this embodiment, the calculation performed by the sum-of-products calculation unit 48 can be represented by Equation 10.

FIG. 12 is an internal block diagram of the accumulator circuit 43. As shown in FIG.
The accumulator circuit 43 accumulates the sum-of-products operation result O(di) of the multiplier 42 in the second memory 2 . The accumulator circuit 43 has Bd accumulator units 48 and can accumulate Bd product-sum operation results O(di) in parallel in the second memory 2 .

FIG. 13 is an internal block diagram of the accumulator unit 48. As shown in FIG.
The accumulator unit 48 has an adder 48a and a mask portion 48b. The adder 48 a adds the element O(di) of the sum-of-products operation result O and the partial sum, which is the intermediate progress of the convolution operation shown in Equation 1, stored in the second memory 2 . The addition result is 16 bits per element. The addition result is not limited to 16 bits per element, and may be, for example, 15 bits or 17 bits per element.

The adder 48a writes the addition result to the same address in the second memory 2. FIG. The mask unit 48b masks the output from the second memory 2 and zeros the addition target for the element O(di) when the initialization signal clear is asserted. The initialization signal clear is asserted when the intermediate partial sum is not stored in the second memory 2 .

When the convolution operation by the multiplier 42 and the accumulator circuit 43 is completed, the output data f(x, y, do) are stored in the second memory.

State controller 44 controls the states of multiplier 42 and accumulator circuit 43 . Also, the state controller 44 is connected to the controller 6 via an internal bus IB. The state controller 44 has an instruction queue 45 and a control circuit 46 .

The instruction queue 45 is a queue in which the instruction command C4 for the convolution operation circuit 4 is stored, and is composed of a FIFO memory, for example. An instruction command C4 is written to the instruction queue 45 via the internal bus IB.

The control circuit 46 is a state machine that decodes the instruction command C4 and controls the multiplier 42 and the accumulator circuit 43 based on the instruction command C4. The control circuit 46 has the same configuration as the control circuit 34 of the state controller 32 of the DMAC3.

In the present embodiment, one product-sum operation unit 48 has shown an example corresponding to a circuit that performs a 1×1 size convolution operation, but the present invention is not limited to this. For example, an appropriate division weight w(i, j, co, do) is read out from the weight memory 41 to perform a plurality of 1×1 size convolution operations, and each result is accumulated in the accumulator unit 48 to obtain a larger size weight. A convolution operation using w can also be implemented.

[Quantization arithmetic circuit 5]
FIG. 14 is an internal block diagram of the quantization arithmetic circuit 5. As shown in FIG.
Quantization operation circuit 5 has quantization parameter memory 51 , vector operation circuit 52 , quantization circuit 53 , and state controller 54 . It has a dedicated state controller 54, and when an instruction command is input, it can perform quantization operations without the need for an external controller.

The quantization parameter memory 51 is a memory that stores the quantization parameter q used in the quantization calculation, and is a rewritable memory such as a volatile memory such as an SRAM (Static RAM). The DMAC 3 writes the quantization parameter q required for the quantization calculation into the quantization parameter memory 51 by DMA transfer.

FIG. 15 is an internal block diagram of the vector operation circuit 52 and the quantization circuit 53. As shown in FIG.
The vector computation circuit 52 computes the output data f(x, y, do) stored in the second memory 2 . The vector operation circuit 52 has Bd number of operation units 57 and performs SIMD operations on output data f(x, y, do) in parallel.

FIG. 16 is a block diagram of the arithmetic unit 57. As shown in FIG.
The arithmetic unit 57 has, for example, an ALU 57a, a first selector 57b, a second selector 57c, a register 57d, and a shifter 57e. The arithmetic unit 57 may further include other calculators and the like that a known general-purpose SIMD arithmetic circuit has.

The vector operation circuit 52 performs a pooling layer 221 in the quantization operation layer 220, a batch normalization layer 222, and a At least one operation among the operations of the activation function layer 223 is performed.

The arithmetic unit 57 can add the data stored in the register 57d and the element f(di) of the output data f(x, y, do) read from the second memory 2 by the ALU 57a. The arithmetic unit 57 can store the addition result by the ALU 57a in the register 57d. The arithmetic unit 57 can initialize the addition result by inputting "0" to the ALU 57a instead of the data stored in the register 57d by selecting the first selector 57b. For example, when the pooling area is 2×2, the shifter 57e can output the average value of the addition result by shifting the output of the ALU 57a to the right by 2 bits. The vector operation circuit 52 can perform the average pooling operation shown in Equation 2 by repeating the above operations and the like by the Bd number of operation units 57 .

The arithmetic unit 57 can compare the data stored in the register 57d with the element f(di) of the output data f(x, y, do) read from the second memory 2 by the ALU 57a.
The arithmetic unit 57 can control the second selector 57c according to the result of comparison by the ALU 57a to select the larger one of the data stored in the register 57d and the element f(di). The arithmetic unit 57 can initialize the comparison target to the minimum value by inputting the minimum value of the possible values of the element f(di) to the ALU 57a by selecting the first selector 57b. Since the element f(di) is a 16-bit signed integer in this embodiment, the minimum possible value of the element f(di) is "0x8000". The vector operation circuit 52 can implement the MAX pooling operation of Equation 3 by repeating the above operations and the like by the Bd number of operation units 57 . Note that the shifter 57e does not shift the output of the second selector 57c in the MAX pooling calculation.

The arithmetic unit 57 can subtract the data stored in the register 57d and the element f(di) of the output data f(x, y, do) read from the second memory 2 by the ALU 57a. Shifter 57e can left shift (ie, multiply) or right shift (ie, divide) the output of ALU 57a. The vector operation circuit 52 can perform the operation of Batch Normalization of Equation 4 by repeating the above operation and the like by the Bd number of operation units 57 .

The arithmetic unit 57 can compare the element f(di) of the output data f(x, y, do) read from the second memory 2 with "0" selected by the first selector 57b by the ALU 57a. The arithmetic unit 57 can select and output either the element f(di) or the constant value "0" previously stored in the register 57d according to the comparison result by the ALU 57a. The vector operation circuit 52 can perform the ReLU operation of Equation 5 by repeating the above operations and the like by the Bd number of operation units 57 .

The vector operation circuit 52 can perform average pooling, MAX pooling, batch normalization, activation function operations, and combinations of these operations. Since vector arithmetic circuit 52 is capable of performing general-purpose SIMD operations, it may also perform other operations required for operations in quantization operations layer 220 . Also, the vector operation circuit 52 may perform operations other than the operations in the quantization operation layer 220 .

Note that the quantization arithmetic circuit 5 may not have the vector arithmetic circuit 52 . If the quantization operation circuit 5 does not have the vector operation circuit 52 , the output data f(x, y, do) are input to the quantization circuit 53 .

A quantization circuit 53 quantizes the output data of the vector operation circuit 52 . As shown in FIG. 15, the quantization circuit 53 has Bd quantization units 58 and performs operations on the output data of the vector operation circuit 52 in parallel.

FIG. 17 is an internal block diagram of quantization unit 58. As shown in FIG.
A quantization unit 58 quantizes the element in(di) of the output data of the vector operation circuit 52 . Quantization unit 58 comprises a comparator 58a and an encoder 58b. The quantization unit 58 performs the operation (formula 6) of the quantization layer 224 in the quantization operation layer 220 on the output data (16 bits/element) of the vector operation circuit 52 . The quantization unit 58 reads the necessary quantization parameters q (th0, th1, th2) from the quantization parameter memory 51, and the comparator 58a compares the input in(di) with the quantization parameter q. Quantization unit 58 quantizes the result of comparison by comparator 58a to 2 bits/element by encoder 58b. Since α(c) and β(c) in Equation 4 are different parameters for each variable c, the quantization parameter q(th0, th1, th2) reflecting α(c) and β(c) is in( d) different parameters for each;

In this embodiment, the comparator 58a has an offset calculation function for the quantization parameter q (th0, th1, th2). Specifically, Off(di) is added to each threshold included in the quantization parameter q. Off(di) can be expressed by Equation 11.

Here, popcount(w) is obtained by counting the number of 1s as a binary value included in the division weight w(i, j, co, do) used in the convolution operation. More specifically, the output data f(x, y, do) read from the second memory 2 corresponding to in(di) to be compared in the comparator 58a, and the output data f(x, y, do ) is obtained by counting the number of 1s as a binary value included in the division weight w used in the convolution operation when obtaining For example, using the division weight w used in the convolution operation of the layer 2M-1, Off(di) depending on popcount(w) is added to the quantization parameter q of the quantization operation of the layer 2M. In the vector operation circuit 52, if the output data f(x, y, do) is subjected to processing such as pooling processing, normalization processing, activation processing, etc., also for popcount(w) A similar treatment is preferably performed. As an example, when the vector operation circuit 52 performs bit shift processing on the output data f(x, y, do), the same amount of bit shift processing can also be performed on popcount(w). preferable.

In this embodiment, the elements of input data a are 2-bit unsigned integers (0, 1, 2, 3), the elements of weight w are 1-bit signed integers (0, 1), and the value A "0" represents +1 and is exemplified as a value of "1". In this case, the convolution operation shown in Equation 1 can be represented by Equation 12.

As an example, if the output data f(x, y, do) is data having a size of 16 bits, possible values range from -32768 to 32767. Therefore, it is necessary to provide a sign bit. On the other hand, comparing Equation 10 and Equation 12, Equation 10 does not have a term that depends on the third term popcount(w) that takes a negative value. Therefore, no sign bit is required. As a result, if O(di), which is the operation result of Equation 10, has a size of 16 bits, the possible values are 0 to 65535 because there is no need to provide a sign bit. In other words, by omitting the terms that take negative values, the possible range of f(x, y, do) can be expanded. In other words, the target range of the convolution operation can be expanded. More specifically, the size of the weight w in the xy direction is determined by the size of the filter used for the convolution operation, but the size in the c direction is limited within a range in which the result of addition does not overflow. Therefore, an increase in the maximum value that can be taken by the output data f(x, y, do) corresponds to an expansion of the range in which overflow does not occur, and it is possible to further improve the computational efficiency.

Furthermore, the term dependent on popcount(w) required in Equation 10 is canceled as an offset calculation function for the quantization parameter q(th0, th1, th2) as described above. In particular, since popcount(w) does not depend on the input data a, it can be determined in advance for each layer as a learning result. In this embodiment, the comparator 58a has an offset operation function for the quantization parameter q (th0, th1, th2), but the operation of the operation unit 57 may include the offset operation function. . Alternatively, the result of pre-computing the quantization parameter q stored in the quantization parameter memory 51 may be stored in the quantization parameter memory 51 again. In other words, the quantization parameter q may be updated using the corresponding weight w when generating the trained model based on the learned result.

Quantization unit 58 compares input in(di) with three thresholds th0, th1, and th2 after offset arithmetic processing, thereby dividing input in(di) into four regions (for example, in≤th0, th0 <in≤th1, th1<in≤th2, th2<in), and encodes the result of classification into 2 bits and outputs it. The quantization unit 58 can also perform batch normalization and calculation of an activation function together with quantization by setting quantization parameters q (th0, th1, th2).

The quantization unit 58 performs quantization by setting the threshold th0 as β(c) in Equation 4 and threshold differences (th1−th0) and (th2−th1) as α(c) in Equation 4, The Batch Normalization operation shown in Equation 4 can be performed together with quantization. α(c) can be reduced by increasing (th1-th0) and (th2-th1). α(c) can be increased by decreasing (th1-th0) and (th2-th1).

Quantization unit 58 may perform a ReLU operation of the activation function in conjunction with quantization of the input in(di). For example, quantization unit 58 saturates the output values in regions where in(di)≤th0 and th2<in(di). Quantization unit 58 may perform activation function computation in conjunction with quantization by setting the quantization parameter q such that the output is non-linear.

State controller 54 controls the states of vector operation circuit 52 and quantization circuit 53 . Also, the state controller 54 is connected to the controller 6 via an internal bus IB. The state controller 54 has an instruction queue 55 and a control circuit 56 .

The instruction queue 55 is a queue in which the instruction command C5 for the quantization arithmetic circuit 5 is stored, and is composed of a FIFO memory, for example. The instruction command C5 is written to the instruction queue 55 via the internal bus IB.

The control circuit 56 is a state machine that decodes the instruction command C5 and controls the vector operation circuit 52 and the quantization circuit 53 based on the instruction command C5. The control circuit 56 has the same configuration as the control circuit 34 of the state controller 32 of the DMAC3.

The quantization operation circuit 5 writes quantization operation output data having Bd elements into the first memory 1 . Formula 10 shows a suitable relationship between Bd and Bc. In Equation 10, n is an integer.

In this way, as a result of learning the quantization parameter q and the weight w, for example, using the weight w in the convolution operation of the layer 2M-1, a quantization operation such as updating the quantization parameter q of a different layer such as the layer 2M , the overall computational efficiency can be improved. More generally, when one layer included in a plurality of layers is defined as a first layer and a different layer is defined as a second layer, Calculation efficiency can be improved by using the parameters (first parameters) used for the calculation of the second layer (second calculation).

As described above, the first embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are included within the scope of the present invention. . Also, the constituent elements shown in the above-described embodiment and modifications can be combined as appropriate.
(Modification 1)
In this embodiment, the weight w used in the convolution operation is used to reflect the quantization parameter used in the quantization operation, but the method for improving the efficiency of the operation is not limited to this. For example, it is possible to reflect the weight w used in the convolution calculation in the calculation of the calculation unit 57 . In addition, since the convolution operation is mainly based on linear calculation, the commutative law holds. It is possible.

(Modification 2)
In the above embodiment, the first memory 1 and the second memory 2 are different memories, but the aspect of the first memory 1 and the second memory 2 is not limited to this. The first memory 1 and the second memory 2 may be, for example, a first memory area and a second memory area in the same memory.

(Modification 3)
For example, the data input to the NN circuit 100 described in the above embodiment is not limited to a single format, and can be composed of still images, moving images, voices, characters, numerical values, and combinations thereof. The data input to the NN circuit 100 can be mounted on the edge device where the NN circuit 100 is provided. It is not limited to the measurement result in the instrument. Peripheral information such as base station information, vehicle/vessel information, weather information, and congestion information received from peripheral devices via wired or wireless communication, and different information such as financial information and personal information may be combined.

(Modification 4)
Edge devices provided with the NN circuit 100 are assumed to be communication devices such as mobile phones driven by batteries, smart devices such as personal computers, digital cameras, game devices, mobile devices such as robot products, but are limited to these. not a thing Unprecedented effects can also be obtained by using power on Ethernet (PoE), etc., to limit the peak power that can be supplied, reduce product heat generation, or use it for products that require long-time operation. For example, by applying it to in-vehicle cameras installed in vehicles and ships, surveillance cameras installed in public facilities and roads, etc., it is possible not only to realize long-time shooting, but also to contribute to weight reduction and durability. . Similar effects can be obtained by applying the present invention to display devices such as televisions and displays, medical equipment such as medical cameras and surgical robots, and work robots used at manufacturing sites and construction sites.

(Modification 5)
NN circuit 100 may implement part or all of NN circuit 100 using one or more processors. For example, the NN circuit 100 may implement part or all of the input layer or the output layer by software processing by a processor. A part of the input layer or output layer realized by software processing is, for example, data normalization and transformation. This allows for various input or output formats. The software executed by the processor may be rewritable using communication means or external media.

(Modification 6)
The NN circuit 100 may implement part of the processing in the CNN 200 by combining a graphics processing unit (GPU) on the cloud. The NN circuit 100 performs further processing on the cloud in addition to the processing performed by the edge device provided with the NN circuit 100, or performs processing on the edge device in addition to the processing on the cloud. More complex processing can be realized with fewer resources. With such a configuration, the NN circuit 100 can reduce the amount of communication between the edge device and the cloud due to processing distribution.

(Modification 7)
Although the operation performed by the NN circuit 100 is at least part of the learned CNN 200, the target of the operation performed by the NN circuit 100 is not limited to this. The calculations performed by the NN circuit 100 may be at least part of a trained neural network that repeats two types of calculations, such as convolution calculations and quantization calculations.

Also, the effects described herein are merely illustrative or exemplary, and are not limiting. In other words, the technology according to the present disclosure can produce other effects that are obvious to those skilled in the art from the description of this specification, in addition to or instead of the above effects.

The present invention can be applied to computation of neural networks.

200 convolutional neural network 100 neural network circuit (NN circuit)
1 first memory 2 second memory 3 DMA controller (DMAC)
4 convolution operation circuit 42 multiplier 43 accumulator circuit 5 quantization operation circuit 52 vector operation circuit 53 quantization circuit 6 controller 61 register

Claims (8)

A neural network device comprising computing means for a plurality of layers performing computations on input data,
The plurality of layers includes a first layer that performs a first operation using at least a first parameter and a second layer that performs a second operation different from the first operation,
The neural network device, wherein the second calculation is performed based on the first parameter and a second parameter different from the first parameter. 2. The neural network device according to claim 1, wherein said first layer and said second layer are layers for successively performing operations in said plurality of layers. 3. The neural network device according to claim 1, wherein said first operation includes a convolution operation, and said second operation includes a quantization operation. 4. The neural network apparatus according to claim 3, wherein said first parameters include weights used in convolution calculations, and include quantization parameters used in said second parameter quantization calculations. 5. The neural network device according to claim 4, wherein each element of the input data for the convolution operation is represented by 2 bits or less, and each element included in the weight is represented by 1 bit. the arithmetic means includes a first circuit for performing a convolution operation;
6. The neural network device according to claim 5, wherein said first circuit includes one or more exclusive OR circuits and counter circuits. 7. The neural network device according to any one of claims 1 to 6, wherein said computing means performs the first computation in the first layer and the second computation in the second layer in parallel. An edge device comprising the neural network device according to any one of claims 1 to 7.

Images