TW201928793A - Neural network forward operation method and related product having the advantages of small calculation amount and low power consumption - Google Patents

Abstract

The present disclosure provides a neural network forward operation method performed on an integrated circuit chip device, the neural network comprising multiple layers, wherein the method includes the following steps: receiving a first calculation instruction, and parsing the first calculation instruction to obtain the first operation of the first calculation instruction included in the ith layer of the forward operation and the input data and weight data corresponding to the first calculation instruction; determining a first complexity of the first operation according to the input data, the weight data, and the first operation, and determining, according to the first complexity, the first data type of the input data and the weight data when performing the first operation, the first data type comprising: a floating point type or a fixed point type; and applying the input data and the weight data to a first operation included in the first layer of the forward operation using the first data type. The technical solution provided by the present disclosure has the advantages of small calculation amount and low power consumption.

Description

Neural network forward calculation method and related products

The present disclosure relates to the field of neural networks, and in particular, to a neural network forward operation method and related products.

Artificial neural network (Artificial Neural Network, ANN) is a research hotspot that has emerged in the field of artificial intelligence since the 1980s. It abstracts the human brain neuron network from the perspective of information processing, establishes some simple model, and forms different networks according to different connection methods. In engineering and academia, it is often referred to as neural network or neural network. A neural network is a computing model that consists of a large number of nodes (or neurons) connected to each other. The existing neural network operations are based on the CPU (Central Processing Unit, Central Processing Unit) or GPU (Graphics Processing Unit, Graphics Processor) to implement the forward operation of the neural network. This forward operation has a large amount of calculation and high power consumption. .

The embodiments of the present disclosure provide a neural network forward operation method and related products, which can improve the processing speed and efficiency of a computing device.

In a first aspect, a neural network forward operation method performed on an integrated circuit chip device is provided. The neural network includes multiple layers. The method includes the following steps:

Receiving a first calculation instruction, and analyzing the first calculation instruction to obtain a first operation included in the i-th layer of the forward operation by the first calculation instruction, and input data and weight data corresponding to the first calculation instruction; The value of i is an integer greater than or equal to 1, and if the i is greater than or equal to 2, the input data is the output data of the i-1th layer;

A first complexity of the first operation is determined according to the input data, weight data, and the first operation, and a first data type of the input data and weight data when the first operation is performed is determined according to the first complexity. The first data type includes: a floating-point type or a fixed-point type;

Performing the first operation included in the first layer of the forward operation with the input data and the weight data in the first data type.

According to a second aspect, an integrated circuit chip device is provided. The integrated circuit chip device is configured to perform a forward operation of a neural network. The neural network includes multiple layers. The device includes a processing circuit and an external interface.

The external interface is configured to receive a first calculation instruction;

The processing circuit is configured to parse a first calculation instruction to obtain a first operation included in the i-th layer of the forward operation by the first calculation instruction, input data corresponding to the first calculation instruction, and weight data; The value range of i is an integer greater than or equal to 1, and if the i is greater than or equal to 2, the input data is the output data of the i-1th layer;

The processing circuit is further configured to determine a first complexity of the first operation according to the input data, weight data, and the first operation, and determine that the input data and weight data are performing the first operation according to the first complexity. The first data type of the time, the first data type includes: a floating point type or a fixed point type;

The processing circuit is further configured to perform the first operation included in the i-th layer of the forward operation by using the input data and the weight data as the first data type.

In a third aspect, a neural network computing device is provided. The neural network computing device includes one or more integrated circuit chip devices provided in the second aspect.

According to a fourth aspect, a combined processing device is provided. The combined processing device includes: a neural network computing device, a universal interconnection interface, and a universal processing device provided in the third aspect;

The neural network computing device is connected to the universal processing device through the universal interconnection interface.

According to a fifth aspect, a chip is provided, and the chip integrates the device of the second aspect, the device of the third aspect, or the device of the fourth aspect.

According to a sixth aspect, an electronic device is provided, and the electronic device includes the chip of the fourth aspect.

It can be seen that, according to the embodiment of the present disclosure, a data conversion operation circuit is provided to perform a conversion operation on the type of the data block, which saves transmission resources and calculation resources, so it has the advantages of low power consumption and small calculation amount.

In order to enable those skilled in the art to better understand the disclosure scheme, the technical solutions in the embodiments of the present disclosure will be described clearly and completely in combination with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are merely These embodiments are part of, but not all of the embodiments of this disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by persons with ordinary knowledge in the technical field without making creative labor fall into the scope of protection of the present disclosure.

In the method provided by the first aspect, determining the first data type of the input data and the weight data when performing the first operation according to the first complexity includes:

Compare the first complexity with a preset threshold, if the first complexity is higher than the preset threshold, determine that the first data type is a fixed-point type, such that the first complexity is lower than or equal to The preset threshold determines that the first data type is a floating point type.

In the method provided by the first aspect, after the determining the first data type of the input data and the weight data when performing the first operation according to the first complexity, the method further includes:

Determining that the input data and weight data belong to the second data type, and if the second data type is different from the first data type, the input data belonging to the second data type and all data belonging to the second data type The weight data is converted into the input data belonging to the first data type and the weight data belonging to the first data type.

In the method provided by the first aspect, if the first operation is a convolution operation, the input data is convolution input data, and the weight data is a convolution kernel,

First complexity = α * C * kW * kW * M * N * W * C * H;

Among them, α is the convolution coefficient, and the value range is greater than 1. C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data;

If the first complexity is greater than a set threshold, determine whether the convolution input data and the convolution kernel are floating point data. If the convolution input data and the convolution kernel are not floating point data, input the convolution input. The data is converted into floating-point data, the convolution kernel is converted into floating-point data, and then the convolution input data and the convolution kernel are used to perform convolution operations in the floating-point data type.

In the method provided by the first aspect, if the first operation is: a matrix multiplication matrix operation, the input data is a first matrix of the matrix multiplication matrix operation, and the weight value is Second matrix

The first complexity = β * F * G * E * F; where β is a matrix coefficient and the value range is 1 or more, F and G are the row and column values of the first matrix, and E and F are the second matrix Row and column values

If the first complexity is greater than a set threshold, determine whether the first matrix and the second matrix are floating-point data. If the first matrix and the second matrix are not floating-point data, convert the first matrix into For floating-point data, the second matrix is converted into floating-point data, and then the first matrix and the second matrix are performed as matrix floating-matrix operations in the floating-point data type.

In the method provided by the first aspect, if the first operation is: a matrix multiplying vector operation, the input data is a first matrix of the matrix multiplying vector operation, and the weight value is vector;

The first complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F, G are the row and column values of the first matrix, and F is the column value of the vector;

If the first complexity is greater than a set threshold, determine whether the first matrix and vector are floating point data. If the first matrix and vector are not floating point data, convert the first matrix to floating point data. , Convert the vector to floating-point data, and then perform matrix multiplication vector operations on the first matrix and vector with floating-point data type.

In the method provided by the first aspect, the i-th layer may further include one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and activation operation.

In the apparatus provided by the second aspect, the processing circuit is specifically configured to compare the first complexity with a preset threshold, and if the first complexity is higher than the preset threshold, the computing device determines the The first data type is a fixed-point type. If the first complexity is lower than or equal to the preset threshold, the computing device determines that the first data type is a floating-point type.

In the device provided by the second aspect, the integrated circuit chip device further includes: a data type conversion circuit;

The processing circuit is further configured to determine a second data type to which the input data and weight data belong. If the second data type is different from the first data type, send a conversion command to the data type conversion circuit. ,

The data type conversion circuit is configured to convert the input data belonging to the second data type and the weight data belonging to the second data type into the input data belonging to the first data type according to the conversion command, and The weight data belonging to the first data type.

In the apparatus provided by the second aspect, if the first operation is a convolution operation, the input data is convolution input data, and the weight data is a convolution kernel,

The processing circuit is configured to calculate a first complexity, and the first complexity = α * C * kW * kW * M * N * W * C * H;

Among them, α is the convolution coefficient, and the value range is greater than 1. C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data;

The processing circuit is further configured to determine whether the convolution input data and the convolution kernel are floating point data if the first complexity is greater than a set threshold, such as the convolution input data and the convolution kernel are not floating point. Data, the convolution input data is converted into floating point data, the convolution kernel is converted into floating point data, and then the convolution input data and the convolution kernel are used to perform a convolution operation in a floating point data type.

In the apparatus provided by the second aspect, if the first operation is: a matrix multiplication matrix operation, the input data is a first matrix of the matrix multiplication matrix operation, and the weight value is Second matrix

The processing circuit is configured to calculate a first complexity;

The first complexity = β * F * G * E * F; where β is a matrix coefficient and the value range is 1 or more, F and G are the row and column values of the first matrix, and E and F are the second matrix Row and column values

The processing circuit is further configured to determine whether the first matrix and the second matrix are floating-point data if the first complexity is greater than a set threshold, such as the first matrix and the second matrix are not floating-point data. , Convert the first matrix into floating-point data, convert the second matrix into floating-point data, and then perform a matrix multiplication matrix operation on the first matrix and the second matrix in a floating-point data type.

In the apparatus provided by the second aspect, if the first operation is: a matrix multiplying vector operation, the input data is a first matrix of the matrix multiplying vector operation, and the weight value is vector;

The processing circuit is configured to calculate a first complexity;

The first complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F, G are the row and column values of the first matrix, and F is the column value of the vector;

The processing circuit is further configured to determine whether the first matrix and vector are floating-point data if the first complexity is greater than a set threshold, and if the first matrix and vector are not floating-point data, A matrix is converted into floating-point data, a vector is converted into floating-point data, and a first matrix and a vector are performed as a floating-point data type by performing a matrix multiplication vector operation.

In the device provided by the second aspect, the i-layer may further include one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and activation operation.

As shown in FIG. 1, a forward operation of a neural network is provided according to an embodiment of the present disclosure. Each layer uses its own input data and weights to calculate corresponding output data according to an operation rule specified by the type of the layer.

The neural network's forward computing process (also called inference) is a process of processing the input data of each layer layer by layer and obtaining the output data after a certain calculation. It has the following characteristics:

Inputs for a layer:

The input of a certain layer can be the input data of a neural network;

The input of one layer can be the output of other layers;

The input of a certain layer can be the output of the previous moment in this layer (corresponding to the case of recurrent neural network);

A layer can obtain input from multiple input sources at the same time;

Output of a layer:

The output of a layer can be used as the output of a neural network;

The output of one layer can be the input of other layers;

The output of a layer can be the input of this layer at the next moment (in the case of a recurrent neural network);

The output of a certain layer can output results to the above multiple output directions;

Specifically, the types of operations of the layers in the neural network include but are not limited to the following:

Convolutional layer (ie performing a convolution operation);

Fully connected layer (that is, performing a fully connected operation);

Normalization (regularization) layer: including LRN (Local Response Normalization) layer, BN (Batch Normalization) layer and other types;

Pooling layer

Activation layer: including but not limited to the following types of Sigmoid layer, ReLU layer, PReLu layer, LeakyReLu layer, Tanh layer;

Layer inversion operation, each layer needs to perform two parts of the operation: one is to use the output data gradient that may be a sparse representation and the input data that may be a sparse representation to calculate the weight gradient (used in the "weight Update "step to update the weights of this layer), the other part is to calculate the gradient of the input data using the output data gradient that may be sparse representation and the weight value that may be sparse representation (for the output data of the next layer in the inverse operation) Gradient for inverse operation);

The reverse operation transfers gradients in the reverse order from the last layer, starting from the last layer.

In an optional solution, the gradient of the output data obtained by inverse calculation of a certain layer can come from:

The gradient returned by the last loss function or cost function of the neural network;

Input data gradients of other layers;

The input data gradient at the previous moment in this layer (corresponding to the case of recurrent neural network);

A certain layer can obtain output data gradients from multiple sources at the same time;

After performing the inverse operation of the neural network, the gradient of the weights of each layer is calculated. In this step, the first input buffer and the second input buffer of the device are used to store the weights and Weight gradient, and then use the weight gradient in the arithmetic unit to update the weight;

The operations mentioned above are all one-level operations in neural networks. For multi-layer neural networks, the implementation process is that in the forward operation, after the execution of the artificial neural network in the previous layer is completed, the operation instructions in the next layer will calculate the operation. The output data calculated in the unit is used as the input data of the next layer (or some operation is performed on the output data and then used as the input data of the next layer), and the weight is also replaced by the weight of the next layer; In the inverse operation, after the inverse operation of the artificial neural network in the previous layer is completed, the operation instructions in the next layer will calculate the input data gradient calculated in the operation unit as the output data gradient in the next layer (or the input data). The gradient performs some operations and then acts as the output data gradient of the next layer), and at the same time replaces the weight with the weight of the next layer; (represented by the following figure, the dotted arrow in the following figure represents the reverse operation, and the solid arrow represents (Forward operation, the label below each figure indicates the meaning of the figure)

Representation of fixed-point data

The fixed-point method refers to converting the data representation of a data block in the network into a specific data representation of a fixed decimal point position (mapped to the 0/1 bit position of the data on the circuit device);

In an optional solution, a plurality of data is combined into a data block as a whole to perform fixed-point representation using the same fixed-point representation method;

FIG. 1 a shows a specific representation method of a short-bit fixed-point data structure for storing data according to an embodiment of the present invention. Among them, 1Bit is used to represent a symbol, M is used to represent an integer part, and N is used to represent a decimal part. Compared to a 32-bit floating-point data representation, the short-bit fixed-point data representation used in the present invention is in addition to occupying bits In addition to fewer digits, for the data of the same layer and the same type in the neural network, such as the ownership value data of the first convolution layer, a flag point location is also set to record the position of the decimal point, which can be based on the actual data. The accuracy and range of data that can be represented by the distribution adjustment data.

The floating-point number is represented by 32bit, but for this technical solution, the use of fixed-point numbers can reduce the number of bits of a numerical value, thereby reducing the amount of data transmitted and the amount of data calculated.

The input data is shown in Figure 2a (N samples, each sample has C channels, and the height of the feature map of each channel is H and width W), and the weight, that is, the convolution kernel is shown in Figure 2b (M Convolution kernel, each convolution kernel has C channels, height and width are KH and KW respectively). For N samples of the input data, the rules of the convolution operation are the same. The process of performing the convolution operation on one sample is explained below. On one sample, each of the M convolution kernels must be the same. Operation, each convolution kernel operates to obtain a planar feature map, and M convolution kernels finally calculate to obtain M planar feature maps (for a sample, the output of the convolution is M feature maps), for a convolution kernel , To perform an inner product operation at each plane position of a sample, and then slide along the H and W directions. For example, Figure 2c shows a convolution kernel that performs an inner product operation at the lower right corner of a sample of input data. Correspondence map; Figure 2d shows the position of the convolution slide one grid to the left and Figure 2e shows the position of the convolution slide one grid up.

When the first operation is a convolution operation, the input data is convolution input data, and the weight data is a convolution kernel,

First complexity = α * C * kW * kW * M * N * W * C * H;

Among them, α is the convolution coefficient, and the value range is greater than 1. C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data;

If the first complexity is greater than a set threshold, determine whether the convolution input data and the convolution kernel are floating point data. If the convolution input data and the convolution kernel are not floating point data, input the convolution input. The data is converted into floating-point data, the convolution kernel is converted into floating-point data, and then the convolution input data and the convolution kernel are used to perform convolution operations in the floating-point data type.

Specifically, the convolution processing method may adopt a chip structure processing as shown in FIG. 3a. The data conversion operation circuit of the main processing circuit (also referred to as the main unit) may convert the weight when the first complexity is greater than a set threshold. The data in some or all of the convolution kernels is converted into fixed-point data. The control circuit of the main processing circuit sends the data in some or all of the weighted convolution kernels to be directly connected to the main processing circuit through the horizontal data input interface. Those basic processing circuits (also called basic units) (for example, the gray-filled vertical data path at the top in Figure 3b);

In an optional solution, the control circuit of the main processing circuit sends data of a certain convolution kernel in the weight value to a certain basic processing circuit at a time; (for example, for a certain basic processing circuit, the first Send the first number of the 3rd line once, send the 2nd number of the 3rd line data for the 2nd time, send the 3rd number of the 3rd line for the 3rd time ... or before the 3rd line of the 1st time Two numbers, the second time sends the 3rd and 4th numbers of the 3rd line, the third time sends the 5th and 6th numbers of the 3rd line ...;)

In another alternative, the control circuit of the main processing circuit sends the data of some convolution kernels in the weight value to the basic processing circuit one by one each time; (for example, for a certain A basic processing circuit that sends the first number of each line 3,4,5 for the first time, the second number of each line 3,4,5 for the second time, and the third number of 3, The 3rd number of each line 4,5 ... or the first two numbers of 3,4,5 lines are sent for the first time, the 3rd and 4th of 5th lines are sent for the second time 4 numbers, 3rd, 4th, 5th lines send 5th and 6th numbers per line ...;)

The control circuit of the main processing circuit divides the input data according to the position of the convolution. The control circuit of the main processing circuit sends the data in some or all of the convolution positions in the input data to the main processing circuit directly through the vertical data input interface. Connected basic processing circuits (for example, the gray-filled horizontal data path to the left of the basic processing circuit array in Figure 3b);

In an optional solution, the control circuit of the main processing circuit sends data of a convolution position in the input data to a basic processing circuit at a time or a portion of the data each time; (for example, for a basic processing circuit, the first Send the first number in the third column once, send the second number in the third column data for the second time, send the third number in the third column for the third time ..., or before the first column 3 Two numbers, the third and third numbers are sent in the third column, the third and third and fifth numbers are sent in the third column ...;)

In another optional solution, the control circuit of the main processing circuit sends data of some convolution positions in the input data to a basic processing circuit each time or a part of the data; (for example, for a certain A basic processing circuit that sends the first number of each of the 3, 4, and 5 columns for the first time, sends the second number of each of the 3, 4, 5 columns for the second time, and sends the third number of 3, The 3rd number of each column in 4,5 columns ..., or the first two numbers of 3,4,5 columns are sent for the first time, and the 3rd and 4th of 5th columns are sent for the second time. 4 numbers, send the 3rd, 4th, 5th columns for the third time, 5th and 6th numbers for each column ...;)

After the basic processing circuit receives the weighted data, it transmits the data to its next basic processing circuit through its horizontal data output interface (for example, the white-filled horizontal data path in the middle of the basic processing circuit array in Figure 3b). ); After receiving the input data, the basic processing circuit transmits the data to the next basic processing circuit connected to it through its vertical data output interface (for example, the white filled pad in the middle of the basic processing circuit array in Figure 3b) Vertical data path);

Each basic processing circuit performs operations on the received data;

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

After the basic processing circuit calculates the result, the result can be transmitted from the data output interface;

In an optional solution, the calculation result may be a final result or an intermediate result of the inner product operation;

Specifically, if the basic processing circuit has an output interface directly connected to the main processing circuit, the result is transmitted from the interface; if not, the result is output to the direction of the basic processing circuit that can directly output to the main processing circuit (for example, FIG. In 3b, the bottom line of the basic processing circuit directly outputs its output result to the main processing circuit, and the other basic processing circuits transmit the calculation results downward from the vertical output interface).

After receiving the calculation results from other basic processing circuits, the basic processing circuit transmits the data to other basic processing circuits or main processing circuits connected to the data;

Output the result in a direction that can be directly output to the main processing circuit (for example, the bottom row of the basic processing circuit outputs its output result directly to the main processing circuit, and the other basic processing circuits transmit the calculation result downward from the vertical output interface);

The main processing circuit receives the result of the inner product operation of each basic processing circuit and can obtain the output result.

Referring to FIG. 4a, FIG. 4a is a matrix-by-matrix operation. For example, the first operation is: matrix-by-matrix operation, the input data is the first matrix of the matrix-by-matrix operation, and the weight is The second matrix of matrix multiplication matrix operation;

The first complexity = β * F * G * E * F; where β is a matrix coefficient and the value range is 1 or more, F and G are the row and column values of the first matrix, and E and F are the second matrix Row and column values

If the first complexity is greater than a set threshold, determine whether the first matrix and the second matrix are floating-point data. If the first matrix and the second matrix are not floating-point data, convert the first matrix into For floating-point data, the second matrix is converted into floating-point data, and then the first matrix and the second matrix are performed as matrix floating-matrix operations in the floating-point data type.

Referring to FIG. 4b, the matrix multiplication matrix operation is completed using the apparatus shown in FIG. 3b;

The following describes the calculation of a multiplication of a matrix S whose size is M rows and L columns and a matrix P whose size is L rows and N columns. (Each row in matrix S has the same length as each column of matrix P, as shown in Figure 2d.) The neural network computing device has K basic processing circuits:

Step S401b: When the first complexity is greater than a set threshold, the main processing circuit converts the matrix S and the matrix P into fixed-point type data, and the control circuit of the main processing circuit distributes each row of data in the matrix S to the K basic processing circuits. On one of them, the basic processing circuit stores the received data in an on-chip buffer and / or register; specifically, it can be sent to the basic processing circuits connected to the main processing circuit among the K basic processing circuits.

In an optional solution, if the number of rows of S M <= K, the control circuit of the main processing circuit distributes one row of the S matrix to the M basic processing circuits respectively;

In an alternative, if the number of rows of S is M> K, the control circuit of the main processing circuit distributes one or more rows of data in the S matrix to each basic processing circuit.

The Mi line in S is distributed to the i-th basic processing circuit. This set of Mi lines is called Ai, and Figure 2e shows the calculation to be performed on the i-th basic processing circuit.

In an alternative, in each basic processing circuit, for example, in the i-th basic processing circuit:

The received matrix Ai distributed by the main processing circuit stores the matrix Ai in the i-th basic processing circuit register and / or on-chip cache; the advantage is that the subsequent data transmission amount is reduced, the calculation efficiency is improved, and the power consumption is reduced.

Step S402b: The control circuit of the main processing circuit transmits each part of the matrix P to each basic processing circuit in a broadcast manner;

In an optional solution, each part of the matrix P may be broadcast only once to a register or an on-chip buffer of each basic processing circuit, and the i-th basic processing circuit fully multiplexes the data of the matrix P obtained this time. Complete the inner product operation corresponding to each row in the matrix Ai; the multiplexing in this embodiment may specifically be used repeatedly by the basic processing circuit in the calculation, for example, the multiplexing of the data of the matrix P may be the multiple of the data of the matrix P Times of use.

In an optional solution, the control circuit of the main processing circuit may broadcast the various parts of the matrix P to the registers or on-chip buffers of each basic processing circuit multiple times. Without multiplexing, the inner product operation corresponding to each row in the matrix Ai is completed in stages;

In an optional solution, the control circuit of the main processing circuit may broadcast the various parts of the matrix P to the registers or on-chip buffers of each basic processing circuit multiple times. Perform partial multiplexing to complete the inner product operation corresponding to each row in the matrix Ai;

In an optional solution, each basic processing circuit, for example, the i-th basic processing circuit, calculates an inner product of the data of the matrix Ai and the data of the matrix P;

Step S403b: The accumulator circuit of each basic processing circuit accumulates the result of the inner product operation and transmits the result to the main processing circuit.

In an optional solution, the basic processing circuit may transfer the part obtained by performing the inner product operation each time and transfer it to the main processing circuit for accumulation;

In an optional solution, the part obtained by the inner product operation performed by each basic processing circuit may also be stored in a register and / or an on-chip buffer of the basic processing circuit, and transferred to the main processing circuit after the accumulation is completed;

In an optional solution, the part obtained by the inner product operation performed by the basic processing circuit and in some cases may be stored in a register of the basic processing circuit and / or an on-chip buffer for accumulation, and may be transmitted to the host in some cases. The processing circuit performs accumulation, and after the accumulation is completed, it is transmitted back to the main processing circuit.

Refer to FIG. 4c, which is a schematic diagram of a matrix multiplying a vector. For example, the first operation is: a matrix multiplying vector operation, the input data is a first matrix of the matrix multiplying vector operation, and the weight value is a vector of the matrix multiplying vector operation;

The first complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F, G are the row and column values of the first matrix, and F is the column value of the vector;

If the first complexity is greater than a set threshold, determine whether the first matrix and vector are floating point data. If the first matrix and vector are not floating point data, convert the first matrix to floating point data. , Convert the vector to floating-point data, and then perform matrix multiplication vector operations on the first matrix and vector with floating-point data type.

Referring to FIG. 4d, FIG. 4d provides a method for implementing a matrix multiplication vector, which may specifically include:

Step S401: The data conversion operation circuit of the main processing circuit converts each row of data in the matrix S into data of a fixed point type, and the control circuit of the main processing circuit is distributed to one of the K basic processing circuits, and the basic processing circuit will receive The obtained distribution data is stored in the on-chip buffer and / or register of the basic processing circuit;

In an optional solution, if the number of rows of the matrix S is M <= K, the control circuit of the main processing circuit distributes one row of the S matrix to the K basic processing circuits respectively;

In an alternative, if the number of rows M of the matrix S is greater than K, the control circuit of the main processing circuit distributes one or more rows of data in the S matrix to each basic processing circuit.

The set of rows distributed to S in the i-th basic processing circuit is Ai, with a total of Mi rows. As shown in FIG. 2c, the calculation to be performed on the i-th basic processing circuit is shown.

In an optional solution, in each basic processing circuit, for example, the i-th basic processing circuit, the received distribution data, such as a matrix Ai, may be stored in a register and / or an on-chip buffer of the i-th basic processing circuit. ; The advantage is that it reduces the data transmission amount of the distributed data in the future, improves the calculation efficiency, and reduces the power consumption.

Step S402: The data type operation circuit of the main processing circuit converts the vector P into data of a fixed point type, and the control circuit of the main processing circuit transmits the parts of the vector P of the fixed point type to the K basic processing circuits in a broadcast manner;

In an optional solution, the control circuit of the main processing circuit may broadcast each part of the vector P to the registers or on-chip buffers of the basic processing circuits only once, and the data of the vector P obtained this time by the ith basic processing circuit Perform sufficient multiplexing to complete the inner product operation corresponding to each row in the matrix Ai. The advantage is that the data transmission amount of repeated transmission of the vector P from the main processing circuit to the basic processing circuit is reduced, the execution efficiency is improved, and the transmission power consumption is reduced.

In an optional solution, the control circuit of the main processing circuit may broadcast the parts of the vector P to the registers or on-chip buffers of each basic processing circuit multiple times. Without multiplexing, the inner product operation corresponding to each row in the matrix Ai is completed in stages; the advantage is that the data transmission amount of a single transmission vector P inside the basic processing circuit is reduced, and the basic processing circuit cache and / Or register capacity, improve execution efficiency, reduce transmission power consumption, and reduce costs.

In an optional solution, the control circuit of the main processing circuit may broadcast the parts of the vector P to the registers or on-chip buffers of each basic processing circuit multiple times, and the i-th basic processing circuit may obtain the data of the vector P each time. Perform partial multiplexing to complete the inner product operation corresponding to each row in the matrix Ai; the advantage is that it reduces the amount of data transmission from the main processing circuit to the basic processing circuit, and also reduces the amount of data transmission inside the basic processing circuit, improving execution efficiency To reduce transmission power consumption.

Step S403: The inner product operator circuit of the K basic processing circuits calculates the inner product of the data of the matrix S and the vector P, for example, the i-th basic processing circuit calculates the inner product of the data of the matrix Ai and the data of the vector P;

Step S404: The accumulator circuits of the K basic processing circuits accumulate the results of the inner product operation to obtain an accumulation result, and transmit the accumulation result to the main processing circuit in a fixed-point type.

In an optional solution, the partial sum (partial sum is a part of the accumulated result) obtained when the inner product operation is performed each time by the basic processing circuit, for example, the accumulated result is: F1 * G1 + F2 * G2 + F3 * G3 + F4 * G4 + F5 * G5, then the partial sum can be: the value of F1 * G1 + F2 * G2 + F3 * G3) is transmitted back to the main processing circuit for accumulation; the advantage is that the internal calculation amount of the basic processing circuit is reduced and the operation efficiency of the basic processing circuit is improved.

In an optional solution, the part obtained by the inner product operation performed by each basic processing circuit may be stored in a register and / or an on-chip buffer of the basic processing circuit, and then transferred back to the main processing circuit after the accumulation is completed; the advantage is, The data transmission amount between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, and the data transmission power consumption is reduced.

In an optional solution, the part obtained by the inner product operation performed by the basic processing circuit and in some cases may be stored in a register of the basic processing circuit and / or an on-chip buffer for accumulation, and may be transmitted to the host in some cases. The processing circuit accumulates and transmits back to the main processing circuit after the accumulation; the advantage is that the data transmission amount between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, the data transmission power consumption is reduced, and the basic processing circuit is reduced The internal calculation amount improves the calculation efficiency of the basic processing circuit.

The present disclosure also provides an integrated circuit chip device for performing a forward operation of a neural network. The neural network includes multiple layers. The device includes: a processing circuit and an external interface.

The external interface is configured to receive a first calculation instruction;

The processing circuit is configured to parse a first calculation instruction to obtain a first operation included in the i-th layer of the forward operation by the first calculation instruction, input data corresponding to the first calculation instruction, and weight data; i The value of can be 1. If it is 1, the input data can be the original input data. When i is greater than or equal to 2, the input data can be the output data of the previous layer, such as the output data of the i-1 layer.

The processing circuit is further configured to determine a first complexity of the first operation according to the input data, weight data, and the first operation, and determine that the input data and weight data are performing the first operation according to the first complexity. The first data type of the time, the first data type includes: a floating point type or a fixed point type;

The processing circuit is further configured to perform the first operation included in the i-th layer of the forward operation by using the input data and the weight data as the first data type.

This disclosure also discloses a neural network computing device, which includes one or more chips as shown in FIG. 3a or FIG. 3b, for obtaining data to be calculated and control information from other processing devices, and executing a specified neural network. The calculation and execution results are passed to the peripheral device through the I / O interface. Peripheral equipment such as camera, monitor, mouse, keyboard, network card, wifi interface, server. When more than one chip shown in Figure 3a or Figure 3b is included, the chips shown in Figure 3a or Figure 3b can be linked and transmitted through a specific structure, for example, interconnected and transmitted through the PCIE bus. Data to support larger-scale neural network operations. At this time, you can share the same control system, or you can have separate control systems; you can share memory, or each accelerator can have its own memory. In addition, its interconnection method can be any interconnection topology.

The neural network computing device has high compatibility and can be connected to various types of servers through a PCIE interface.

The present disclosure also discloses a combined processing device, which includes the aforementioned neural network computing device, a universal interconnection interface, and other processing devices (ie, general processing devices). The neural network computing device interacts with other processing devices to complete a user-specified operation. For example, 5a is a schematic diagram of a combined processing device.

Other processing devices include one or more types of processors such as a central processing unit CPU, a graphics processor GPU, and a neural network processor. The number of processors included in other processing devices is not limited. Other processing devices serve as the interface between the neural network computing device and external data and control, including data transfer, to complete the basic control of the neural network computing device, such as start and stop; other processing devices can also cooperate with the neural network computing device to complete computing tasks.

A universal interconnection interface for transmitting data and control instructions between the neural network computing device and other processing devices. The neural network computing device obtains required input data from other processing devices and writes it to a storage device on the neural network computing device chip; it can obtain control instructions from other processing devices and write it to the control buffer on the neural network computing device chip; also The data in the storage module of the neural network computing device can be read and transmitted to other processing devices.

As shown in FIG. 5b, optionally, the structure further includes a storage device for storing data required by the operation unit / operation device or other operation units, and particularly suitable for the data required for operation in the neural network operation device. Or all data that cannot be saved in the internal storage of other processing devices.

The combined processing device can be used as an SOC system-on-chip for devices such as mobile phones, robots, drones, and video surveillance equipment, effectively reducing the core area of the control section, increasing processing speed, and reducing overall power consumption. In this case, the universal interconnection interface of the combined processing device is connected to some parts of the equipment. Some components such as camera, monitor, mouse, keyboard, network card, wifi interface.

Please refer to FIG. 5c, which is a schematic structural diagram of a neural network processor board provided by an embodiment of the present disclosure. As shown in FIG. 5 c, the neural network processor board 10 includes a neural network chip package structure 11, a first electrical and non-electrical connection device 12, and a first substrate 13.

The present disclosure does not limit the specific structure of the neural network chip package structure 11. Optionally, as shown in FIG. 5d, the aforementioned neural network chip package structure 11 includes: a neural network chip 111, a second electrical and non-electrical connection device 112, a first Two substrates 113.

The specific form of the neural network chip 111 involved in this disclosure is not limited. The aforementioned neural network chip 111 includes, but is not limited to, a neural network chip that integrates a neural network processor. The above chip may be made of silicon material, germanium material, quantum material, or molecule. Materials. According to the actual situation (for example: harsh environment) and different application requirements, the above neural network chip can be packaged so that most of the neural network chip is wrapped, and the pins on the neural network chip are passed through gold wires. The isoconductor is connected to the outer side of the package structure for circuit connection with the outer layer.

The present disclosure does not limit the specific structure of the neural network chip 111. For optional, please refer to the device shown in FIG. 1a.

The disclosure does not limit the types of the first substrate 13 and the second substrate 113, and may be a printed circuit board (PCB) or a printed wiring board (PWB), or may be other circuit boards. There are no restrictions on the materials used to make the PCB.

The second substrate 113 according to the present disclosure is used to carry the neural network chip 111, and a neural network chip package structure 11 obtained by connecting the neural network chip 111 and the second substrate 113 through a second electrical and non-electrical connection device 112. It is used to protect the neural network chip 111 and facilitate the further packaging of the neural network chip packaging structure 11 and the first substrate 13.

There is no limitation on the above-mentioned specific packaging method of the second electrical and non-electrical connection device 112 and the corresponding structure of the packaging method. A suitable packaging method can be selected and simply improved according to the actual situation and different application needs, such as flip chip Flip Chip Ball Grid Array Package (FCBGAP), Low-profile Quad Flat Package (LQFP), Quad Flat Package with Heat sink (HQFP), None Packaging methods such as Quad Flat Non-lead Package (QFN) or Fine-pitch Ball Grid Package (FBGA).

Flip chip (Flip Chip) is suitable for the case where the area after packaging is high or the inductance of the wire and the signal transmission time are sensitive. In addition, wire bonding can be used to reduce the cost and improve the flexibility of the packaging structure.

Ball Grid Array, which can provide more pins, and the average lead length of the pins is short, which has the function of transmitting signals at high speed. Among them, the package can be packaged with a pin grid array (Pin Grid Array, PGA) , Zero Insertion Force (ZIF), Single Edge Contact Connection (SECC), and Land Grid Array (LGA).

Optionally, the Flip Chip Ball Grid Array (Flip Chip Ball Grid Array) packaging method is used to package the neural network chip 111 and the second substrate 113. For a schematic diagram of a specific neural network chip packaging structure, refer to FIG. 6. As shown in FIG. 6, the aforementioned neural network chip package structure includes: a neural network chip 21, a pad 22, a solder ball 23, a second substrate 24, a connection point 25 on the second substrate 24, and a pin 26.

Among them, the pad 22 is connected to the neural network chip 21, and a solder ball 23 is formed by welding between the pad 22 and the connection point 25 on the second substrate 24, and the neural network chip 21 and the second substrate 24 are connected. Packaging of the neural network chip 21.

Pin 26 is used to connect with the external circuit of the package structure (for example, the first substrate 13 on the neural network processor board 10), and can realize the transmission of external data and internal data, which is convenient for the neural network chip 21 or the neural network chip 21 The corresponding neural network processor processes the data. The type and quantity of pins are not limited in this disclosure. Different pin forms can be selected according to different packaging technologies and arranged in accordance with certain rules.

Optionally, the aforementioned neural network chip package structure further includes an insulating filler placed in a gap between the pad 22, the solder ball 23, and the connection point 25 to prevent interference between the solder ball and the solder ball.

The material of the insulating filler may be silicon nitride, silicon oxide, or silicon oxynitride; interference includes electromagnetic interference, inductive interference, and the like.

Optionally, the aforementioned neural network chip package structure further includes a heat dissipation device for dissipating heat during operation of the neural network chip 21. The heat dissipation device may be a metal sheet, a heat sink, or a heat sink with good thermal conductivity, such as a fan.

For example, as shown in FIG. 6a, the neural network chip package structure 11 includes: a neural network chip 21, a pad 22, a solder ball 23, a second substrate 24, a connection point 25, a pin 26, The insulating filler 27, the heat dissipation paste 28, and the metal case heat sink 29. Among them, the heat dissipation paste 28 and the metal shell heat sink 29 are used to dissipate heat during the operation of the neural network chip 21.

Optionally, the aforementioned neural network chip package structure 11 further includes a reinforcing structure connected to the pad 22 and embedded in the solder ball 23 to enhance the connection strength between the solder ball 23 and the pad 22.

The reinforcing structure may be a metal wire structure or a columnar structure, which is not limited herein.

The present disclosure also does not limit the specific form of the first electrical and non-electrical device 12, and may refer to the description of the second electrical and non-electrical device 112, that is, the neural network chip packaging structure 11 is packaged by soldering, and connection may also be adopted. The method of connecting the second substrate 113 and the first substrate 13 in a line connection or plugging manner is convenient for subsequent replacement of the first substrate 13 or the neural network chip package structure 11.

Optionally, the first substrate 13 includes an interface of a memory unit for expanding the storage capacity, such as: synchronous dynamic random access memory (SDRAM), double-rate synchronous dynamic random access memory (Double Date Rate SDRAM, DDR), etc., to improve the processing capacity of the neural network processor by expanding the memory.

The first substrate 13 may further include a Peripheral Component Interconnect-Express (PCI-E or PCIe) interface, a Small Form-factor Pluggable (SFP) interface, an Ethernet interface, Controller Area Network (CAN) interfaces, etc., are used for data transmission between the package structure and external circuits, which can improve the operation speed and the convenience of operation.

The neural network processor is packaged as a neural network chip 111, the neural network chip 111 is packaged as a neural network chip package structure 11, the neural network chip package structure 11 is packaged as a neural network processor board 10, and an interface on the board ( Slots or inserts) perform data interaction with external circuits (for example, computer motherboards), that is, the function of the neural network processor is realized directly by using the neural network processor board 10 and the neural network chip 111 is protected. In addition, other modules can be added to the neural network processor board 10, which improves the application range and operation efficiency of the neural network processor.

In one embodiment, the present disclosure discloses an electronic device including the neural network processor board 10 or the neural network chip package structure 11 described above.

Electronic devices include data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cameras, camcorders, projectors, watches, headphones, mobile storage , Wearables, vehicles, home appliances, and / or medical devices.

The vehicles include airplanes, ships, and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, cooker hoods, and the medical equipment includes nuclear magnetic resonance Instrument, B-mode and / or electrocardiograph.

The specific embodiments described above further describe the purpose, technical solution and beneficial effects of the present disclosure. It should be understood that the above are only specific embodiments of the present disclosure and are not used to limit the present disclosure. Any modification, equivalent replacement, or improvement made within the spirit and principle of this disclosure shall be included in the protection scope of this disclosure.

A, B, S‧‧‧ Matrix

P‧‧‧ vector

S401b, S402b, S403b, S401, S402, S403, S404‧‧‧ steps

10‧‧‧ Neural Network Processor Card Board

11‧‧‧ neural network chip packaging structure

12‧‧‧First electrical and non-electrical connection device

13‧‧‧First substrate

111‧‧‧ neural network chip

112‧‧‧Second electrical and non-electrical connection device

113‧‧‧second substrate

1111‧‧‧Storage Unit

1112‧‧‧Direct Memory Access Unit

1113‧‧‧Instruction cache unit

1114‧‧‧ Rights Cache Unit

1115‧‧‧Input neuron buffer unit

1116‧‧‧Output neuron buffer unit

1117‧‧‧Control Unit

1118‧‧‧ Computing Unit

21‧‧‧Neural Network Chip

22‧‧‧ pad

23‧‧‧Solder Ball

24‧‧‧second substrate

25‧‧‧ Connection points on the second substrate 24

26‧‧‧pin

27‧‧‧Insulation filler

28‧‧‧ Thermal Paste

29‧‧‧ metal case heat sink

Figure 1 is a schematic diagram of the forward operation of a neural network.

FIG. 1a is a schematic structural diagram of a fixed-point data type.

Figure 2a is a schematic diagram of convolution input data.

Figure 2b is a schematic diagram of a convolution kernel.

FIG. 2c is a schematic diagram of a calculation window of a three-dimensional data block of input data.

FIG. 2d is a schematic diagram of another operation window of a three-dimensional data block of input data.

Figure 2e is a schematic diagram of another calculation window of a three-dimensional data block of the input data.

Figure 3a is a schematic structural diagram of a neural network chip.

FIG. 3b is a schematic structural diagram of another neural network chip.

Figure 4a is a schematic diagram of matrix multiplication by matrix.

FIG. 4b is a flowchart of a method of matrix by matrix.

Figure 4c is a schematic diagram of a matrix multiplied by a vector.

FIG. 4d is a flowchart of a method of matrix multiplication by a vector.

FIG. 5a is a schematic structural diagram of a combination processing device disclosed in the present disclosure.

FIG. 5b also discloses another structure diagram of a combined processing device in the present disclosure.

5c is a schematic structural diagram of a neural network processor board provided in an embodiment of the present disclosure;

5d is a schematic structural diagram of a neural network chip package structure provided by an embodiment of the present disclosure;

5e is a schematic structural diagram of a neural network chip according to an embodiment of the present disclosure;

6 is a schematic diagram of a neural network chip package structure provided by the embodiment of the present disclosure;

FIG. 6a is a schematic diagram of another neural network chip package structure provided by the embodiment of the present disclosure.

Claims (18)

A neural network forward operation method executed on an integrated circuit chip device. The neural network includes multiple layers, wherein the method includes the following steps: receiving a first calculation instruction, parsing the first calculation instruction, and obtaining the first calculation instruction in the normal operation; The first operation included in the i-th layer of the operation, and an input data and a weight data corresponding to the first calculation instruction, the value of i is an integer greater than or equal to 1, and if the i is greater than or equal to 2, the input The data is the output data of layer i-1; the first complexity of the first operation is determined according to the input data, the weight data, and the first operation; the input data and the weight data are determined according to the first complexity; A first data type when performing a first operation, the first data type includes: a floating-point type or a fixed-point type; and the i-th layer that performs the forward operation on the input data and the weight data with the first data type The first operation. The method according to item 1 of the scope of patent application, wherein determining the input data and the first data type of the weight data when performing the first operation according to the first complexity includes: combining the first complexity with a pre- Set a threshold comparison. If the first complexity is higher than the preset threshold, determine that the first data type is a fixed-point type. If the first complexity is lower than or equal to the preset threshold, determine that the first data type is floating. Point type. The method according to item 2 of the patent application scope, wherein after determining the input data according to the first complexity and the first data type of the weight data when performing the first operation, the method further includes: determining the input data And the weight data belongs to the second data type. If the second data type is different from the first data type, the input data belonging to the second data type and the weight data belonging to the second data type are converted to belong to the first data type. The input data of a data type and the weight data belonging to a first data type. The method according to item 1 of the scope of patent application, wherein if the first operation is a convolution operation, the input data is a convolution input data, the weight data is a convolution kernel, and the first complexity = α * C * kW * kW * M * N * W * C * H; Among them, α is the convolution coefficient, and the value range is greater than 1; C, kW, kW, and M are the values of the four dimensions of the convolution kernel, N, W, C, H are the values of the four dimensions of the convolution input data; if the first complexity is greater than a set threshold, determine whether the convolution input data and the convolution kernel are floating point data, such as the convolution The input data and the convolution kernel are not floating point data. The convolution input data is converted into floating point data, the convolution kernel is converted into floating point data, and then the convolution input data, the convolution The kernel performs convolution operations on floating-point data types. The method according to item 1 of the scope of patent application, wherein if the first operation is a matrix multiplication matrix operation, the input data is the first matrix multiplication of the matrix multiplication matrix operation, and the weight value is the second matrix multiplication of the matrix multiplication matrix operation Matrix; first complexity = β * F * G * E * F; where β is a matrix coefficient, the value range is greater than or equal to 1, F and G are the row and column values of the first matrix, and E and F are Row and column values of the second matrix; if the first complexity is greater than a set threshold, determine whether the first matrix and the second matrix are floating point data, such as the first matrix and the second matrix are not floating point Data, convert the first matrix into floating point data, convert the second matrix into floating point data, and then perform matrix multiplication matrix operations on the first matrix and the second matrix with floating point data types. The method according to item 1 of the scope of patent application, wherein if the first operation is a matrix multiplication vector operation, the input data is a first matrix of the matrix multiplication vector operation, and the weight is a vector of the matrix multiplication vector operation; The first complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F and G are row and column values of the first matrix, and F is a column value of the vector; If the first complexity is greater than a set threshold, determine whether the first matrix and the vector are floating point data. If the first matrix and the vector are not floating point data, convert the first matrix to floating point Data, convert the vector into floating point data, and then perform a matrix multiplying vector operation on the first matrix and the vector in the floating point data type. The method according to any one of claims 1 to 6, wherein the i-th layer further includes one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and active operation. An integrated circuit chip device, wherein the integrated circuit chip device is used to perform a forward operation of a neural network, the neural network includes multiple layers, the device includes: a processing circuit and an external interface; the external interface is configured to receive a first calculation instruction A processing circuit for parsing the first calculation instruction to obtain a first operation included in the i-th layer of the forward operation by the first calculation instruction, an input data corresponding to the first calculation instruction, and a weight data, The value of i is an integer greater than or equal to 1. If the value of i is greater than or equal to 2, the input data is the output data of the i-1th layer. The processing circuit is further configured to use the input data, the weight data, and The first operation determines a first complexity of the first operation, and determines a first data type of the input data and the weight data when performing the first operation according to the first complexity. The first data type includes: a floating point type Or fixed-point type; the processing circuit is further configured to perform the first operation included in the first layer of the forward operation on the input data and the weight data with the first data type . The integrated circuit chip device according to item 8 of the scope of patent application, wherein the processing circuit is specifically configured to compare the first complexity with a preset threshold, and if the first complexity is higher than the preset threshold, the computing device It is determined that the first data type is a fixed-point type. If the first complexity is lower than or equal to the preset threshold, the computing device determines that the first data type is a floating-point type. The integrated circuit chip device according to item 9 of the scope of patent application, wherein the integrated circuit chip device further includes a data type conversion circuit; the processing circuit is further configured to determine the second data type to which the input data and the weight data belong. If the second data type is different from the first data type, a conversion command is sent to the data type conversion circuit, and the data type conversion circuit is configured to, according to the conversion command, the input data that belongs to the second data type and belong to The weight data of the second data type is converted into the input data belonging to the first data type and the weight data belonging to the first data type. According to the integrated circuit chip device according to item 8 of the scope of patent application, if the first operation is a convolution operation, the input data is a convolution input data, the weight data is a convolution kernel, and the processing circuit uses To calculate the first complexity, the first complexity = α * C * kW * kW * M * N * W * C * H; where α is a convolution coefficient and the value range is greater than 1; C, kW, kW, M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data; the processing circuit is further configured to: Determine whether the convolution input data and the convolution kernel are floating point data. If the convolution input data and the convolution kernel are not floating point data, convert the convolution input data into floating point data, and The convolution kernel is converted into floating-point data, and then the convolution input data is performed, and the convolution kernel performs a convolution operation in a floating-point data type. The integrated circuit chip device according to item 8 of the scope of patent application, wherein if the first operation is a matrix multiplication matrix operation, the input data is the first matrix of the matrix multiplication matrix operation, and the weight value is the matrix multiplication matrix operation A second matrix; the processing circuit is used to calculate the first complexity; the first complexity = β * F * G * E * F; where β is a matrix coefficient, and the value range is greater than or equal to 1, F, G Are the row and column values of the first matrix, and E and F are the row and column values of the second matrix; the processing circuit is further configured to determine the first matrix and the first matrix if the first complexity is greater than a set threshold; Whether the second matrix is floating-point data. If the first matrix and the second matrix are not floating-point data, convert the first matrix into floating-point data and convert the second matrix into floating-point data. , And then perform a matrix multiplication matrix operation on the first matrix and the second matrix with a floating point data type. The integrated circuit chip device according to item 8 of the scope of patent application, wherein if the first operation is a matrix multiplication vector operation, the input data is a first matrix of the matrix multiplication vector operation, and the weight value is the matrix multiplication vector operation The vector is used to calculate the first complexity; the first complexity = β * F * G * F; where β is a matrix coefficient, and the value range is greater than or equal to 1, F and G are the first The row and column values of a matrix, where F is the column value of the vector; the processing circuit is further configured to determine whether the first matrix and the vector are floating-point data if the first complexity is greater than a set threshold, such as The first matrix and the vector are not floating point data, the first matrix is converted into floating point data, the vector is converted into floating point data, and then the first matrix and the vector are converted into floating point data Type performs matrix multiplication vector operations. The integrated circuit chip device according to any one of claims 8-13, wherein the i-th layer further includes one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and active operation. A neural network computing device, wherein the neural network computing device includes one or more integrated circuit chip devices according to any one of claims 8-14 of the scope of the patent application. A combined processing device, wherein the combined processing device includes: a neural network computing device, a universal interconnection interface, and a universal processing device, such as item 15 of the scope of patent application; the neural network computing device communicates with the universal through the universal interconnection interface The processing device is connected. A chip, wherein the chip integrates a device such as any one of items 8-14 of the scope of patent application. An electronic device, wherein the electronic device includes a chip as in item 17 of the scope of patent application.