CN111710356A - Encoding type flash memory device and encoding method - Google Patents

Abstract

The invention discloses a coding type flash memory device and a coding method, wherein the coding type flash memory device comprises: at least one flash memory array structural unit, multiple comparators and multiple adders, each flash memory array in the at least one flash memory array structural unit The structural unit is a 3D NAND FLASH array structural unit, which is used to implement an encoding operation to generate a source line voltage on each of the multiple source lines of the flash memory array structural unit; each comparator in the multiple comparators is related to each source. The lines are correspondingly connected, and are used to convert the source line voltage of each source line correspondingly connected to an output result in a binary form; and each adder in the plurality of adders corresponds to at least two of the plurality of comparators. Each of the source lines of , is connected to each other, and is used to perform sum operation on at least two output results corresponding to at least two comparators. The coded flash memory device and the coding method of the present invention can realize efficient and accurate full-connection layer or convolutional layer operations, thereby realizing a deep neural network.

Description

Encoding type flash memory device and encoding method

technical field

The present invention relates to the technical field of semiconductor devices and integrated circuits, in particular to a coding flash memory device and coding method for realizing a deep neural network.

Background technique

Deep neural networks are currently widely used in image processing and speech recognition and have shown excellent performance. Most of the deep neural networks in the prior art rely on the traditional von Neumann architecture to perform operations, but due to the separation of storage units and computing units in the von Neumann computing architecture, data transmission through the bus will cause delays. The increase of energy consumption and the increase of energy consumption make the data processing capacity and energy efficiency ratio improvement of deep neural network bottlenecks. To this end, the prior art further proposes an encoded flash memory device that performs analog operations based on a 3D NAND FLASH structure. However, there are still three main problems in the implementation of the deep neural network in the coded flash memory device as follows:

First, when the threshold voltage fluctuation between different devices is high, the calculation accuracy will decrease;

Second, the result of the analog operation needs to go through a complex analog-to-digital conversion circuit, which reduces the energy efficiency ratio brought by the in-memory operation of 3D NANDFLASH to a certain extent;

Third, limited by the operation mode of 3D NAND FLASH, that is, only data on a single word line WL can be operated on at the same time, the efficiency of the entire computing architecture has not been maximized, and the parallelism has not reached saturation.

SUMMARY OF THE INVENTION

(1) Technical problems to be solved

In order to solve the technical problems in the prior art that the coded flash memory device based on the 3D NAND FLASH structure for analog operation has low calculation accuracy, reduced energy efficiency ratio, and the efficiency of the calculation architecture is not maximized and the parallelism is not saturated. An encoding type flash memory device and encoding method.

(2) Technical solutions

One aspect of the present invention discloses an encoded flash memory device, which includes: at least one flash memory array structural unit, multiple comparators and multiple adders, and each flash memory array structural unit in the at least one flash memory array structural unit is 3D The NAND FLASH array structure unit is used to implement encoding operations to generate the source line voltage on each of the multiple source lines of the flash memory array structure unit; each of the multiple comparators is connected to each source line correspondingly, It is used to convert the source line voltage of each connected source line into an output result in binary form; and each adder in the plurality of adders and at least two comparators in the plurality of comparators pass through the corresponding source The lines are connected to each other, and are used for summing at least two output results corresponding to at least two comparators, so as to implement a deep neural network.

According to an embodiment of the present invention, each flash memory array structural unit includes: multiple operation units and multiple redundant units; each operation unit in the multiple operation units is each word line and each word line in the multiple word lines. Transistors at the intersection of source lines in the flash memory array structural unit, wherein each source line is provided with a corresponding operation unit; the plurality of redundant units are transistors of non-operational units in each flash memory array structural unit, which are used to implement coding On during operation.

According to an embodiment of the present invention, wherein each flash memory array structural unit further includes: a string selection line and a ground selection line, the string selection line is connected to the bit line end of the flash memory array structural unit; the ground selection line is connected to the source of the flash memory array structural unit The line ends are connected; wherein, the string selection line and the ground selection line are used to apply a high level when implementing the encoding operation.

According to an embodiment of the present invention, wherein the encoding operation is a fully connected operation or a convolutional layer operation, and the number of the multiple adders is consistent with the number of operation results of the summation operation of the fully connected operation or the convolutional layer operation.

Another aspect of the present invention discloses an encoding method, implemented based on the above-mentioned encoded flash memory device, wherein the encoding method includes: performing an encoding operation based on at least one flash memory array structural unit, and generating a plurality of flash memory array structural units for each of the flash memory array structural units. source line voltage on each of the source lines; a plurality of comparators to convert the source line voltage of each source line corresponding to each comparator to an output result in binary form; and a plurality of adders to compare the plurality of At least two output results corresponding to at least two comparators in the device are added and added to implement a deep neural network.

According to an embodiment of the present invention, the encoding operation is a fully connected operation or a convolutional layer operation.

According to an embodiment of the present invention, when the encoding operation is a fully connected operation, the encoding operation is performed on at least one flash memory array structural unit to generate a source line voltage on each of the multiple source lines of each flash memory array structural unit. Before, the encoding method further includes: pre-storing each non-zero bit element of the multiple non-zero bit elements in the weight matrix vector of the fully connected layer into a corresponding operation unit of the multiple operation units in the encoded flash memory device.

According to an embodiment of the present invention, when the encoding operation is a convolution layer operation, the encoding operation is performed on at least one flash memory array structural unit to generate a source line on each of the multiple source lines of each flash memory array structural unit. Before applying the voltage, the encoding method further includes: pre-storing each convolution kernel element of the plurality of convolution kernel elements in the convolution matrix of the convolution layer into a corresponding operation unit of the plurality of operation units in the encoded flash memory device.

According to an embodiment of the present invention, wherein each non-zero bit element of the multiple non-zero bit elements in the weight matrix vector of the fully connected layer is pre-stored into the corresponding operation in the multiple operation units in the coded flash memory device After each convolution kernel element in the convolution matrix of the convolution layer is pre-stored in the corresponding operation unit of the plurality of operation units in the coded flash memory device, the encoding The method further includes: correspondingly inputting input elements of the corresponding input vector into each word line of the plurality of word lines to generate a word line voltage on each corresponding word line, so that the plurality of redundant cells of the coded flash memory device is on.

According to an embodiment of the present invention, wherein the input elements of the corresponding input vector are input to each word line of the plurality of word lines correspondingly to generate a word line voltage on each word line, so that the coded flash memory device After the plurality of redundant cells are in an on state, the encoding method further includes: applying a high level to a string selection line and a ground selection line of the encoded flash memory device to generate each of the plurality of source lines in each of the flash memory array structural cells. The source line voltage on the source line.

According to an embodiment of the present invention, when the encoding operation is a fully connected operation, the encoding operation is performed based on at least one flash memory array structural unit to generate a source line on each of the multiple source lines of each flash memory array structural unit. Voltage, including: simultaneously performing encoding operations on corresponding input vectors and weight matrix vectors based on multiple flash memory array structural units, or encoding corresponding input vectors and weight matrix vectors based on one flash memory array structural unit in a time-division multiplexing manner operation; wherein, the input vector is multi-bit data.

According to an embodiment of the present invention, when the encoding operation is a convolutional layer operation, the encoding method further includes: shifting and summing the addition results output by the addition operations of multiple adders to implement a deep neural network; wherein, The convolution matrix of the convolution layer is multi-bit data.

(3) Beneficial effects

The invention discloses a coding type flash memory device and a coding method, wherein the coding type flash memory device comprises: at least one flash memory array structural unit, multiple comparators and multiple adders, each flash memory array in the at least one flash memory array structural unit The structural unit is a 3D NAND FLASH array structural unit, which is used to implement an encoding operation to generate a source line voltage on each of the multiple source lines of the flash memory array structural unit; each comparator in the multiple comparators is related to each source. The lines are correspondingly connected, and are used to convert the source line voltage of each source line correspondingly connected to an output result in a binary form; and each adder in the plurality of adders corresponds to at least two of the plurality of comparators. Each source line of , is connected to each other, and is used to perform sum operation on at least two output results corresponding to at least two comparators. The coded flash memory device and the coding method of the present invention can realize efficient and accurate full-connection layer or convolutional layer operations, thereby realizing a deep neural network.

Description of drawings

FIG. 1 is a schematic diagram of the composition of a coded flash memory device corresponding to a single flash memory array structural unit according to an embodiment of the present invention;

FIG. 2 is a schematic flowchart of an encoding method for an encoded flash memory device according to an embodiment of the present invention;

3 is a schematic diagram of the composition of a coded flash memory device applied to a fully connected operation according to another embodiment of the present invention;

4 is a schematic partial flowchart of an encoding method for an encoded flash memory device applied to a fully connected operation according to yet another embodiment of the present invention;

5 is a schematic diagram of the composition of a coded flash memory device applied to a convolutional layer operation according to another embodiment of the present invention;

FIG. 6 is a schematic partial flowchart of an encoding method for an encoded flash memory device applied to convolutional layer operations according to another embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

In order to solve the technical problems in the prior art that the coded flash memory device based on the 3D NAND FLASH structure for analog operation has low calculation accuracy, reduced energy efficiency ratio, and the efficiency of the calculation architecture is not maximized and the parallelism is not saturated. An encoding type flash memory device and encoding method.

The encoding type flash memory device and encoding method of the present invention can realize parallel in-memory computing based on the 3D NAND FLASH structure. The encoding operation is based on the following technical principle: on the same word line WL, when all memory cells (ie transistors) are in the on state , the corresponding source line SL can read the source line current. Based on this principle, in order to realize the parallel in-memory calculation based on the 3D NAND FLASH structure, it is different from the traditional serial operation word line WL to realize the coding method of 3D NAND FLASH in-memory calculation. The technical idea of the coding method of the flash memory device is as follows:

On each source line SL of the 3D NAND FLASH structure, it is preset that only one storage unit is responsible for data operation (ie, operation unit).

The value that can be stored in the operation unit is 0 or 1, and the corresponding threshold voltage on the operation unit is V _th =V _{th_Low} (that is, the value stored in the operation unit is 0) or V _th =V _{th_High} (that is, the value stored in the operation unit is 0) is 1);

The remaining storage units (non-operational units) on the 3D NAND FLASH structure are set as redundant units, and the values stored on the redundant units are all 0, then the threshold voltage on the corresponding redundant units is V _th =V _{th_Low} ;

The input value 0 or 1 on the operation unit corresponds to the word line voltage V _g =V _{g_High} or V _g =V _{g_Low} applied on the word line WL where the operation unit is located;

It can be seen that no matter the input value on the operation unit is 0 or 1, other redundant units are in the open state. Therefore, the source line current of the source line SL will correspond to the following operation results:

When the source line current read by the source line SL is 0, it means that the operation unit is in an off state, that is, V _th =V _{th_High} (1), V _g =V _{g_Low} (1), indicating that the operation result is 1;

When the source line current read by SL is 1, it means that the operation unit is in the open state, that is, there are one of the following three situations: V _th =V _{th_High} (1), V _g =V _{g_High} (0); V _th =V _{th_Low} (0), V _g =V _{g_High} (0); V _th =V _{th_Low} (0), V _g =V _{g_Low} (1), indicating that the operation result is 0.

Therefore, by adopting the encoding type flash memory device of the present invention, the encoding method for parallel in-memory computing can be realized in the 3D NAND FLASH structure. Specifically, the coded flash memory device of the present invention can perform the functions of each computing layer in parallel by using the integrated storage and computing technology through a deep neural network based on 3D NANDFLASH assisted by an adder, and obtain accurate computing results.

One aspect of the present invention discloses an encoded flash memory device, as shown in FIG. 1 , FIG. 3 and FIG. 5 , wherein the encoded flash memory device includes: at least one flash memory array structure unit, multiple comparators and multiple adders , which is used to implement the above-mentioned deep neural network based on 3D NAND FLASH.

The coded flash memory device includes: at least one flash memory array structural unit 100, such as the flash memory array structural unit 100-1 to the flash memory array structural unit 110-N connected in parallel with each other as shown in FIG. 3 and FIG. 4, a total of N flash memory array structural units, N≥1. Each flash memory array structural unit 100 in the N flash memory array structural units is a 3D NAND FLASH array structural unit, and the 3D NAND FLASH array structural unit has a plurality of memory cells, a plurality of bit lines BL, a plurality of word lines WL and a plurality of Source line SL.

As shown in FIG. 1 , the number of the plurality of storage cells may be M×M, wherein the M storage cells are connected in series with each other according to a certain arrangement rule to form M storage cell strings, and one end of each storage cell string is connected to a corresponding bit The other end of the line BL is connected to a source line SL correspondingly, and each word line WL is perpendicular to the memory cell strings, and connects the corresponding memory cells on the plurality of memory cell strings. Therefore, the number of bit lines BL and source lines SL is M. , the number of word lines WL corresponds to the number of memory cells connected in series on the memory cell string, which is also M, and each flash memory array structural unit 100 is used to implement an encoding operation to generate a plurality of source lines of the flash memory array structural unit. the source line voltage.

Specifically, the preset arithmetic unit corresponding to each source line SL stores a value of 0 or 1, and other redundant units store a value of 0. At this time, the redundant unit will be in an open state, corresponding to the word where the arithmetic unit is located. The word line voltage applied on the line WL is V _g =V _{g_High} or V _g =V _{g_Low} , at this time, the source line voltage corresponding to the source line SL where the operation unit is located can be generated.

Each comparator in the plurality of comparators is correspondingly connected to each source line, and is used to convert the source line voltage of each source line correspondingly connected to an output result in binary form. The comparator of the present invention can be in the field of integrated circuits. The standard device is a circuit that compares the analog voltage signal with the reference voltage. It can input two analog signals to realize the output as binary value 0 or 1, and when the input voltage fluctuates, the output value remains constant. In another embodiment of the present invention, the above-mentioned comparator may be replaced by a sampling circuit.

Each adder in the plurality of adders is connected to at least two comparators in the plurality of comparators through each corresponding source line, and is used for summing at least two output results corresponding to the at least two comparators, to implement deep neural networks. An adder is a logic device in the field of computer technology, which is used to perform digital addition operations. The deep neural network includes the processing of multiple convolution layer data or fully connected layer data.

Therefore, the coded flash memory device of the present invention outputs the output result in binary form through the comparator, which can not only maintain a high calculation accuracy when the threshold voltage fluctuation between different devices is relatively high, but also can avoid the similar traditional output The complexity of the resulting numerical value avoids the complex analog-to-digital conversion circuit required by traditional analog operations, and improves the energy efficiency ratio brought by the in-memory operation of the 3DNAND FLASH structure. Efficiency and parallelism, parallel encoding operations can be completed within one clock cycle.

According to an embodiment of the present invention, wherein, as shown in FIG. 1 , FIG. 3 and FIG. 5 , each flash memory array structural unit includes: a plurality of operation units C and a plurality of redundant units; Unit C is a transistor at the intersection of each word line and each source line in the multiple word lines in the flash memory array structural unit, wherein each source line is correspondingly provided with an arithmetic unit C; the arithmetic unit C is in the 3D NAND FLASH structure. A memory cell is preset in the memory cell string on each source line SL, and only this memory cell is responsible for data operation in the memory cell string of the source line SL. The value that can be stored in the operation unit C is 0 or 1, and the threshold voltage corresponding to the operation unit C is V _th =V _{th_Low} (that is, the value stored in the operation unit C is 0) or V _th =V _{th_High} (that is, the operation unit C stores a value of 0). Cell C stores the value 1). The operation unit is used for the operation of the fully connected layer or the operation of the convolution layer. During the operation of the fully connected layer, each non-zero bit element in the weight matrix vector of the fully connected layer is correspondingly stored in the corresponding operation. In the unit C; when performing the convolution layer operation, each convolution kernel element in the multiple convolution kernel elements in the convolution matrix of the convolution layer is correspondingly stored in the corresponding operation unit C.

The plurality of redundant units are transistors of the non-operation unit C in each flash memory array structural unit 100, and are used to be in an open state when the encoding operation is implemented. The redundant cell is the memory cell corresponding to the non-operational cell C in the memory cell string on each source line, and is equivalent to the transistor at the intersection of each word line and each source line, that is, the redundant cell. When the coding type flash memory device of the present invention performs coding budget, no matter the input value on the operation unit C is 0 or 1, the redundant unit is in an open state.

According to an embodiment of the present invention, as shown in FIG. 1 , FIG. 3 and FIG. 5 , wherein each flash memory array structural unit 100 further includes: a serial select line SSL (serial select line) and a ground select line GSL (ground select line) , the string selection line SSL is connected to the bit line end of the flash memory array structural unit 100. Specifically, the string selection line SSL is provided with a plurality of selection tubes, one end of each selection tube is connected to the bit line BL, and the other end is connected to the memory cell string. At the bit end, the string selection line SSL connects a plurality of selection tubes in series along the direction perpendicular to the source line SL.

The ground selection line GSL is connected to the source line end of the flash memory array structure unit 100; specifically, the ground selection line GSL is provided with a plurality of selection tubes, one end of each selection tube is connected to the source line SL, and the other end is connected to the source of the memory cell string. At the end, the ground selection line GSL connects a plurality of selection tubes in series along the direction perpendicular to the source line SL.

Among them, the string selection line SSL and the ground selection line GSL are used to apply a high level when implementing an encoding operation.

According to an embodiment of the present invention, wherein the encoding operation is a fully connected operation or a convolutional layer operation, and the number of the multiple adders is consistent with the number of operation results of the summation operation of the fully connected operation or the convolutional layer operation. Specifically, the number l of adders on each flash memory array structural unit 100 is consistent with the number of operation results of the full connection operation or the sum operation of the convolution layer operation on the flash memory array structural unit 100. The encoded flash memory of the present invention The number L of adders of the device is L=1×N, where N is the number of flash memory array structural units 100 connected in parallel in the coded flash memory device.

For the fully connected layer operation, as shown in Figure 3, the input vector X (1×M) is multiplied by the weight matrix vector K (M×N) to obtain the output vector Y (1×N). The mathematical expression is as follows (1 ):

Y _i =X ₁ ·K _1,i +X ₂ ·K _2,i +...+X _M ·K _M,i

where 1≤i≤N.

Therefore, the output results Y of the full connection operation may have N, and the number of adders required for the corresponding full connection operation is equal to N.

For the convolutional layer operation, as shown in Figure 5, for the fully connected layers of multiple local regions, each part of the matrix X (M×N) of the input vector is respectively subjected to a matrix-vector multiplication operation with the convolution matrix K (k×k) , complete the convolution operation and get the matrix Y(m×n) of the output vector, the mathematical expression is as follows (2):

Y _i,j =X _i,j ·K _k,k +X _i,j+1 ·K _k,k-1 +X _i+1,j ·K _k-1,k +...+X _{i+k- 1,j+k-1} ·K _1,1

where 1≤i≤M-k+1(m), 1≤j≤N-k+1(n).

Therefore, the output results Y of the convolutional layer operation can have (M-k+1)*(N-k+1), which constitute a complete output matrix, which corresponds to the number of adders required for the convolutional layer operation. Equal to (M-k+1)*(N-k+1).

Whether it is a convolutional layer operation or a fully connected operation, it is equivalent to a vector-matrix multiplication operation. In contrast, the convolutional layer increases the shift operation of the convolution kernel than the fully connected layer. When using the flash memory array structure unit 100 of the 3D NAND FLASH structure to implement two types of vector-matrix multiplication operations, the main difference is that the bit line BL input of the convolution layer is updated after each cycle to the next input image to be convolved. Partial area.

The embodiments of the coded flash memory device of the present invention have been described in detail above with reference to FIG. 1 , FIG. 3 and FIG. 5 .

The process of applying 3D NAND FLASH to vector-matrix multiplication and realizing parallelized data processing will now be described in detail.

Another aspect of the present invention discloses an encoding method, as shown in FIG. 2 , implemented based on the above-mentioned encoding type flash memory device, wherein the encoding method includes:

Step S410: Perform an encoding operation based on at least one flash memory array structural unit to generate a source line voltage on each of the multiple source lines of each flash memory array structural unit; The set operation unit stores a value of 0 or 1, and other redundant units store a value of 0. At this time, the redundant unit will be in an open state, corresponding to the word line voltage applied on the word line WL where the operation unit is located. V _g = V _{g_High} or V _g =V _{g_Low} , at this time, the source line voltage corresponding to the source line SL where the operation unit is located can be generated.

Step S420 : a plurality of comparators convert the source line voltage of each source line corresponding to each comparator into an output result in binary form; the comparator of the present invention can input analog signals in two channels, and realize the output as a binary value 0 or 1, and the output value remains constant when the input voltage fluctuates numerically.

Step S430 : The multiple adders perform an addition operation on at least two output results corresponding to at least two of the multiple comparators, so as to implement a deep neural network. An adder is a logic device in the field of computer technology, which is used to perform digital addition operations. The deep neural network includes the processing of multiple convolution layer data or fully connected layer data.

Based on the encoding method of the encoding type flash memory device of the present invention, the encoding method of parallel in-memory computing can be realized in the 3D NAND FLASH structure. Specifically, the coded flash memory device of the present invention can perform the functions of each computing layer in parallel and obtain accurate computing results by using the integrated storage and computing technology through the 3D NANDFLASH-based deep neural network assisted by the adder.

Therefore, the coding method of the present invention outputs the output result in binary form through the comparator, which can not only maintain a high calculation accuracy rate when the threshold voltage fluctuation between different devices is high, but also can avoid similar traditional output result values. It avoids the complex analog-to-digital conversion circuit required by traditional analog operations, improves the energy efficiency ratio brought by the in-memory operation of the 3D NANDFLASH structure, and finally maximizes the efficiency and efficiency of the entire computing architecture with the help of the adder. Parallelism, parallel coding operations can be completed within one clock cycle.

The coding method of the coded flash memory device having a single flash memory array structural unit of the present invention will be further explained below.

According to an embodiment of the present invention, as shown in FIG. 1 , for the above-mentioned encoded flash memory device of the present invention, a certain flash memory array structure unit 100 converts the weight matrix vector K (M×N) to the value matrix before the encoding operation starts. The non-zero bit elements in each element (K _{1 -KM} ) are respectively stored in the corresponding operation unit C on the M source lines SL of the 3D NAND FLASH structural unit, that is, the non-zero bit element K ₁ is stored in the word line WL1 At the intersection position with the source line SL1, the non-zero bit element K ₂ is stored in the intersection position of the word line WL2 and the source line SL2... and so on, the non-zero bit element K _M is stored in the word line WL _M and the source line SL _M. cross position.

As mentioned above, the value 0 or 1 stored in the operation unit C corresponds to the threshold voltage of the operation unit V _th =V _{th_Low} or V _{th_High} , only one device on each source line SL is the operation unit C, and the rest of the units It is set as a redundant unit, and the value stored in each redundant unit is 0, that is, the threshold voltage V _th =V _{th_Low} corresponding to the redundant unit, so that the redundant unit remains on.

The elements (X ₁ ˜X M ) of the input vector (1× _M ) are respectively converted into corresponding voltage values and applied to the word lines WL1 ˜ WL-M, and the corresponding voltage values correspond to the input 0 or 0 on the word line WL. 1, so that the word line voltage applied on the word line WL is V _g =V _{g_High} or V _{g_Low} . In this way, it can be ensured that the input data of the word line WL corresponding to the redundant cell will not affect the state of the redundant cell, that is, the redundant cell is always in the ON state.

Finally, the string selection line SSL and the ground selection line GSL of the flash memory array structure unit 100 are simultaneously increased to a high level, and the output of the source line SL only depends on the state of the corresponding operation unit C. Specifically: when the operation unit When the weight stored in C and the input data input through the word line WL are both 1, that is, the threshold voltage of the operation unit C is V _th =V _{th_High} , the word line voltage of the word line WL where it is located is V _g =V _{g_Low} , At this time, the operation unit C is in the off state, and the corresponding source line SL has a passive line current, which means that the operation result is 1; in other cases: V _th =V _{th_High} (1), V _g =V _{g_High} (0); V _th =V _{th_Low} (0), V _g =V _{g_High} (0); V _th =V _{th_Low} (0), V _g =V _{g_Low} (1), the operation units are all in the open state, the corresponding source line SL has active line current, representing The result of the operation is 0.

Through the comparator 200 connected to the source line at the end of the source line SL, the presence or absence of the source line current can be read out and converted into a calculation result of 0 or 1, that is, as the output result of the comparator 200 in binary form; the operation of different source lines SL The output result obtained by the unit C is added by the adder, and the summation result of the input data and the weight data can be obtained. At this time, the deep neural network is realized.

The coding method of the coded flash memory device having a plurality of flash memory array structural units of the present invention will be further explained below.

According to an embodiment of the present invention, the encoding operation is a fully connected operation or a convolutional layer operation.

For the fully connected layer operation, as shown in Figure 3, the input vector X (1×M) is multiplied by the weight matrix vector K (M×N) to obtain the output vector Y (1×N). The mathematical expression is as follows (1 ):

Y _i =X ₁ ·K _1,i +X ₂ ·K _2,i +...+X _M ·K _M,i

where 1≤i≤N.

For the convolutional layer operation, as shown in Figure 5, for the fully connected layers of multiple local regions, each part of the matrix X (M×N) of the input vector is respectively subjected to a matrix-vector multiplication operation with the convolution matrix K (k×k) , complete the convolution operation and get the matrix Y(m×n) of the output vector, the mathematical expression is as follows (2):

Y _i,j =X _i,j ·K _k,k +X _i,j+1 ·K _k,k-1 +X _i+1,j ·K _k-1,k +...+X _{i+k- 1,j+k-1} ·K _1,1

where 1≤i≤M-k+1(m), 1≤j≤N-k+1(n).

According to another embodiment of the present invention, as shown in FIG. 3 and FIG. 4 , wherein the encoding method of the embodiment of the present invention is implemented based on the encoding flash memory device shown in FIG. 3 , when the encoding operation is a fully connected operation, before step S410 , the encoding method also includes:

Step S510: Pre-store each non-zero bit element of the multiple non-zero bit elements in the weight matrix vector of the fully connected layer into a corresponding operation unit C of the multiple operation units C in the coded flash memory device.

According to yet another embodiment of the present invention, after step S510, the encoding method further includes:

Step S520: Input the input elements of the corresponding input vector into each word line of the plurality of word lines, so as to generate a word line voltage on each word line, so that the plurality of redundant cells of the coded flash memory device are in the open state.

According to yet another embodiment of the present invention, after step S520, it further includes:

Step S530: Apply a high level to the string selection line and the ground selection line of the coded flash memory device to generate a source line voltage on each of the plurality of source lines of each flash memory array structural unit.

Specifically, when the encoding operation is a fully connected operation, as shown in FIG. 3 and FIG. 4 , a flash memory array structural unit 100 (ie, a 3D NAND FLASH structural unit) in the encoded flash memory device can be used as a single “serial” "Responsible for processing the full connection operation of the single-bit input vector X (1×M) and the single-bit weight matrix K (M×N).

There are N groups of storage and arithmetic modules on the same string, corresponding to output vectors Y-1 to Y-N respectively. Each set of memory modules includes M source lines SL, which correspond to the input word lines WL-1 to WL-M respectively. Before the operation starts, it is necessary to configure the storage and computing resources of the 3D NAND FLASH structural unit, and write the non-zero elements of the weight matrix into the 3D NAND FLASH according to the corresponding relationship between the input vector and each element of the weight matrix vector when performing vector-matrix multiplication. The corresponding operation unit C in the structural unit corresponds to step S510.

The input elements of the input vectors (X ₁ ˜X _M ) of different strings are applied to the corresponding word lines WL1 ˜WL-M, and are respectively converted into the input word line voltages on the corresponding WL1 ˜WL-M, corresponding to step 520 , so that Redundant cells on the 3D NAND FLASH structural cells are on.

When the string selection line SSL line and the ground selection line GSL line are at high level, the encoding operation starts: because the same source line SL has only one FLASH unit (ie the operation unit C), the FLASH unit stores the weight value (ie the aforementioned weight value). The non-zero element of the matrix) and perform operation, the operation result (0 or 1) can be reflected by the source line voltage, that is, corresponding to step S530.

The source line current value corresponding to the source line voltage is read out by the comparator and converted into an output result in binary form. The N adders respectively add the operation results of the output results of the corresponding N groups of comparators (each group of comparators corresponds to M pieces of SL) to obtain the output results Y1 to Y-N of the fully connected layer.

According to yet another embodiment of the present invention, different strings (SSL-1 to SSL-N) cooperatively process multi-bit inputs and fully-connected operations of multi-bit weight matrices. When the encoding operation is a fully connected operation, the input vector is multi-bit data, and the multi-bit data may be data of a multi-bit input vector and a multi-bit weight matrix vector. Step S410 includes:

Based on a plurality of flash memory array structural units performing encoding operations corresponding to the input vector and the weight matrix vector at the same time, since the plurality of flash memory array structural units are connected in parallel in the encoded flash memory device of the present invention, it can simultaneously realize multi-bit encoding operations. The full connection operation of the input and the multi-bit weight matrix, that is, each flash memory array structural unit of the multiple flash memory array structural units realizes the full connection operation of single-bit data.

Or based on a flash memory array structural unit, the coding operation of the corresponding input vector and the weight matrix vector is performed in a time-division multiplexing manner; for the case where the number of flash memory array structural units is relatively limited, it can be based on one flash memory array structural unit at different times. , which is divided into multiple times to perform the full connection operation of the multi-bit input and the multi-bit weight matrix, that is, each time the flash memory array structural unit implements the full connection operation of the single-bit data.

It can be seen that the encoding method according to the embodiment of the present invention, based on the in-memory computing architecture of the 3D NAND FLASH structural unit, can complete parallel fully connected operations within one clock cycle.

According to another embodiment of the present invention, as shown in FIG. 5 and FIG. 6 , the encoding method of the embodiment of the present invention is implemented based on the encoding flash memory device shown in FIG. 5 , and when the encoding operation is a convolution layer operation, in step S410 Previously, encoding methods also included:

Step S610: Pre-store each convolution kernel element in the multiple convolution kernel elements in the convolution matrix of the convolution layer into a corresponding operation unit among the multiple operation units in the coded flash memory device.

According to another embodiment of the present invention, after step S610, the encoding method further includes:

Step S620: Input the input elements of the corresponding input vector into each word line of the plurality of word lines, so as to generate a word line voltage on each corresponding word line, so that the plurality of redundant cells of the encoded flash memory device are in the open state.

According to another embodiment of the present invention, after step S620, the encoding method further includes:

Step S630: Apply a high level to the string selection line and the ground selection line of the coded flash memory device to generate a source line voltage on each of the plurality of source lines of each flash memory array structural unit.

Specifically, when the encoding operation is a convolution layer operation, as shown in Figure 5 and Figure 6, each string of the 3D NAND FLASH structural unit is responsible for processing the single-bit input matrix X (M × N) and the convolution kernel K (k ×k) convolution operation. Taking Figure 5 as an example, input a 3*3 image (X _1,1 ,X _1,2 ,X _1,3 ,X _2,1 ,X _2,2 ,X _2,3 ,X _3,1 ,X _{3, 2} ,X _3,3 ) and the 2*2 convolution kernel (K _1,1 ,K _1,0 ,K _0,1 ,K _0,0 )The 2*2 output image obtained after convolution is (Y _{1, 1} ,Y _1,2 ,Y _2,1 ,Y _2,2 ). To implement the above convolution operation, the input image (that is, the corresponding input matrix vector) needs to be split into four sub-matrices (X _1,1 ,X _1,2 ,X _2,1 ,X _2,2 ), (X _1,2 ,X _1,3 ,X _2,2 ,X _2,3 ),(X _2,1 ,X _2,2 ,X _3,1 ,X _3,2 ),(X _2,2 ,X _2,3 , X _3,2 ,X _3,3 ) and the convolution kernel (K _1,1 ,K _1,0 ,K _0,1 ,K _0,0 ) vector-matrix multiplication respectively to obtain Y _1,1 ,Y _1,2 , Y _2,1 ,Y _2,2 .

During the encoding operation, each convolution kernel element in the convolution matrix of the convolution layer is mapped to the corresponding operation unit C of the input vector (X _1,1 ,X _1,2 ,X _2,1 ,X _2,2 ) , that is, the intersection positions of the bit lines WL1 , WL2 , WL4 , and WL5 corresponding to the source lines SL, that is, corresponding to step S610 . Considering that each convolutional layer in the deep neural network has multiple convolution kernels to perform multi-dimensional feature extraction on the input image, other convolution kernels can be written into the corresponding operation units C of WL1-WL9 respectively according to the same principle.

The input image is split into row vectors (X _1,1 ,X _1,2 ,…,X _1,N ,…,,,X _M,1 ,X _M,2 ,…,X _M,N ) in the row direction and It is applied to the word line WL of the 3D NAND FLASH structure unit as an input to be converted into an input word line voltage on the corresponding word line WL, corresponding to step S620.

When the string selection line SSL line and the ground selection line GSL line add high level, the operation starts: because the same source line SL has only one FLASH unit (ie, the operation unit C), the FLASH unit stores the convolution kernel element value and performs the operation, The operation result (0 or 1) can be reflected by the source line voltage, which corresponds to step S630. Finally, as shown in FIG. 5 , the four adders 300 respectively add the output results of the corresponding four groups of comparators 200 (each group of four source lines SL) to obtain the output results of the convolution layer operation. (Y _1,1 ,Y _1,2 ,Y _2,1 ,Y _2,2 ), which implements a deep neural network.

According to another embodiment of the present invention, when the encoding operation is a convolution layer operation, the convolution matrix of the convolution layer is multi-bit data. The encoding method of the present invention also includes:

Step S440: Shifting and summing the summation results output by the summation operations of multiple adders to implement a deep neural network; The result is shifted and summed, that is, the output image after the input image is convolved with the multi-bit convolution kernel can be obtained.

It can be seen that the encoding method of the present invention is based on the above encoding flash memory device, and can realize parallel convolution operations in one clock cycle on the in-memory computing architecture based on the 3D NAND FLASH structure.

It can be seen that, in the encoding method of the present invention, before the encoding operation starts, the non-zero-bit elements in the weight matrix vector or the convolution kernel elements in the convolution vector are written into the position corresponding to the 3D NAND FLASH structural unit according to a certain mapping rule (operation unit), the value (0 or 1) of different elements in the input vector determines the input voltage on the word line WL in the structural unit of the flash memory array (ie the word line voltage V _g =V _{g_High} or V _{g_Low} ), and then the corresponding source line SL The output source line current reflects the calculation result of the arithmetic unit. Finally, through a plurality of comparators arranged at the ends of each source line SL in the 3D NAND FLASH structural unit, the calculation result is converted into binary form (0 or 1) and read out in parallel. On this basis, the adder connected to the comparator adds the output results of the comparators from multiple source lines to obtain the final summation result, which realizes efficient and accurate fully connected or convolutional layer operations. So as to realize the deep neural network.

So far, the embodiments of the present invention have been described in detail with reference to the accompanying drawings.

It should be noted that, in the accompanying drawings or the text of the description, the implementations that are not shown or described are in the form known to those of ordinary skill in the technical field, and are not described in detail. In addition, the above definitions of various elements and methods are not limited to various specific structures, shapes or manners mentioned in the embodiments, and those of ordinary skill in the art can simply modify or replace them.

It should also be noted that the directional terms mentioned in the embodiments, such as "up", "down", "front", "rear", "left", "right", etc., only refer to the directions of the drawings, not used to limit the scope of protection of the present invention. Throughout the drawings, the same elements are denoted by the same or similar reference numbers. Conventional structures or constructions will be omitted when it may lead to obscuring the understanding of the present invention.

Moreover, the shapes and sizes of the components in the figures do not reflect the actual size and proportion, but merely illustrate the contents of the embodiments of the present invention. Furthermore, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.

Furthermore, the word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

The ordinal numbers such as "first", "second", "third", etc. used in the description and the claims are used to modify the corresponding elements, which themselves do not mean that the elements have any ordinal numbers, nor do they Representing the order of an element from another element or the order in the manufacturing method, the use of these ordinal numbers is only used to clearly distinguish an element with a certain name from another element with the same name.

Those skilled in the art will appreciate that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Also, in a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware.

Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together , or in its description. This disclosure, however, should not be construed as reflecting an intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, disclosed aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in further detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (12)

1. An encoding type flash memory device, comprising:

at least one flash memory array structure unit, wherein each flash memory array structure unit is a 3D NAND FLASH array structure unit, and is used for realizing coding operation to generate source line voltage on each source line in a plurality of source lines of the flash memory array structure unit;

a plurality of comparators, wherein each comparator is correspondingly connected with each source line and is used for converting the source line voltage of each correspondingly connected source line into an output result in a binary form; and

and each adder is connected with at least 2 comparators in the comparators through corresponding source lines and is used for summing at least two output results corresponding to the at least 2 comparators so as to realize the deep neural network.

2. The coded flash memory device according to claim 1, wherein each of the flash array structure units comprises:

the memory comprises a plurality of operation units, wherein each operation unit is a transistor at the crossing position of each word line in a plurality of word lines and each source line in the flash memory array structure unit, and one operation unit is correspondingly arranged on each source line;

and the redundant units are transistors of non-operation units in each flash array structure unit and are used for being in an on state when the coding operation is realized.

3. The coded flash memory device according to claim 2, wherein each of said flash array structure units further comprises:

the string selection line is connected with the bit line end of the flash memory array structure unit;

the ground selection line is connected with a source line end of the flash memory array structure unit;

wherein the string select line and the ground select line are used to apply a high level when implementing the encoding operation.

4. The coded flash memory device of claim 1, wherein the coding operation is a full concatenation operation or a convolutional layer operation,

the number of the adders is equal to the number of operation results of the addition operation of the full-concatenation operation or the convolution layer operation.

5. An encoding method implemented based on the encoding type flash memory device of any one of claims 1 to 4, wherein the encoding method comprises:

performing encoding operation based on at least one flash memory array structure unit to generate source line voltage on each source line in a plurality of source lines of each flash memory array structure unit;

a plurality of comparators convert the source line voltage of each source line corresponding to each comparator into an output result in a binary form; and

and the adders add at least 2 output results corresponding to at least 2 comparators in the comparators to realize the deep neural network.

6. The encoding method of claim 5, wherein the encoding operation is a fully concatenated operation or a convolutional layer operation.

7. The encoding method of claim 6, wherein when the encoding operation is a full-link operation, before the encoding operation performed on at least one flash memory array structure unit generates a source line voltage on each source line of a plurality of source lines of each flash memory array structure unit, the method further comprises:

and pre-storing each non-zero element in a plurality of non-zero elements in the weight matrix vector of the full connection layer into a corresponding operation unit in a plurality of operation units in the coding type flash memory device.

8. The encoding method of claim 6, wherein when the encoding operation is a convolutional layer operation, before the encoding operation performed by at least one flash memory array structure unit generates a source line voltage on each source line in a plurality of source lines of each flash memory array structure unit, the method further comprises:

and pre-storing each convolution kernel element in a plurality of convolution kernel elements in the convolution matrix of the convolution layer into a corresponding operation unit in a plurality of operation units in the coding type flash memory device.

9. The encoding method according to claim 7 or 8,

after pre-storing each non-zero element in a plurality of non-zero elements in the weight matrix vector of the fully-connected layer into a corresponding operation unit in a plurality of operation units in the coded flash memory device, or after pre-storing each convolution kernel element in a plurality of convolution kernel elements in the convolution matrix of the convolutional layer into a corresponding operation unit in a plurality of operation units in the coded flash memory device, the method further includes:

correspondingly inputting the input elements of the corresponding input vector into each word line of the plurality of word lines to generate word line voltages on each word line, so that the plurality of redundant units of the coding type flash memory device are in an on state.

10. The encoding method of claim 9, wherein after correspondingly inputting the input elements of the corresponding input vector into each of the plurality of word lines to generate a word line voltage on the corresponding each word line such that the plurality of redundant cells of the encoding-type flash memory device are in an on state, further comprising:

applying a high level to a string select line and a ground select line of the encoding type flash memory device to generate a source line voltage on each of a plurality of source lines of each flash memory array structural unit.

11. The encoding method of claim 5, wherein when the encoding operation is a full join operation, the performing the encoding operation based on at least one flash memory array structural unit to generate a source line voltage on each source line of a plurality of source lines of each flash memory array structural unit comprises:

based on multiple flash memory array structure units, simultaneously performing coding operation corresponding to input vector and weight matrix vector, or

Performing coding operation of corresponding input vectors and weight matrix vectors by adopting a time division multiplexing mode based on a flash memory array structure unit;

wherein the input vector is multi-bit data.

12. The encoding method of claim 5, wherein when the encoding operation is a convolutional layer operation, the encoding method further comprises:

shifting and summing the summation result output by the summation operation of the summers to realize a deep neural network;

wherein the convolution matrix of the convolutional layer is multi-bit data.

Images