KR20200067632A - Method and apparatus for allocating memory space for driving a neural network - Google Patents

Abstract

The present invention relates to a method and a device for allocating a memory space for driving a neural network. The method for allocating a memory space for driving a neural network comprises: allocating a space to store an input feature map in a memory based on an initial address value of the memory and capacity information of the input feature map; and allocating a space to store an output feature map in the memory based on a last address value of the memory and capacity information of the output feature map.

Description

Method and apparatus for allocating memory space for driving a neural network}

It relates to a method and apparatus for allocating memory space for driving a neural network. Specifically, it relates to a method and apparatus for allocating memory space for each of a plurality of layers of a neural network.

A neural network refers to a model in which artificial neurons that form a network by combining synapses change the binding strength of synapses through learning, thereby having a problem-solving ability.

Devices that process neural networks require large amounts of computation on complex input data. For example, the neural network performs many operations on each image and generates a lot of intermediate result data.

Accordingly, in a process in which a processor of a neural network reads or writes many intermediate result data from an external memory, performance of the neural network may be deteriorated due to a limitation of the bandwidth of the external memory.

Disclosed is a method and apparatus for allocating memory space for driving a neural network.

The technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may be inferred from the following embodiments.

A first aspect of the present disclosure includes a first capacity information regarding a capacity of a space to store an input feature map of a first layer among a plurality of layers of a neural network and a space to store an output feature map of the first layer. Obtaining second capacity information about the capacity; And a first storage space to store the input feature map in the memory based on the initial address value of the memory and the first capacity information, and based on the last address value of the memory and the second capacity information. And allocating a second storage space to store the output feature map in a memory, to provide a method for allocating memory space for a plurality of layers of a neural network.

A second aspect of the present disclosure includes a memory; And And a processor for driving a neural network by executing at least one program, wherein the processor includes first capacity information regarding a capacity of a space to store an input feature map of a first layer among a plurality of layers of the neural network and the First storage for acquiring second capacity information regarding a capacity of a space to store the output feature map of the first layer, and storing the input feature map in the memory based on the initial address value of the memory and the first capacity information It is possible to provide a neural network device that allocates space and allocates a second storage space to store the output feature map in the memory based on the last address value of the memory and the second capacity information.

A third aspect of the present disclosure can provide a computer-readable recording medium recording a program for executing the method of the first aspect on a computer.

The present disclosure can provide a method and apparatus for allocating memory space for driving a neural network. Specifically, for each of the plurality of layers of the neural network, an internal memory space located in the processor may be allocated. Specifically, space for storing the input feature map and the output feature map of the layer may be allocated to both ends of the internal memory space. Since the output feature map of the current layer becomes the input feature map of the next layer, if space for storing the input feature map and the output feature map is allocated to both ends of the internal memory space, access to external memory can be reduced. have. Therefore, it is possible to minimize a phenomenon in which the performance of the neural network is degraded due to the bandwidth limitation of the external memory.

1 is a view for explaining an example of a neural network architecture.
2 is a diagram for explaining an example of the relationship between an input feature map and an output feature map in a neural network.
3 is a diagram illustrating an example of a neural network device.
4 is a diagram illustrating an example of a memory space allocation method.
5 is a diagram illustrating an example of a process in which overhead occurs due to a memory fragmentation phenomenon.
6 is a view showing an example of dividing a layer into a plurality of sub-layers.
7 is a diagram illustrating an example of allocating memory by dividing a layer into a plurality of sub-layers.
8 is a diagram illustrating an example of allocating memory by grouping tiles in a plurality of layers.
9 is a flowchart illustrating an example of a process of allocating memory space in a neural network device.
10 is a flowchart illustrating another example of a process of allocating memory space in a neural network device.
11 is a flowchart illustrating an example of a process of allocating a space for storing an input feature map and an output feature map in memory in each layer of a neural network.

The terminology used in the present embodiments was selected from general terms that are currently widely used as possible while considering functions in the present embodiments, but this may vary depending on the intention or precedent of a person skilled in the art or the appearance of new technology. Can be. In addition, in certain cases, some terms are arbitrarily selected, and in this case, their meanings will be described in detail in the description of the corresponding embodiment. Therefore, the terms used in the present embodiments should be defined based on the meaning of the terms and the contents of the present embodiments, not simply the names of the terms.

In the descriptions of the embodiments, when a part is connected to another part, this includes not only the case of being directly connected, but also the case of being electrically connected with another component in between. . Also, when a part includes a certain component, this means that other components may be further included rather than excluding other components unless otherwise specified.

The terms "consisting of" or "comprising" as used in the embodiments should not be construed as including all of the various components, or various steps described in the specification, and some of them or It should be construed that some steps may not be included, or may further include additional components or steps.

The description of the following embodiments should not be construed as limiting the scope of rights, and those that can be easily inferred by those skilled in the art should be interpreted as belonging to the scope of the embodiments. Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

1 is a view for explaining an example of a neural network architecture.

Referring to FIG. 1, the neural network 1 may be an architecture of a deep neural network (DNN) or an n-layers neural network. The DNN or n-layer neural network may correspond to Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Belief Networks, Restricted Boltzman Machines, and the like. For example, the neural network 1 may be implemented as a convolutional neural network (CNN). In FIG. 1, in the convolutional neural network corresponding to the example of the neural network 1, in addition to the convolutional layer, a subsampling layer (or a pooling layer), a fully connected layer, etc. may be further included. have.

The neural network 1 may be implemented with an architecture having multiple layers including input images, feature maps and outputs. In the neural network 1, the input image is subjected to a convolution operation with a weight map, and as a result, feature maps are output. The weight map is a parameter for finding characteristics of an input image, and is also called a kernel or filter. At this time, the generated output feature maps are input feature maps, and convolution operation with the weight map is performed again, and new feature maps are output. As a result of such a convolution operation being repeatedly performed, a result of recognition of characteristics of the input image through the neural network 1 may be finally output.

For example, when an image having a size of 24x24 pixels is input to the neural network 1 of FIG. 1, the input image may be output as feature channels of 4 channels having a size of 20x20 through convolution operation with a weight map. Also, through the subsampling process, only some of the pixel values of the feature map of 4 channels having a size of 20x20 may be used to output feature maps of 4 channels having a size of 10x10. As a subsampling method, a method such as max-pooling or average-pooling may be applied.

Afterwards, the 10x10 feature maps are reduced in size through iterative convolution and subsampling operations with the weight map, and finally, global features can be output. The neural network 1 filters and outputs robust features that can represent the entire image from the input image by repeatedly performing convolution and subsampling (or pooling) operations on multiple layers. By inputting to the connected layer, the recognition result for the input image can be finally obtained.

2 is a diagram for explaining an example of the relationship between an input feature map and an output feature map in a neural network.

Referring to FIG. 2, in a layer 2 of a neural network, the first feature map FM1 may correspond to an input feature map, and the second feature map FM2 may correspond to an output feature map. The feature map may mean a data set in which various characteristics of input data are expressed. The feature maps FM1 and FM2 may have elements of a 2D matrix or elements of a 3D matrix, and a pixel value may be defined for each element. The feature maps FM1 and FM2 have a width W (or a column), a height H (or a row), and a depth D. At this time, the depth D may correspond to the number of channels.

The convolution operation for the first feature map FM1 and the weight map WM may be performed, and as a result, the second feature map FM2 may be generated. The weight map filters characteristics of the first feature map FM1 by performing a convolution operation with the first feature map FM1 with a weight parameter defined in each element. The weight map shifts the first feature map FM1 in a sliding window manner and performs convolutional operations with the windows of the first feature map FM1 (also referred to as tiles). During each shift, each of the weight parameters included in the weight map may be multiplied and added to each of the pixel values of the overlapped window in the first feature map FM1. As the first feature map FM1 and the weight map are convolved, one channel of the second feature map FM2 may be generated. Although only one weight map is shown in FIG. 1, in reality, a plurality of weight maps may be convolved with the first feature map FM1, so that a second feature map FM2 of a plurality of channels may be generated.

Meanwhile, the second feature map FM2 may correspond to an input feature map of the next layer. For example, the second feature map FM2 may be an input feature map of a pooling (or subsampling) layer.

1 and 2 are shown only for the schematic architecture of the neural network 1 for convenience of explanation. However, the neural network 1 can be implemented with more or fewer layers, feature maps, weight maps, and the like, as shown, and its sizes can also be variously modified. Those skilled in the art can understand.

3 is a diagram illustrating an example of a neural network device.

The neural network apparatus 300 may be implemented with various types of devices such as a personal computer (PC), a server device, a mobile device, and an embedded device. As a specific example, voice recognition, image recognition, image classification, etc. using a neural network Smartphones, tablet devices, AR (Augmented Reality) devices, Internet of Things (IoT) devices, autonomous vehicles, robotics, medical devices, and the like, which are performed, but are not limited thereto. Further, the neural network device 300 may correspond to a dedicated hardware accelerator (HW accelerator) mounted on the above device, and the neural network device 300 may be a neural processing unit (NPU), which is a dedicated module for driving the neural network. It may be a hardware accelerator such as a TPU (Tensor Processing Unit), a Neural Engine, etc., but is not limited thereto.

Referring to FIG. 3, the neural network device 300 may include a processor 310, an internal memory 320 and an external memory 330. In the neural network device 300 illustrated in FIG. 3, only components related to the present embodiments are illustrated. Accordingly, it is apparent to those skilled in the art that the neural network device 300 may further include other general-purpose components in addition to the components shown in FIG. 3.

The processor 310 serves to control overall functions for executing the neural network device 300. For example, the processor 310 controls the neural network device 300 as a whole by executing programs stored in the neural network device 300. The processor 310 may be implemented as a central processing unit (CPU), graphics processing unit (GPU), or application processor (AP) provided in the neural network device 300, but is not limited thereto.

The external memory 330 is hardware storing various data processed in the neural network device 300. For example, the external memory 330 stores data processed in the neural network device 300 and data to be processed. Can be saved. The external memory 330 may store applications, drivers, and the like to be driven by the neural network device 300. The external memory 330 includes random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and CD -ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory, but is not limited thereto.

For example, the external memory 330 may store an output feature map that is an intermediate calculation result in each of the plurality of layers of the neural network or weight maps that are not used in the current layer on which the operation is performed. On the other hand, the neural network can perform giga-operation per second (GOPS) for each input image, use approximately hundreds to hundreds of billions of weight maps, and generate hundreds of gigabytes of intermediate calculation results. Accordingly, when the frequency of access to the external memory 330 of the processor 310 increases, the execution speed of the neural network device 300 may be reduced by consuming a significant portion of the bandwidth of the external memory 330. .

Accordingly, the neural network device 300 may further include an internal memory 320 in the processor 310 to access data faster than accessing the external memory 330 directly. The internal memory 320 may be a cache-to-memory or a static random access memory (SRAM) of the processor 310, but is not limited to this and may include various types of memory.

The processor 310 may efficiently allocate space in the internal memory 320 to minimize access to the external memory 330 in each of the plurality of layers.

Specifically, the processor 310 may acquire capacity information regarding a capacity of a space for storing various data used or generated in each layer, for each of the plurality of layers. The processor 310 may obtain first capacity information regarding the capacity of the space to store the input feature map of the layer and second capacity information regarding the capacity of the space to store the output feature map of the layer. The first capacity information is a capacity of space required to store the input feature map, and may correspond to the size of the input feature map data. Likewise, the second capacity information is a capacity of space required to store the output feature map, and may correspond to the size of the output feature map data.

The processor 310 may allocate a space to store each of the input feature map and the output feature map of the layer in the internal memory 320 based on the acquired capacity information. The processor 310 allocates a first storage space to store the input feature map in the internal memory 320 based on the initial address value of the internal memory 320 and the first capacity information, and the last of the internal memory 320 A second storage space to store the output feature map in the internal memory 320 may be allocated based on the address value and the second capacity information.

The first storage space corresponds to the space from the initial address value of the internal memory 320 to the first address value, and the second storage space corresponds to the space from the second address value to the last address value of the internal memory 320. can do. The first address value and the second address value may be any address value between the initial address value and the last address value in memory. On the neural network, since the output feature map of the current layer becomes the input feature map of the next layer, the space to store the input feature map and the output feature map of the layer is allocated to both ends of the internal memory 320, respectively, thereby the external memory 330. Access to the server can be minimized.

4 is a diagram illustrating an example of a memory space allocation method.

4 illustrates an example in which the processor 310 allocates memory 400 space in each of the first, second, and third layers of the neural network. The memory 400 of FIG. 4 may correspond to the internal memory 320 of FIG. 3.

The memory 400 in the neural network device 300 may have a memory 400 capacity of 10K (byte), for example. At this time, the memory 400 can distinguish data by numbering from 0 (0K) to 10000 (10K), and this number may be referred to as an address value of the memory 400. That is, the location of the data stored in the memory can be specified using address values from 0K to 10K. Meanwhile, the above-described capacity of the memory 400 is only an example, and the memory 400 may have various sizes. In addition, the address value of the memory 400 is not limited as described above, and may be represented in various ways.

First, the processor 310 may obtain first capacity information regarding a capacity of a space to store an input feature map of the first layer and second capacity information about a capacity of a space to store the output feature map of the first layer.

The first capacity information may be determined based on the width, height and number of channels of the input feature map, and the second capacity information may also be determined based on the width, height and number of channels of the output feature map. For example, the processor 310 may provide information that the capacity required to store the input feature map of the first layer is 3K (byte), and the capacity required to store the output feature map of the first layer is 2K (byte). Can be obtained.

Processor 310 is based on the capacity information that the capacity required to store the input feature map of the first layer is 3K (byte), the memory corresponding to the space from the initial address value 0K to 3K address of the memory 400 One storage space 410 may be allocated in the memory 400. Further, based on the capacity information of 2K (byte), the capacity required to store the output feature map of the first layer corresponds to a space from 8K address of the memory 400 to 10K address, which is the last address value of the memory 400. The second storage space 420 may be allocated in the memory 400.

In addition, the processor 310 may further obtain fourth capacity information regarding the capacity of the space to store the input feature map of the first layer and the weight map on which the calculation is performed. The fourth capacity information is a capacity of space required to store the weight map of the first layer, and may be determined based on the size and number of weight maps. The processor 310 may allocate space for storing a weight map between the first memory 400 space and the second memory 400 space based on the acquired fourth capacity information. Specifically, the space to store the weight map may be at least part of the space from the first address value to the second address value in memory.

Meanwhile, the processor 310 may allocate space in the memory 400 to store various types of data generated in the process of convolution of a layer, in addition to the input feature map, the output feature map, and the weight map, and working data (working) data). For example, the working data may include intermediate results in a convolution operation process.

Depending on how the processor 310 allocates space in the memory 400 in the first layer, the input feature map and the weight map of the first layer may be stored in the memory 400. When the convolution operation between the input feature map and the weight map is completed, the output feature map of the first layer may be stored in the second storage space 420.

Thereafter, the processor 310 may reallocate the memory 400 space for the second layer following the first layer. In this case, the output feature map generated as a result of the convolution operation in the first layer may be an input feature map of the second layer. Accordingly, the processor 310 may allocate a space to store the input feature map of the second layer to the second storage space 420, and the output feature map of the first layer stored in the second memory 400 space to the second It can be used as it is as an input feature map of a layer.

Also, as in the first layer, the processor 310 may acquire third capacity information regarding the capacity of the space to store the output feature map of the second layer. The processor 310 may allocate a third storage space 430 to store the output feature map of the second layer in the memory 400 based on the initial address value of the memory 400 and the acquired third capacity information. For example, when the capacity required to store the output feature map of the first layer is 1K (byte), the processor 310 may control the space corresponding to the space from the address 0K to address 1K of the initial address of the memory 400. 3 The storage space 430 may be allocated in the memory 400.

Similarly, when the processor 310 allocates memory 400 space for the third layer, the output feature map of the second layer becomes an input feature map of the third layer, and therefore, a space to store the input feature map of the third layer Can be allocated to the third storage space 430.

On the other hand, when each of the spaces for storing the input feature map and the output feature map of each layer is allocated to both ends in the memory 400, memory fragmentation may be prevented. Hereinafter, the memory fragmentation phenomenon will be described in more detail with reference to FIG. 5.

5 is a diagram illustrating an example of a process in which overhead occurs due to a memory fragmentation phenomenon.

5 illustrates an example of memory 500 space allocation of the processor 310 in each of the first layer and the second layer of the neural network. The memory 500 of FIG. 5 may correspond to the internal memory 320 of FIG. 3.

The processor 310 may allocate space from 4K to 7K of the memory 500 to store the output feature map of the first layer. When the convolution operation between the input feature map and the weight map is completed in the first layer, the output feature map of the first layer may be stored in a space from 4K to 7K of the memory 500.

Thereafter, the processor 310 may reallocate the memory 500 space for the second layer following the first layer. Since the input feature map of the second layer is the same as the output feature map generated by the convolution operation in the first layer, the output feature map of the first layer stored in the space from 4K to 7K of the memory 500 is second. It can be used as it is as an input feature map of a layer.

Thereafter, the processor 310 may obtain capacity information regarding a capacity of a space to store each of the output feature map, weight map, and walking data of the second layer. For example, the processor 310 may obtain capacity information of 1K (byte), 5K (byte), and 1K (byte), respectively, of capacity required to store the output feature map, weight map, and walking data, respectively.

The processor 310 allocates space from the address 0K to 1K, which is the initial address value of the memory 500, to store the output feature map of the second layer, and from the 9K address of the memory 500 to store working data Space up to 10K, the last address value, can be allocated. At this time, the space from the 1K address to the 4K address and the space from the 7K address to the 9K address may be available in the memory 500, but the space required to store the weight map is 5K (byte). 500) A memory fragmentation phenomenon that cannot be allocated occurs.

Accordingly, a process in which the processor 310 is distributed in the memory 500 and is present in a fragmented memory 500 space is combined and the memory 500 space may be rearranged, and overhead may be required in this process. Can occur.

Returning to FIG. 4 again, in the case of FIG. 4, each of the spaces for storing the input feature map and the output feature map of each layer is allocated at both ends in the memory 400. Therefore, since the memory fragmentation phenomenon as shown in FIG. 5 does not occur, it is possible to solve the overhead problem.

6 is a view showing an example of dividing a layer into a plurality of sub-layers.

Referring to FIG. 6, the processor 310 may generate an output feature map 620 by performing a convolution operation between the input feature map 600 and the weight map 610 of the first layer. However, when calculating in the first layer, the capacity of the space for storing various data necessary for calculation such as the input feature map 600, the weight map 610, and the output feature map 620, or generated in the calculation process, is the processor 310. ) It can be larger than the size of the memory space located inside. In this case, the frequency at which the processor 310 reads or writes data from the external memory increases, and execution speed may decrease.

Accordingly, the processor 310 divides the first layer into a first sub-layer, a second sub-layer, and a third sub-layer, and allocates memory space to each of the divided sub-layers and performs calculation on the external memory. Access can be reduced. For example, when the size of the weight map 610 is dominant, the processor 310 may perform a method of dividing the layer into sub-layers.

The input feature map 600 of the first layer and the weight map 610 on which calculation is performed may also be divided into a plurality of sub weight maps and allocated to each sub layer. The processor 310 allocates memory for each of the divided sub-layers, and may perform calculation based on the memory.

For example, a first sub-weight map 611 is assigned to the first sub-layer, and convolution operation between the input feature map 600 and the first sub-weight map 611 may be performed in the first sub-layer. . Similarly, the second sub-weight map 612 is allocated to the second sub-layer, and convolutional calculation with the input feature map 600 can be performed, and the third sub-weight map 613 is allocated to the third sub-layer. As a result, the input feature map 600 and the convolution operation may be performed.

As a result of the operation, the processor 310 may generate each of the channels of the output feature map 620 for each of the divided sub-layers. Specifically, each of the sub-weight maps can traverse each channel of the input feature map, calculate a composite product, and synthesize the calculation results to make each of the channels of the output feature map 620.

For example, the first channel output feature map 621 may be generated by an operation between the input feature map 600 and the first sub weight map 611 in the first sub layer, and the second channel output feature map ( 622) may be generated by an operation between the input feature map 600 and the second sub weight map 612 in the second sub layer. Similarly, the third channel output feature map 623 may be generated by an operation between the input feature map 600 and the third sub weight map 613 in the third sub layer. When all of the first channel output feature map 621, the second channel output feature map 622, and the third channel output feature map 623 are combined, the same result as the output feature map 620 can be obtained.

Meanwhile, the number of sub-layers and the number of sub-weight maps are not limited to those described above, and may have various values.

7 is a diagram illustrating an example of allocating memory by dividing a layer into a plurality of sub-layers.

Referring to FIG. 7, the processor 310 may allocate memory 700 space for each of the first sub-layer, the second sub-layer, and the third sub-layer. The memory 700 of FIG. 7 may correspond to the internal memory 320 of FIG. 3.

As described above with reference to FIG. 6, when the processor 310 divides the first layer into a plurality of sub-layers and allocates a memory 700 for each of the sub-layers, the sub-weight is not a weight map of the first layer. Maps can be assigned.

For example, when allocating the memory 700 to the first sub-layer, the processor 310 may store a first storage space 710 to store an input feature map in the memory 700 and a second to store an output feature map. Storage space 720 may be allocated. Also, the processor 310 may allocate a fourth storage space 730 to store the first sub-weight map allocated to the first sub-layer between the first storage space 710 and the second storage space 720. .

Similarly, even in the case of allocating the memory 700 for each of the second sub-layer and the third sub-layer, the space for storing the input feature map is allocated to the first storage space 710 and output to the second storage space 720 You can allocate space to store feature maps. In addition, in the second sub-layer, space to store the second sub-weight map may be allocated to the fourth storage space 730, and in the third sub-layer, space to store the third sub-weight map in the fourth storage space 730. Can be assigned.

In this case, the input feature maps stored in the first storage space 710 in each of the first sub-layer, the second sub-layer, and the third sub-layer may be the same. In the fourth storage space 730, while moving from the first sub-layer to the third sub-layer, the sub weight map allocated to each sub layer may be overwritten and stored.

In the second storage space 720, each channel of the output feature map, which is a result of the convolution operation between the input feature map and the allocated sub weight map in each sub layer, may be sequentially stored. For example, the first channel output feature map may be stored in the second storage space 720 in the first sub-layer, and the second channel output feature map may be stored in the first channel output feature map in the second sub-layer. Can accumulate. Finally, when the third channel output feature map is accumulated in the first channel output feature map and the second channel output feature map in the third sub-layer, generation of all channels of the output feature map is completed and convolution in the first layer The solution operation may be terminated.

8 is a diagram illustrating an example of allocating memory by grouping tiles in a plurality of layers.

When calculating at a layer in the neural network, the capacity of a space for storing various data necessary for calculation such as an input feature map, a weight map, and an output peter map is greater than the size of the memory space located inside the processor 310. It can grow. In addition, as described above with reference to FIGS. 6 and 7, even when the memory is allocated by dividing the layer into a plurality of sub-layers, the capacity of the space to store various data is greater than the size of the memory space located inside the processor 310. It can grow.

In this case, the processor 310 may tile each of the predetermined layers in order to reduce access to the external memory by maximizing the memory capacity of the internal memory. The processor 310 may process tiles by grouping tiles in predetermined layers corresponding to the input tile. For example, when the size of the input feature map is dominant, the processor 310 may group tiles in predetermined layers to allocate memory and process operations.

Referring to FIG. 8, an input feature map 810 of the first layer, an input feature map 820 of the second layer, and an output feature map 830 of the second layer are illustrated. The input feature map 820 of the second layer includes four channels 820-1, 820-2, 820-3 and 820-4, and the input feature map 830 of the second layer is six channels Fields 830-1, 830-2, 830-3, 830-4, 830-5 and 830-6. Here, for each of the input feature map 820 of the second layer and the output feature map 830 of the second layer, the number of channels illustrated in FIG. 8 is not limited to the above, and may have various values.

First, the processor 310 may select an input tile 812-1 in the input feature map 810 from the input feature map 810 of the first layer. The input tile 812-1 may correspond to a part of the input feature map 810. A tile 822-1 may be generated through an operation between the input tile 812-1 and the weight map 840 in the first layer, and between the tile 822-1 and the weight map 850 in the second layer. Tile 832-1 may be generated through the operation. The tile 822-1 may include a portion of the upper left of each of the channels 820-1, 820-2, 820-3, and 820-4 of the input feature map 820, and the tile 822-1 ) May include a portion of the upper left of each of the channels 830-1, 830-2, 830-3, 830-4, 830-5, and 830-6 of the output feature map 830.

When the processor 310 wants to group and process the tiles of the first layer and the second layer at the same time, the processor 310 includes the input tiles 812-1, the tiles 822-1 and 822-1, and the first The capacity information for the capacity of the space to store each of the weight map 840 of the layer, the weight map 850 of the second layer, and other working data may be obtained. When the sum of each obtained capacity is within the size of the memory capacity located inside the processor 310, the processor 310 inputs the tile 812-1 into the memory located inside the processor 310 based on each acquired capacity information. ), a tile 822-1, a tile 832-1, a weight map 840 of the first layer and a weight map 850 of the second layer may be allocated.

The processor 310 may collectively process operations in the first layer and the second layer corresponding to the input tile 812-1 based on the allocated memory space. Likewise, the processor 310 may collectively process operations in the first layer and the second layer for the remaining input tiles 812-2, 812-3, and 812-4.

Meanwhile, the size of the input tile and the number of predetermined layers to be grouped are not limited to those described above, and may have various values.

9 is a flowchart illustrating an example of a process for allocating memory space in a neural network device.

In operation 910, the neural network device 300 may allocate memory for the n-th layer of the neural network. The neural network device 300 may first allocate memory for the first layer of the neural network. The memory may be located in the processor 310 of the neural network device 300.

In operation 920, the neural network device 300 may acquire the capacity of each of the input feature map, output feature map, weight map, and walking data of the n-th layer.

In operation 930, the neural network device 300 may allocate a first storage space to store the input feature map in memory, and a second storage space to store the output feature map. The first storage space may correspond to a space from the initial address value of the memory to the first address value, and the second storage space may correspond to the space from the second address value to the last address value of the memory.

In operation 940, the neural network device 300 may determine whether the total sum of the acquired capacities is smaller than the size of the memory space. If the total sum of the acquired capacities is smaller than the size of the memory space, the process proceeds to step 950. However, if the total sum of the acquired capacities is not smaller than the size of the memory space, the process proceeds to step 960. Specifically, the neural network device 300 sums all of the capacity of the space for storing the input feature map, the output feature map, the weight map, and the walking data of the n-th layer, and whether the resultant value is smaller than the size of the memory space I can judge.

In operation 950, the neural network device 300 may allocate space for storing the weight map between the first storage space and the second storage space.

In operation 960, the neural network device 300 may divide the weight map into a plurality of sub weight maps. The neural network device 300 may divide the weight map into a plurality of sub weight maps so that each of the plurality of channels of the output feature map can be generated from the calculation between each of the sub weight maps and the input feature map.

In operation 970, the neural network device 300 divides the n-th layer into a plurality of sub-layers, and allocates space to store a sub-weight map between the first storage space and the second storage space for each of the sub-layers. Can be.

In operation 980, the neural network device 300 may allocate memory for the next layer of the n-th layer.

In operation 990, the neural network device 300 may determine whether the n-th layer corresponds to the last layer. When the n-th layer corresponds to the last layer, memory allocation is ended. If the n-th layer does not correspond to the last layer, the process returns to step 920. That is, steps 920 to 980 may be repeated until memory is allocated for all layers.

10 is a flowchart illustrating another example of a process of allocating memory space in a neural network device.

In step 1010, the neural network device 300 may allocate memory for the n-th layer of the neural network. The neural network device 300 may first allocate memory for the first layer of the neural network. The memory may be located in the processor 310 of the neural network device 300.

In step 1015, the neural network device 300 may determine whether the total sum of the acquired capacities is smaller than the size of the memory space. If the total sum of the acquired capacities is smaller than the size of the memory space, step 1025 is performed. However, if the total sum of the acquired capacities is not smaller than the size of the memory space, the process proceeds to step 1030. Specifically, the neural network device 300 sums all of the capacity of the space for storing the input feature map, the output feature map, the weight map, and the walking data of the n-th layer, and whether the resultant value is smaller than the size of the memory space I can judge.

In operation 1025, the neural network device 300 may allocate a first storage space to store the input feature map in memory, and a second storage space to store the output feature map. The first storage space may correspond to a space from the initial address value of the memory to the first address value, and the second storage space may correspond to the space from the second address value to the last address value of the memory.

In step 1030, the neural network device 300 may acquire the sum S of the capacity of the space to store the input tile, the output tile, the weight map, and the working data in the input feature map of the n-th layer, respectively. The input tile may correspond to a part of the input feature map, and the output tile may be generated by an operation between the input tile and the weight map in the n-th layer.

In step 1035, the neural network device 300 may determine whether S is smaller than the size of the memory space. If S is smaller than the size of the memory space, the process proceeds to step 1040. However, if S is not smaller than the size of the memory space, the process proceeds to step 1055.

In step 1040, the neural network device 300 may perform tiling on the next layer of the n-th layer.

In step 1045, the neural network device 300 may acquire a sum S _n of the capacity of the space to store the output tile and weight map of the n-th layer. Specifically, the neural network device 300 is a space for storing an output tile generated through the next layer of the n-th layer and a weight map used for calculation of the next layer of the n-th layer by using the output tile of the n-th layer as an input. The sum of the doses of can be obtained.

In step 1050, the neural network device 300 may accumulate S by adding Sn to S. The neural network device 300 may determine whether the accumulated S is smaller than the size of the memory space. That is, steps 1035 to 1050 may be repeated until S is not smaller than the size of the memory space.

In operation 1055, the neural network device 300 may divide the weight map into a plurality of sub weight maps. The neural network device 300 may divide the weight map into a plurality of sub weight maps so that each of the plurality of channels of the output feature map can be generated from the calculation between each of the sub weight maps and the input feature map.

In operation 1060, the neural network device 300 may divide the n-th layer into a plurality of sub-layers, and allocate a sub-weight map to each of the sub-layers.

In step 1065, the neural network device 300 may determine whether S is smaller than the size of the memory space. If S is smaller than the size of the memory space, the process proceeds to step 1070. However, if S is not smaller than the size of the memory space, the process proceeds to step 1075.

In operation 1070, the neural network device 300 may allocate space for storing input tiles, output tiles, weight maps, and working data in the memory.

In operation 1075, the neural network device 300 may store a portion of the input tile, output tile, weight map, and walking data in an external memory. That is, since S is larger than the size of the memory space, the processor 310 has no choice but to read or write data from the external memory during the calculation process.

In operation 1080, the neural network device 300 may allocate memory for the next layer of the n-th layer.

In operation 1085, the neural network device 300 may determine whether the n-th layer corresponds to the last layer. When the n-th layer corresponds to the last layer, memory allocation is ended. If the n-th layer does not correspond to the last layer, the process returns to step 1015. That is, steps 1015 to 1080 may be repeated until memory is allocated for all layers.

11 is a flowchart illustrating an example of a process of allocating a space for storing an input feature map and an output feature map in memory in each layer of a neural network.

In operation 1110, the neural network device 300 first capacity information regarding a capacity of a space to store an input feature map of the first layer among a plurality of layers of the neural network and a capacity of a space to store the output feature map of the first layer. It is possible to obtain the second capacity information about. . The first capacity information is a capacity of a space required to store the input feature map of the first layer, and may correspond to the size of the input feature map data. Similarly, the second capacity information is a capacity of space required to store the output feature map of the second layer, and may correspond to the size of the output feature map data.

In step 1120, the neural network device 300 allocates a first storage space to store an input feature map in the memory based on the initial address value of the memory and the first capacity information, and the last address value and the second capacity information of the memory A second storage space to store the output feature map in memory can be allocated based on the. The first storage space may correspond to a space from the initial address value of the memory to the first address value, and the second storage space may correspond to the space from the second address value to the last address value of the memory. The first address value and the second address value may be any address value between the initial address value and the last address value in memory. The neural network device 300 can minimize access to external memory by allocating spaces for storing the input feature map and the output feature map of the layers to both ends of the memory.

The embodiments can also be embodied in the form of a recording medium containing instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, other data in a modulated data signal, such as program modules, or other transport mechanisms, and includes any information delivery media.

Further, in the present specification, the “unit” may be a hardware component such as a processor or circuit, and/or a software component executed by a hardware component such as a processor.

The above description of the present specification is for illustration only, and those skilled in the art to which the contents of this specification belong may understand that it can be easily modified to other specific forms without changing the technical spirit or essential features of the present invention. Will be able to. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present embodiment is indicated by the claims, which will be described later, rather than by the detailed description, and should be interpreted to include all modified or modified forms derived from the meaning and scope of the claims and their equivalent concepts.

Claims (17)

A method for allocating memory space for a plurality of layers of a neural network,
Acquiring first capacity information about a capacity of a space to store an input feature map of a first layer among a plurality of layers of the neural network and second capacity information about a capacity of a space to store an output feature map of the first layer step; And
Allocating a first storage space to store the input feature map in the memory based on the initial address value of the memory and the first capacity information, and based on the last address value of the memory and the second capacity information And allocating a second storage space to store the output feature map within. According to claim 1,
Allocating a space to store the input feature map of the second layer following the first layer to the second storage space;
Obtaining third capacity information regarding a capacity of a space to store the output feature map of the second layer; And
And allocating a third storage space to store the output feature map of the second layer in the memory based on the initial address value of the memory and the third capacity information. According to claim 1,
The first storage space corresponds to a space from an initial address value of the memory to a first address value,
The second storage space corresponds to a space from the second address value to the last address value in the memory. According to claim 1,
The acquiring step further acquires fourth capacity information regarding a capacity of a space to store the input feature map and a weight map on which calculation is performed,
The allocating step further allocates a space for storing the weight map between the first storage space and the second storage space based on the fourth capacity information. According to claim 1,
Dividing the weight map of the first layer into a plurality of sub weight maps;
Dividing the first layer into a plurality of sub-layers, and assigning each of the plurality of sub-weight maps to each of the divided sub-layers;
Obtaining sub-capacity information regarding a capacity of a space to store each of the plurality of sub-weight maps; And
And for each of the plurality of sub-layers, allocating a space for storing a sub-weight map allocated based on the sub-capacity information between the first storage space and the second storage space. The method of claim 5,
Each of the plurality of channels of the output feature map is generated from the operation between each of the plurality of sub weight maps and the input feature map,
And storing each of the plurality of channels of the output feature map sequentially in the second storage space. According to claim 1,
Selecting an input tile in the input feature map of the first layer; Obtaining capacity information for a capacity of a space to store each of the input tile, the output tile corresponding to the input tile, and the weight map of the first layer; And
And allocating a space to store each of the input tile, the output tile, and the weight map in the memory based on the acquired capacity information. According to claim 1,
Wherein the memory is located within a processor of a device driving the neural network. A computer-readable recording medium recording a program for executing the method of claim 1 on a computer. For neural network devices,
Memory; And
And a processor running the neural network by executing at least one program,
The processor,
Acquiring first capacity information about a capacity of a space to store an input feature map of a first layer among a plurality of layers of the neural network and second capacity information about a capacity of a space to store an output feature map of the first layer, , Allocate a first storage space to store the input feature map in the memory based on the initial address value of the memory and the first capacity information, and based on the last address value of the memory and the second capacity information A device for allocating a second storage space to store the output feature map in memory. The method of claim 10,
The processor allocates space to store the input feature map of the second layer following the first layer to the second storage space, and receives third capacity information about the capacity of the space to store the output feature map of the second layer. A device for acquiring and allocating a third storage space to store an output feature map of the second layer in the memory based on the initial address value of the memory and the third capacity information. The method of claim 10,
The first storage space corresponds to a space from an initial address value of the memory to a first address value, and the second storage space corresponds to a space from a second address value to the last address value of the memory. The method of claim 10,
The processor further acquires fourth capacity information regarding a capacity of a space to store the input feature map and a weight map on which calculation is performed, and based on the fourth capacity information, the first storage space and the second storage space An apparatus for allocating space for storing the weight map therebetween. The method of claim 10,
The processor divides the weight map of the first layer into a plurality of sub weight maps, divides the first layer into a plurality of sub layer layers, and divides the first layer into a plurality of sub weight maps, and the plurality of sub weight maps in each of the divided sub layers. Allocating each of them, obtaining sub-capacity information about the capacity of the space to store each of the plurality of sub-weight maps, and, for each of the plurality of sub-layers, sub-map allocated based on the sub-capacity information An apparatus for allocating space for storage between the first storage space and the second storage space. The method of claim 14,
The processor generates each of a plurality of channels of the output feature map from an operation between each of the plurality of sub-weight maps and the input feature map, and in the second storage space, a plurality of channels of the output feature map. A device that stores each sequentially. The method of claim 10,
The processor selects an input tile in the input feature map of the first layer, and stores the input tile, an output tile corresponding to the input tile, and a space to store each weight map of the first layer. An apparatus for acquiring capacity information for a device and allocating a space to store each of the input tile, the output tile, and the weight map in the memory based on the acquired capacity information. The method of claim 10,
And the memory is in the processor of the neural network device.

Images