CN116775149A - A method and device for cold start of neural network - Google Patents

Abstract

The embodiment of the present application discloses a method and device for cold starting a neural network, using an edge device with a multi-core processor, where the multi-core processor adopts a large and small core architecture, and the operation of the neural network is combined with the operator core of the neural network as a unit. The process is divided into the running processes of multiple operator kernels. According to the running order of the operator kernels, the operation process of reading the parameters of the first operator kernel, the reading and conversion of the corresponding weights are The process and the operation process of running the first operator core are scheduled to be completed in the large core processor; the operation process of running the other operator cores is scheduled to be completed in the large core processor, and the operation processes of the remaining operator cores are scheduled to be completed in the large core processor. The operation process of parameter reading, and the operation process of reading and converting corresponding weights are scheduled to be completed on the selected small core processor. In this way, the delay time can be reduced without affecting the accuracy of the neural network operation.

Description

A method and device for cold start of neural network

Technical field

The present application relates to the field of neural network technology, and in particular to a method and device for cold starting a neural network.

Background technique

With the development of artificial intelligence technology, neural networks have been applied to business recognition in countless fields such as computer vision technology and natural language understanding. When applying neural networks for image or semantic recognition, in order to pursue low processing latency and privacy of processed data, the deployment of neural networks has shifted from large data centers to edge devices. Edge devices are devices that provide entry points to the core network of enterprises or service providers, such as smartphones, Internet of Things access devices, wearable devices, and autonomous vehicle access devices, etc. These edge devices have computing power, and it is necessary to make full use of the computing power of these edge devices for processing based on neural networks.

There are two significant trends in deploying neural networks in edge devices: 1) the number and type of neural networks per edge device has exploded; 2) the structural complexity of neural networks is also increasing, such as deploying deep neural networks, deep Neural network is a multi-layer unsupervised multi-layer neural network. These two trends highlight the crowding of neural networks on resource-constrained edge devices. Therefore, it is unlikely that all neural networks will be pre-loaded into the memory of the edge device and then waiting to be run. In other words, the cold start, that is, loading, and subsequent initialization and execution of neural networks on edge devices are becoming very important. The speed of cold start of the neural network on the edge device, like the hot start, is crucial to the service quality and user experience of the edge device.

However, the cold start time of neural networks, especially deep neural networks, on edge devices is much longer than the warm start time. Neural networks often experience delays when edge devices are cold-started. How to reduce the delay time without affecting the accuracy of the neural network operation is an urgent problem that needs to be solved.

Contents of the invention

In view of this, embodiments of the present application provide a method for cold starting a neural network on an edge device. This method can reduce the delay time without affecting the operation accuracy of the neural network.

Embodiments of the present application also provide a device for cold starting a neural network on an edge device, which can reduce the delay time without affecting the operation accuracy of the neural network.

This application is implemented as follows:

In one embodiment of the present application, a method for cold starting a neural network on an edge device is provided. The method includes:

Taking the operator kernel of the neural network as a unit, split the operation of the neural network into the operation of multiple operator kernels;

According to the operation sequence of the operator kernel, the operation process of reading the parameters of the first operator kernel, the operation process of reading and converting the corresponding weights, and the operation process of running the first operator kernel are scheduled to the edge. Executed in the large-core processor in the device;

The operation process of running the remaining operator kernels is scheduled to be completed in the large core processing; the operation process of reading the parameters of the remaining operator kernels, as well as the operation process of reading and converting the corresponding weights, is scheduled to the edge device Executed in the selected small core processor;

The large-core processor and the small-core processor of the edge device respectively execute the corresponding operation process based on the schedule.

In the above method, the operation process of reading the parameters of the remaining operator cores and the operation process of reading and converting the corresponding weights is scheduled to be executed in the small core processor selected in the edge device, including:

According to the operation sequence of the operator cores, the operator cores are scheduled one by one to the small core processors selected in sequence, and the operation process of parameter reading of the operator cores and the corresponding weights are scheduled to be executed. The operation process of reading and conversion.

In the above method, the scheduling to the small core processor selected in the edge device for execution includes:

After the small core processor obtains the information of scheduling the operator core, it initializes its own operation list, and stores the operation information of reading the parameters of the operator core and the operation information of reading and converting the corresponding weights. in the action list;

The small core processor executes the corresponding operation process based on scheduling, including:

The small core processor sequentially executes the corresponding operation processes according to the operation list.

In the above method, the scheduling is executed in the large-core processor in the edge device, and further includes:

In addition to the first operator core, for each operator core, the operation process of reading the parameters of the operator core, the operation process of reading and converting the corresponding weights, and running the operator are calculated on the large core processor The total time of the operation process of the kernel is determined by whether the total time is less than the operation process of reading the parameters of the operator kernel in the small core processor, and the operation process of reading and converting the corresponding weights in the small core processor. , and the total time for the operation process of running the operator kernel on the large-core processor;

If so, the operation process of reading the parameters of the operator kernel on the small core processor, and the operation process of reading and converting the corresponding weights on the small core processor are scheduled to be completed on the large core processor;

If not, the small core processor is scheduled to complete the operation process of reading the parameters of the operator kernel, and the small core processor is scheduled to complete the operation process of reading and converting the corresponding weights.

In the above method, the scheduling is executed in the small core processor selected in the edge device, and further includes:

Determine the small-core processor that completes the operation first, and determine whether the small-core processor can complete the operation process of reading the parameters of the operator core with the longest operation time, and the operation process of reading and converting the corresponding weights. ;

If so, the operation process of reading the parameters of the operator kernel with the longest operation time and the operation process of reading and converting the corresponding weights are scheduled to be completed in the small core process.

In the above method, the operation process of determining whether the small core processor can complete the parameter reading of the operator core with the longest operation time, and the operation process of reading and converting the corresponding weights include:

Determine whether the small core processor completes the operation process of parameter reading of the operator core with the longest operation time and the operation process of reading and conversion of the corresponding weights before running the operation process of the operator core with the longest operation time. Before the time of the operation process of the sub-kernel, if so, the operation process of parameter reading of the operator kernel with the longest operation time, and the operation process of reading and conversion of the corresponding weights will be scheduled to the small core processor middle.

In another embodiment of the present application, a device for cold starting a neural network on an edge device is provided. The device includes: a decision-making module and an execution module, wherein,

The decision-making module is used to divide the operation of the neural network into the operation of multiple operator kernels based on the operator kernel of the neural network; according to the operation sequence of the operator kernels, divide the first operator kernel among them The operation process of parameter reading, the operation process of reading and converting the corresponding weights, and the operation process of running the first operator kernel are scheduled to be executed in the large-core processor in the edge device; the operation process of running the remaining operator kernels The operation process scheduling is completed in the large core processing, and the operation process of reading the parameters of the remaining operator cores, as well as the operation process of reading and converting the corresponding weights, is scheduled to the small core processor selected in the edge device in execution;

The execution module is used to control the large-core processor and the small-core processor to execute the corresponding operation process based on scheduling.

In yet another embodiment of the present application, an electronic device is provided, including:

processor;

The memory stores a program, and the program is configured to implement the above-mentioned method of cold starting a neural network on an edge device when executed by the processor.

In yet another embodiment of the present application, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores instructions, wherein the instructions, when executed by a processor, cause the processor to execute The above-mentioned method of cold starting a neural network on an edge device.

In yet another embodiment of the present application, a computer program product is provided, including a computer program or instructions. When the computer program or instructions are executed by a processor, a neural network according to any one of the above is implemented to perform a cold start on an edge device. steps of the method.

As can be seen above, the embodiment of the present application uses an edge device with a multi-core processor. The multi-core processor adopts a large and small core architecture. Taking the operator core of the neural network as a unit, the running process of the neural network is divided into multiple said operators. The running process of the sub-kernel, according to the running order of the operator kernel, includes the operation process of reading the parameters of the first operator kernel, the operation process of reading and converting the corresponding weights, and running the first operator kernel. The operation process is scheduled to be completed in the large-core processor; the operation process of running the remaining operator kernels is scheduled to be completed in the large-core processor, the operation process of reading the parameters of the remaining operator kernels, and the corresponding The weight reading and conversion operations are scheduled to be completed on the selected small core processor. In this way, during the cold start process of the edge device, the neural network is executed using a combination of the large-core processor cycle and the small-core processor cycle of the edge device, reducing the delay time without affecting the operation accuracy of the neural network.

Description of drawings

Figure 1 is a process flow chart when cold starting on an edge device provided by an embodiment of the present application.

Figure 2 is a flow chart of a method for cold starting a neural network on an edge device provided by an embodiment of the present application;

Figure 3 is a schematic diagram of the process of executing a cold start of the entire neural network on an edge device using a pipeline scheduling strategy provided by an embodiment of the present application;

Figure 4 is a schematic structural diagram of a device for cold starting a neural network on an edge device according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an electronic device provided by another embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects without necessarily using Used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein can, for example, be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units need not be limited to those steps or units that are expressly listed, but may include steps or units that are not expressly listed or that are not specific to the process, method, product, or device. Other steps or units inherent to the equipment.

The technical solutions of the present application are described in detail below with specific examples. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

During the cold start process of neural networks, especially deep neural networks, on edge devices, the bottlenecks that take a long time mainly include: reading the original weights of the neural network from disk to the memory on the edge device during loading, and during initialization. A running process of converting the original weights into weights in an executable format and inputting data into a neural network based on the weights in the executable format at execution time. The above processes are also called the cold inference process of deep neural networks on edge devices.

Currently, there are two ways to mitigate cold inference latency. One is to share the original weights of the neural network, with the purpose of packaging and storing more neural networks into the memory of the edge device in order to warm up each neural network to be run subsequently, but this method is not scalable. Sexuality, because when the number and types of neural networks to be run on edge devices increase, it will lead to a significant decrease in the operating accuracy of the neural network model. Another way is to estimate the execution time of the neural network and read and store the original weights in advance. However, when the edge device needs to run multiple neural networks, it is difficult to estimate the execution time of each neural network. . Both of the above methods solve the cold start delay problem in an indirect way, but they rely on changing the neural network structure or external knowledge, making them difficult to implement.

Therefore, the embodiments of the present application use a direct method to optimize the delay of the neural network when cold starting on the edge device, without relying on any assumptions about the structure or execution environment of the neural network, and can ensure zero accuracy loss. The embodiment of the present application uses an edge device with a multi-core processor. The multi-core processor adopts a large and small core architecture and may include: multiple large core processors and multiple small core processors. Taking the operator core of the neural network as a unit, The operation process of the neural network is divided into the operation processes of multiple operator kernels. According to the operation sequence of the operator kernels, the operation process of reading the parameters of the first operator kernel and reading the corresponding weights are and the operation process of conversion, as well as the operation process of running the first operator kernel, are scheduled to be completed in the large-core processor; the operation process of running the remaining operator kernels is scheduled to be completed in the large-core processor, and the remaining operator kernels are scheduled to be completed in the large-core processor. The operation process of parameter reading of the operator kernel and the operation process of reading and converting the corresponding weights are scheduled to be completed on the selected small core processor.

Here, selecting a small core processor is to schedule the relevant operation processes of the operator core to the sequentially selected small core processors one by one according to the operation sequence of the operator core, and schedule the completion of the operator core The operation process of parameter reading, and the operation process of reading and conversion of corresponding weights.

In this way, during the cold start process of the edge device, the neural network is executed using a combination of the large-core processor cycle and the small-core processor cycle of the edge device, reducing the delay time without affecting the operation accuracy of the neural network.

When performing the small-core processor cycle in the embodiment of the present application, the premise is to balance the workload of each small-core processor. When this application uses a small core processor to cycle through the operation process of reading parameters of each operator core, and the operation process of reading and converting corresponding weights, it also includes: determining the small core processor that completes the operation first, Determine whether the small core processor can complete the operation process of parameter reading of the operator core with the longest operation time, and the operation process of reading and conversion of the corresponding weights. If so, the operation process of the operator core with the longest operation time will be completed. The operation process of parameter reading of the operator kernel and the operation process of reading and converting the corresponding weights are scheduled to be completed in the small core processing. Here, when it is judged whether the small core processor can complete the operation process of parameter reading of the operator core with the longest operation time and the operation process of reading and conversion of the corresponding weights, it is judged that the small core processor Is the time to complete the operation process of parameter reading of the operator kernel with the longest operation time and the operation process of reading and conversion of the corresponding weights within the time of running the operation process of the operator kernel with the longest operation time? Previously, if so, the operation process of reading the parameters of the operator core with the longest operation time and the operation process of reading and converting the corresponding weights are scheduled to the small core processor. In this way, the workload of each small core processor can be more balanced.

In the embodiment of the present application, when the small core processor cycles, after the small core processor obtains the information of scheduling the operator core, it initializes its own operation list and reads the parameters of the operator core. The operation information, and the operation information corresponding to the reading and conversion of weights are stored in the operation list; when performing scheduling, the small core processor executes sequentially according to the operation list.

This application uses the large-core processor cycle and the small-core processor cycle of the edge device to cooperate to complete the neural network during the cold start of the edge device, balancing the workload of the large-core processor and the small-core processor to minimize the large-core processor. The completion time of the core processor. In the large-core processor cycle, it also includes: in addition to the first operator core, for each operator core, calculate the operation process of parameter reading of the operator core performed by the large-core processor, and the reading of corresponding weights. The operation process of retrieval and transformation, and the total time of the operation process of running the operator kernel are judged whether it is less than the operation process of executing the parameter reading of the operator kernel on the small core processor, and the operation process of executing the corresponding weight on the small core processor. The operation process of reading and conversion, and the total time of the operation process of running the operator kernel on the large-core processor. If so, the operation process of reading the parameters of the operator kernel on the small-core processor will be executed on the small-core processor. The processor performs the operation process of reading and converting the corresponding weights and schedules it to the large-core processor for completion. Otherwise, the operation process of reading the parameters of the operator core is still scheduled to be completed on the small-core processor. Complete the operation process of reading and converting the corresponding weights. In this way, the workload among all core processors can be balanced to a large extent.

In the embodiment of the present application, the operator kernel of the neural network refers to the basic mathematical operation unit used by each neural network layer (neural network layer) in the neural network. The operator kernel constitutes the basic calculation unit of the neural network. Different operator kernels correspond to the calculation logic of different neural network layers. They are usually presented in the form of calculation matrices. For example, the convolution layer is an operator that executes the convolution layer. The computing unit is an operator core. The weight summation process in the fully connected layer is an operator. The computing unit that performs the fully connected layer operation is another operator core.

In the embodiment of this application, the large-core processor of the edge device is the performance core (PerformanceCores) central processing unit (CPU) or graphics processing unit (GPU) in the edge device, which refers to a high-performance processor core and is the edge The main computing power of the device; the small-core processor of the edge device is the Efficiency cores CPU, which refers to the core CPU for the purpose of saving energy and is mainly used when the edge device is under low load.

The embodiments of the present application are described in detail below.

As shown in Figure 1, Figure 1 is a process flow chart for cold start on an edge device provided by an embodiment of the present application, including a decision-making phase and an execution phase. In the decision-making phase, the operator core and cache of the neural network are selected. The converted weights and the adopted scheduling strategy enter the execution stage under the control of the scheduler; in the execution stage, the data is input to the corresponding operator kernel for execution, and the execution result is obtained. Among them, the operator kernel is scheduled by the scheduler. , run based on the corresponding converted weights. Among them, the execution of the operator kernel is called inference in the diagram.

The embodiment of this application studies the optimization space of neural networks when cold starting on edge devices, and determines the following three effective optimization methods.

The first optimization method

Select the best operator kernel implementation from the neural network. In neural networks, especially deep neural networks, there are usually many different implementations of each operator kernel. These operator kernels divided in the neural network are to improve the running speed, and the current operator kernel selection strategy is completely based on the running speed of the neural network during hot start. However, the operator kernel that runs the fastest during a hot start may not necessarily show the best performance during a cold start. For example, some operator kernels run very fast, but the corresponding weight reading and conversion are The operation process takes a long time, which will delay the cold start speed. In this case, the selection strategy of the operator kernel in the neural network should be a strategy to reduce the cold start delay, so as to obtain the optimal operator kernel in the neural network.

The second optimization method

Cache the converted weights of the corresponding operator kernel. The operational process of converting weights can be bypassed by storing this converted weight on disk on the edge device for direct reading and execution. However, the converted weights may take up more storage space and result in higher disk read and write costs. Whether to perform the operation process of reading and converting the weights corresponding to the operator core or to perform the operation process of reading the converted weights corresponding to the operator core requires a trade-off between disk read and write costs and computing costs.

The third optimization method

Optimal pipeline scheduling strategy. The execution process of the operator kernel based on pipeline technology and the binding method of the kernel of the edge device. The execution process of an operator kernel includes: the operation process of reading and converting the corresponding weights, and the operation process of running the operator kernel. The blocking time of transfers between disk and memory in edge devices can be reduced through pipeline scheduling strategies. The pipeline scheduling strategy can also schedule asymmetric core processors on edge devices to process different operations, such as between central processing units (CPUs) and graphics processing units (GPUs), or between large-core CPUs or small-core CPUs. operating process.

Taking the above three optimization methods into consideration, the delay reduction effect of the neural network when cold starting on the edge device is tightly coupled. For example, selecting different operator kernels may have different pipeline scheduling strategies. In order to set up a comprehensive and effective cold-start solution for neural networks on edge devices, this application addresses the following two challenges.

First, the search space is too large. The setting is formulated as the selection of combined operator kernels, the bypass of the operation process of the conversion of the corresponding operator kernel weights, and the operation process of multiple running operator kernels to obtain the optimal scheduling operator kernel problem. This problem is Non-deterministic (NP) problems of polynomial complexity are difficult to solve. Secondly, due to the limited disk and memory capacity of edge devices, different operating processes will interfere with each other, further complicating the problem.

Therefore, this application adopts a heuristic optimal pipeline scheduling strategy, which is inspired by some key observations during the cold start of neural networks on edge devices, such as: 1) The operation process runs on different core processors Differences in processing. 2) Multi-threading on multiple cores is more efficient in performing operations than multi-threading on other cores. Therefore, this application uses the large-core processor of the edge device to run the operation process of the operator kernel in multi-threads, and the operation process of reading the parameters of the operator kernel, and the operation process of reading and converting the corresponding weights are performed in a small Core processor executes. The embodiments of this application also take advantage of a feature, that is, the operation process of reading and converting the weights corresponding to the operator kernel requires fewer storage resources than the operation process of reading the converted weights of the corresponding operator kernel. dependencies, therefore, the execution schedule of operator kernels can be easily arranged.

Based on the above inspiration, the embodiment of this application sets up an intuitive and effective optimal pipeline scheduling strategy. The key is to balance the workload on different core processors in the edge device to minimize the total execution time of cold start. At the same time, during the scheduling planning process, each operation process is rescheduled by analyzing each operation process during the execution of the operator kernel to improve the performance of each operation process and facilitate better scheduling planning.

Figure 2 is a flow chart of a method for cold starting a neural network on an edge device provided by an embodiment of the present application. The method is applied to an edge device with a multi-core processor. The specific steps include:

Step 201: Taking the operator kernel of the neural network as a unit, split the operation of the neural network into the operation of multiple operator kernels;

Step 202: According to the operation sequence of the operator kernel, the operation process of reading the parameters of the first operator kernel, the operation process of reading and converting the corresponding weights, and the operation process of running the first operator kernel, Scheduled to the large-core processor in the edge device for execution;

Step 203: Schedule the operation process of running the remaining operator kernels to be completed in the large core processing, and schedule the operation process of reading the parameters of the remaining operator kernels and the operation process of reading and converting the corresponding weights to all Executed in the small core processor selected in the edge device;

Step 204: The large-core processor and the small-core processor of the edge device respectively execute the corresponding operation process based on scheduling.

In the above method, the operation process of reading the parameters of the remaining operator cores and the operation process of reading and converting the corresponding weights is scheduled to be executed in the small core processor selected in the edge device, including:

According to the running order of the operator cores, the operator cores are scheduled one by one to the sequentially selected small core processors, and the operation process of parameter reading of the operator cores and the corresponding weights are scheduled to be executed. The operation process of reading and conversion.

In the above method, the scheduling to the small core processor selected in the edge device for execution includes:

After the small core processor obtains the information of scheduling the operator core, it initializes its own operation list, and stores the operation information of reading the parameters of the operator core and the operation information of reading and converting the corresponding weights. in the action list;

The small core processor executes the corresponding operation process based on scheduling, including:

The small core processor sequentially executes the corresponding operation processes according to the operation list.

In the above method, the scheduling is executed in the large-core processor in the edge device, and further includes:

In addition to the first operator core, for each operator core, the operation process of reading the parameters of the operator core, the operation process of reading and converting the corresponding weights, and running the operator are calculated on the large core processor The total time of the operation process of the kernel is determined by whether the total time is less than the operation process of reading the parameters of the operator kernel in the small core processor, and the operation process of reading and converting the corresponding weights in the small core processor. , and the total time for the operation process of running the operator kernel on the large-core processor;

If so, the operation process of reading the parameters of the operator kernel on the small core processor, and the operation process of reading and converting the corresponding weights on the small core processor are scheduled to be completed on the large core processor;

If not, the small core processor is scheduled to complete the operation process of reading the parameters of the operator kernel, and the small core processor is scheduled to complete the operation process of reading and converting the corresponding weights.

In the above method, the scheduling is executed in the small core processor selected in the edge device, and further includes:

Determine the small-core processor that completes the operation first, and determine whether the small-core processor can complete the operation process of reading the parameters of the operator core with the longest operation time, and the operation process of reading and converting the corresponding weights. ;

If so, the operation process of reading the parameters of the operator kernel with the longest operation time and the operation process of reading and converting the corresponding weights are scheduled to be completed in the small core process.

Here, the operation process of determining whether the small-core processor can complete the parameter reading of the operator core with the longest operation time, and the operation process of reading and converting the corresponding weights includes:

Determine whether the small core processor completes the operation process of parameter reading of the operator core with the longest operation time and the operation process of reading and conversion of the corresponding weights before running the operation process of the operator core with the longest operation time. Before the time of the operation process of the sub-kernel, if so, the operation process of parameter reading of the operator kernel with the longest operation time, and the operation process of reading and conversion of the corresponding weights will be scheduled to the small core processor middle.

Using the above method, the cold start process of the neural network on the edge device is executed based on the small-core processor loop and the large-core processor loop. The operator core of the neural network is used as the unit, and the execution results are output. As shown in Figure 3, Figure 3 is a schematic diagram of the process of using a pipeline scheduling strategy to perform a cold start of the entire neural network on an edge device according to an embodiment of the present application.

Figure 4 is a schematic structural diagram of a device for cold starting a neural network on an edge device according to an embodiment of the present application. The device includes: a decision-making module and an execution module, where,

The decision-making module is used to divide the operation of the neural network into the operation of multiple operator kernels based on the operator kernel of the neural network; according to the operation sequence of the operator kernels, divide the first operator kernel among them The operation process of parameter reading, the operation process of reading and converting the corresponding weights, and the operation process of running the first operator kernel are scheduled to be executed in the large-core processor in the edge device; the operation process of running the remaining operator kernels The operation process scheduling is completed in the large core processing, and the operation process of reading the parameters of the remaining operator cores, as well as the operation process of reading and converting the corresponding weights, is scheduled to the small core processor selected in the edge device in execution;

The execution module is used to control the large-core processor and the small-core processor to execute the corresponding operation process based on scheduling.

In another embodiment of the present application, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the foregoing A method for cold starting a neural network on an edge device in an embodiment.

FIG. 5 is a schematic diagram of an electronic device provided by another embodiment of the present application. As shown in Figure 5, another embodiment of the present application also provides an electronic device, which may include a processor 501, where the processor 501 is configured to perform the above-mentioned step of cold starting a neural network on an edge device. It can also be seen from Figure 5 that the electronic device provided by the above embodiment also includes a non-transitory computer-readable storage medium 502. The non-transitory computer-readable storage medium 502 stores a computer program and a neural network model, wherein the computer program When the processor 501 is running, the steps in the above method for cold starting a neural network on an edge device are executed.

Specifically, the non-transitory computer-readable storage medium 502 can be a general storage medium, such as a removable disk, a hard disk, FLASH, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), or a portable Compact disk read-only memory (CD-ROM), etc., when the computer program on the non-transitory computer-readable storage medium 502 is run by the processor 501, it can cause the processor 801 to execute the above-mentioned neural network to perform cooling on the edge device. The various steps in the method that are initiated.

In practical applications, the non-transitory computer-readable storage medium 502 may be included in the equipment/device/system described in the above embodiments, or may exist independently without being assembled into the equipment/device/system. . The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed, each step in the above-mentioned method for cold starting a neural network on an edge device can be executed.

Yet another embodiment of the present application further provides a computer program product, including a computer program or instructions, which when executed by a processor implements each of the above-mentioned methods for cold starting a neural network on an edge device. step.

The flowcharts and block diagrams in the drawings of this application illustrate the possible implementation architecture, functions and operations of systems, methods and computer program products according to various embodiments disclosed in this application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the standard order noted in the figures. For example, two blocks shown connected may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagram or flowchart illustration, and combinations of blocks in the block diagram or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or may be implemented by special purpose hardware-based systems that perform the specified functions or operations. Achieved by a combination of specialized hardware and computer instructions.

It will be understood by those skilled in the art that the features described in the various embodiments and/or claims of the present disclosure may be combined and/or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and/or claims of the present application may be combined and/or combined in various ways without departing from the spirit and teachings of the present application, and all of these combinations and/or combinations fall within the disclosure scope of this application.

This article uses specific embodiments to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the present application, and is not intended to limit the present application. For those skilled in the art, changes can be made in the specific implementation modes and application scope according to the ideas, spirit and principles of this application. Any modifications, equivalent substitutions, improvements, etc. shall be included in this application. within the scope of protection.

Claims (10)

1. A method for cold start of a neural network on an edge device, the method comprising:

splitting the operation of the neural network into a plurality of operation of operator cores by taking the operator cores of the neural network as units;

according to the operator kernel running sequence, scheduling an operation process of parameter reading of a first operator kernel, an operation process of reading and converting corresponding weight and an operation process of running the first operator kernel into a big kernel processor in edge equipment for execution;

scheduling the operation process of running the other operator kernels to be completed in the large kernel processing; scheduling the operation process of parameter reading of other operator kernels and the operation process of reading and converting of corresponding weights to a small kernel processor selected from the edge equipment for execution;

and the big core processor and the small core processor of the edge equipment execute the corresponding operation process based on the scheduling respectively.

2. The method of claim 1, wherein the scheduling the operations of reading parameters of the remaining operator kernels and the operations of reading and translating the corresponding weights to the selected corelet processor in the edge device comprises:

and according to the operator kernel operation sequence, dispatching the operator kernels to the sequentially selected small kernel processors one by one, and dispatching and executing the operation process of parameter reading of the operator kernels and the operation process of reading and converting the corresponding weights.

3. The method of claim 1, wherein the scheduling to execute in a corelet processor selected in the edge device comprises:

after the small core processor acquires the information for scheduling the operator kernel, initializing an operation list of the small core processor, and storing operation information read by parameters of the operator kernel and operation information read and converted by corresponding weights in the operation list;

the small core processor executing the corresponding operation process based on scheduling comprises:

and the small core processor sequentially executes the corresponding operation processes according to the operation list.

4. The method of claim 1, wherein the scheduling is performed in a large core processor in an edge device, further comprising:

calculating, for each operator core, a total time of an operation process of executing parameter reading of the operator core, an operation process of reading and converting corresponding weights, and an operation process of operating the operator core at a large core processor, and judging whether the total time is smaller than a total time of an operation process of executing parameter reading of the operator core at a small core processor, an operation process of executing reading and converting corresponding weights at a small core processor, and an operation process of operating the operator core at a large core processor, except for a first operator core;

if yes, scheduling an operation process of executing parameter reading of an operator kernel in the small core processor and an operation process of executing reading and conversion of corresponding weight in the small core processor to the large core processor to finish;

if not, the operation process of parameter reading of the operator kernel is scheduled and completed in the small core processor, and the operation process of reading and converting of the corresponding weight is scheduled and completed in the small core processor.

5. The method of claim 1, wherein the scheduling is performed in a corelet processor selected in the edge device, further comprising:

determining a small core processor which finishes the operation first, judging whether the small core processor can finish the operation process of parameter reading of an operator kernel with the longest operation time and the operation process of reading and converting corresponding weights;

if yes, the operation process of completing parameter reading of the operator kernel with the longest operation time and the operation process of reading and converting the corresponding weight are scheduled to be completed in the small kernel processing.

6. The method of claim 5, wherein the operation of determining whether the corelet processor can re-complete the parameter reading of the operator kernel with the longest operation time, and the operation of reading and translating the corresponding weights, comprises:

judging whether the time of the operation process of parameter reading of the operator core with the longest operation time and the operation process of reading and converting of the corresponding weight of the small core processor is before the time of operating the operation process of the operator core with the longest operation time, if so, scheduling the operation process of parameter reading of the operator core with the longest operation time and the operation process of reading and converting of the corresponding weight into the small core processor.

7. An apparatus for cold starting a neural network on an edge device, the apparatus comprising: the decision module and the execution module, wherein,

the decision module is used for splitting the operation of the neural network into a plurality of operation of operator cores by taking the operator cores of the neural network as units; according to the operator kernel running sequence, scheduling an operation process of parameter reading of a first operator kernel, an operation process of reading and converting corresponding weight and an operation process of running the first operator kernel into a big kernel processor in edge equipment for execution; scheduling the operation process of operating the other operator cores in the large core processing, and scheduling the operation process of parameter reading of the other operator cores and the operation process of reading and converting corresponding weights to a small core processor selected from the edge equipment for execution;

and the execution module is used for controlling the large-core processor and the small-core processor to execute the corresponding operation process based on the scheduling respectively.

8. An electronic device, comprising:

a processor;

a memory storing a program configured to implement a method of cold starting a neural network on an edge device as claimed in any one of claims 1 to 6 when executed by the processor.

9. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method of cold starting a neural network on an edge device according to any one of claims 1 to 6.

10. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the steps of a method of cold starting a neural network on an edge device as claimed in any one of claims 1 to 6.