CN113887722A - Neural network testing method and device, computer equipment and storage medium - Google Patents
Neural network testing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113887722A CN113887722A CN202111168642.9A CN202111168642A CN113887722A CN 113887722 A CN113887722 A CN 113887722A CN 202111168642 A CN202111168642 A CN 202111168642A CN 113887722 A CN113887722 A CN 113887722A
- Authority
- CN
- China
- Prior art keywords
- neural network
- network
- deployed
- result data
- network layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 289
- 238000012360 testing method Methods 0.000 title claims abstract description 146
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 74
- 238000012795 verification Methods 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 51
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010998 test method Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 23
- 230000006870 function Effects 0.000 description 19
- 238000013139 quantization Methods 0.000 description 15
- 238000013135 deep learning Methods 0.000 description 14
- 101100317378 Mus musculus Wnt3 gene Proteins 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure provides a method, an apparatus, a computer device and a storage medium for testing a neural network, wherein the method comprises: acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed; processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer; determining a target network layer from the neural network to be deployed based on the first result data and the second result data. Therefore, by determining the target network layer, parameters of the target network layer can be adjusted in a targeted manner, so that time consumption for adjusting the parameters of the neural network to be deployed is reduced, and the adjusting efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of neural network technologies, and in particular, to a method and an apparatus for testing a neural network, a computer device, and a storage medium.
Background
With the development of artificial intelligence, deep learning networks are gradually applied to production and living practices. Before deploying the trained deep learning network to the embedded device, further adjusting parameters of the deep learning network; the current adjusting mode has the problems of long time consumption and low efficiency.
Disclosure of Invention
The embodiment of the disclosure at least provides a test method and device of a neural network, computer equipment and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a method for testing a neural network, including: acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed; processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer; determining a target network layer from the neural network to be deployed based on the first result data and the second result data.
Therefore, the target network layer causing the problem of the precision of the neural network to be deployed can be determined from the neural network to be deployed by using the standard network corresponding to the neural network to be deployed, so that the parameters of the target network layer can be adjusted in a targeted manner, the time consumption for adjusting the parameters of the neural network to be deployed is reduced, and the adjusting efficiency is improved.
In an optional implementation manner, in a hardware environment of the to-be-deployed neural network, processing the first verification data by using the to-be-deployed neural network to obtain first result data corresponding to each network layer of the to-be-deployed neural network includes: analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed; determining operators to be replaced which are not supported by the hardware environment from the first operators of the neural network to be deployed based on the hardware environment and the operator information of each first operator; replacing the operator to be replaced by using the second operator adapted to the hardware environment to generate a testing neural network; and inputting the first verification data into the test neural network to obtain the first result data.
In an optional embodiment, the operator information includes: operator type and/or operator context; analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed, including: analyzing the structure definition file corresponding to the neural network to be deployed to obtain the operator types respectively corresponding to the operators of each network layer of the neural network to be deployed; and/or analyzing the model parameter file corresponding to the neural network to be deployed to obtain operator internal parameters of operators respectively corresponding to each network layer in the neural network to be deployed.
In an optional embodiment, determining a target network layer from the neural network to be deployed based on the first result data and the second result data includes: for each network layer in the plurality of network layers of the neural network to be deployed, inversely quantizing first result data corresponding to the network layer to a bit number consistent with the second result data to obtain third result data corresponding to the network layer; determining the distance between the second result data and the third result data corresponding to the network layer; and determining a target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers respectively.
In an optional implementation manner, the determining a target network layer from the plurality of network layers according to distances corresponding to the plurality of network layers, respectively, includes: comparing the distances respectively corresponding to the plurality of network layers with a preset distance threshold; determining the network layer with the corresponding distance greater than the preset distance threshold value as an alternative network layer; and determining a target network layer from the alternative network layers based on the positions of the alternative network layers in the neural network to be deployed.
In an optional embodiment, the testing method further includes: and adjusting the target network layer to obtain a target neural network.
In an optional implementation manner, before the adjusting the target network layer, the method further includes: randomly generating first test data input to the target network layer; processing second verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain fourth result data corresponding to a previous network layer of the target network layer; taking the fourth result data as second test data, and testing the target network layer by using the first test data and the second test data to obtain a test result; the adjusting the target network layer to obtain the target neural network includes: and adjusting the target network layer based on the test result to obtain the target neural network.
In a second aspect, an embodiment of the present disclosure further provides a testing apparatus for a neural network, including: the system comprises an acquisition module, a deployment module and a deployment module, wherein the acquisition module is used for acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed; the first processing module is used for processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer; a determining module, configured to determine a target network layer from the neural network to be deployed based on the first result data and the second result data.
In an optional implementation manner, in a hardware environment of the to-be-deployed neural network, when the first processing module processes the first verification data by using the to-be-deployed neural network to obtain first result data corresponding to each network layer of the to-be-deployed neural network, the first processing module is configured to: analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed; determining operators to be replaced which are not supported by the hardware environment from the first operators of the neural network to be deployed based on the hardware environment and the operator information of each first operator; replacing the operator to be replaced by using the second operator adapted to the hardware environment to generate a testing neural network; and inputting the first verification data into the test neural network to obtain the first result data.
In an optional embodiment, the operator information includes: operator type and/or operator context; the first processing module is configured to, when analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed: analyzing the structure definition file corresponding to the neural network to be deployed to obtain the operator types respectively corresponding to the operators of each network layer of the neural network to be deployed; and/or analyzing the model parameter file corresponding to the neural network to be deployed to obtain operator internal parameters of operators respectively corresponding to each network layer in the neural network to be deployed.
In an optional embodiment, the determining module, when determining the target network layer from the neural network to be deployed based on the first result data and the second result data, is configured to: for each network layer in the plurality of network layers of the neural network to be deployed, inversely quantizing first result data corresponding to the network layer to a bit number consistent with the second result data to obtain third result data corresponding to the network layer; determining the distance between the second result data and the third result data corresponding to the network layer; and determining a target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers respectively.
In an optional embodiment, when determining the target network layer from the plurality of network layers according to respective distances corresponding to the plurality of network layers, the determining module is configured to: comparing the distances respectively corresponding to the plurality of network layers with a preset distance threshold; determining the network layer with the corresponding distance greater than the preset distance threshold value as an alternative network layer; and determining a target network layer from the alternative network layers based on the positions of the alternative network layers in the neural network to be deployed.
In an optional implementation manner, the testing apparatus further includes a second processing module, configured to: and adjusting the target network layer to obtain a target neural network.
In an optional embodiment, before the adjusting the target network layer, the second processing module is further configured to: randomly generating first test data input to the target network layer; processing second verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain fourth result data corresponding to a previous network layer of the target network layer; taking the fourth result data as second test data, and testing the target network layer by using the first test data and the second test data to obtain a test result; the second processing module is configured to, when the target network layer is adjusted to obtain the target neural network: and adjusting the target network layer based on the test result to obtain the target neural network.
In a third aspect, this disclosure also provides a computer device, a processor, and a memory, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions are executed by the processor to perform the steps in the first aspect or any one of the possible implementations of the first aspect.
In a fourth aspect, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, where the computer program is executed to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.
For the description of the effects of the testing apparatus, the computer device, and the computer-readable storage medium of the neural network, reference is made to the description of the testing method of the neural network, and details are not repeated here.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 shows a flow chart of a testing method of a neural network provided by an embodiment of the present disclosure;
fig. 2 is a schematic diagram illustrating a testing apparatus for a neural network provided in an embodiment of the present disclosure;
fig. 3 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of embodiments of the present disclosure, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
Research shows that the training process of the deep learning network is usually realized in the environment with high calculation power and high data precision; when the deep learning network is deployed in the embedded device, because the computational power and the data accuracy of the embedded device are limited to a certain extent, the trained deep learning network needs to be quantized according to the specific hardware environment of the embedded device, so that the quantized deep learning network can adapt to the hardware environment. Due to the reduction of data precision, the precision of the deep learning network is obviously reduced after the deep learning network is deployed in the embedded device, and the use requirement cannot be met.
In order to reduce the problem of accuracy degradation of the deep learning network after the deep learning network is deployed to the embedded device, it is generally necessary to further adjust parameters in the deep learning network before the deep learning network is deployed to the embedded device. Currently, a Random Search (Random Search), Bayesian Optimization (Bayesian Optimization), and other manners based on hyper-parameter adjustment are usually adopted to perform parameter adjustment in a global manner, and such manners are time-consuming and inefficient.
Based on the above research, the present disclosure provides a testing method for a neural network, which can determine, from a neural network to be deployed, a target network layer causing a problem in accuracy of the neural network to be deployed by using a standard network corresponding to the neural network to be deployed, so as to adjust parameters of the target network layer in a targeted manner, reduce time consumption required for adjusting the parameters of the neural network to be deployed, reduce computing resources and storage resources and data transmission resources required by the neural network in a process of executing a processing task, reduce operating environment requirements required for deploying the neural network, and improve adjustment efficiency.
The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
For the understanding of the present embodiment, a detailed description will be given to a testing method of a neural network disclosed in the embodiments of the present disclosure. In some possible implementations, the testing method of the neural network may be implemented by a processor calling computer-readable instructions stored in a memory.
The following describes a method for testing a neural network provided in an embodiment of the present disclosure.
Referring to fig. 1, a flowchart of a testing method for a neural network provided in an embodiment of the present disclosure is shown, where the method includes steps S101 to S103, where:
s101: acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed;
s102: processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer;
s103: determining a target network layer from the neural network to be deployed based on the first result data and the second result data.
According to the method and the device for adjusting the parameters of the neural network to be deployed, the time consumed for adjusting the parameters of the neural network to be deployed is reduced, and the adjusting efficiency is improved.
According to different functions of the neural network to be deployed, the neural network to be deployed can have corresponding functions and structures; illustratively, for a face detection function, the neural network to be deployed is a face detection neural network, and the corresponding verification data is an image containing a face; if the three-dimensional reconstruction function is achieved, the neural network to be deployed is a three-dimensional reconstruction neural network, and the corresponding verification data is an image for three-dimensional reconstruction; and if the verification data is a voice recognition function, the neural network to be deployed is a voice recognition network, and the corresponding verification data is voice. Specifically, the function and structure of the neural network to be deployed may be determined according to actual conditions, and the embodiments of the present disclosure are not limited.
The following describes details of S101 to S103.
For the above S101, when the neural network to be deployed is obtained, the structure of the obtained neural network to be deployed may be determined according to actual application and requirements, for example.
For example, when the method is actually applied to image processing, the selected Neural Network to be deployed may include a Convolutional Neural Network (RNN), or a self-attention Neural Network (Transformer), for example; when the word is processed, the selected Neural Network to be deployed may include, for example, a Recurrent Neural Network (RNN). The specific method can be determined according to actual conditions, and is not limited herein.
The specific function of the neural network to be deployed may also be determined according to actual needs, and the function may include any one of the following: face recognition, motion recognition, voice conversion, character recognition, and the like.
In addition, when the neural network to be deployed is determined, for example, the network model may be trained by using sample data to obtain the neural network to be deployed. For example, when it is determined that the network type of the neural network to be deployed is a convolutional neural network applied to image processing, for example, a plurality of sample images may be acquired as sample data, the convolutional neural network is trained, and the trained convolutional neural network is obtained as the above-described neural network to be deployed. That is, the determined neural network to be deployed may be, for example, a neural network capable of accurately performing corresponding task processing.
In another embodiment of the present disclosure, a standard network corresponding to a neural network to be deployed may also be obtained. The neural network to be deployed is a neural network which has a specific structure of the neural network. In order to actually deploy the neural network to be deployed in the hardware environment, a corresponding standard network can be obtained, so that whether a network layer with inaccurate setting exists in the neural network to be deployed is determined in the test environment, a target neural network is obtained by adjusting the neural network to be deployed, and the target neural network is specifically deployed in the hardware environment.
The hardware environment refers to a target neural network obtained by adjusting a neural network to be deployed, and is a use environment when the application is realized in different application scenes. For the test environment, for example, a clone replication environment of a hardware environment, that is, a transition environment from a development environment of the neural network to be deployed to the hardware environment, is included, and in the test environment, it is mainly tested whether a target network layer that needs to be adjusted exists in the neural network to be deployed.
The standard network is determined by using a Fast convolution structure (context) for Feature Embedding as an example. Wherein, Caffe is a deep learning framework, and can perform calculation processing on the input data of the network layer in the form of a network layer (layer) to obtain a processing result which can be considered as correct. Therefore, the network layers corresponding to the network layers in the neural network to be deployed can be determined through Caffe, and the standard network is constructed by utilizing the network layers determined by Caffe.
That is, each network layer in the determined standard network may determine a corresponding network layer in the neural network to be deployed, and the functions that the pair of network layers may specifically implement are the same. Therefore, when the standard network and the neural network to be deployed are used for respectively processing the same data and comparing result data output by each network layer, the standard network can correspondingly output a correct result, so that a target network layer which possibly has problems in the neural network to be deployed can be determined by using the comparison result.
For the above S102, after obtaining the neural network to be deployed and the standard network corresponding to the neural network to be deployed, for example, the neural network to be deployed and the standard network may be used to respectively process the first verification data, so as to obtain first result data respectively corresponding to each network layer of the neural network to be deployed and second result data respectively corresponding to each network layer in the standard network.
Next, data processing of the first verification data by the neural network to be deployed and the standard network is described. Here, the first verification data may be, for example, a data set, which may include one or more sets of data, and each set of data may include one or more data therein.
For the case that the to-be-deployed neural network performs data processing on the first verification data, specifically, for example, the to-be-deployed neural network may be used to process the first verification data in a hardware environment of the to-be-deployed neural network, so as to obtain first result data corresponding to each network layer of the to-be-deployed neural network.
The first verification data may include, for example, random data, and the acquired first verification data may include, for example, a set of random data with a mean value of 0 and a standard deviation of 1. Alternatively, the first verification data may also include target data actually used in the hardware environment, such as experimental data in the hardware environment, or data to be processed in an actual task, for example. Specifically, the manner of determining the first verification data is only a few examples, and is not limited.
After the first verification data is determined, for example, the following method may be specifically used to determine first result data obtained by processing the first verification data by the neural network to be deployed: analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed; determining operators to be replaced which are not supported by the hardware environment from the first operators of the neural network to be deployed based on the hardware environment and the operator information of each first operator; replacing the operator to be replaced by using the second operator adapted to the hardware environment to generate a testing neural network; and inputting the first verification data into the test neural network to obtain the first result data.
Here, since the neural network to be deployed obtained in S101 only can obtain a better processing result when performing data processing on software, and cannot guarantee that the neural network to be deployed can be well adapted to a hardware environment to be deployed, even if the determined neural network to be deployed has a better processing result when performing processing of a corresponding task, it is necessary to perform quantization processing on network parameters thereof according to the hardware environment, so as to obtain the neural network that can be actually deployed in the hardware environment.
Specifically, before quantization processing is performed on a network parameter, that is, a first operator, in the neural network to be deployed, for example, the neural network to be deployed may be analyzed to determine operator information of each first operator in the neural network to be deployed. Wherein the operator information comprises: operator type and/or operator context.
When the neural network to be deployed is analyzed, for example, the neural network to be deployed may be analyzed to obtain operator information of each first operator in the neural network to be deployed; (ii) a And/or analyzing the model parameter file corresponding to the neural network to be deployed to obtain operator internal parameters of operators respectively corresponding to each network layer in the neural network to be deployed.
Illustratively, the structure definition file may comprise, for example, a ". prototxt" type file. The structure definition file may include, for example, definitions of structures corresponding to the neural network to be deployed, such as the number of network layers of the neural network to be deployed, and structure types corresponding to the network layers, such as convolutional layers, pooling layers, full-link layers, and the like, and further include operator types corresponding to operators respectively for the network layers. For operators corresponding to different network layers, the operators corresponding to part of the network layers may further include operator internal parameters, such as convolution kernels corresponding to convolution layers, convolution step lengths and the like, and full-connection weight matrices corresponding to full-connection layers, so that a model parameter file corresponding to the neural network to be deployed may be further analyzed to determine the operator internal parameters in the neural network to be deployed. The model parameter files may include, for example, files for defining or setting operator types of operators in the model, such as model. Analyzing the model parameter file, wherein an operator can be a convolution operator, and the corresponding operator internal parameter comprises a convolution kernel and a convolution step 2 of the convolution kernel; or a fully connected weight matrix comprising fully connected layers, the corresponding additional parameters for example comprising a set of weight parameters.
In a possible case, the operator types of the first operators in the neural network to be deployed are the same, for example, in a case that the quantity of the neural network to be deployed is small, it is determined that the operator types of the first operators in the neural network to be deployed are int 4. In fact, the quantization bit number corresponding to the hardware environment to be deployed is int8 or float16 (which represents a floating point type that occupies 16 bits in a computer), so the neural network to be deployed with the int4 as the operator type cannot be applied to the hardware environment.
Accordingly, since the quantization bit number of the hardware environment can be determined, the operator to be replaced, which is not supported by the hardware environment, can be determined from the first operators of the neural network to be deployed according to the operator type of each first operator.
Specifically, it may be determined whether an operator of a function corresponding to the first operator exists in the function library by a detection method. If the operator with the corresponding function cannot be found in the function library by the first operator, that is, the operator to be replaced, for example, the function and the name corresponding to the operator to be replaced may be output, the corresponding second operator may be determined again for the operator to be replaced by programming or the like, and the operator with the function corresponding to the second operator may be added to the function library. Therefore, after the second operator replaces the corresponding operator to be replaced, the test neural network can be generated under the condition that the first operator and the second operator can both determine the corresponding operator in the function library. In a possible case, in order to determine the test neural network by using the first operator and the second operator, quantization processing may be further performed on the operators of the first operator and the second operator, and the quantization processing is performed to be an operator which can be used in a hardware environment. Taking the above example as an example, if the operator type in the hardware environment is int8, and the operator type in the neural network to be deployed is int4, that is, the operator types of the first operator and the second operator are int4, quantization processing may be performed on the first operator and the second operator, and quantization is performed to int8, so that the quantized operator can be used in the hardware environment.
After the test neural network is generated, since the test neural network can be used in a hardware environment, the first verification data can be processed in the hardware environment by using the test neural network. The obtained first result data can be used for verifying whether the testing neural network can show better processing capacity in a hardware environment, namely, whether the neural network to be deployed can have better data processing capacity in the hardware environment can be further determined.
The first result data comprises result data respectively corresponding to each network layer in the neural network to be deployed. For example, if the neural network to be deployed has n (n is a positive integer) network layers, the first result data corresponds to each network layer in the n network layers. For example, corresponding to an i-th layer network layer, the first result data corresponding to this layer network layer may be represented as si(ii) a Corresponding to the i-1 th layer network layer, the first result data corresponding to this layer network layer can be represented as si-1. Wherein, 1 is not less than i-1<i≤n。
Here, in order to determine whether the neural network to be deployed has better data processing capability through the first result data, the processing result of the first verification data (i.e., the second result data) may be determined by comparing the first result data with the standard network. In order to obtain the second result data and to make the first result data and the second result data comparable, for example, the same first verification data may be used. The first verification data is processed by using the standard network, and similarly, second result data corresponding to each network layer in the standard network can be obtained.
Here, the second resultThe data is similar to the first result data and includes result data corresponding to respective network layers in the standard network. Taking the to-be-deployed neural network described above as an example, since the to-be-deployed neural network has a similar structure to the standard network, for the standard network, each network layer in the corresponding n-layer network layers corresponds to the second result data. For example, corresponding to a layer of the i-th network layer, the second result data corresponding to that layer of the network may be denoted as p, for examplei(ii) a Corresponding to the i-1 th layer network layer, the corresponding second result data of this layer network layer may be denoted as p, for examplei-1。
In this way, by comparing the first result data and the second result data respectively corresponding to the corresponding network layers in the neural network to be deployed and the standard network, it can be determined that there may be a network layer of an operator which is not adapted to hardware in the neural network to be deployed.
For the above S103, when determining the target network layer in the neural network to be deployed by using the first result data and the second result data, for example, the following manner may be adopted: for each network layer in the plurality of network layers of the neural network to be deployed, inversely quantizing first result data corresponding to the network layer to a bit number consistent with the second result data to obtain third result data corresponding to the network layer; determining the distance between the second result data and the third result data corresponding to the network layer; and determining a target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers respectively.
In a specific implementation, since the second result data is obtained by processing the first verification data level by the standard network, the type of the second result data is, for example, corresponding to the standard network, for example, float32, and the first result data is obtained by processing the first verification data in the hardware environment by the network to be deployed, and therefore, the type of the first result data is, for example, corresponding to the hardware environment, for example, int8 or float 16.
Illustratively, the type of the first result data is int8, and the type of the second result data is float 32. To make it possible to compare the first result data with the second result dataThe two result data are compared, and first result data corresponding to each network layer in a plurality of network layers in the neural network to be deployed can be dequantized to the bit number consistent with the second result data. For example, for the i-th layer network layer, the corresponding first result data is siAnd the first result data siThe corresponding type is int 8; the corresponding second result data is piAnd the second result data is float 32. By inverse quantizing the first result data to the type of float32, third result data q may be obtained, for examplei. In this way, the third result data is obtained by inverse quantizing the first result data to the bit number consistent with the second result data, and because the type of the third result data is consistent with that of the second result data, the third result data can be compared with the second result data to determine whether the layer i network layer in the neural network to be deployed can complete the corresponding data processing task similar to the standard network.
The number of bits of the first result data which are inversely quantized to be consistent with the number of bits of the second result data are selected to obtain third result data, and the third result data and the second result data are compared in a mode of high-precision standard. Compared with a method of quantizing the second result data to the number of bits consistent with the first result data and then comparing the second result data with the first result data, the inverse quantization method adopted by the embodiment of the disclosure can meet the requirement of accurately comparing the second result data under a higher precision standard on the one hand, and can avoid the introduction of quantization errors under the quantization method on the other hand, so that whether the network layer can complete the corresponding data processing task similarly to a standard network or not can not be determined through the comparison of the result data under the same precision.
Wherein, aiming at the i-th layer network layer, the second result data p is processediAnd the third result data qiWhen the comparison is performed, for example, the second result data p may be determined firstiAnd the third result data qiThe distance is compared with a preset distance threshold value by using the distance. Wherein the second result data p is calculatediAnd the third result data qiBetweenCan be calculated, for example, the second result data piAnd the third result data qiAt least one of cosine distance and euclidean distance, and the specific calculation manner and calculation process are not limited, and are not described herein again.
Similarly, for other network layers in the neural network to be deployed, for example, distances corresponding to the network layers of the i-th layer may be determined in a manner corresponding to the network layers of the i-th layer. In this way, distances corresponding to a plurality of network layers in the neural network to be deployed can be determined.
In a specific implementation, for example, the target network layer may be determined from the plurality of network layers according to respective distances corresponding to the plurality of network layers. Specifically, for example, the following manner may be adopted: comparing the distances respectively corresponding to the plurality of network layers with a preset distance threshold; determining the network layer with the corresponding distance smaller than the preset distance threshold value as an alternative network layer; and determining a target network layer from the alternative network layers based on the positions of the alternative network layers in the neural network to be deployed.
Illustratively, with a corresponding distance l of the i-th network layeriThe description is given for the sake of example. When determining the preset distance threshold Th for comparison, a specific value of the preset distance threshold may be determined according to actual conditions or experience, for example. Specifically, the distance l corresponding to the i-th network layeriFor example, the cosine distance is used, and in order to ensure that data output by an i-layer network layer in the neural network to be deployed is substantially consistent with a standard network, a preset distance threshold Th may be set to be 0.99. Here, the distance may represent similarity of network layers, for example, and if data corresponding to the distance is large, the two network layers may be considered to be more similar. If the distance l isiIf the value is less than 0.99, the i-th network layer can be considered as an alternative network layer which may need to be adjusted; if the distance l isiIf the number of the i-th layer is greater than or equal to 0.99, the i-th layer can be considered as a network layer which does not need to be adjusted, i.e. as a network layer without errors in configuration.
For other network layers in the neural network to be deployed, it can be determined whether other network layers are alternative network layers according to a similar manner as for whether the i-th network layer can be used as an alternative network layer in the above example. In a possible case, because there is a problem in the configuration of a network layer of the neural network to be deployed, the difference between the first result data output by the layer and the corresponding second result data is large (that is, the distance corresponding to the layer network layer is large), which may cause that a network with correct subsequent configuration cannot obtain result data with a small difference from the second result data when the first result data is continuously processed, and therefore, in order to more clearly locate a network layer with a configuration error, a target network layer may be determined from the alternative network layers based on the positions of the alternative network layers in the neural network to be deployed.
Specifically, when the target network layer is determined from the candidate network layers, for example, a first network layer in the candidate network layers may be directly used as the target network layer. For example, when the alternative network layers include i-th to n-th layers, the i-th network layer may be a target network layer, for example.
In another embodiment of the present disclosure, after determining the target network layer, the target network layer may be adjusted, for example, to obtain the target neural network.
In a specific implementation, when determining the target neural network, for example, the following manner may be adopted: randomly generating first test data input to the target network layer; processing second verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain fourth result data corresponding to a previous network layer of the target network layer; taking the fourth result data as second test data, and testing the target network layer by using the first test data and the second test data to obtain a test result; and adjusting the target network layer based on the test result to obtain the target neural network.
For example, after the test result is determined, for example, an operator that may be abnormal in the target network layer may be reversely deduced according to the test result, and the operator that may be abnormal is adjusted, so as to achieve the purpose of adjusting the target network layer. After one adjustment, for example, the above manner may be repeated to determine new first test data, so as to determine whether the target network layer is adjusted by the correct operator according to the new test result.
In determining the first test data input to the target network layer, the first test data may be determined based on input data of the target network layer when data is actually processed, for example. For example, when the target network layer is a first network layer in the neural network to be deployed, the first test data input to the target layer may be obtained in a manner similar to the determination of the first verification data in S102. When the target network layer is another layer network layer in the neural network to be deployed, the first test data input to the target network layer may randomly generate another data similar in type and structure as the first test data according to the first result data corresponding to the previous network layer of the target network layer of the layer, for example.
In addition, the second verification data can be processed by using the neural network to be deployed under the hardware environment of the neural network to be deployed, so that fourth result data corresponding to a network layer above the target network layer is obtained. Here, the second verification data may be, for example, partially the same as the first verification data described above, or completely different from the first verification data determined in a similar manner to the determination of the first verification data described above. Similarly, the second verification data is processed by using the neural network to be deployed, so that processing results respectively corresponding to each network layer can be obtained, wherein fourth result data corresponding to a previous network layer of the target network layer is selected as second test data.
After the first test data and the second test data are determined, the target network layer can be tested by using the first test data and the second test data to obtain output data of the target network layer as a test result. The test result may include, for example, a first test result corresponding to the first test data and a second test result corresponding to the second test data.
Here, for the first number of testsAccording to the corresponding first test result, the type corresponding to the first test data is the same as the type of the first result data. In order to determine whether the neural network to be deployed can accurately process the randomly generated test data or not through the first test data, the test result a corresponding to the first test data may be usediAnd b, processing the first test data with a target network layer in the standard network to obtain a result biAnd (6) carrying out comparison. Here, the specific pair aiAnd biThe comparison may be performed by, for example, comparing the distance between the first result data and the second result data with a preset distance threshold to obtain the distance l'i。
Here, the preset distance threshold Th' may be the same as the distance threshold Th described above, for example, or may be set to a value different from the distance threshold Th in accordance with the actual situation.
At a determined distance l'iAfter, if distance l'iIf the distance is smaller than the preset distance threshold, the target network layer is considered to be incapable of well processing the randomly generated test data; l'iIf the distance is greater than or equal to the preset distance threshold, the target network layer is considered to be capable of better processing the randomly generated test data.
That is, the first test result corresponding to the first test data may include, for example: a1. the target network layer can better process the randomly generated test data; and a2, the target network layer cannot well process the randomly generated test data.
For the second test data, the corresponding second test result may be determined in the same processing manner as the first test data. Since the second test data includes fourth result data corresponding to a previous network layer of the target network layer, the second test result that may be determined may include, for example: b1. the target network layer can better process the output result data of the previous network layer; and b2. the target network layer cannot process the output result data of the previous network layer well.
Thus, the target network layer is determined, and the determined first test result and the second test result are combined to obtain the test result for testing the target network layer. Specifically, under different first test results and second test results, the test result corresponding to the target network layer may include:
the first test result is a2 and the second test result is b2.
In this case, it is reflected that the target network layer has poor processing on both the randomly generated data and the data result data of the previous network layer, and it can be correspondingly considered that the target network layer of the layer has a problem and further parameter adjustment is required.
② the first test result is a1, the second test result is b2.
In this case, it is reflected that the target network layer processes the randomly generated data well, but processes the data result data of the previous network layer poorly, and it can be considered that a problem occurs in the quantization step in the quantization process for distance comparison, for example, a bit setting error of quantization is not a parameter error of the target network layer itself. Therefore, this problem can be eliminated by reviewing the quantization step, and the detailed process is not described herein.
③ the first test result is a1 and the second test result is b1.
Under the condition, the target network layer can better process the randomly generated data and the data result data of the previous network layer, and accordingly the parameters in the target network layer can be considered to be better suitable for the requirement of the hardware environment. At this point, for example, an attempt may be made to replace the target network layer with more accurate data for further testing, such as data of the float16 or float32 type, to more specifically determine whether there are parameters in the target network layer that need to be adjusted.
Thus, according to the test result determined for the target network layer, the target network layer can be adjusted to determine the target neural network with higher accuracy in data processing under the hardware environment.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, the embodiment of the present disclosure further provides a testing apparatus for a neural network corresponding to the testing method for the neural network, and since the principle of the apparatus in the embodiment of the present disclosure for solving the problem is similar to the testing method for the neural network described above in the embodiment of the present disclosure, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not described again.
Referring to fig. 2, a schematic diagram of a testing apparatus for a neural network provided in an embodiment of the present disclosure is shown, the testing apparatus including: the device comprises an acquisition module 21, a first processing module 22 and a determination module 23; wherein,
an obtaining module 21, configured to obtain a neural network to be deployed and a standard network corresponding to the neural network to be deployed;
the first processing module 22 is configured to, in a hardware environment of the neural network to be deployed, process the first verification data by using the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer;
a determining module 23, configured to determine a target network layer from the neural network to be deployed based on the first result data and the second result data.
In an optional implementation manner, in a hardware environment of the neural network to be deployed, when the first processing module 22 processes the first verification data by using the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed, the first processing module is configured to: analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed; determining operators to be replaced which are not supported by the hardware environment from the first operators of the neural network to be deployed based on the hardware environment and the operator information of each first operator; replacing the operator to be replaced by using the second operator adapted to the hardware environment to generate a testing neural network; and inputting the first verification data into the test neural network to obtain the first result data.
In an optional embodiment, the operator information includes: operator type and/or operator context; the first processing module 22 is configured to, when analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed: analyzing the structure definition file corresponding to the neural network to be deployed to obtain the operator types respectively corresponding to the operators of each network layer of the neural network to be deployed; and/or analyzing the model parameter file corresponding to the neural network to be deployed to obtain operator internal parameters of operators respectively corresponding to each network layer in the neural network to be deployed.
In an optional embodiment, the determining module 23, when determining the target network layer from the neural network to be deployed based on the first result data and the second result data, is configured to: for each network layer in the plurality of network layers of the neural network to be deployed, inversely quantizing first result data corresponding to the network layer to a bit number consistent with the second result data to obtain third result data corresponding to the network layer; determining the distance between the second result data and the third result data corresponding to the network layer; and determining a target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers respectively.
In an optional embodiment, when determining the target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers, the determining module 23 is configured to: comparing the distances respectively corresponding to the plurality of network layers with a preset distance threshold; determining the network layer with the corresponding distance greater than the preset distance threshold value as an alternative network layer; and determining a target network layer from the alternative network layers based on the positions of the alternative network layers in the neural network to be deployed.
In an optional embodiment, the testing apparatus further includes a second processing module 24, configured to: and adjusting the target network layer to obtain a target neural network.
In an optional embodiment, the second processing module 24 is further configured to, before performing the adjustment to the target network layer: randomly generating first test data input to the target network layer; processing second verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain fourth result data corresponding to a previous network layer of the target network layer; taking the fourth result data as second test data, and testing the target network layer by using the first test data and the second test data to obtain a test result; the second processing module 24 is configured to, when the target network layer is adjusted to obtain the target neural network: and adjusting the target network layer based on the test result to obtain the target neural network.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
An embodiment of the present disclosure further provides a computer device, as shown in fig. 3, which is a schematic structural diagram of the computer device provided in the embodiment of the present disclosure, and the computer device includes:
a processor 10 and a memory 20; the memory 20 stores machine-readable instructions executable by the processor 10, the processor 10 being configured to execute the machine-readable instructions stored in the memory 20, the processor 10 performing the following steps when the machine-readable instructions are executed by the processor 10:
acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed; processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer; determining a target network layer from the neural network to be deployed based on the first result data and the second result data.
The storage 20 includes a memory 210 and an external storage 220; the memory 210 is also referred to as an internal memory, and temporarily stores operation data in the processor 10 and data exchanged with the external memory 220 such as a hard disk, and the processor 10 exchanges data with the external memory 220 through the memory 210.
For the specific execution process of the instruction, reference may be made to the steps of the neural network testing method described in the embodiments of the present disclosure, and details are not described here.
The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of the method for testing a neural network described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the method for testing a neural network in the above method embodiments, which may be referred to specifically in the above method embodiments, and are not described herein again.
The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (10)
1. A method for testing a neural network, comprising:
acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed;
processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; and
processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer;
determining a target network layer from the neural network to be deployed based on the first result data and the second result data.
2. The testing method according to claim 1, wherein the obtaining first result data corresponding to each network layer of the neural network to be deployed by processing the first verification data with the neural network to be deployed in the hardware environment of the neural network to be deployed comprises:
analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed;
determining operators to be replaced which are not supported by the hardware environment from the first operators of the neural network to be deployed based on the hardware environment and the operator information of each first operator;
replacing the operator to be replaced by using the second operator adapted to the hardware environment to generate a testing neural network;
and inputting the first verification data into the test neural network to obtain the first result data.
3. The testing method of claim 2, wherein the operator information comprises: operator type and/or operator context;
analyzing the neural network to be deployed to obtain operator information of each first operator in the neural network to be deployed, including:
analyzing the structure definition file corresponding to the neural network to be deployed to obtain the operator types respectively corresponding to the operators of each network layer of the neural network to be deployed; and/or the presence of a gas in the gas,
analyzing the model parameter file corresponding to the neural network to be deployed to obtain operator internal parameters of operators respectively corresponding to each network layer in the neural network to be deployed.
4. The testing method according to any one of claims 1 to 3, wherein the determining a target network layer from the neural network to be deployed based on the first result data and the second result data comprises:
for each network layer in the plurality of network layers of the neural network to be deployed, inversely quantizing first result data corresponding to the network layer to a bit number consistent with the second result data to obtain third result data corresponding to the network layer;
determining the distance between the second result data and the third result data corresponding to the network layer;
and determining a target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers respectively.
5. The method according to claim 4, wherein the determining a target network layer from the plurality of network layers according to the distances corresponding to the plurality of network layers comprises:
comparing the distances respectively corresponding to the plurality of network layers with a preset distance threshold;
determining the network layer with the corresponding distance greater than the preset distance threshold value as an alternative network layer;
and determining a target network layer from the alternative network layers based on the positions of the alternative network layers in the neural network to be deployed.
6. The test method of any one of claims 1-5, further comprising: and adjusting the target network layer to obtain a target neural network.
7. The method of claim 6, wherein before adjusting the target network layer, further comprising:
randomly generating first test data input to the target network layer;
processing second verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain fourth result data corresponding to a previous network layer of the target network layer;
taking the fourth result data as second test data, and testing the target network layer by using the first test data and the second test data to obtain a test result;
the adjusting the target network layer to obtain the target neural network includes:
and adjusting the target network layer based on the test result to obtain the target neural network.
8. An apparatus for testing a neural network, comprising:
the system comprises an acquisition module, a deployment module and a deployment module, wherein the acquisition module is used for acquiring a neural network to be deployed and a standard network corresponding to the neural network to be deployed;
the first processing module is used for processing first verification data by using the neural network to be deployed under the hardware environment of the neural network to be deployed to obtain first result data corresponding to each network layer of the neural network to be deployed; processing the first verification data by using the standard network under a test environment to obtain second result data respectively corresponding to each network layer;
a determining module, configured to determine a target network layer from the neural network to be deployed based on the first result data and the second result data.
9. A computer device, comprising: a processor, a memory storing machine readable instructions executable by the processor, the processor for executing the machine readable instructions stored in the memory, the machine readable instructions when executed by the processor, the processor performing the steps of the method of testing a neural network of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when executed by a computer device, performs the steps of the method of testing a neural network as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111168642.9A CN113887722A (en) | 2021-09-30 | 2021-09-30 | Neural network testing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111168642.9A CN113887722A (en) | 2021-09-30 | 2021-09-30 | Neural network testing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113887722A true CN113887722A (en) | 2022-01-04 |
Family
ID=79005240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111168642.9A Pending CN113887722A (en) | 2021-09-30 | 2021-09-30 | Neural network testing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113887722A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114818600A (en) * | 2022-04-29 | 2022-07-29 | 上海阵量智能科技有限公司 | Chip verification method and device, electronic equipment and storage medium |
-
2021
- 2021-09-30 CN CN202111168642.9A patent/CN113887722A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114818600A (en) * | 2022-04-29 | 2022-07-29 | 上海阵量智能科技有限公司 | Chip verification method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11790212B2 (en) | Quantization-aware neural architecture search | |
| US20220283820A1 (en) | Data parallelism in distributed training of artificial intelligence models | |
| US20210019151A1 (en) | Executing large artificial intelligence models on memory-constrained devices | |
| US20210019634A1 (en) | Dynamic multi-layer execution for artificial intelligence modeling | |
| CN112200296A (en) | Network model quantification method and device, storage medium and electronic equipment | |
| CN112784981A (en) | Training sample set generation method, and training method and device for deep generation model | |
| CN116991711A (en) | Test case generation method and device, terminal equipment and storage medium | |
| CN115081613A (en) | Method and device for generating deep learning model, electronic equipment and storage medium | |
| US20220222927A1 (en) | Apparatus, system, and method of generating a multi-model machine learning (ml) architecture | |
| CN113887722A (en) | Neural network testing method and device, computer equipment and storage medium | |
| CN114818600A (en) | Chip verification method and device, electronic equipment and storage medium | |
| CN115496144A (en) | Power distribution network operation scene determining method and device, computer equipment and storage medium | |
| CN113554097B (en) | Model quantization method and device, electronic equipment and storage medium | |
| CN111832693A (en) | Neural network layer operation and model training method, device and equipment | |
| EP3324346B1 (en) | Parallelization approaches of modified teaching learning based search optimization technique for variable selection | |
| EP4322064A1 (en) | Method and system for jointly pruning and hardware acceleration of pre-trained deep learning models | |
| CN111797220A (en) | Dialog generation method and device, computer equipment and storage medium | |
| KR20210141252A (en) | Electronic apparatus and method for controlling thereof | |
| CN117472431A (en) | Code annotation generation method, device, computer equipment, storage medium and product | |
| CN115438575A (en) | Analysis method for high-precision airfoil flow field prediction | |
| CN117642752A (en) | Quantum neural network for noisy medium quantum device | |
| CN108664368B (en) | Processor performance index evaluation method and device | |
| US11868304B1 (en) | Auto-configuration of hardware non-linear function acceleration | |
| KR102731304B1 (en) | Apparatus, method and recording medium storing instructions for optimizing hyperparameter | |
| US20240249114A1 (en) | Search space limitation apparatus, search space limitation method, and computer-readable recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: Floor 1-3, No. 24, Lane 315, Fenggu Road, Xuhui District, Shanghai, 201103 Applicant after: Shanghai Qianshi Technology Co.,Ltd. Address before: 201103 unit 6-78, building 6, No. 1900, Hongmei Road, Xuhui District, Shanghai Applicant before: Shanghai shangtangqian Technology Co.,Ltd. Country or region before: China |