CN111353585A - Structure searching method and device of neural network model - Google Patents
Structure searching method and device of neural network model Download PDFInfo
- Publication number
- CN111353585A CN111353585A CN202010116705.5A CN202010116705A CN111353585A CN 111353585 A CN111353585 A CN 111353585A CN 202010116705 A CN202010116705 A CN 202010116705A CN 111353585 A CN111353585 A CN 111353585A
- Authority
- CN
- China
- Prior art keywords
- network structure
- candidate network
- candidate
- image
- yuv image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003062 neural network model Methods 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 73
- 238000012545 processing Methods 0.000 claims abstract description 45
- 230000004044 response Effects 0.000 claims abstract description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 230000000875 corresponding effect Effects 0.000 description 26
- 238000010586 diagram Methods 0.000 description 11
- 238000013528 artificial neural network Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000002596 correlated effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/086—Learning methods using evolutionary algorithms, e.g. genetic algorithms or genetic programming
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Physiology (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure relates to the field of artificial intelligence. The embodiment of the disclosure discloses a structure searching method and device of a neural network model. The method comprises the following steps: searching a candidate network structure from a preset network structure searching space by adopting a controller; training a candidate network structure by using YUV image training data; acquiring performance information of the trained candidate network structure, generating feedback information according to the performance information of the trained candidate network structure, and feeding the feedback information back to the controller, so that the controller searches a new candidate network structure from the network structure search space after updating based on the feedback information; and in response to determining that the feedback information generated based on the performance information of the candidate network structure finished with the current training reaches a preset convergence condition, generating a neural network model structure for the YUV image data based on the candidate network structure finished with the current training. The method can automatically search out the neural network model structure suitable for processing YUV image data.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to the technical field of artificial intelligence, and particularly relates to a structure searching method and device of a neural network model.
Background
With the development of artificial intelligence technology, deep neural networks have achieved important achievements in many fields. The deep neural network has incomparable advantages compared with the traditional data processing technology in the processing task of large-batch and various data. For example, for processing of image data, the trained deep neural network can process various scene images of various resolutions, various sizes, and various degrees of definition.
Existing neural network models for image or video processing are designed for RGB (red, yellow and blue) type input data. However, in some common image or video applications, the acquired data is in YUV (a luminance-chrominance color coding) format. Because the physical meanings of the YUV data and the RGB data are greatly different, a neural network model designed for the RGB data is difficult to obtain a good processing effect on the YUV data.
Disclosure of Invention
The embodiment of the disclosure provides a structure searching method and device of a neural network model, an electronic device and a computer readable medium.
In a first aspect, an embodiment of the present disclosure provides a structure search method for a neural network model, including: searching a candidate network structure from a preset network structure searching space by adopting a controller; acquiring YUV image training data, and training a candidate network structure by using the YUV image training data; acquiring performance information of the trained candidate network structure, generating feedback information according to the performance information of the trained candidate network structure, and feeding the feedback information back to the controller, so that the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information; and in response to determining that the feedback information generated based on the performance information of the candidate network structure finished with the current training reaches a preset convergence condition, generating a neural network model structure for the YUV image data based on the candidate network structure finished with the current training.
In some embodiments, the preset network structure search space includes candidate sizes of the image; and the above method further comprises: and determining the image size corresponding to the candidate network structure from the candidate sizes of the image by adopting a controller.
In some embodiments, the above method further comprises: and determining the image size corresponding to the candidate network structure determined currently as the target image size corresponding to the neural network model structure for the YUV image data in response to the fact that the feedback information generated based on the performance information of the candidate network structure trained currently reaches the preset convergence condition.
In some embodiments, the above method further comprises: acquiring YUV image data to be processed; and converting the size of the YUV image data to be processed into the size of a target image, and inputting the size of the target image into a neural network model structure for processing the YUV image data.
In some embodiments, the acquiring YUV image training data includes: and converting the acquired RGB image sample data into YUV image sample data, and converting the image size of the YUV image sample data into the image size corresponding to the candidate network structure determined from the preset network structure search space.
In a second aspect, an embodiment of the present disclosure provides a structure searching apparatus for a neural network model, including: a first searching unit configured to search out a candidate network structure from a preset network structure searching space by using a controller; the training unit is configured to acquire YUV image training data and train a candidate network structure by using the YUV image training data; the feedback unit is configured to acquire performance information of the trained candidate network structure, generate feedback information according to the performance information of the trained candidate network structure, and feed the feedback information back to the controller, so that the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information; a generating unit configured to generate a neural network model structure for the YUV image data based on the currently trained candidate network structure in response to determining that feedback information generated based on the performance information of the currently trained candidate network structure reaches a preset convergence condition.
In some embodiments, the preset network structure search space includes candidate sizes of the image; and the above apparatus further comprises: and the second searching unit is configured to determine the image size corresponding to the candidate network structure from the candidate sizes of the images by adopting the controller.
In some embodiments, the generating unit is further configured to determine an image size corresponding to the currently determined candidate network structure as a target image size corresponding to a neural network model structure for the YUV image data in response to determining that the feedback information generated based on the performance information of the currently trained candidate network structure reaches a preset convergence condition.
In some embodiments, the above apparatus further comprises: a processing unit configured to: acquiring YUV image data to be processed; and converting the size of the YUV image data to be processed into the size of a target image, and inputting the size of the target image into a neural network model structure for processing the YUV image data.
In some embodiments, the training unit is configured to obtain YUV image training data as follows: and converting the acquired RGB image sample data into YUV image sample data, and converting the image size of the YUV image sample data into the image size corresponding to the candidate network structure determined from the preset network structure search space.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; a storage device for storing one or more programs which, when executed by one or more processors, cause the one or more processors to implement the structure search method for a neural network model as provided in the first aspect.
In a fourth aspect, an embodiment of the present disclosure provides a computer readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the structure search method of the neural network model provided in the first aspect.
The structure searching method and apparatus of the neural network model according to the above embodiments of the present disclosure searches a candidate network structure from a preset network structure search space by using a controller, then obtains YUV image training data, trains the candidate network structure using the YUV image training data, then obtains performance information of the trained candidate network structure, generates feedback information according to the performance information of the trained candidate network structure, and feeds back the feedback information to the controller, so that the controller searches a new candidate network structure from the preset network structure search space after updating based on the feedback information, generates a neural network model structure for image data based on the currently trained candidate network structure in response to determining that the feedback information generated based on the performance information of the currently trained candidate network structure reaches a preset YUV convergence condition, the automatic search of the neural network model structure suitable for processing the YUV image data is realized, so that the neural network model structure suitable for processing the YUV image data can be automatically constructed. In addition, the controller is updated through performance information of the candidate network structure trained by the YUV image training data in the searching process, so that the optimization efficiency of the controller can be accelerated, the searching speed of the model structure is increased, and the processing capacity of the searched network structure on YUV data is ensured.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which embodiments of the present disclosure may be applied;
FIG. 2 is a flow diagram of one embodiment of a structure search method of a neural network model according to the present disclosure;
FIG. 3 is a flow diagram of another embodiment of a structure search method of a neural network model according to the present disclosure;
FIG. 4 is a schematic structural diagram of an embodiment of a structure searching apparatus of a neural network model of the present disclosure;
FIG. 5 is a schematic block diagram of a computer system suitable for use in implementing an electronic device of an embodiment of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which the structure search method of the neural network model or the structure search apparatus of the neural network model of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 101, 102, 103 interact with a server 105 via a network 104 to receive or send messages or the like. The end devices 101, 102, 103 may be customer premises devices on which various client applications may be installed. Such as image processing type applications, video interaction type applications, etc.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server running various services, such as a server running image or video based object detection, object tracking, image optimization, etc. services. The server 105 may obtain image data collected from the terminal devices 101, 102, 103 or obtain image data from a database to construct training samples, automatically search and optimize the model structure of the neural network for performing the deep learning task.
The server 105 may also be a backend server providing backend support for applications installed on the terminal devices 101, 102, 103. For example, the server 105 may receive image or video data to be processed sent by the terminal devices 101, 102, 103, process the image or video data using a neural network model, and return the processing result to the terminal devices 101, 102, 103.
In a real scenario, a user may use the terminal devices 101, 102, 103 to send an image or video processing request to the server 105. The server 105 may run thereon a neural network model that has been trained for a corresponding image or video processing task, with which the image or video is processed.
It should be noted that the structure searching method of the neural network model provided by the embodiment of the present disclosure is generally executed by the server 105, and accordingly, the structure searching device of the neural network model is generally disposed in the server 105.
In some scenarios, server 105 may retrieve source data (e.g., training samples, non-optimized neural networks, etc.) required for model generation from a database, memory, or other device, in which case exemplary system architecture 100 may be absent of terminal devices 101, 102, 103 and network 104.
The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a structure search method of a neural network model in accordance with the present disclosure is shown. The structure searching method of the neural network model comprises the following steps:
In this embodiment, the structure search method of the neural network model may search out the neural network structure model for the YUV image data by iteratively performing a plurality of search operations. Specifically, the feedback information may be initialized, for example, the feedback value is initialized to 0, and in each iteration, after the controller is updated based on the current feedback information, the controller is adopted to search out the candidate network structure from the preset network structure search space.
Here, the controller may be implemented as a recurrent neural network, the parameters of which are adjusted in each iteration based on feedback information back propagation.
Alternatively, the controller may be implemented as a reinforcement learning algorithm with feedback information as reward to direct the updating of reinforcement learning based neural networks to re-search for better candidate network structures in each iteration.
Alternatively, the controller may be implemented as a simulated annealing algorithm, a genetic algorithm, or other algorithm based on a feedback information iterative optimization strategy.
The preset network structure search space may be a search space which is constructed in advance and contains various network structure units. The network structure unit may be a basic unit for constructing a neural network model, and specifically may be a single network layer, such as a single convolutional layer or a full connection layer; or may be a structural unit formed by combining a plurality of network layers, such as a block structure (block) formed by combining a convolutional layer, a Batch Normalization layer (Batch Normalization), and a nonlinear layer (e.g., Relu).
The sequence code output by the controller is the code of the network structure unit in the preset network structure search space, and the candidate network structure searched in each iteration can be obtained by decoding the sequence code.
In this embodiment, image data in YUV format may be acquired from a database as training data, or may be connected to a terminal device to acquire acquired YUV image data to construct training data.
In some optional implementation manners, the collected RGB image data may be converted into image data in a YUV format, so that training data in the YUV format may be constructed based on the commonly used RGB image data, and the difficulty in obtaining and the construction cost of the YUV training data are reduced.
The candidate network structure searched out in step 201 may be trained using YUV image training data. In particular, the candidate network structure may be utilized to perform corresponding processing tasks on the YUV image training data, such as object recognition, sharpness conversion, transformation or optimization of a specified object in the image, and so on. Iteratively adjusting parameters of the candidate network structure based on error back-propagation of the candidate network structure performing the respective processing task.
The trained candidate network structure can be tested by using a test data set to obtain performance information of the trained candidate network structure. Here, the performance information of the trained candidate network structure may include one or more of accuracy, error, delay, processor power consumption, and computational efficiency of the trained candidate network structure. The performance information may also be related to the number of search operations (i.e., the number of iterative updates of the controller).
Feedback information may be generated based on performance information of the trained candidate network structures. And generating positive feedback information if the performance of the trained candidate network structure is good, and generating negative feedback information if the performance of the trained candidate network structure is poor. Specifically, the feedback value representing the feedback information may be positively correlated with the accuracy and the computational efficiency of the trained candidate network structure, and negatively correlated with the error, the delay, and the power consumption of the processor of the trained candidate network structure. Alternatively, the feedback value may also be positively correlated with the number of search operations that have been completed. The generated feedback information may be fed back to the controller, and the controller updates and searches out a new candidate network structure based on the feedback information. In this way, the performance of the candidate network searched in the current iteration can be propagated back to the controller after training, and the controller performs iterative update with the optimized feedback information (maximized feedback value) as a target and searches out a new candidate network structure. In this way, by performing the search operation a number of iterations, candidate network structures that achieve better performance may be continually searched.
And 204, in response to the fact that the feedback information generated based on the performance information of the candidate network structure which is trained currently reaches the preset convergence condition, generating a neural network model structure for the YUV image data based on the candidate network structure which is trained currently.
If the searched candidate network structure is trained in the current iteration, and the feedback information generated according to the performance information of the candidate network structure reaches a preset condition, for example, the feedback value reaches a preset threshold value, the operation of searching the network structure by iteration can be stopped, the candidate network structure searched in the current iteration is taken as the finally searched network structure, and the candidate network structure trained in the current iteration can be taken as the neural network model structure for the YUV image data, or the candidate network structure trained in the current iteration is continuously trained to generate the neural network model structure for the YUV image data.
If the feedback information generated based on the performance information of the candidate network structure that is currently trained does not reach the preset convergence condition, for example, the feedback value does not reach the preset threshold value, the controller may be updated based on the feedback value, and the next search operation may be performed.
The above-described embodiment searches the candidate network structure from the preset network structure search space by using the controller, then YUV image training data is obtained, the candidate network structure is trained by utilizing the YUV image training data, then performance information of the trained candidate network structure is obtained, generating feedback information according to the trained performance information of the candidate network structure, feeding the feedback information back to the controller, the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information, generates a neural network model structure for YUV image data based on the candidate network structure finished by the current training in response to determining that the feedback information generated based on the performance information of the candidate network structure finished by the current training reaches a preset convergence condition, and realizes automatic search of the neural network model structure suitable for processing the YUV image data. In addition, the controller is updated through performance information of the candidate network structure trained by the YUV image training data in the searching process, so that the optimization efficiency of the controller can be accelerated, the searching speed of the model structure is increased, and the processing capacity of the searched network structure on YUV data is ensured.
In some image processing scenarios, the image data acquired by the acquisition device is data in YUV format, for example, YUV image data is acquired by a video application at the mobile phone end. Image data in YUV format is generally converted into RGB format for storage when storing an image. The existing neural network model for processing image data is designed for data in RGB format, and because the physical meanings of the data in RGB format and the data in YUV format are greatly different, the effect of directly applying the neural network model designed for the data in RGB format to the processing of the data in YUV format is poor. After the neural network model structure suitable for the YUV image data is searched out, the acquired image data in the YUV format can be processed directly by using the searched neural network model structure suitable for the YUV image data without converting the encoding format of the acquired image. Therefore, when the method is applied to online image processing, the time delay overhead caused by converting YUV data into RGB data can be saved.
With continued reference to fig. 3, a flow diagram of another embodiment of a structure search method of a neural network model in accordance with the present disclosure is shown. As shown in fig. 3, the process 300 of the structure searching method of the neural network model of the present embodiment includes the following steps:
In this embodiment, the preset network structure search space may include candidate sizes of the image. The candidate size represents the size of YUV image data which can be processed by the searched candidate network structure.
On the basis of step 201 in the foregoing method flow 200, in each search operation, the controller of the present embodiment searches for an image size corresponding to the candidate network structure searched in the current iteration, in addition to performing an operation of searching for the candidate network structure. Specifically, a sequence code generated by the controller and a decoding rule of the network structure and the image size may be predefined, and the candidate network structure and the corresponding image size may be obtained by decoding the sequence code generated by the controller according to the rule. For example, one of the bits (e.g., the first or last bit) of the sequence code generated by the controller may be decoded to the picture size and the other bits of the sequence code decoded to the corresponding candidate network structure.
Step 302 of this embodiment may be implemented with reference to the specific implementation described in step 202 of the foregoing embodiment.
Further, YUV image training data may be acquired as follows: and converting the acquired RGB image sample data into YUV image sample data, and converting the image size of the YUV image sample data into the image size corresponding to the candidate network structure determined from the preset network structure search space.
In each iteration, the image size corresponding to the candidate network structure may be searched. When the candidate network structure is trained, the obtained YUV image training data may be converted to the corresponding image size and then input to the candidate network structure for training. Therefore, the YUV image training data is subjected to size conversion, so that the size of the YUV image training data is normalized to be the same, the efficiency of batch processing of the training data by the candidate network structure can be improved, and the training efficiency of the candidate network structure is improved.
Or, a network layer for converting the image size may be added to the candidate network structure, and after the obtained YUV image training data is input to the candidate network structure, the obtained YUV image training data is firstly converted to the image size searched in step 301 through the network layer, and then the image size searched in step 301 is processed through the candidate network structure searched in step 301.
And 303, acquiring performance information of the trained candidate network structure, generating feedback information according to the performance information of the trained candidate network structure, and feeding the feedback information back to the controller, so that the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information.
And 304, in response to determining that the feedback information generated based on the performance information of the candidate network structure finished by the current training reaches a preset convergence condition, generating a neural network model structure for the YUV image data based on the candidate network structure finished by the current training.
Step 303 and step 304 of this embodiment are respectively consistent with step 203 and step 204 of the foregoing embodiment, and specific implementation manners of step 303 and step 304 may refer to descriptions of step 203 and step 204 in the foregoing embodiment, which is not described herein again.
Different model structures have different image sizes suitable for processing, and the embodiment can search the corresponding image size in the process of searching the neural network model suitable for the YUV data by searching the image size corresponding to the candidate network structure in the searching operation, thereby further optimizing the searching result of the neural network model structure.
Optionally, the method flow 300 may further include:
And if the current feedback information reaches a preset convergence condition, stopping updating the controller, wherein the image size output by the controller is the target image size corresponding to the neural network model structure for the YUV image data. Therefore, through the searching of the neural network model structure, the image size suitable for processing of the searched neural network structure model is determined, and data can be processed based on the image size when the searched neural network model structure is applied, so that the performance of processing the image by the searched neural network model structure is improved.
Further, the method flow 300 may further include: the method comprises the steps of obtaining YUV image data to be processed, converting the size of the YUV image data to be processed into a target image size, and inputting the target image size into a neural network model structure for the YUV image data to be processed.
Here, the YUV image data to be processed may be acquired from an on-line image processing application. In an actual scene, after a user submits an image acquisition request in the image processing application, an equipment end running the image processing application acquires YUV image data as to-be-processed YUV image data through image acquisition equipment. The device side running the image processing application may run the neural network model structure for YUV image data to process the YUV image to be processed, or the YUV image data to be processed may be transmitted to a device (e.g., a server of the image processing application) running the neural network model structure for YUV image data to be processed.
Before processing the YUV image to be processed based on the above neural network model structure for YUV image data, the YUV image to be processed may be converted to a target image size. That is, the size of the YUV data to be processed is converted according to the image size suitable for processing of the searched neural network model structure, so that the processing effect of the YUV image data to be processed can be improved.
Referring to fig. 4, as an implementation of the structure searching method for the neural network model, the present disclosure provides an embodiment of a structure searching apparatus for a neural network model, where the apparatus embodiment corresponds to the method embodiments shown in fig. 2 and fig. 3, and the apparatus may be applied to various electronic devices.
As shown in fig. 4, the structure search apparatus 400 of the neural network model of the present embodiment includes a first search unit 401, a training unit 402, a feedback unit 403, and a generation unit 404. Wherein the first searching unit 401 is configured to search out a candidate network structure from a preset network structure searching space by using a controller; the training unit 402 is configured to obtain YUV image training data, train a candidate network structure using the YUV image training data; the feedback unit 403 is configured to obtain performance information of the trained candidate network structure, generate feedback information according to the performance information of the trained candidate network structure, and feed the feedback information back to the controller, so that the controller searches for a new candidate network structure from a preset network structure search space after updating based on the feedback information; the generating unit 404 is configured to generate a neural network model structure for the YUV image data based on the currently trained candidate network structure in response to determining that the feedback information generated based on the performance information of the currently trained candidate network structure reaches a preset convergence condition.
In some embodiments, the preset network structure search space includes candidate sizes of the image; and the above apparatus further comprises: and the second searching unit is configured to determine the image size corresponding to the candidate network structure from the candidate sizes of the images by adopting the controller.
In some embodiments, the generating unit 404 is further configured to determine the image size corresponding to the currently determined candidate network structure as the target image size corresponding to the neural network model structure for the YUV image data in response to determining that the feedback information generated based on the performance information of the currently trained candidate network structure reaches a preset convergence condition.
In some embodiments, the above apparatus further comprises: a processing unit configured to: acquiring YUV image data to be processed; and converting the size of the YUV image data to be processed into the size of a target image, and inputting the size of the target image into a neural network model structure for processing the YUV image data.
In some embodiments, the training unit 402 is configured to obtain YUV image training data as follows: and converting the acquired RGB image sample data into YUV image sample data, and converting the image size of the YUV image sample data into the image size corresponding to the candidate network structure determined from the preset network structure search space.
The units in the apparatus 400 described above correspond to the steps in the method described with reference to fig. 2 and 3. Thus, the operations, features and technical effects described above for the structure search method of the neural network model are also applicable to the apparatus 400 and the units included therein, and are not described herein again.
Referring now to FIG. 5, a schematic diagram of an electronic device (e.g., the server shown in FIG. 1) 500 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; a storage device 508 including, for example, a hard disk; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 5 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program, when executed by the processing device 501, performs the above-described functions defined in the methods of embodiments of the present disclosure. It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: searching a candidate network structure from a preset network structure searching space by adopting a controller; acquiring YUV image training data, and training a candidate network structure by using the YUV image training data; acquiring performance information of the trained candidate network structure, generating feedback information according to the performance information of the trained candidate network structure, and feeding the feedback information back to the controller, so that the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information; and in response to determining that the feedback information generated based on the performance information of the candidate network structure finished with the current training reaches a preset convergence condition, generating a neural network model structure for the YUV image data based on the candidate network structure finished with the current training.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a first search unit, a training unit, a feedback unit, and a generation unit. Where the names of the units do not in some cases constitute a limitation on the units themselves, for example, the first searching unit may also be described as a "unit that searches out a candidate network structure from a preset network structure search space using the controller".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the inventive concept as defined above. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (12)
1. A structure search method of a neural network model comprises the following steps:
searching a candidate network structure from a preset network structure searching space by adopting a controller;
acquiring YUV image training data, and training the candidate network structure by using the YUV image training data;
acquiring performance information of a trained candidate network structure, generating feedback information according to the performance information of the trained candidate network structure, and feeding the feedback information back to the controller, so that the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information;
and in response to determining that the feedback information generated based on the performance information of the candidate network structure finished with the current training reaches a preset convergence condition, generating a neural network model structure for the YUV image data based on the candidate network structure finished with the current training.
2. The method of claim 1, wherein the preset network structure search space comprises candidate sizes of images; and
the method further comprises the following steps:
and determining the image size corresponding to the candidate network structure from the candidate sizes of the image by adopting a controller.
3. The method of claim 2, wherein the method further comprises:
and determining the image size corresponding to the candidate network structure which is determined currently as the target image size corresponding to the neural network model structure for the YUV image data in response to the fact that the feedback information generated based on the performance information of the candidate network structure which is trained currently reaches the preset convergence condition.
4. The method of claim 3, wherein the method further comprises:
acquiring YUV image data to be processed;
and converting the size of the YUV image data to be processed into the size of the target image, and inputting the size of the target image into the neural network model structure for processing.
5. The method of any of claims 2-4, wherein the obtaining YUV image training data comprises:
and converting the acquired RGB image sample data into YUV image sample data, and converting the image size of the YUV image sample data into the image size corresponding to the candidate network structure determined from the preset network structure search space.
6. A structure search apparatus of a neural network model, comprising:
a first searching unit configured to search out a candidate network structure from a preset network structure searching space by using a controller;
a training unit configured to obtain YUV image training data, train the candidate network structure using the YUV image training data;
the feedback unit is configured to acquire performance information of a trained candidate network structure, generate feedback information according to the performance information of the trained candidate network structure, and feed the feedback information back to the controller, so that the controller searches a new candidate network structure from a preset network structure search space after updating based on the feedback information;
a generating unit configured to generate a neural network model structure for the YUV image data based on the currently trained candidate network structure in response to determining that feedback information generated based on the performance information of the currently trained candidate network structure reaches a preset convergence condition.
7. The apparatus of claim 6, wherein the preset network structure search space comprises candidate sizes of images; and
the device further comprises:
and the second searching unit is configured to determine the image size corresponding to the candidate network structure from the candidate sizes of the images by adopting the controller.
8. The apparatus according to claim 7, wherein the generating unit is further configured to determine an image size corresponding to the candidate network structure currently determined as a target image size corresponding to the neural network model structure for YUV image data in response to determining that feedback information generated based on performance information of the candidate network structure currently trained reaches a preset convergence condition.
9. The apparatus of claim 8, wherein the apparatus further comprises: a processing unit configured to:
acquiring YUV image data to be processed;
and converting the size of the YUV image data to be processed into the size of the target image, and inputting the size of the target image into the neural network model structure for processing.
10. The apparatus according to any of claims 7-9, wherein the training unit is configured to obtain YUV image training data as follows:
and converting the acquired RGB image sample data into YUV image sample data, and converting the image size of the YUV image sample data into the image size corresponding to the candidate network structure determined from the preset network structure search space.
11. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010116705.5A CN111353585B (en) | 2020-02-25 | 2020-02-25 | Structure searching method and device of neural network model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010116705.5A CN111353585B (en) | 2020-02-25 | 2020-02-25 | Structure searching method and device of neural network model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111353585A true CN111353585A (en) | 2020-06-30 |
| CN111353585B CN111353585B (en) | 2024-09-06 |
Family
ID=71197226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010116705.5A Active CN111353585B (en) | 2020-02-25 | 2020-02-25 | Structure searching method and device of neural network model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111353585B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111950727A (en) * | 2020-08-06 | 2020-11-17 | 中科智云科技有限公司 | Neural network training and testing method and device for image data |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015083199A1 (en) * | 2013-12-04 | 2015-06-11 | J Tech Solutions, Inc. | Computer device and method executed by the computer device |
| CN106651969A (en) * | 2016-12-28 | 2017-05-10 | 深圳市捷顺科技实业股份有限公司 | Color identification method and apparatus for vehicle |
| CN107122826A (en) * | 2017-05-08 | 2017-09-01 | 京东方科技集团股份有限公司 | Processing method and system and storage medium for convolutional neural networks |
| CA3077830A1 (en) * | 2016-12-05 | 2018-06-05 | Avigilon Coporation | System and method for appearance search |
| CN108364270A (en) * | 2018-05-22 | 2018-08-03 | 北京理工大学 | Colour cast color of image restoring method and device |
| CN110347868A (en) * | 2018-04-02 | 2019-10-18 | 庞德5公司 | Method and system for picture search |
| CN110543944A (en) * | 2019-09-11 | 2019-12-06 | 北京百度网讯科技有限公司 | neural network structure searching method, apparatus, electronic device, and medium |
| CN110766142A (en) * | 2019-10-30 | 2020-02-07 | 北京百度网讯科技有限公司 | Model generation method and device |
| CN110807515A (en) * | 2019-10-30 | 2020-02-18 | 北京百度网讯科技有限公司 | Model generation method and device |
-
2020
- 2020-02-25 CN CN202010116705.5A patent/CN111353585B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015083199A1 (en) * | 2013-12-04 | 2015-06-11 | J Tech Solutions, Inc. | Computer device and method executed by the computer device |
| CA3077830A1 (en) * | 2016-12-05 | 2018-06-05 | Avigilon Coporation | System and method for appearance search |
| CN106651969A (en) * | 2016-12-28 | 2017-05-10 | 深圳市捷顺科技实业股份有限公司 | Color identification method and apparatus for vehicle |
| CN107122826A (en) * | 2017-05-08 | 2017-09-01 | 京东方科技集团股份有限公司 | Processing method and system and storage medium for convolutional neural networks |
| WO2018205676A1 (en) * | 2017-05-08 | 2018-11-15 | 京东方科技集团股份有限公司 | Processing method and system for convolutional neural network, and storage medium |
| CN110347868A (en) * | 2018-04-02 | 2019-10-18 | 庞德5公司 | Method and system for picture search |
| CN108364270A (en) * | 2018-05-22 | 2018-08-03 | 北京理工大学 | Colour cast color of image restoring method and device |
| CN110543944A (en) * | 2019-09-11 | 2019-12-06 | 北京百度网讯科技有限公司 | neural network structure searching method, apparatus, electronic device, and medium |
| CN110766142A (en) * | 2019-10-30 | 2020-02-07 | 北京百度网讯科技有限公司 | Model generation method and device |
| CN110807515A (en) * | 2019-10-30 | 2020-02-18 | 北京百度网讯科技有限公司 | Model generation method and device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111950727A (en) * | 2020-08-06 | 2020-11-17 | 中科智云科技有限公司 | Neural network training and testing method and device for image data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111353585B (en) | 2024-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102342604B1 (en) | Method and apparatus for generating neural network | |
| CN110807515B (en) | Model generation method and device | |
| CN110288049B (en) | Method and apparatus for generating image recognition model | |
| CN108520220B (en) | Model generation method and device | |
| JP7208952B2 (en) | Method and apparatus for generating interaction models | |
| CN110766142A (en) | Model generation method and device | |
| CN111340220B (en) | Method and apparatus for training predictive models | |
| CN111340221B (en) | Neural network structure sampling method and device | |
| CN111368973B (en) | Method and apparatus for training a super network | |
| CN111738010A (en) | Method and apparatus for generating semantic matching model | |
| CN111353601B (en) | Method and apparatus for predicting latency of model structure | |
| CN110009101B (en) | Method and apparatus for generating a quantized neural network | |
| CN111539287B (en) | Method and device for training face image generation model | |
| CN117290477A (en) | Generating type building knowledge question-answering method based on secondary retrieval enhancement | |
| CN111278085B (en) | Method and device for acquiring target network | |
| CN109598344B (en) | Model generation method and device | |
| CN110782016A (en) | Method and apparatus for optimizing neural network architecture search | |
| CN111008213A (en) | Method and apparatus for generating language conversion model | |
| CN111353585B (en) | Structure searching method and device of neural network model | |
| CN110046670B (en) | Feature vector dimension reduction method and device | |
| CN110991661A (en) | Method and apparatus for generating a model | |
| CN115049730B (en) | Component mounting method, component mounting device, electronic apparatus, and storage medium | |
| CN113255819B (en) | Method and device for identifying information | |
| CN113361677B (en) | Quantification method and device for neural network model | |
| CN111680754B (en) | Image classification method, device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240807 Address after: Room 1201, building 1, Juhui garden, 108 Xuanwu Avenue, Xuanwu District, Nanjing, Jiangsu 210000 Applicant after: Nanjing Yufeng Video Technology Co.,Ltd. Country or region after: China Address before: 2 / F, baidu building, 10 Shangdi 10th Street, Haidian District, Beijing 100085 Applicant before: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |