TextNet: An Neural Architecture Search Method based on Rapid Text Processing Network Structure Analysis
Authors: 
Siyuan Chen, Wei Fang, Shuwei ZhuAuthors Info & Claims
Pages 2087 - 2090
Abstract
Evolutionary algorithm (EA) is well-suited for solving neural architecture search (NAS) problems, however, most of existing multi-objective evolutionary NAS algorithms (MOEA-NAS) have limited their search space to a pre-defined collection of network modules, which could not effectively express the best direction for evolution. To overcome this barrier, TextNet is approached to automatically search any possible efficient network structure module by utilizes a fast network structure analysis method. A list of recorded neural network modules are generated each generation, of which dynamically evaluated efficiency values are passed to the search process. TextNet can effectively complement existing EA-Based NAS algorithms, and three tested representative EMO algorithms show that our methods could empower other EMO algorithms to gain better performance. An open-source implementation is available.
References
[1]
Han Cai, Ligeng Zhu, and Song Han. 2019. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In Proc. Int. Conf. Learn. Represent. (ICLR).
[2]
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution. In Proc. Int. Conf. Learn. Represent. (ICLR).
[3]
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural Architecture Search: A Survey. The Journal of Machine Learning Research 20, 1 (2019), 1997--2017.
[4]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR).
[5]
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. 2017. Densely Connected Convolutional Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Honolulu, HI). IEEE, 2261--2269.
[6]
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, and Wei Hua. 2018. Progressive Neural Architecture Search. In Proceedings of the European Conference on Computer Vision (ECCV). 19--34. https://openaccess.thecvf.com/content_ECCV_2018/html/Chenxi_Liu_Progressive_Neural_Architecture_ECCV_2018_paper.html
[7]
Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. In Proc. Int. Conf. Learn. Represent. (ICLR).
[8]
Zhichao Lu, Ran Cheng, Yaochu Jin, Kay Chen Tan, and Kalyanmoy Deb. 2022. Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment. IEEE Transactions on Evolutionary Computation (2022), 1--1.
[9]
Zhichao Lu, Kalyanmoy Deb, Erik Goodman, Wolfgang Banzhaf, and Vishnu Naresh Boddeti. 2020. NSGANetV2: Evolutionary Multi-Objective SurrogateAssisted Neural Architecture Search. http://arxiv.org/abs/2007.10396
[10]
Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, and Kalyanmoy Deb. 2019. NSGA-Net: Neural Architecture Search Using Multi-Objective Genetic Algorithm. In GECCO '19: Genetic and Evolutionary Computation Conference (Prague Czech Republic). ACM, 419--427.
[11]
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. 2018. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. In Computer Vision - ECCV 2018: 15th European Conference, Munich, Germany, September 8--14, 2018, Proceedings, Part XIV (Berlin, Heidelberg). Springer-Verlag, 122--138.
[12]
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. 2019. Regularized Evolution for Image Classifier Architecture Search. In Proc. AAAI Conf. Artif. Intell.
[13]
Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, and Yutaka Leon Suematsu. 2017. Large-Scale Evolution of Image Classifiers. In Proc. Int. Conf. Mach. Learn. (ICML). 2902--2911.
[14]
S. A. R. Shah, Wenji Wu, Qiming Lu, Liang Zhang, and Sajith Sasidharan. 2018. AmoebaNet: An SDN-enabled Network Service for Big Data Science. Journal of Network and Computer Applications 119 (2018), 70--82.
[15]
Masanori Suganuma, Shinichi Shirakawa, and Tomoharu Nagao. 2017. A Genetic Programming Approach to Designing Convolutional Neural Network Architectures. In Proc. Genet. Evol. Comput. Conf. (GECCO). 497--504.
[16]
Yanan Sun, Handing Wang, Bing Xue, Yaochu Jin, and Gary G. Yen. 2020. Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor. IEEE Transactions on Evolutionary Computation 24, 2 (2020), 350--364.
[17]
Yanan Sun, Bing Xue, Mengjie Zhang, Gary G. Yen, and Jiancheng Lv. 2020. Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification. IEEE Transactions on Cybernetics 50, 9 (2020), 3840--3854.
[18]
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, and Mark Sandler. 2019. Mnasnet: Platform-aware Neural Architecture Search for Mobile. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR). 2820--2828.
[19]
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. 2018. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (Salt Lake City, UT). IEEE, 6848--6856.
[20]
Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, and Cheng-Lin Liu. 2018. Practical Block-Wise Neural Network Architecture Generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2423--2432. https://openaccess.thecvf.com/content_cvpr_2018/html/Zhong_Practical_Block-Wise_Neural_CVPR_2018_paper.html
[21]
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. 2018. Learning Transferable Architectures for Scalable Image Recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR).
Index Terms
- TextNet: An Neural Architecture Search Method based on Rapid Text Processing Network Structure Analysis
Recommendations
Novelty driven evolutionary neural architecture search
GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionEvolutionary algorithms (EA) based neural architecture search (NAS) involves evaluating each architecture by training it from scratch, which is extremely time-consuming. This can be reduced by using a supernet for estimating the fitness of an ...
Fitness landscape analysis of graph neural network architecture search spaces
GECCO '21: Proceedings of the Genetic and Evolutionary Computation ConferenceNeural Architecture Search (NAS) is the name given to a set of methods designed to automatically configure the layout of neural networks. Their success on Convolutional Neural Networks inspired its use on optimizing other types of neural network ...
Pareto Local Search is Competitive with Evolutionary Algorithms for Multi-Objective Neural Architecture Search
GECCO '23: Proceedings of the Genetic and Evolutionary Computation ConferenceNeural architecture search (NAS) involves automatically searching for promising deep neural network structures in certain architecture spaces. Depending on the number of criteria being concerned, NAS can be formulated as single-objective optimization ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
2187 pages
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Natural Science Foundation of China
- Natural Science Foundation of Jiangsu Province, China
Conference
GECCO '24 Companion
Sponsor:
GECCO '24 Companion: Genetic and Evolutionary Computation Conference Companion
July 14 - 18, 2024
VIC, Melbourne, Australia
Acceptance Rates
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 29Total Downloads
- Downloads (Last 12 months)29
- Downloads (Last 6 weeks)2
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in