Dynamic Neural Architecture Search for Image Classification
Pages 1554 - 1562
Abstract
Neural architecture search (NAS) has proven to be effective at automating the design of neural network architectures and hyperparameters. The design produced by NAS is generated offline and remains the same throughout the learning process of the neural network. We refer to this as static neural architecture search (SNAS). We hypothesis that changing the design and design options on different epochs of the learning process will be more effective than using the same design throughout the process. This study investigates this hypothesis by introducing the concept of dynamic neural architecture search (DNAS). This research forms part of a larger initiative investigating changing designs in real-time based on feedback on the progression of the search in the space using the design. However, this study focuses on just the first aspect of changing the designs in real-time which we refer to as dynamic neural architecture search. A genetic algorithm is used in both SNAS and DNAS to generate designs comprising the architecture and hyperparameters. The performance of SNAS and DNAS are evaluated on the MNIST, CIFAR-10, CIFAR-100, Celebrity Faces, Movie Success, Mosquito, Melanoma and FruitsGB datasets. For all datasets DNAS was found to produce better accuracies than SNAS with a minimal increase in runtimes at the 95% level of confidence. Furthermore, the results were found to be comparative to state of the art approaches with DNAS producing an improvement over the best know results for the Melanoma dataset.
References
[1]
Mohd Mohsin Ali, Manish Raj, and Deepika Vatsa. 2023. FruizNet Using an Efficient Convolutional Neural Network. In 2023 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE). IEEE, 287--292.
[2]
Hussain Alibrahim and Simone A Ludwig. 2021. Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In 2021 IEEE Congress on Evolutionary Computation (CEC). IEEE, 1551--1559.
[3]
Enes Ayan. 2023. Genetic Algorithm-Based Hyperparameter Optimization for Convolutional Neural Networks in the Classification of Crop Pests. Arabian Journal for Science and Engineering (2023), 1--15.
[4]
Edmund K Burke, Michel Gendreau, Matthew Hyde, Graham Kendall, Barry McCollum, Gabriela Ochoa, Andrew J Parkes, and Sanja Petrovic. 2011. The cross-domain heuristic search challenge-an international research competition. In Learning and Intelligent Optimization: 5th International Conference, LION 5, Rome, Italy, January 17--21, 2011. Selected Papers 5. Springer, 631--634.
[5]
Adam Byerly, Tatiana Kalganova, and Ian Dear. 2021. No routing needed between capsules. Neurocomputing 463 (2021), 545--553.
[6]
DanB. 2017. 5 Celebrity Faces Dataset. https://www.kaggle.com/datasets/dansbecker/5-celebrity-faces-dataset.
[7]
Tom Den Ottelander, Arkadiy Dushatskiy, Marco Virgolin, and Peter A. N. Bosman. 2021. Local Search is a Remarkably Strong Baseline for Neural Architecture Search. In Evolutionary Multi-Criterion Optimization, Hisao Ishibuchi, Qingfu Zhang, Ran Cheng, Ke Li, Hui Li, Handing Wang, and Aimin Zhou (Eds.). Springer International Publishing, Cham, 465--479.
[8]
Shuchao Deng, Yanan Sun, and Edgar Galvan. 2022. Neural architecture search using genetic algorithm for facial expression recognition. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. 423--426.
[9]
Qi Ding and Navid Razmjooy. 2023. An optimal diagnosis system for melanoma dermoscopy images based on enhanced design of horse herd optimizer. International Journal of Imaging Systems and Technology 33, 3 (2023), 1092--1107.
[10]
Nick DiSanto. 2023. ISIC Melanoma Dataset.
[11]
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural architecture search: a survey. J. Mach. Learn. Res. 20, 1 (jan 2019), 1997--2017.
[12]
Bilge S Akkoca Gazioğlu and Mustafa E Kamaşak. 2021. Effects of objects and image quality on melanoma classification using deep neural networks. Biomedical Signal Processing and Control 67 (2021), 102530.
[13]
Mia Gerber. 2021. Automated Design of the Deep Neural Network Pipeline. https://www.cs.up.ac.za/cs/nicog/ThPane.htm. [Online; accessed 25-November-2022].
[14]
Priya Goyal, Quentin Duval, Isaac Seessel, Mathilde Caron, Ishan Misra, Levent Sagun, Armand Joulin, and Piotr Bojanowski. 2022. Vision models are more robust and fair when pretrained on uncurated images without supervision. arXiv preprint arXiv:2202.08360 (2022).
[15]
Ahmed Hassan and Nelishia Pillay. 2021. Dynamic Heuristic Set Selection for Cross-Domain Selection Hyper-heuristics. In Theory and Practice of Natural Computing: 10th International Conference, TPNC 2021, Tsukuba, Japan, December 7--10, 2021, Proceedings 10. Springer, 33--44.
[16]
Naeem Iqbal, Rashid Ahmad, Faisal Jamil, and Do-Hyeun Kim. 2021. Hybrid features prediction model of movie quality using Multi-machine learning techniques for effective business resource planning. Journal of Intelligent & Fuzzy Systems 40, 5 (2021), 9361--9382.
[17]
Naeem Iqbal, Rashid Ahmad, Faisal Jamil, and Do-Hyeun Kim. 2021. Hybrid features prediction model of movie quality using Multi-machine learning techniques for effective business resource planning. Journal of Intelligent & Fuzzy Systems 40, 5 (2021), 9361--9382.
[18]
Yesmina Jaafra, Jean Luc Laurent, Aline Deruyver, and Mohamed Saber Naceur. 2019. Reinforcement learning for neural architecture search: A review. Image and Vision Computing 89 (2019), 57--66.
[19]
Xu Ji, Joao F Henriques, and Andrea Vedaldi. 2019. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9865--9874.
[20]
Randa Kamal, Ezz El-Din Hemdan, and Nawal El-Fishway. 2023. An efficient security system based on cancelable face recognition with blockchain over cognitive IoT. Multimedia Tools and Applications (2023), 1--21.
[21]
Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009).
[22]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (Lake Tahoe, Nevada) (NIPS'12). Curran Associates Inc., Red Hook, NY, USA, 1097--1105.
[23]
KB Shibu Kumar and Philip Samuel. 2023. An Efficient Method to Accurately Cluster Large Number of High Dimensional Facial Images. IEEE Access (2023).
[24]
George Kyriakides and Konstantinos Margaritis. 2020. The effect of reduced training in neural architecture search. Neural Computing and Applications 32, 23 (2020), 17321--17332.
[25]
Yann LeCun. 1998. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/ (1998).
[26]
Jiayu Liang, Hanqi Cao, Yaxin Lu, and Mingming Su. 2024. Architecture search of accurate and lightweight CNNs using genetic algorithm. Genetic Programming and Evolvable Machines 25, 1 (2024), 13.
[27]
Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, Kalyanmoy Deb, Erik Goodman, and Wolfgang Banzhaf. 2019. Nsga-net: neural architecture search using multi-objective genetic algorithm. In Proceedings of the genetic and evolutionary computation conference. 419--427.
[28]
Vishal Meshram, Koravat Thanomliang, Supawadee Ruangkan, Prawit Chumchu, and Kailas Patil. 2020. FruitsGB: Top Indian Fruits with quality.
[29]
Erfan Miahi, Seyed Abolghasem Mirroshandel, and Alexis Nasr. 2022. Genetic Neural Architecture Search for automatic assessment of human sperm images. Expert Systems with Applications 188 (2022), 115937.
[30]
Mustafa Misir, Katja Verbeeck, Patrick De Causmaecker, and Greet Vanden Berghe. 2010. Hyper-heuristics with a dynamic heuristic set for the home care scheduling problem. In IEEE Congress on Evolutionary Computation. IEEE, 1--8.
[31]
Md Jubaer Hossain Pantho, Pankaj Bhowmik, and Christophe Bobda. 2021. Towards an efficient cnn inference architecture enabling in-sensor processing. Sensors 21, 6 (2021), 1955.
[32]
Md Jubaer Hossain Pantho, Joel Mandebi Mbongue, Pankaj Bhowmik, and Christophe Bobda. 2022. Event camera simulator design for modeling attention-based inference architectures. Journal of Real-Time Image Processing 19, 2 (2022), 363--374.
[33]
Nelishia Pillay and Rong Qu. 2018. Hyper-Heuristics: Theory and Applications. Springer.
[34]
Reshma Pise, Mayawadee Aungmaneeporn, Kailas Patil, and Prawit Chumchu. 2020. Image Dataset of Aedes and Culex Mosquito Species.
[35]
Matt Poyser and Toby P Breckon. 2024. Neural architecture search: A contemporary literature review for computer vision applications. Pattern Recognition 147 (2024), 110052.
[36]
Kathiresan Shankar, Sachin Kumar, Ashit Kumar Dutta, Ahmed Alkhayyat, Anwar Ja'afar Mohamad Jawad, Ali Hashim Abbas, and Yousif K Yousif. 2022. An automated hyperparameter tuning recurrent neural network model for fruit classification. Mathematics 10, 13 (2022), 2358.
[37]
Iksoo Shin, Changsik Cho, and Seontae Kim. 2024. Method for expanding search space with hybrid operations in DynamicNAS. IEEE Access (2024).
[38]
Seok Bin Son, Soohyun Park, and Joongheon Kim. 2023. Entropy-Aware Similarity for Balanced Clustering: A Case Study with Melanoma Detection. IEEE Access (2023).
[39]
Le-Anh Tran and Dong-Chul Park. 2023. Feature Embedding Clustering using POCS-based Clustering Algorithm. arXiv preprint arXiv:2305.00001 (2023).
[40]
Lili Wang, Sunit Mistry, Abdulkadir Abdulahi Hasan, Abdiaziz Omar Hassan, Yousuf Islam, and Frimpong Atta Junior Osei. 2023. Implementation of a Collaborative Recommendation System Based on Multi-Clustering. Mathematics 11, 6 (2023), 1346.
[41]
Long Wen, Liang Gao, Xinyu Li, and Hui Li. 2022. A new genetic algorithm based evolutionary neural architecture search for image classification. Swarm and Evolutionary Computation 75 (2022), 101191.
[42]
Long Wen, Liang Gao, Xinyu Li, and Hui Li. 2022. A new genetic algorithm based evolutionary neural architecture search for image classification. Swarm and Evolutionary Computation 75 (2022), 101191.
[43]
Colin White, Willie Neiswanger, and Yash Savani. 2020. BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search. arXiv:1910.11858 [cs.LG]
Index Terms
- Dynamic Neural Architecture Search for Image Classification
Recommendations
VNAS: Variational Neural Architecture Search
AbstractDifferentiable neural architecture search delivers point estimation to the optimal architecture, which yields arbitrarily high confidence to the learned architecture. This approach thus suffers in calibration and robustness, in contrast with the ...
A Training-free Genetic Neural Architecture Search
ACM ICEA '21: Proceedings of the 2021 ACM International Conference on Intelligent Computing and its Emerging ApplicationsThe so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical ...
Graph neural architecture prediction
AbstractGraph neural networks (GNNs) have shown their superiority in the modeling of graph data. Recently, increasing attention has been paid to automatic graph neural architecture search, aiming to overcome the shortcomings of manually constructing GNN ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
2187 pages
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- NRF
- MultiChoice Joint Chair in Machine Learning
Conference
GECCO '24 Companion
Sponsor:
GECCO '24 Companion: Genetic and Evolutionary Computation Conference Companion
July 14 - 18, 2024
VIC, Melbourne, Australia
Acceptance Rates
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 22Total Downloads
- Downloads (Last 12 months)22
- Downloads (Last 6 weeks)9
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in