research-article

Neural Architecture Search with Synchronous Advantage Actor-Critic Methods and Partial Training

Authors:

George Kyriakides,

Konstantinos G. MargaritisAuthors Info & Claims

SETN '18: Proceedings of the 10th Hellenic Conference on Artificial Intelligence

Article No.: 34, Pages 1 - 7

https://doi.org/10.1145/3200947.3208068

Published: 09 July 2018 Publication History

Get Access

Abstract

In recent years, deep neural networks have enabled researchers to solve many difficult learning tasks, including natural language processing, image recognition and translation. Although powerful and flexible tool for many automation tasks, their design requires intensive human effort. Furthermore, recent studies suggest that the architecture itself can contribute more to the network's performance than its training. It is thus easy to see the necessity of automating the task of architecture design, as it will lead to even further improvements in the field. In this paper we implement a synchronous Advantage Actor-Critic Reinforcement Learning method in order to generate fully convolutional architectures, inspired by recent research in Neural Architecture Search and Reinforcement Learning. Furthermore, we explore the possibility of partially training the evaluated architectures, in order to assess the network's quality, greatly reducing the time required to evaluate them. Using Kendall's tau we show that a set of architectures, ordered by their performance retains its relative ranking when evaluated with partial training. Furthermore, the method outperforms random search, when partial training is used, as it finishes faster, produces better results and the overall probability of producing a high-quality architecture is higher.

References

[1]

Moez Baccouche, Franck Mamalet, Christian Wolf, Christophe Garcia, and Atilla Baskurt. 2011. Sequential deep learning for human action recognition. In International Workshop on Human Behavior Understanding. Springer, 29--39.

Digital Library

Google Scholar

[2]

Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. 2016. Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016).

Google Scholar

[3]

James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, Feb (2012), 281--305.

Digital Library

Google Scholar

[4]

Zezhou Cheng, Qingxiong Yang, and Bin Sheng. 2015. Deep colorization. In Proceedings of the IEEE International Conference on Computer Vision. 415--423.

Digital Library

Google Scholar

[5]

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).

Google Scholar

[6]

Kun He, Yan Wang, and John Hopcroft. 2016. A powerful generative model using random weights for the deep image representation. In Advances in Neural Information Processing Systems. 631--639.

Digital Library

Google Scholar

[7]

Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine 29, 6 (2012), 82--97.

Crossref

Google Scholar

[8]

Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat. {n. d.}. Evolving Deep Neural Networks. ({n. d.}). arXiv:cs.NE/1703.00548v2

Google Scholar

[9]

Volodymyr Mnih, AdriÃă PuigdomÃÍnech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. {n. d.}. Asynchronous Methods for Deep Reinforcement Learning. ({n. d.}). arXiv:cs.LG/1602.01783v2

Digital Library

Google Scholar

[10]

Renato Negrinho and Geoff Gordon. 2017. Deeparchitect: Automatically designing and training deep architectures. arXiv preprint arXiv:1704.08792 (2017).

Google Scholar

[11]

OpenAI. 2017. OpenAI Baselines: ACKTR & A2C. (Nov 2017). https://blog.openai.com/baselines-acktr-a2c/

Google Scholar

[12]

Kenneth O Stanley and Risto Miikkulainen. 2002. Evolving neural networks through augmenting topologies. Evolutionary computation 10, 2 (2002), 99--127.

Digital Library

Google Scholar

[13]

Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Reinforcement Learning. Springer, 5--32.

Google Scholar

[14]

Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016).

Google Scholar

Cited By

View all

Tian FXu YDu HGuo XLi XLi J(2022)A Learning-driven Method for Adaptive PID Control2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)10.1109/ITAIC54216.2022.9836627(2185-2189)Online publication date: 17-Jun-2022
https://doi.org/10.1109/ITAIC54216.2022.9836627
Al-Saadi MAl-Greer MShort M(2021)Strategies for Controlling Microgrid Networks with Energy Storage Systems: A ReviewEnergies10.3390/en1421723414:21(7234)Online publication date: 2-Nov-2021
https://doi.org/10.3390/en14217234
Nistor S(2021)An Actor-Critic Approach to Neural Network Architecture Search for Facial Expressions Recognition2021 IEEE 17th International Conference on Intelligent Computer Communication and Processing (ICCP)10.1109/ICCP53602.2021.9733673(171-178)Online publication date: 28-Oct-2021
https://doi.org/10.1109/ICCP53602.2021.9733673
Show More Cited By

Index Terms

Neural Architecture Search with Synchronous Advantage Actor-Critic Methods and Partial Training
1. Computing methodologies
  1. Distributed computing methodologies
  2. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
    2. Machine learning approaches
      1. Neural networks

Recommendations

Supervised Advantage Actor-Critic for Recommender Systems
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Casting session-based or sequential recommendation as reinforcement learning (RL) through reward signals is a promising research direction towards recommender systems (RS) that maximize cumulative profits. However, the direct use of RL algorithms in the ...
Towards automated neural design: an open source, distributed neural architecture research framework
PCI '18: Proceedings of the 22nd Pan-Hellenic Conference on Informatics

NORD (Neural Operations Research & Development) is an open source distributed deep learning architectural research framework, based on PyTorch, MPI and Horovod. It aims to make research of deep architectures easier for experts of different domains, in ...
A Training-free Genetic Neural Architecture Search
ACM ICEA '21: Proceedings of the 2021 ACM International Conference on Intelligent Computing and its Emerging Applications

The so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SETN '18: Proceedings of the 10th Hellenic Conference on Artificial Intelligence

July 2018

339 pages

ISBN:9781450364331

DOI:10.1145/3200947

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

EETN: Hellenic Artificial Intelligence Society
UOP: University of Patras
University of Thessaly: University of Thessaly, Volos, Greece

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SETN '18

SETN '18: 10th Hellenic Conference on Artificial Intelligence

July 9 - 12, 2018

Patras, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
200
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)4

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Tian FXu YDu HGuo XLi XLi J(2022)A Learning-driven Method for Adaptive PID Control2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)10.1109/ITAIC54216.2022.9836627(2185-2189)Online publication date: 17-Jun-2022
https://doi.org/10.1109/ITAIC54216.2022.9836627
Al-Saadi MAl-Greer MShort M(2021)Strategies for Controlling Microgrid Networks with Energy Storage Systems: A ReviewEnergies10.3390/en1421723414:21(7234)Online publication date: 2-Nov-2021
https://doi.org/10.3390/en14217234
Nistor S(2021)An Actor-Critic Approach to Neural Network Architecture Search for Facial Expressions Recognition2021 IEEE 17th International Conference on Intelligent Computer Communication and Processing (ICCP)10.1109/ICCP53602.2021.9733673(171-178)Online publication date: 28-Oct-2021
https://doi.org/10.1109/ICCP53602.2021.9733673
Hajewski JOliveira S(2021)Efficient Evolution of Variational Autoencoders2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC51732.2021.9376167(1541-1550)Online publication date: 27-Jan-2021
https://doi.org/10.1109/CCWC51732.2021.9376167
Hajewski JOliveira SXing X(2021)Distributed Evolution of Deep AutoencodersIntelligent Computing10.1007/978-3-030-80119-9_6(133-153)Online publication date: 13-Jul-2021
https://doi.org/10.1007/978-3-030-80119-9_6
Hajewski JOliveira S(2020)A Scalable System for Neural Architecture Search2020 10th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC47524.2020.9031181(0053-0060)Online publication date: Jan-2020
https://doi.org/10.1109/CCWC47524.2020.9031181
Kyriakides GMargaritis K(2019)Comparison of Neural Network Optimizers for Relative Ranking Retention Between Neural ArchitecturesArtificial Intelligence Applications and Innovations10.1007/978-3-030-19823-7_22(272-281)Online publication date: 12-May-2019
https://doi.org/10.1007/978-3-030-19823-7_22

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Supervised Advantage Actor-Critic for Recommender Systems

Towards automated neural design: an open source, distributed neural architecture research framework

A Training-free Genetic Neural Architecture Search