Abstract
The setting hyperparameters in the support vector machine (SVM) is very important with regard to its accuracy and efficiency. In this paper, we employ a novel definition of the reinforcement learning state, actions and reward function that allows a deep Q-network (DQN) to learn to control the optimization hyperparameters for the SVM deep neural networks by supervised Big-Data. In this framework, the DQN algorithm with experience replay is based on the off-policy reinforcement learning for the expected discounted return of rewards, or q-values, connected to the actions of adjusting the hyperprameters in the SVM. We propose the two deep neural networks, one with the SVM and the other with Q-network (DQN). The SVM deep neural networks learns a policy for the optimization hyperparameters, but differ in the number of allowed actions. The SVM deep neural networks trains the hyperparameters of the SVM simultaneously such as the Lagrangian multiplier. The proposed algorithm is called a Hybrid DQN combined with SVM deep neural networks. This algorithm could be considered as the classifier in the real-world domains such as network anomalies in the distributed server loads, because the SVM is suitable for the application in a classification, especially for the one-against-the others. Algorithm comparisons show that our proposed algorithm leads to good optimization of the Lagrangian multiplier and can prevent overfitting to a certain extent automatically without human system designers. In terms of the classification performance of the proposed algorithm can be compared to the original LIBSVM with no controls of the hyperparameters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machine. Mach Learn 46:389–422
Chao C-F, Horng M-H (2015) The construction of support vector machine classifier using the firefly algorithm. Comput Intell Neurosci Arch 2015(2)
Sutton RS (1988) Learning to predict by the methods of temporal differences. Mach Learn 3(1):9–44
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 1. MIT press Cambridge
Sutton RS, McAllester DA, Singh SP, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation. In: NIPS, vol 99, pp 1057–1063 (1999)
Sutton RS (1984) Temporal credit assignment in reinforcement learning. Doctoral Dissertation
Tieleman T, Hinton G (2012) Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw Mach Learn 4(2)
Silver D, Lever G, Heess N (2014) Deterministic policy gradient algorithms. In: ICML’14 proceedings of the 31st international conference on international conference on machine learning vol 32, pp 387–395
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Driessche GVD, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M (2016) Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489
Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. In: NIPS
Hansen S (2016) Using deep q-learning to control optimization hyperparameters. ArXiv
Tavallaee M, Bagheri E, Lu W, Ghorbani AA (2009) A detailed analysis of the KDD CUP 99 dataset. In: Proceedings of the 2009 IEEE symposium on computational intelligence in security and defense applications (CISDA 2009), pp 53–58
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and Future Planning (No. 2016RIA2B4012386).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kim, C., Kim, Hy. (2019). A Hybrid Deep Q-Network for the SVM Lagrangian. In: Kim, K., Baek, N. (eds) Information Science and Applications 2018. ICISA 2018. Lecture Notes in Electrical Engineering, vol 514. Springer, Singapore. https://doi.org/10.1007/978-981-13-1056-0_63
Download citation
DOI: https://doi.org/10.1007/978-981-13-1056-0_63
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1055-3
Online ISBN: 978-981-13-1056-0
eBook Packages: EngineeringEngineering (R0)