Abstract
We build on the Deep Q-Learning Network (DQN) to solve the N-Queens problem to propose a solution to the Golomb Ruler problem, a popular example of a one dimensional constraint satisfaction problem. A comparison of the DQN approach with standard solution approaches to solve constraint satisfaction problems, such as backtracking and branch-and-bound, demonstrates the efficacy of the DQN approach, with significant computational savings as the order of the problem increases. The convergence behavior of the DQN model has been approximated using Locally Weighted Regression and Cybenko Approximation, demonstrating an improvement in the performance of the DQN with episodes, regardless of the order of the problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Babcock, W.C.: Intermodulation interference in radio systems frequency of occurrence and control by channel selection. Bell Syst J. 32(1), 63–73 (1953)
Bansal, S., Singh, A.K., Gupta, N.: Optimal golomb ruler sequences generation for optical WDM systems: a novel parallel hybrid multi-objective bat algorithm. J. Inst. Eng. India Ser. B 98(1), 43–64 (2017)
Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signal. Syst. 2(4), 303–314 (1989)
Drakakis, K.: A review of the available construction methods for Golomb rulers. Adv. Math. Commun. 3(3), 235 (2009)
Englert, P.: Locally weighted learning. In: Seminar Class on Autonomous Learning Systems (2012)
Kumar, V.: Algorithms for constraint-satisfaction problems: a survey. AI Mag. 13(1), 32–32 (1992)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)
Papavassiliou, V.A., Russell, S.: Convergence of reinforcement learning with general function approximators. In: IJCAI, pp. 748–757 (1999)
Polash, M.A., Newton, M.H., Sattar, A.: Constraint-based search for optimal Golomb rulers. J. Heur. 23(6), 501–532 (2017)
Potapov, A., Ali, M.: Convergence of reinforcement learning algorithms and acceleration of learning. Phys. Rev. E 67(2), 026706 (2003)
Prudhvi Raj, P., Shah, P., Suresh, P.: Faster convergence to N-queens problem using reinforcement learning. In: Saha, S., Nagaraj, N., Tripathi, S. (eds.) MMLA 2019. CCIS, vol. 1290, pp. 66–77. Springer, Singapore (2020). https://doi.org/10.1007/978-981-33-6463-9_6
Rivin, I., Zabih, R.: A dynamic programming solution to the N-Queens problem. Inf. Proces. Lett. 41(5), 253–256 (1992)
Robinson, J.p., Bernstein, A.: A class of binary recurrent codes with limited error propagation. IEEE Trans. Inf. Theory 13(1), 106–113 (1967)
Shearer, J.B.: Some new optimum golomb rulers. IEEE Transactions on Information Theory 36(1), 183–184 (1990)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992). https://doi.org/10.1007/BF00992698, https://doi.org/10.1007/BF00992698
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Prudhvi Raj, P., Saha, S., Srinivasa, G. (2021). Solving the N-Queens and Golomb Ruler Problems Using DQN and an Approximation of the Convergence. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_63
Download citation
DOI: https://doi.org/10.1007/978-3-030-92310-5_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92309-9
Online ISBN: 978-3-030-92310-5
eBook Packages: Computer ScienceComputer Science (R0)