Convergence of a Batch Gradient Algorithm with Adaptive Momentum for Neural Networks

1. Rumelhart DE, McClelland JL, The PDP Research Group: (1986) Parallel distributed processing-explorations in the microstructure of cognition. MIT Press, Cambridge

   Google Scholar 

2. Meybodi MR, Beigy H (2002) A note on learning automata-based schemes for adaptation of BP parameters. Neurocomputing 48: 957–974

   Article  MATH  Google Scholar 

3. Qiu G, Varley MR, Terrell TJ (1992) Accelerated training of backpropagation networks by using adaptive momentum step. IEE Electron Lett 28(4): 377–379

   Article  Google Scholar 

4. Istook E, Martinez T (2002) Improved backpropagation learning in nerural networks with windowed momentum. Int J Neural Syst 12(3–4): 303–318

   Article  Google Scholar 

5. Chan LW, Fallside F (1987) An adaptive training algorithm for backpropagation networks. Comput Speech Lang 2: 205–218

   Article  Google Scholar 

6. Yu X, Loh NK, Miller WC (1993) A new acceleration technique for the backpropagation algorithm. IEEE Int Conf Neural Netw 3: 1157–1161

   Article  Google Scholar 

7. Yu C, Liu B (2002) A backpropagation algorithm with adaptive learning rate and momentum coefficient. IEEE Int Conf Neural Netw 2: 1218–1223

   Google Scholar 

8. Shao HM, Zheng GF (2009) A new BP algorithm with adaptive momentum for FNNs training. WRI Glob Congr Intell Syst 4: 16–20

   Article  Google Scholar 

9. Bhaya A, Kaszkurewicz E (2004) Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method. Neural Netw 17: 65–71

   Article  MATH  Google Scholar 

10. Torii M, Hagan MT (2002) Stability of steepest descent with momentum for quadratic functions. IEEE Trans Neural Netw 13(3): 752–756

    Article  Google Scholar 

11. Zhang NM, Wu W, Zheng GF (2006) Convergence of gradient method with momentum for two-layer feedforward neural networks. IEEE Trans Neural Netw 17(2): 522–525

    Article  Google Scholar 

12. Wu W, Xu YS (2002) Deterministic convergence of an online gradient method for neural networks. J Comput Appl Math 144(1–2): 335–347

    Article  MathSciNet  MATH  Google Scholar 

13. Zhang C, Wu W, Xiong Y (2007) Convergence analysis of batch gradient algorithm for three classes of sigma-pi neural networks. Neural Process Lett 26: 177–189

    Article  MATH  Google Scholar 

14. Xu DP, Zhang HS, Liu LJ (2010) Convergence analysis of three classes of split-complex gradient algorithms for complex-valued recurrent neural networks. Neural Comput 22(10): 2655–2677

    Article  MathSciNet  MATH  Google Scholar 

15. Yuan YX, Sun WY (2001) Optimization theory and methods. Science Press, Beijing

    Google Scholar 

Download references