On descent spectral CG algorithms for training recurrent neural networks Print

I.E. Livieris D.G. Sotiropoulos and P. Pintelas, On Descent Spectral CG Algorithms for Training Recurrent Neural Networks. In Proceedings of 13th Panhellenic Conference on Informatics (PCI 2009), 2009.


Abstract - In this paper, we evaluate the performance of descent conjugate gradient methods and we propose a new algorithm for training recurrent neural networks. The presented algorithm preserves the advantages of classical conjugate gradient methods while simultaneously avoids the usually inefficient restarts. Simulation results are also presented using three different recurrent neural network architectures in a variety of benchmarks.