I.E. Livieris, D.G. Sotiropoulos and P. Pintelas. On descent spectral CG algorithms for training recurrent neural networks. In Proceedings of 13th Panellenic Conference on Informatics (PCI’09), Corfu, pp. 65-69, 2009.
Abstract: In this paper, we evaluate the performance of descent conjugate gradient methods and we propose a new algorithm for training recurrent neural networks. The presented algorithm preserves the advantages of classical conjugate gradient methods while simultaneously avoids the usually inefficient
restarts. Simulation results are also presented using three different recurrent neural network architectures in a variety of benchmarks.
restarts. Simulation results are also presented using three different recurrent neural network architectures in a variety of benchmarks.