M.S. Apostolopoulou, D.G. Sotiropoulos, I.E. Livieris and P. Pintelas. A memoryless BFGS neural network training algorithm. In Proceedings of 7th IEEE International Conference on Industrial Informatics (INDIN’09), Cardiff, U.K., pp. 216-221, 2009.
Abstract: We present a new curvilinear algorithmic model for training neural networks which is based on a modifications of the memoryless BFGS method that incorporates a curvilinear search. The proposed model exploits the nonconvexity of the error surface based on information provided by the eigensystem of memoryless BFGS matrices using a pair of directions; a memoryless quasi-Newton direction and a direction of negative curvature. In addition, the computation of the negative curvature direction is accomplished
by avoiding any storage and matrix factorization. Simulations results verify that the proposed modification significantly improves the efficiency of the training process.
by avoiding any storage and matrix factorization. Simulations results verify that the proposed modification significantly improves the efficiency of the training process.