A Variable Memory Quasi-Newton Training Algorithm

摘要

A new neural network training algorithm which optimises performance in relation to the available memory is described. Numerically it has equivalent properties to Full Memory BFGS optimisation (FM) when there are no restrictions on memory and to FM with periodic reset when memory is limited. Achievable performance is determined by the ratio between available memory and problem size and accordingly varies between that of the full and memory-less versions of the BFGS algorithm.