SCESN, SPESN, SWESN: Three recurrent neural echo state networks with clustered reservoirs for prediction of nonlinear and chaotic time series

摘要

Echo state networks (ESNs) with very simple and linear learning algorithm are a new approach to recurrent neural network training. Recently, these networks have aroused a lot of interest in their nonlinear dynamic system modeling capacities. In previous studies, the largest eigenvalue of the reservoir connectivity matrix (spectral radius) is used as a predictor for the stable network dynamics, but recent evidences show that in the presence of reservoir substructures like clusters, stability criteria in these kind of networks are altered. Some researchers have also demonstrated that network approximation ability in ESN networks is improved by the characteristics of small-world and scale-free. In this paper, we used three classic clustering algorithms called K-Means (C- Centeriod), partitioning Around Medoids (PAM) and ward algorithm, for clustering the internal neurons. After that, we refer to mean nodes in each cluster as backbone units and refer to the other neurons in a cluster as local neurons. Connections between neurons are such that the resulting networks have small-world topology and neurons in the new networks follow a power-law distribution. At first, we demonstrate that resulting clustered networks have some characteristics of biological neural system like power-law distribution, small-word feature, community structure, and distributed architecture. For investigating the prediction power and the range of spectral radius of resulting networks, we use new ESNs on the Mackey-Glass dynamic system and the laser time series prediction problem and compared the results with the previous works. Then we evaluate echo state property and performance of approximating highly complex nonlinear dynamic systems of proposed networks rather than previous approaches. Results show that the proposed methods outperform the previous ones in terms of prediction accuracy of chaotic time series.