Self-adaptation of learning rate in XCS working in noisy and dynamic environments

摘要

An extended classifier system (XCS) is an adaptive rule-based technique that uses evolutionary search and reinforcement learning to evolve complete, accurate, and maximally general payoff map of an environment. The payoff map is represented by a set of condition-action rules called classifiers. Despite this insight, till now parameter-setting problem associated with LCS/XCS has important drawbacks. Moreover, the optimal values of some parameters are strongly influenced by properties of the environment like its complexity, changeability, and the level of noise. The aim of this paper is to overcome some of these difficulties by a self-adaptation of a learning rate parameter, which plays a key role in reinforcement learning, since it is used for updates of classifier parameters: prediction, prediction error, fitness, and action set estimation. Self-adaptive control of prediction learning rate is investigated in the XCS, whereas the fitness and error learning rates remain fixed. Simultaneous self-adaptation of prediction learning rate and mutation rate also undergo experiments. Self-adaptive XCS solves one-step problems in noisy and dynamic environments.