Hierarchical Correlations Replay for Continual Learning

Highlights：

• We present an Instance-level Correlation Replay (ICR) module to achieve instance-level replay, in which a correlation matrix is used to integrate information between inter-instance and instance itself, and consolidate old knowledge by constraining it during the rehearsal process.

• We develop a Class-level Correlation Replay (CCR) module for classification as well as alleviating the catastrophic forgetting by constructing the class-level correlation and constraining their consistency at different moments.

• Extensive experiments are conducted on three continual learning settings, namely Task Incremental Learning (Task-IL), Class Incremental Learning (Class-IL) and General Continual Learning (GCL). The results show that the HCR approach is quite competitive under diverse continual learning settings.

摘要

•We propose a Hierarchical Correlations Replay (HCR) method consisting of an instance-level correlation replay module and a class-level correlation replay module, which complements traditional experience replay by building and replaying the correlations of instance-level and class-level.•We present an Instance-level Correlation Replay (ICR) module to achieve instance-level replay, in which a correlation matrix is used to integrate information between inter-instance and instance itself, and consolidate old knowledge by constraining it during the rehearsal process.•We develop a Class-level Correlation Replay (CCR) module for classification as well as alleviating the catastrophic forgetting by constructing the class-level correlation and constraining their consistency at different moments.•Extensive experiments are conducted on three continual learning settings, namely Task Incremental Learning (Task-IL), Class Incremental Learning (Class-IL) and General Continual Learning (GCL). The results show that the HCR approach is quite competitive under diverse continual learning settings.