End-to-end kernel learning via generative random Fourier features

Highlights：

• In random features learning, this is the first one-stage method. It is different from the existing two-stage methods that first solve a kernel alignment problem and then a classification problem.

• Compared with two-stage methods, the one-stage method could improve performance since the features are directly driven by the classification loss, which has been evaluated by numerical experiments.

• The random resampling mechanism in the proposed method could alleviate the performance decrease brought by adversarial attack and achieve over 30% accuracy on adversarial examples of MNIST (The attack could destroy the accuracy of CNN with the same architecture to almost 0%).

摘要

•A one-stage, end-to-end kernel learning method based on random Fourier features is proposed. This method involves a generative network to learn the distribution of kernel and to build random features, which are then followed by a linear layer to categorize the features.•In random features learning, this is the first one-stage method. It is different from the existing two-stage methods that first solve a kernel alignment problem and then a classification problem.•Compared with two-stage methods, the one-stage method could improve performance since the features are directly driven by the classification loss, which has been evaluated by numerical experiments.•The random resampling mechanism in the proposed method could alleviate the performance decrease brought by adversarial attack and achieve over 30% accuracy on adversarial examples of MNIST (The attack could destroy the accuracy of CNN with the same architecture to almost 0%).