Surrogate dropout: Learning optimal drop rate through proxy

作者：

Highlights：

• A simple and effective regularization method called surrogate dropout is proposed, which regards the surrogate module as a proxy for approximating the optimal drop rate of each neuron.

• Compared with conventional dropout, the surrogate dropout method has fewer restrictions. Both the shallow and deep layers in CNNs can benefit from the usage of surrogate dropout.

• The superior regularization effect of surrogate dropout has been empirically verified using multiple datasets and networks with various depths.

摘要

•A simple and effective regularization method called surrogate dropout is proposed, which regards the surrogate module as a proxy for approximating the optimal drop rate of each neuron.•Compared with conventional dropout, the surrogate dropout method has fewer restrictions. Both the shallow and deep layers in CNNs can benefit from the usage of surrogate dropout.•The superior regularization effect of surrogate dropout has been empirically verified using multiple datasets and networks with various depths.

论文关键词：Deep neural networks,Dropout,Regularization

论文评审过程：Received 22 April 2020, Revised 16 July 2020, Accepted 28 July 2020, Available online 4 August 2020, Version of Record 13 August 2020.

论文官网地址：https://doi.org/10.1016/j.knosys.2020.106340