LightweightNet: Toward fast and lightweight convolutional neural networks via architecture distillation

作者：

Highlights：

• We present a new framework of deep convolutional neural network architecture distillation, namely LightweightNet, for acceleration and compression.

• We exploit the prior knowledge of pre-defined network architecture to guide the efficient design of acceleration/compression strategies, while not using pre-trained model.

• The proposed framework consists of network parameter compression, network structure acceleration, and non-tensor layer improvement.

• The proposed framework demonstrates a higher acceleration/compression rate than previous methods in experiments, including a large category handwritten Chinese character recognition task with state-of-the-art performance.

摘要

•We present a new framework of deep convolutional neural network architecture distillation, namely LightweightNet, for acceleration and compression.•We exploit the prior knowledge of pre-defined network architecture to guide the efficient design of acceleration/compression strategies, while not using pre-trained model.•The proposed framework consists of network parameter compression, network structure acceleration, and non-tensor layer improvement.•The proposed framework demonstrates a higher acceleration/compression rate than previous methods in experiments, including a large category handwritten Chinese character recognition task with state-of-the-art performance.

论文关键词：Deep network acceleration and compression,Architecture distillation,Lightweight network

论文评审过程：Received 11 February 2018, Revised 4 July 2018, Accepted 28 October 2018, Available online 2 November 2018, Version of Record 27 November 2018.

论文官网地址：https://doi.org/10.1016/j.patcog.2018.10.029