Deep CNN Co-design for HEVC CU Partition Prediction on FPGA–SoC

摘要

Convolutional neural networks (CNNs) are widely used, due to their excellent performance, in many computer vision applications, such as facial recognition, image classification tasks, speech recognition programs, video gaming, etc. However, CNNs require a large number of memory resources and they are also computationally intensive. Field Programmable Gate Arrays (FPGAs), especially the new technology FPGA–SoC, are considered as the most promising platforms for accelerating CNNs, due to their high performance capabilities, energy efficiency, and reconfigurable property. This paper proposes an accelerated CNN model for video compression application based on hardware-software architecture. We first accelerate the CNN layers to build an Intellectual Property (IP) cores using Vivado High Level Synthesis (HLS). Then, we create a hardware-software architecture based on a CNN’s IP cores designed and integrated in the programmable logic zone (PL) which is connected to the Xilinx Processing System (PS) that manage all processing tasks on the FPGA–SoC board. The experimental results demonstrate that our proposed co-design achieves an on-chip power consumption of 1.69 W under a 142 MHz PL frequency and 525 MHz PS frequency. The comparative study with existing methods shows that the design we proposed has obvious advantages in terms of power consumption and hardware cost requirements.