Human emotion recognition by optimally fusing facial expression and speech feature

作者：

Highlights：

• Subsequently, we leverage the MFCC to convert speech signal to images.

• We utilize the weighted decision fusion method to fuse facial expression and speech signal to achieve speech emotion recognition.

• Comprehensive experimental results have demonstrated that, compared with the uni-modal emotion recognition, bimodal features-based emotion recognition achieves a better performance.

摘要

•Subsequently, we leverage the MFCC to convert speech signal to images.•We utilize the weighted decision fusion method to fuse facial expression and speech signal to achieve speech emotion recognition.•Comprehensive experimental results have demonstrated that, compared with the uni-modal emotion recognition, bimodal features-based emotion recognition achieves a better performance.

论文关键词：Facial expression recognition,Speech emotion recognition,Bimodal fusion,Feature fusion,RNN

论文评审过程：Received 18 August 2019, Revised 21 January 2020, Accepted 4 March 2020, Available online 13 March 2020, Version of Record 18 March 2020.

论文官网地址：https://doi.org/10.1016/j.image.2020.115831