Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition

作者:

Highlights:

• A novel neural network based approach to handwritten mathematical expression recognition.

• An end-to-end encoder-decoder framework to alleviate the problem caused by an explicit symbol segmentation and the computational demands of employing a mathematical expression grammar.

• The deep fully convolutional neural network as the encoder.

• The coverage-based attention model to incorporate the attention history.

• Attention visualization to show the link between the input image and output symbol sequence in latex format.

• To the best of our knowledge, achieving the best published expression recognition accuracy on CROHME 2014 competition set by only using the official training data.

摘要

•A novel neural network based approach to handwritten mathematical expression recognition.•An end-to-end encoder-decoder framework to alleviate the problem caused by an explicit symbol segmentation and the computational demands of employing a mathematical expression grammar.•The deep fully convolutional neural network as the encoder.•The coverage-based attention model to incorporate the attention history.•Attention visualization to show the link between the input image and output symbol sequence in latex format.•To the best of our knowledge, achieving the best published expression recognition accuracy on CROHME 2014 competition set by only using the official training data.

论文关键词:Handwritten mathematical expression recognition,Neural network,Attention

论文评审过程:Received 19 November 2016, Revised 26 April 2017, Accepted 7 June 2017, Available online 10 June 2017, Version of Record 5 July 2017.

论文官网地址:https://doi.org/10.1016/j.patcog.2017.06.017