Discriminating ambiguous symbols in online handwritten mathematical expression is difficult without context. We propose a Bidirectional Recurrent Neural Network for segmenting and classifying online handwritten mathematical symbols. The context from forward and backward directions helps the classification model discriminate ambiguous symbols and improve recognition rates. The classification model is integrated into the Stochastic Context-Free Grammar recognition system for recognizing mathematical expressions. We show the effectiveness of the approach for improving symbol classification and segmentation on the CROHME 2016 dataset.

This paper presents an improvement in recognizing offline handwritten mathematical expressions (HMEs) by deep neural networks. We train it end-to-end using weakly supervised learning. The network has three parts: an encoder using a Convolutional Neural Network to encode high-level features from an input HME image; a decoder using gated recurrent units with attention to parse the high-level features and generate an output expression in the LaTeX format; and a symbol classifier to improve the localization and classification of the high-level features. Besides, we use the model ensemble method for the beam search process to average the probabilities from multiple models. For the dataset of the Competition on Recognition of Online Handwritten Mathematical Expressions (CROHME) 2014 and 2016, we have achieved an expression recognition rate of 53.65% and 51.96% correspondingly, which is 6 points better than without weakly supervised learning. Furthermore, when ensembling several models, the recognition rate of our method is increased to 55.68% for the CROHME 2014 testing set.