There are a variety of grand challenges for multi-orientation text detection in scene videos, where the typical issues include skew distortion, low contrast, and arbitrary motion. Most conventional video text detection methods using individual frames have limited performance. In this paper, we propose a novel tracking based multi-orientation scene text detection method using multiple frames within a unified framework via dynamic programming. First, a multi-information fusion-based multi-orientation text detection method in each frame is proposed to extensively locate possible character candidates and extract text regions with multiple channels and scales. Second, an optimal tracking trajectory is learned and linked globally over consecutive frames by dynamic programming to finally refine the detection results with all detection, recognition, and prediction information. Moreover, the effectiveness of our proposed system is evaluated with the state-of-the-art performances on several public data sets of multi-orientation scene text images and videos, including MSRA-TD500, USTB-SV1K, and ICDAR 2015 Scene Videos.

We construct a robust and precise multi-orientation text detection system in scene images which can extensively locate possible characters with multi-information fusion. In our method, an adaptive multi-channel character grouping algorithm is first proposed to extract all possible character candidates robustly, and an AdaBoost classifier is then to properly identify character candidates as characters or non-characters. A single-link clustering with distance metric learning is thereafter used to adaptively group characters into text regions, and an effective hybrid filter with Convolution Neural Networks (CNN), AdaBoost and Bayesian classifiers is finally designed to precisely verify the extracted text regions. Our proposed technology is extensively evaluated on several public multi-orientation scene text datasets, e.g., MSRA-TD500 and USTB-SV1K, and is much better than state-of-the-art methods.

Text detection and tracking in scene videos are important prerequisites for content-based video analysis and retrieval, wearable camera systems and mobile devices augmented reality translators. Here, we present a novel multi-strategy tracking based text detection approach in scene videos. In this approach, a state-of-the-art scene text detection module [1] is first used to detect text in each video frame. Then a multi-strategy text tracking technique is proposed, which uses tracking by detection, spatio-temporal context learning, and linear prediction to predict the candidate text location sequentially, and adaptively integrates and selects the best matching text block from the candidate blocks with a rule-based method. This multi-strategy tracking technique can combine the advantages of the three different tracking techniques and afterwards make remedies to the disadvantages of them. Experiments on a variety of scene videos show that our proposed approach is effective and robust to reduce false alarm and improve the accuracy of detection.

Text detection in natural scene images is an important prerequisite for many content-based image analysis tasks, while most current research efforts only focus on horizontal or near horizontal scene text. In this paper, first we present a unified distance metric learning framework for adaptive hierarchical clustering, which can simultaneously learn similarity weights (to adaptively combine different feature similarities) and the clustering threshold (to automatically determine the number of clusters). Then, we propose an effective multi-orientation scene text detection system, which constructs text candidates by grouping characters based on this adaptive clustering. Our text candidates construction method consists of several sequential coarse-to-fine grouping steps: morphology-based grouping via single-link clustering, orientation-based grouping via divisive hierarchical clustering, and projection-based grouping also via divisive clustering. The effectiveness of our proposed system is evaluated on several public scene text databases, e.g., ICDAR Robust Reading Competition data sets (2011 and 2013), MSRA-TD500 and NEOCR. Specifically, on the multi-orientation text data set MSRA-TD500, the f measure of our system is 71 percent, much better than the state-of-the-art performance. We also construct and release a practical challenging multi-orientation scene text data set (USTB-SV1K), which is available at http://prir.ustb.edu.cn/TexStar/MOMV-text-detection/.

After being kick-started with major breakthrough in 2006 by Hinton, LeCun and Bengio respectively, deep learning has been becoming the mainstream for challenging classification systems, which, however always were with "shallow" discriminative classifiers in the past. In this paper, we argue that in common classification cases with plenty but not enough training examples, mixed-quality examples for dozens of categories, deep learning and shallow classification may have complementary performance. Then, we design a hybrid recognition strategy with classification switching to adaptively fuse deep learning and shallow classification technologies. Finally, we present a variety of experiments with visual recognition tasks, i.e., USPS character recognition, Caltech101 visual object classification, and ICDAR scene text recognition. Specifically, we perform word recognition by dynamically combing the conventional open source OCR engine with the present popular convolutional neural networks, and construct an effective end-to-end scene text recognition system with open-vocabulary. This end-to-end system is evaluated on ICDAR 2011 Robust Reading Competition (Challenge 2) dataset, the f measure of which is 54.5%, much better than 45.2% of the latest state-of-the-art performance.