We describe Photo OCR, a system for text extraction from images. Our particular focus is reliable text extraction from smartphone imagery, with the goal of text recognition as a user input modality similar to speech recognition. Commercially available OCR performs poorly on this task. Recent progress in machine learning has substantially improved isolated character classification, we build on this progress by demonstrating a complete OCR system using these techniques. We also incorporate modern data center-scale distributed language modelling. Our approach is capable of recognizing text in a variety of challenging imaging conditions where traditional OCR systems fail, notably in the presence of substantial blur, low resolution, low contrast, high image noise and other distortions. It also operates with low latency, mean processing time is 600 ms per image. We evaluate our system on public benchmark datasets for text extraction and outperform all previously reported results, more than halving the error rate on multiple benchmarks. The system is currently in use in many applications at Google, and is available as a user input modality in Google Translate for Android.

We outline an approach for using concentration inequalities to perform rapid approximate multi-hypothesis testing. In a scenario where multiple hypotheses are ranked according to a large set of features, our scheme improves the efficiency of selecting the best hypothesis by providing a “bail-out threshold” at which unpromising hypotheses can be excluded from further evaluation. We show how concentration inequalities can be used to derive principled bail-out thresholds, subject to a user-specified error tolerance. The technique is similar to the sequential probability ratio test, but is applicable in more general conditions. We apply the technique to improve the speed of the fast-appearance-based mapping system for appearance-based place recognition and mapping. The speed increase provided by the new approach is data dependent, but we demonstrate speed improvements of between 25x - 50x on real data, with only a slight degradation in accuracy.

Continuous, real-time mapping of an environment using a camera requires a constant-time estimation engine. This rules out optimal global solving such as bundle adjustment. In this article, we investigate the precision that can be achieved with only local estimation of motion and structure provided by a stereo pair. We introduce a simple but novel representation of the environment in terms of a sequence of relative locations. We demonstrate precise local mapping and easy navigation using the relative map, and importantly show that this can be done without requiring a global minimisation after loop closure. We discuss some of the issues that arise from using a relative representation, and evaluate our system on long sequences processed at a constant 30-45 Hz, obtaining precisions down to a few metres over distances of a few kilometres.

This paper describes a probabilistic bail-out condition for multihypothesis testing based on Bennett's inequality. We investigate the use of the test for increasing the speed of an appearance-only SLAM system where locations are recognised on the basis of their sensory appearance. The bail-out condition yields speed increases between 25x-50x on real data, with only slight degradation in accuracy. We demonstrate the system performing real-time loop closure detection on a mobile robot over multiple-kilometre paths in initially unknown outdoor environments.

In this paper we present a loop closure method for a handheld single-camera SLAM system based on our previous work on relocalization. By finding correspondences between the current image and the map, our system is able to reliably detect loop closures. We compare our algorithm to existing techniques for loop closure in single-camera SLAM based on both image-to-image and map-to-map correspondences and discuss both the reliability and suitability of each algorithm in the context of monocular SLAM.

This paper describes a probabilistic framework for navigation using only appearance data. By learning a generative model of appearance, we can compute not only the similarity of two observations, but also the probability that they originate from the same location, and hence compute a pdf over observer location. We do not limit ourselves to the kidnapped robot problem (localizing in a known map), but admit the possibility that observations may come from previously unvisited places. The principled probabilistic approach we develop allows us to explicitly account for the perceptual aliasing in the environment - identical but indistinctive observations receive a low probability of having come from the same place. Our algorithm complexity is linear in the number of places, and is particularly suitable for online loop closure detection in mobile robotics.