Our planet is photographed on a daily basis by dozens of imaging satellites, hundreds of airplanes and drones, and thousands of cars collecting street-level imagery. This imagery is critical to consumer products such as Google Earth or Google Street View, which let users travel virtually and explore any destination around the world. In addition, it is used by governments and commercial entities to make financial decisions, drive policy, or enforce laws, among other use cases. Increasingly though, it is not the geo-referenced pixels themselves that are consumed by end-users, it is knowledge that has been derived from these pixels. For example, searching for an address on Google Maps translates into calling a geocoding service whose geographic index of addresses is increasingly derived from Street View pixels house numbers observed in Street View's accurately geo-referenced panoramas are used to generate this index, yet these panoramas do not need to be shown to users as a result of their search. In this talk, I will give an overview of Google's efforts to capture and organize high-quality, fresh geographic imagery where it matters most. I'll then explain how recent advances in computer vision and deep learning in particular are trans-forming the way this imagery is being used to derive map data, local data, business intelligence, and power new consumer and business use cases.

The last two years have witnessed the introduction and rapid expansion of products based upon large, systematically-gathered, street-level image collections, such as Google Street View, EveryScape, and Mapjack. In the process of gathering images of public spaces, these projects also capture license plates, faces, and other information considered sensitive from a privacy standpoint. In this work, we present a system that addresses the challenge of automatically detecting and blurring faces and license plates for the purpose of privacy protection in Google Street View. Though some in the field would claim face detection is “solved”, we show that state-of-the-art face detectors alone are not sufficient to achieve the recall desired for large-scale privacy protection. In this paper we present a system that combines a standard sliding-window detector tuned for a high recall, low-precision operating point with a fast post-processing stage that is able to remove additional false positives by incorporating domain-specific information not available to the sliding-window detector. Using a completely automatic system, we are able to sufficiently blur more than 89% of faces and 94 - 96% of license plates in evaluation sets sampled from Google Street View imagery.

A frequent query in geospatial planning and decision making domains (e.g., emergency response, data acquisition, street cleaning), is to find an optimal traversal plan (OTP) that traverses an entire area (e.g., a city) by navigating through all its streets. The optimality is defined in terms of the time it takes to complete the traversal. This time depends on the number of times each street segment is traversed as well as the navigation time such as the time spent on changing direction at each intersection.

Unveiled in late 2004, Google Book Search is an ambitious program to make all the world's books discoverable online. The sheer scale of the problem brings a number of unique document analysis and understanding challenges that are outlined in this paper. We also go over some of the ways that Google is working with the document analysis research community to help push the state of the art.

An algorithm is proposed for extracting high quality, polygonal contours from connected objects in binary images. This is can be useful for OCR algorithms working on scanned text. It is also particularly interesting for raster-to-vector conversion of engineering drawings or other types of line drawings, where simple contours (known as Freeman chains) are grossly inadequate. Our algorithm is based on a relaxation technique, in which an initial polygonal approximation is iteratively refined until stability. Experiments demonstrate that this method consistently produces high quality results: the polygonal approximations obtained have a minimal number of corner points and remain extremely close to the original Freeman chain code.

Granulometries constitute one of the most useful and versatile sets of tools of morphological image analysis. They can be applied to a wide range of tasks, such as feature extraction, texture characterization, size estimation, image segmentation, etc

We describe a new approach for evaluating page segmentation algorithms. Unlike techniques that rely on OCR output, our method is region-based: the segmentation output, described as a set of regions together with their types, output order etc., is matched against the pre-stored set of ground-truth regions. Misclassifications, splitting, and merging of regions are among the errors that are detected by the system. Each error is weighted individually for a particular application and a global estimate of segmentation quality is derived. The system can be customized to benchmark specific aspects of segmentation (e.g., headline detection) and according to the type of error correction that might follow (e.g., re-typing). Segmentation ground-truth files are quickly and easily generated and edited using GroundsKeeper, an X-Window based tool that allows one to view a document, manually draw regions (arbitrary polygons) on it, and specify information about each region (e.g., type, parent).

The authors complement the standard algebraic view of mathematical morphology with a geometric, differential view. Three observations underlie this approach. (1) Certain structuring elements (convex) are scalable in that a sequence of repeated operations is equivalent to a single operation, but with a larger structuring element of the same shape. (2) To determine the outcome of the operation, it is sufficient to consider how the boundary is modified. (3) The modifications of the boundary are such that each point can be moved along the normal by a certain amount, which is dependent on the structuring element. Taken together, these observations, when the size of the structuring element shrinks to zero, assert that mathematical morphology operations with a convex structuring element are captured by a differential deformation of the boundary along the normal, governed by a Hamilton-Jacobi partial differential equation (PDE). A second theme is to show that mathematical morphology operations can be numerically implemented in a highly accurate fashion as the solution of these PDEs.>

Two different formal definitions of gray-scale reconstruction are presented. The use of gray-scale reconstruction in various image processing applications discussed to illustrate the usefulness of this transformation for image filtering and segmentation tasks. The standard parallel and sequential approaches to reconstruction are reviewed. It is shown that their common drawback is their inefficiency on conventional computers. To improve this situation, an algorithm that is based on the notion of regional maxima and makes use of breadth-first image scannings implemented using a queue of pixels is introduced. Its combination with the sequential technique results in a hybrid gray-scale reconstruction algorithm which is an order of magnitude faster than any previously known algorithm.

Gray-scale reconstruction is formally defined for discrete images. A brief summary of the existing techniques to compute it is provided, and a hybrid algorithm that is an order of magnitude faster than any other algorithm is introduced. Some of its application to image filtering and segmentation are listed.>

Up to now, all the known Euclidean distance function algorithms are either excessively slow or inaccurate, and even Danielsson's method (1980) produces errors in some configurations. The author shows that these problems are due to the local way distances are propagated in images by this algorithm. To remedy these drawbacks, an algorithm which encodes the objects boundaries as chains and propagates these structures in the image using rewriting rules is introduced. The chains convey Euclidean distances and can be written above one another, thus yielding exact results. In addition, the proposed algorithm is particularly efficient. Some of its applications to skeletons and neighborhood graphs are described.>