One of the most popular and challenging tasks in remote sensing applications is the generation of digitized representations of Earth's objects from satellite raster image data. A common approach to tackle this challenge is a two-step method that first involves performing a pixel-wise classification of the raster data, then vectorizing the obtained classification map. We propose a novel approach, which recasts the polygonization problem as a mesh-based approximation of the input classification map, where binary labels are assigned to the mesh triangles to represent the building class. A dense initial mesh is decimated and optimized using local edge and vertex-based operators in order to minimize an objective function that models a balance between fidelity to the classification map in l1 norm sense, right angle regularity for polygonized buildings, and final mesh complexity. Experiments show that adding the right angle objective yields better representations quantitatively and qualitatively than previous work and commonly used polygon generalization methods in remote sensing literature for similar number of vertices.

The ultimate goal of land mapping from remote sensing image classification is to produce polygonal representations of Earth's objects, to be included in geographic information systems. This is most commonly performed by running a pixelwise image classifier and then polygonizing the connected components in the classification map. We here propose a novel polygonization algorithm, which uses a labeled triangular mesh to approximate the input classification maps. The mesh is optimized in terms of an l1norm with respect to the classifiers's output. We use a rich set of optimization operators, which includes a vertex relocator, and add a topology preservation strategy. The method outperforms current approaches, yielding better accuracy with fewer vertices.

New challenges in remote sensing impose the necessity of designing pixel classification methods that, once trained on a certain dataset, generalize to other areas of the earth. This may include regions where the appearance of the same type of objects is significantly different. In the literature it is common to use a single image and split it into training and test sets to train a classifier and assess its performance, respectively. However, this does not prove the generalization capabilities to other inputs. In this paper, we propose an aerial image labeling dataset that covers a wide range of urban settlement appearances, from different geographic locations. Moreover, the cities included in the test set are different from those of the training set. We also experiment with convolutional neural networks on our dataset.

We address the pixelwise classification of high-resolution aerial imagery. While convolutional neural networks (CNNs) are gaining increasing attention in image analysis, it is still challenging to adapt them to produce fine-grained classification maps. This is due to a well-known trade-off between recognition and localization: the impressive capability of CNNs to recognize meaningful objects comes at the price of losing spatial precision. We here propose an architecture that addresses this issue. It learns features at different levels of detail and also learns a function to combine them. By integrating local and global information in an efficient and flexible manner, it outperforms previous techniques.

While initially devised for image categorization, convolutional neural networks (CNNs) are being increasingly used for the pixelwise semantic labeling of images. However, the proper nature of the most common CNN architectures makes them good at recognizing but poor at localizing objects precisely. This problem is magnified in the context of aerial and satellite image labeling, where a spatially fine object outlining is of paramount importance. Different iterative enhancement algorithms have been presented in the literature to progressively improve the coarse CNN outputs, seeking to sharpen object boundaries around real image edges. However, one must carefully design, choose, and tune such algorithms. Instead, our goal is to directly learn the iterative process itself. For this, we formulate a generic iterative enhancement process inspired from partial differential equations, and observe that it can be expressed as a recurrent neural network (RNN). Consequently, we train such a network from manually labeled data for our enhancement task. In a series of experiments, we show that our RNN effectively learns an iterative process that significantly improves the quality of satellite image classification maps.

We propose an end-to-end framework for the dense, pixelwise classification of satellite imagery with convolutional neural networks (CNNs). In our framework, CNNs are directly trained to produce classification maps out of the input images. We first devise a fully convolutional architecture and demonstrate its relevance to the dense classification problem. We then address the issue of imperfect training data through a two-step training approach: CNNs are first initialized by using a large amount of possibly inaccurate reference data, and then refined on a small amount of accurately labeled data. To complete our framework, we design a multiscale neuron module that alleviates the common tradeoff between recognition and precise localization. A series of experiments show that our networks consider a large amount of context to provide fine-grained classification maps.

The problem of dense semantic labeling consists in assigning semantic labels to every pixel in an image. In the context of aerial image analysis, it is particularly important to yield high-resolution outputs. In order to use convolutional neural networks (CNNs) for this task, it is required to design new specific architectures to provide fine-grained classification maps. Many dense semantic labeling CNNs have been recently proposed. Our first contribution is an in-depth analysis of these architectures. We establish the desired properties of an ideal semantic labeling CNN, and assess how those methods stand with regard to these properties. We observe that even though they provide competitive results, these CNNs often underexploit properties of semantic labeling that could lead to more effective and efficient architectures. Out of these observations, we then derive a CNN framework specifically adapted to the semantic labeling problem. In addition to learning features at different resolutions, it learns how to combine these features. By integrating local and global information in an efficient and flexible manner, it outperforms previous techniques. We evaluate the proposed framework and compare it with state-of-the-art architectures on public benchmarks of high-resolution aerial image labeling.

We propose a convolutional neural network (CNN) model for remote sensing image classification. Using CNNs provides us with a means of learning contextual features for large-scale image labeling. Our network consists of four stacked convolutional layers that downsample the image and extract relevant features. On top of these, a deconvolutional layer upsamples the data back to the initial resolution, producing a final dense image labeling. Contrary to previous frameworks, our network contains only convolution and deconvolution operations. Experiments on aerial images show that our network produces more accurate classifications in lower computational time.

A partition tree is a hierarchical representation of an image. Once constructed, it can be repeatedly processed to extract information. Multi-object multi-class image segmentation with shape priors is one of the tasks that can be efficiently done upon an available tree. The traditional construction approach is a greedy clustering based on color similarities. However, not considering higher level cues during the construction phase leads to trees that might not accurately represent the underlying objects in the scene, inducing mistakes in the later segmentation. We propose a method to optimize a tree based both on color distributions and shape priors. It consists in pruning and regrafting tree branches in order to minimize the energy of the best segmentation that can be extracted from the tree. Theoretical guarantees help reducing the search space and make the optimization efficient. Our experiments show that we succeed in incorporating shape information to restructure a tree, which in turn enables to extract from it good quality multi-object segmentations with shape priors.

We propose a new binary partition tree (BPT)-based framework for multi-class segmentation of remote sensing images. In the literature, BPTs are typically computed in a bottom-up manner based on spectral similarities, then analyzed to extract image objects. When image objects exhibit a considerable internal spectral variability, it often happens that such objects are composed of several disjoint regions in the BPT, yielding errors in object extraction. We pose the multi-class segmentation problem as an energy minimization task and solve it by using BPTs. Our main contribution consists in introducing a new dissimilarity function for the tree construction, which combines both spectral discrepancies and supervised class-specific information to take into account the within-class spectral variability. The experimental validation proved that the proposed method constitutes a competitive alternative for object-based image classification.

Tensor voting is a computational framework that addresses the problem of perceptual organisation. It was designed to convey human perception principles into a unified framework that can be adapted to extract visually salient elements from possibly noisy or corrupted images. The original formulation featured some concerns that made it difficult or impractical to be applied directly. Therefore, several partial or total theoretical reformulations or augmentations have been proposed. These almost parallel publication were presented in different directions, with different priorities and even in a different notation. Thus, the authors observed the need for a coherent description and comparison of the different proposals. This work, after describing the original approach of tensor voting, reviews and explains a number of high impact theoretical modifications in a self‐contained manner and including possible future directions of work. The authors have selected and organised a number of formulations and unified the way the problem is addressed across the different proposals. The aim of this study is to contribute with a modern comprehensive source of information on the theoretical aspects of tensor voting.

We comment on a paper that describes a closed-form formulation to Tensor Voting, a technique to perceptually group clouds of points, usually applied to infer features in images. The authors proved an analytic solution to the technique, a highly relevant contribution considering that the original formulation required numerical integration, a time-consuming task. Their work constitutes the first closed-form expression for the Tensor Voting framework. In this work we first observe that the proposed formulation leads to unexpected results which do not satisfy the constraints for a Tensor Voting output, hence they cannot be interpreted. Given that the closed-form expression is said to be an analytic equivalent solution, unexpected outputs should not be encountered unless there are flaws in the proof. We analyzed the underlying math to find which were the causes of these unexpected results. In this commentary we show that their proposal does not in fact provide a proper analytic solution to Tensor Voting and we indicate the flaws in the proof.