Extracting buildings from aerial scene images is an important task with many applications. However, this task is highly difficult to automate due to extremely large variations of building appearances, and still heavily relies on manual work. To attack this problem, we design a deep convolutional network with a simple structure that integrates activation from multiple layers for pixel-wise prediction, and introduce the signed distance function of building boundaries to represent output, which has an enhanced representation power. To train the network, we leverage abundant building footprint data from geographic information systems (GIS) to generate large amounts of labeled data. The trained model achieves a superior performance on datasets that are significantly larger and more complex than those used in prior work, demonstrating that the proposed method provides a promising and scalable solution for automating this labor-intensive task.

Very large overhead imagery associated with ground truth maps has the potential to generate billions of training image patches for machine learning algorithms. However, random sampling selection criteria often leads to redundant and noisy-image patches for model training. With minimal research efforts behind this challenge, the current status spells missed opportunities to develop supervised learning algorithms that generalize over wide geographical scenes. In addition, much of the computational cycles for large scale machine learning are poorly spent crunching through noisy and redundant image patches. We demonstrate a potential framework to address these challenges specifically, while evaluating a human settlement detection task. A novel binary search tree sampling scheme is fused with a kernel based hashing procedure that maps image patches into hash-buckets using binary codes generated from image content. The framework exploits inherent redundancy within billions of image patches to promote mostly high variance preserving samples for accelerating algorithmic training and increasing model generalization.

Establishing up-to-date nationwide building maps is essential to understand urban dynamics, such as estimating population and urban planning and many other applications. However, an efficient and effective solution is yet to be developed. In this paper, for the first time we evaluate three state-of-the-art CNNs for detecting buildings across entire United States using aerial images. The three CNN architectures, fully convolutional neural network, conditional random field as recurrent neural network, and SegNet, support semantic pixel-wise labeling and focus on capturing textural information at multi-scale. We use 1-meter resolution NAIP images as the test data set, and compare the detection results across the three methods. In addition, we propose to combine signed distance function labels with SegNet, which is the preferred CNN architecture identified by our extensive evaluations. The results are further improved in terms of precision, recall rate and the number of building detected. On average, model inference on test images is less than one minute for an area of size ∼ 56 km2. With these promising results and the time required to process images, the framework offers great potential toward country scale building mapping with remote sensing imagery.

Up-to-date maps of installed solar photovoltaic panels are a critical input for policy and financial assessment of solar distributed generation. However, such maps for large areas are not available. With high coverage and low cost, aerial images enable large-scale mapping, but it is highly difficult to automatically identify solar panels from images, which are small objects with varying appearances dispersed in complex scenes. We introduce a new approach based on deep convolutional networks, which effectively learns to delineate solar panels in aerial scenes. The approach is applied to mapping solar panels in imagery covering 200 square kilometers in two cities, using only 12 square kilometers of training data that are manually labeled. Results are generated efficiently with an accuracy comparable to manual mapping, demonstrating the effectiveness and scalability of our approach.

This paper introduces a factorization-based approach that efficiently segments textured images. We use local spectral histograms as features, and construct an M × N feature matrix using M-dimensional feature vectors in an N-pixel image. Based on the observation that each feature can be approximated by a linear combination of several representative features, we factor the feature matrix into two matrices--one consisting of the representative features and the other containing the weights of representative features at each pixel used for linear combination. The factorization method is based on singular value decomposition and nonnegative matrix factorization. The method uses local spectral histograms to discriminate region appearances in a computationally efficient way and at the same time accurately localizes region boundaries. The experiments conducted on public segmentation data sets show the promise of this simple yet powerful approach.

Determining the number of buildings in aerial images is an important problem because the information greatly benefits applications such as population estimation, change detection, and urbanization monitoring. In this paper, we address this problem by learning the relationship between low-level image features and building counts. Building footprints from public cartographic databases are used as labeled data. We first extract straight line segments from images. A classifier is then trained to identify line segments corresponding to building edges. Although there exist mismatches between resulting line segments and building edges, we observe a strong linear relationship between building numbers and line numbers for similar types of buildings. Based on this observation, we predict the building count for a given image using the following method. We find top k images with the most similar appearances from training samples and learn a linear regression model from this image set. The building count is computed based on the model. Our method avoids the difficulty in building detection and produces reliable results on large, diverse datasets.

We present a new method for remote sensing image segmentation, which utilizes both spectral and texture information. Linear filters are used to provide enhanced spatial patterns. For each pixel location, we compute combined spectral and texture features using local spectral histograms, which concatenate local histograms of all input bands. We regard each feature as a linear combination of several representative features, each of which corresponds to a segment. Segmentation is given by estimating combination weights, which indicate segment ownership of pixels. We present segmentation solutions where representative features are either known or unknown. We also show that feature dimensions can be greatly reduced via subspace projection. The scale issue is investigated, and an algorithm is presented to automatically select proper scales, which does not require segmentation at multiple-scale levels. Experimental results demonstrate the promise of the proposed method.

The trace data generated from GPS enabled vehicles is highly valuable for applications such as map inference and traffic analysis. However, the data tends to be noisy due to signal interference. In this paper, we introduce aerial images in GPS trace analysis. Computer vision techniques are developed that effectively integrate image information with GPS data to generate road networks. An image is first segmented by an efficient factorization-based algorithm. A structure tensor approach is proposed to measure the orientation difference between a trace segment and the corresponding image patch. The segmentation result and orientation measures lead to significantly reducing the traces not aligning with roads. The traces are further processed to produce high-quality road networks. We show that our method produces promising results for very noisy GPS data with a low sampling rate and also outperforms the leading method of map inference from GPS traces.

This letter presents a benchmarking study for aerial image segmentation. We construct an image data set consisting of various aerial scenes. Segmentations generated by different human subjects are used as ground truth. We analyze the consistency between segmentations from different subjects. We select six leading segmentation algorithms, which include not only the algorithms specifically designed for aerial images but also more generally applicable algorithms. We also select a recently proposed algorithm due to its promising performance in handling texture regions. We apply these algorithms to the aerial image data set and quantitatively evaluate their performance. We interpret the evaluation results based on the characteristics of algorithms, which provide general guidance for selecting proper algorithms in specific applications.

We present a novel method for segmenting images with texture and nontexture regions. Local spectral histograms are feature vectors consisting of histograms of chosen filter responses, which capture both texture and nontexture information. Based on the observation that the local spectral histogram of a pixel location can be approximated through a linear combination of the representative features weighted by the area coverage of each feature, we formulate the segmentation problem as a multivariate linear regression, where the solution is obtained by least squares estimation. Moreover, we propose an algorithm to automatically identify representative features corresponding to different homogeneous regions, and show that the number of representative features can be determined by examining the effective rank of a feature matrix. We present segmentation results on different types of images, and our comparison with another spectral histogram based method shows that the proposed method gives more accurate results.

An automatic method for road extraction from satellite imagery is presented. The core of the proposed method is locally excitatory globally inhibitory oscillator networks (LEGION). The road extraction task is decomposed into three stages. The first stage is image segmentation by LEGION. In the second stage, the medial axis of each segment is computed, and the medial axis points corresponding to narrow regions are selected. The third is the road grouping stage. Alignment-dependent connections between selected points are established, and LEGION is utilized to group well-aligned points, which represent the extracted roads. Due to the selective gating mechanism of LEGION, different roads in an image are grouped separately. Road extraction results on synthetic and real images are presented. A comparison with other methods shows that the proposed method produces very competitive extraction results.

We present an automatic method for road extraction from satellite imagery. The core of the proposed method is locally excitatory globally inhibitory oscillator networks (LEGION). We decompose the road extraction task into three stages. The first stage is image segmentation by LEGION. In the second stage, we compute the medial axis of each segment and select the segments with narrow widths. The third is the road grouping stage. With the medial axes, alignment-dependent connections between medial axis points are established and LEGION is utilized to group the well-aligned medial axes, which represent extracted road segments. Due to the selective gating mechanism of LEGION, different roads in an image are grouped separately. Experimental results on synthetic and real images show the effectiveness of this method.