The modern road network topology comprises intricately designed structures that introduce complexity when automatically reconstructing road networks. While open resources like OpenStreetMap (OSM) offer road networks with well-defined topology, they may not always be up to date worldwide. In this paper, we propose a fully automated pipeline for extracting road networks from very-high-resolution (VHR) satellite imagery. Our approach directly generates road line-strings that are seamlessly connected and precisely positioned. The process involves three key modules: a CNN-based neural network for road segmentation, a graph optimization algorithm to convert road predictions into vector line-strings, and a machine learning model for classifying road materials. Compared to OSM data, our results demonstrate significant potential for providing the latest road layouts and precise positions of road segments.

The recent introduction of very high resolution satellite imagery (30 cm/pixel) allows us to propose a pipeline for automated cartography of major man-made structures such as buildings and especially the novel class of wall/fence. Extraction is performed by deep learning then geometry is reconstructed and regularized to fit mapping specifications. The final step is an algorithm to ensure the coherence of the map between classes (here building and wall/fence), which is very important for mapping, avoiding gaps or overlaps of objects.

BrightEarth City Texturing is an automatic and resource-efficient 3D urban modeling pipeline, producing quality low-poly 3D models with high-resolution seamless textures suitable for real-time rendering. From a single satellite image it extracts building geometry with footprint polygons along with attributes such as building height, roof shape, and roof texture. If ground images are available it can also faithfully texture facades in terms of architectural style.

Identifying and mapping fractures and faults are important in geosciences, especially in earthquake hazard and geological reservoir studies. This mapping can be done manually in optical images of the earth surface, yet it is time consuming and it requires an expertise that may not be available. Building upon a recent prior study, we develop a deep learning approach, based on a variant of a U-Net neural network, and apply it to automate fracture and fault mapping in optical images and topographic data. We show that training the model with a realistic knowledge of fracture and fault uneven distributions and trends, and using a loss function that operates at both pixel and larger scales through the combined use of weighted Binary Cross Entropy and Intersection over Union, greatly improves the predictions, both qualitatively and quantitatively. As we apply the model to a site differing from those used for training, we demonstrate its enhanced generalization capacity.

Learning - based approaches are now typically used to extract building rooftop from overhead imagery. However, converting boundaries of segmented objects from raster format to vector coordinates remains a challenging problem. Using re-cent advances in multi-task learning, we propose a fast and scalable approach, based on a polygonal partitioning of the space and discrete optimization, to deliver accurate and sim-ple vectorized building rooftops, that are compared to those produced by state-of-the-art techniques.

Recent technologies have enabled a significant growth of geographic datasets with different levels of detail and specifications. Subsequent analysis and mapping tasks may require to get the best out of the diversity of proposed sources. One solution is to integrate such maps through conflation. The purpose of this merging technique is to combine data that represent the same features from multiple datasets, into a new, richer dataset. Vector data conflation was intensively applied on linear networks like roads, streets and waterways, however combining polygonal building shapes has been relatively overlooked by the literature. In this paper, we propose an idea to aggregate a set of vector building maps to obtain a single fused representation. The proposed method takes as input two or more vector maps (more inputs lead to much more reliable maps), and decomposes the 2D space into a polygonal partition. A binary labelling procedure is applied using maps reliability weights, yielding contours of sets of connected buildings. Finally, a slicing algorithm decomposes contours into separate building instances. We show that our pipeline generates more accurate maps in terms of both IoU and F1 scores than any of the maps used as an input.

While state of the art image segmentation models typically output segmentations in raster format, applications in geographic information systems often require vector polygons. To help bridge the gap between deep network output and the format used in downstream tasks, we add a frame field output to a deep segmentation model for extracting buildings from remote sensing images. We train a deep neural network that aligns a predicted frame field to ground truth contours. This additional objective improves segmentation quality by leveraging multi-task learning and provides structural information that later facilitates polygonization; we also introduce a polygonization algorithm that that utilizes the frame field along with the raster segmentation. Our code is available at https://github.com/Lydorn/Polygonization-by-Frame-Field-Learning.

Automatic footprint extraction from remote sensing images remains a popular yet challenging research topic, driven by numerous applications such as telecommunications and urban management. In this paper, we propose an automatic pipeline for reliable building footprint extraction from high-resolution satellite images. The proposed method takes as the input two images with the corresponding rational polynomial coefficient models, applies deep learning to each image for building segmentation and estimation of contour orientations, then matches extracted building footprints from two images for estimating reliability of each footprint. The pipeline has proven to be efficient for reconstructing accurate building footprints, and estimated reliability scores help to efficiently validate and correct predicted maps.

The domain adaptation of satellite images has recently gained increasing attention to overcome the limited generalization abilities of machine learning models when segmenting large-scale satellite images. Most of the existing approaches seek for adapting the model from one domain to another. However, such single-source and single-target setting prevents the methods from being scalable solutions since, nowadays, multiple sources and target domains having different data distributions are usually available. Besides, the continuous proliferation of satellite images necessitates the classifiers to adapt to continuously increasing data. We propose a novel approach, coined DAugNet, for unsupervised, multisource, multitarget, and life-long domain adaptation of satellite images. It consists of a classifier and a data augmentor. The data augmentor, which is a shallow network, is able to perform style transfer between multiple satellite images in an unsupervised manner, even when new data are added over time. In each training iteration, it provides the classifier with diversified data, which makes the classifier robust to large data distribution difference between the domains. Our extensive experiments prove that DAugNet significantly better generalizes to new geographic locations than the existing approaches.

We add a frame field output to an image segmentation neural network to improve segmentation quality and provide structural information for a subsequent polygonization step. A frame field encodes two directions up to sign at every point of an image. To improve segmentation, we train a network to align an output frame field to the tangents of ground truth contours. In addition to increasing performance by leveraging the multi-task learning effect, our method produces more regular segmentations with sharp building corners. GitHub: github.com/Lydorn/Polygonization-by-Frame-Field-Learning.

Although convolutional neural networks have been proven to be an effective tool to generate high quality maps from remote sensing images, their performance significantly deteriorates when there exists a large domain shift between training and test data. To address this issue, we propose a new data augmentation approach that transfers the style of test data to training data using generative adversarial networks. Our semantic segmentation framework consists in first training a U-net from the real training data and then fine-tuning it on the test stylized fake training data generated by the proposed approach. Our experimental results prove that our framework outperforms the existing domain adaptation methods.

Automatic 3D reconstruction of urban scenes from stereo pairs of satellite images remains a popular yet challenging research topic, driven by numerous applications such as telecommunications and defense. The quality of reconstruction results depends particularly on the quality of the available stereo pair. In this paper, we propose an operational pipeline for large-scale 3D reconstruction of buildings from stereo satellite images. The proposed chain uses U-net to extract contour polygons of buildings, and the combination of optimization and computational geometry techniques to reconstruct a digital terrain model and a digital height model, and to correctly estimate the position of building footprints. The pipeline has proven to be efficient for 3D building reconstruction, even if the close-to-nadir image is not available.

Due to the various reasons, such as atmospheric effects and differences in acquisition, it is often the case that there exists a large difference between the spectral bands of satellite images collected from different geographic locations. The large shift between the spectral distributions of training and test data causes the current state-of-the-art supervised learning approaches to output unsatisfactory maps. We present a novel semantic segmentation framework that is robust to such a shift. The key component of the proposed framework is color mapping generative adversarial networks (ColorMapGANs) that can generate fake training images that are semantically exactly the same as training images, but whose spectral distribution is similar to the distribution of the test images. We then use the fake images and the ground truth for the training images to fine-tune the already trained classifier. Contrary to the existing generative adversarial networks (GANs), the generator in ColorMapGAN does not have any convolutional or pooling layers. It learns to transform the colors of the training data to the colors of the test data by performing only one elementwise matrix multiplication and one matrix-addition operation. Due to the architecturally simple but powerful design of ColorMapGAN, the proposed framework outperforms the existing approaches with a large margin in terms of both accuracy and computational complexity.

In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for which available annotations are misaligned polygons, resulting in noisy supervision. We subsequently set up a multiple-rounds training scheme which corrects the ground truth annotations at each round to better train the model at the next round. We show that it is possible to reduce the noise of the dataset by iteratively training a better alignment model to correct the annotation alignment.

In this work, we propose a novel multi-task framework, to learn satellite image pansharpening and segmentation jointly. Our framework is based on the encoder-decoder architecture, where both tasks share the same encoder but each one has its own decoder. We compare our framework against single-task models with different architectures. Results show that our framework outperforms all other approaches in both tasks.

In dense labeling problem, the major drawback of the convolutional neural networks is their inability to learn new classes without affecting performance for the old classes on the data, having no annotations for the previous classes. In this work, we address the issue of adding new classes continually to the already trained network from a stream of data. Our approach comprises two main components: adaptation and remembering. For adaptation, we keep a clone of the previously trained network, which serves as a memory for the old classes in absence of their annotations on the new data. The updated network learns new as well as old classes on the current data using output of the memory network and the new ground-truth. For remembering, we store a little portion of the previous data, from which we systematically feed samples to the updated network during training. Our results prove that segmentation capabilities for the new classes can be added to the already trained network without catastrophically forgetting the previously learned information.

Given a trained neural network, we aim at understanding how similar it considers any two samples. For this, we express a proper definition of similarity from the neural network perspective (i.e. we quantify how undissociable two inputs A and B are), by taking a machine learning viewpoint: how much a parameter variation designed to change the output for A would impact the output for B as well? We study the mathematical properties of this similarity measure, and show how to estimate sample density with it, in low complexity, enabling new types of statistical analysis for neural networks. We also propose to use it during training, to enforce that examples known to be similar should also be seen as similar by the network. We then study the self-denoising phenomenon encountered in regression tasks when training neural networks on datasets with noisy labels. We exhibit a multimodal image registration task where almost perfect accuracy is reached, far beyond label noise variance. Such an impressive self-denoising phenomenon can be explained as a noise averaging effect over the labels of similar examples. We analyze data by retrieving samples perceived as similar by the network, and are able to quantify the denoising effect without requiring true labels.

While geographic information systems typically use polygonal representations to map Earth's objects, most state-of-the-art methods produce maps by performing pixelwise classification of remote sensing images, then vectorizing the outputs. This paper studies if one can learn to directly output a vectorial semantic labeling of the image. We here cast a mapping problem as a polygon prediction task, and propose a deep learning approach which predicts vertices of the polygons outlining objects of interest. Experimental results on the Solar photovoltaic array location dataset show that the proposed network succeeds in learning to regress polygon coordinates, yielding directly vectorial map outputs.

Over the recent years, there has been an increasing interest in large-scale classification of remote sensing images. In this context, the Inria Aerial Image Labeling Benchmark has been released online in December 2016. In this paper, we discuss the outcomes of the first year of the benchmark contest, which consisted in dense labeling of aerial images into building / not building classes, covering areas of five cities not present in the training set. We present four methods with the highest numerical accuracies, all four being convolutional neural network approaches. It is remarkable that three of these methods use the U-net architecture, which has thus proven to become a new standard in image dense labeling.

One of the most popular and challenging tasks in remote sensing applications is the generation of digitized representations of Earth's objects from satellite raster image data. A common approach to tackle this challenge is a two-step method that first involves performing a pixel-wise classification of the raster data, then vectorizing the obtained classification map. We propose a novel approach, which recasts the polygonization problem as a mesh-based approximation of the input classification map, where binary labels are assigned to the mesh triangles to represent the building class. A dense initial mesh is decimated and optimized using local edge and vertex-based operators in order to minimize an objective function that models a balance between fidelity to the classification map in l1 norm sense, right angle regularity for polygonized buildings, and final mesh complexity. Experiments show that adding the right angle objective yields better representations quantitatively and qualitatively than previous work and commonly used polygon generalization methods in remote sensing literature for similar number of vertices.

The ultimate goal of land mapping from remote sensing image classification is to produce polygonal representations of Earth's objects, to be included in geographic information systems. This is most commonly performed by running a pixelwise image classifier and then polygonizing the connected components in the classification map. We here propose a novel polygonization algorithm, which uses a labeled triangular mesh to approximate the input classification maps. The mesh is optimized in terms of an l1norm with respect to the classifiers's output. We use a rich set of optimization operators, which includes a vertex relocator, and add a topology preservation strategy. The method outperforms current approaches, yielding better accuracy with fewer vertices.

New challenges in remote sensing impose the necessity of designing pixel classification methods that, once trained on a certain dataset, generalize to other areas of the earth. This may include regions where the appearance of the same type of objects is significantly different. In the literature it is common to use a single image and split it into training and test sets to train a classifier and assess its performance, respectively. However, this does not prove the generalization capabilities to other inputs. In this paper, we propose an aerial image labeling dataset that covers a wide range of urban settlement appearances, from different geographic locations. Moreover, the cities included in the test set are different from those of the training set. We also experiment with convolutional neural networks on our dataset.

We address the pixelwise classification of high-resolution aerial imagery. While convolutional neural networks (CNNs) are gaining increasing attention in image analysis, it is still challenging to adapt them to produce fine-grained classification maps. This is due to a well-known trade-off between recognition and localization: the impressive capability of CNNs to recognize meaningful objects comes at the price of losing spatial precision. We here propose an architecture that addresses this issue. It learns features at different levels of detail and also learns a function to combine them. By integrating local and global information in an efficient and flexible manner, it outperforms previous techniques.

While initially devised for image categorization, convolutional neural networks (CNNs) are being increasingly used for the pixelwise semantic labeling of images. However, the proper nature of the most common CNN architectures makes them good at recognizing but poor at localizing objects precisely. This problem is magnified in the context of aerial and satellite image labeling, where a spatially fine object outlining is of paramount importance. Different iterative enhancement algorithms have been presented in the literature to progressively improve the coarse CNN outputs, seeking to sharpen object boundaries around real image edges. However, one must carefully design, choose, and tune such algorithms. Instead, our goal is to directly learn the iterative process itself. For this, we formulate a generic iterative enhancement process inspired from partial differential equations, and observe that it can be expressed as a recurrent neural network (RNN). Consequently, we train such a network from manually labeled data for our enhancement task. In a series of experiments, we show that our RNN effectively learns an iterative process that significantly improves the quality of satellite image classification maps.

We propose an end-to-end framework for the dense, pixelwise classification of satellite imagery with convolutional neural networks (CNNs). In our framework, CNNs are directly trained to produce classification maps out of the input images. We first devise a fully convolutional architecture and demonstrate its relevance to the dense classification problem. We then address the issue of imperfect training data through a two-step training approach: CNNs are first initialized by using a large amount of possibly inaccurate reference data, and then refined on a small amount of accurately labeled data. To complete our framework, we design a multiscale neuron module that alleviates the common tradeoff between recognition and precise localization. A series of experiments show that our networks consider a large amount of context to provide fine-grained classification maps.

The problem of dense semantic labeling consists in assigning semantic labels to every pixel in an image. In the context of aerial image analysis, it is particularly important to yield high-resolution outputs. In order to use convolutional neural networks (CNNs) for this task, it is required to design new specific architectures to provide fine-grained classification maps. Many dense semantic labeling CNNs have been recently proposed. Our first contribution is an in-depth analysis of these architectures. We establish the desired properties of an ideal semantic labeling CNN, and assess how those methods stand with regard to these properties. We observe that even though they provide competitive results, these CNNs often underexploit properties of semantic labeling that could lead to more effective and efficient architectures. Out of these observations, we then derive a CNN framework specifically adapted to the semantic labeling problem. In addition to learning features at different resolutions, it learns how to combine these features. By integrating local and global information in an efficient and flexible manner, it outperforms previous techniques. We evaluate the proposed framework and compare it with state-of-the-art architectures on public benchmarks of high-resolution aerial image labeling.

We propose a convolutional neural network (CNN) model for remote sensing image classification. Using CNNs provides us with a means of learning contextual features for large-scale image labeling. Our network consists of four stacked convolutional layers that downsample the image and extract relevant features. On top of these, a deconvolutional layer upsamples the data back to the initial resolution, producing a final dense image labeling. Contrary to previous frameworks, our network contains only convolution and deconvolution operations. Experiments on aerial images show that our network produces more accurate classifications in lower computational time.

A partition tree is a hierarchical representation of an image. Once constructed, it can be repeatedly processed to extract information. Multi-object multi-class image segmentation with shape priors is one of the tasks that can be efficiently done upon an available tree. The traditional construction approach is a greedy clustering based on color similarities. However, not considering higher level cues during the construction phase leads to trees that might not accurately represent the underlying objects in the scene, inducing mistakes in the later segmentation. We propose a method to optimize a tree based both on color distributions and shape priors. It consists in pruning and regrafting tree branches in order to minimize the energy of the best segmentation that can be extracted from the tree. Theoretical guarantees help reducing the search space and make the optimization efficient. Our experiments show that we succeed in incorporating shape information to restructure a tree, which in turn enables to extract from it good quality multi-object segmentations with shape priors.

In this work, we propose a stochastic model for curvilinear structure reconstruction using morphological profiles of path opening operator. We apply the support vector machine classifier to obtain initial probabilities to belong to line network for each pixel. Then, we formulate a stochastic optimization problem that detects line segments corresponding to the latent curvilinear structure in a scene. Experimental results on DNA filament and remote sensing images validate the effectiveness of the proposed algorithm when compared to other recent methods.

We propose a new binary partition tree (BPT)-based framework for multi-class segmentation of remote sensing images. In the literature, BPTs are typically computed in a bottom-up manner based on spectral similarities, then analyzed to extract image objects. When image objects exhibit a considerable internal spectral variability, it often happens that such objects are composed of several disjoint regions in the BPT, yielding errors in object extraction. We pose the multi-class segmentation problem as an energy minimization task and solve it by using BPTs. Our main contribution consists in introducing a new dissimilarity function for the tree construction, which combines both spectral discrepancies and supervised class-specific information to take into account the within-class spectral variability. The experimental validation proved that the proposed method constitutes a competitive alternative for object-based image classification.

We propose a new model for wrinkle detection in human faces using a marked point process. In order to detect an arbitrary shape of wrinkles, we represent them as a set of line segments, where each segment is characterized by its length and orientation. We propose a probability density of wrinkle model which exploits local edge profile and geometric properties of wrinkles. To optimize the probability density of wrinkle model, we employ reversible jump Markov chain Monte Carlo sampler with delayed rejection. Experimental results demonstrate that the new algorithm detects facial wrinkles more accurately than a recent state-of-the-art method.

We propose a new spectral-spatial method for hyperspectral image classification based on a graph cut. The classification task is formulated as an energy minimization problem on the graph of image pixels, and is solved by using the graph-cut α-expansion approach. The energy to optimize is computed as a sum of data and interaction energy terms, respectively. The data energy term is computed using the outputs of the probabilistic support vector machines classification. The second energy term, which expresses the interaction between spatially adjacent pixels, is computed by using dissimilarity measures between spectral vectors, such as vector norms, spectral angle mapper and spectral information divergence. Experimental results on hyperspectral images captured by the RO-SIS and the AVIRIS sensors reveal that the proposed method yields higher classification accuracies when compared to the recent state-of-the-art approaches.

A Markov random field is a graphical model that is commonly used to combine spectral information and spatial context into image classification problems. The contributions of the spatial versus spectral energies are typically defined by using a smoothing parameter, which is often set empirically. We propose a new framework to estimate the smoothing parameter. For this purpose, we introduce the new concepts of dynamic blocks and class label cooccurrence matrices. The estimation is then based on the analysis of the balance of spatial and spectral energies computed using the spatial class co-occurrence distribution and dynamic blocks. Moreover, we construct a new spatially weighted parameter to preserve the edges, based on the Canny edge detector. We evaluate the performance of the proposed method on three data sets: a multispectral DigitalGlobe WorldView-2 and two hyperspectral images, recorded by the AVIRIS and the ROSIS sensors, respectively. The experimental results show that the proposed method succeeds in estimating the optimal smoothing parameter and yields higher classification accuracy values when compared with state-of-the-art methods.

We propose a new method for joint segmentation of monotonously growing or shrinking shapes in a time sequence of noisy images. The task of segmenting the image time series is expressed as an optimization problem using the spatio-temporal graph of pixels, in which we are able to impose the constraint of shape growth or of shrinkage by introducing monodirectional infinite links connecting pixels at the same spatial locations in successive image frames. The globally optimal solution is computed with a graph cut. The performance of the proposed method is validated on three applications: segmentation of melting sea ice floes and of growing burned areas from time series of 2D satellite images, and segmentation of a growing brain tumor from sequences of 3D medical scans. In the latter application, we impose an additional intersequences inclusion constraint by adding directed infinite links between pixels of dependent image structures.

We propose a new method based on graph cuts for joint segmentation of monotonously growing or shrinking shapes in time series of noisy images. By introducing directed infinite links connecting pixels at the same spatial locations in successive image frames, we impose shape growth/shrinkage constraint in graph cuts. Minimization of energy computed on the resulting graph of the image sequence yields globally optimal segmentation. We validate the proposed approach on two applications: segmentation of melting sea ice floes from a time series of multimodal satellite images and segmentation of a growing brain tumor from sequences of 3D multimodal medical scans. In the latter application, we impose an additional inter-sequences inclusion constraint by adding directed infinite links between pixels of dependent image structures.

We propose a new method based on graph cuts for the segmentation of burned areas in time series of satellite images. The method consists in rewriting a segmentation problem as a (s, t)-min-cut on the spatio-temporal image graph and computing this minimal cut. As burned areas grow in time, we introduce growth constraint in graph cuts by using directed infinite links connecting pixels at the same spatial locations in successive image frames. This method guarantees to find the globally optimal segmentation satisfying the growth constraint in small time complexity. Experimental results on a set of MODIS measurements over the Northern Australia demonstrated that the new approach succeeded in combining both spatial and temporal information for accurate segmentation of burned areas.

Recent advances in spectral-spatial classification of hyperspectral images are presented in this paper. Several techniques are investigated for combining both spatial and spectral information. Spatial information is extracted at the object (set of pixels) level rather than at the conventional pixel level. Mathematical morphology is first used to derive the morphological profile of the image, which includes characteristics about the size, orientation, and contrast of the spatial structures present in the image. Then, the morphological neighborhood is defined and used to derive additional features for classification. Classification is performed with support vector machines (SVMs) using the available spectral information and the extracted spatial information. Spatial postprocessing is next investigated to build more homogeneous and spatially consistent thematic maps. To that end, three presegmentation techniques are applied to define regions that are used to regularize the preliminary pixel-wise thematic map. Finally, a multiple-classifier (MC) system is defined to produce relevant markers that are exploited to segment the hyperspectral image with the minimum spanning forest algorithm. Experimental results conducted on three real hyperspectral images with different spatial and spectral resolutions and corresponding to various contexts are presented. They highlight the importance of spectral-spatial strategies for the accurate classification of hyperspectral images and validate the proposed methods.

The melting of sea ice is correlated to increases in sea surface temperature and associated climatic changes. Therefore, it is important to investigate how rapidly sea ice floes melt. For this purpose, a new TempoSeg method for multitemporal segmentation of multiyear ice floes is proposed. The microwave radiometer is used to track the position of an ice floe. Then, a time series of MODIS images are created with the ice floe in the image center. A TempoSeg method is performed to segment these images into two regions: Floe and Background. First, morphological feature extraction is applied. Then, the central image pixel is marked as Floe, and shape-constrained best merge region growing is performed. The resulting two-region map is post-filtered by applying morphological operators. We have successfully tested our method on a set of MODIS images and estimated the area of a sea ice floe as a function of time.

A new spectral-spatial method for classification of hyperspectral images is proposed. The HSegClas method is based on the integration of probabilistic classification and shape analysis within the hierarchical step-wise optimization algorithm. First, probabilistic support vector machines classification is applied. Then, at each iteration two neighboring regions with the smallest Dissimilarity Criterion (DC) are merged, and classification probabilities are recomputed. The important contribution of this work consists in estimating a DC between regions as a function of statistical, classification and geometrical (area and rectangularity) features. Experimental results are presented on a 102-band ROSIS image of the Center of Pavia, Italy. The developed approach yields more accurate classification results when compared to previously proposed methods.

Best merge region growing normally produces segmentations with closed connected region objects. Recognizing that spectrally similar objects often appear in spatially separate locations, we present an approach for tightly integrating best merge region growing with nonadjacent region object aggregation, which we call hierarchical segmentation or HSeg. However, the original implementation of nonadjacent region object aggregation in HSeg required excessive computing time even for moderately sized images because of the required intercomparison of each region with all other regions. This problem was previously addressed by a recursive approximation of HSeg, called RHSeg. In this paper, we introduce a refined implementation of nonadjacent region object aggregation in HSeg that reduces the computational requirements of HSeg without resorting to the recursive approximation. In this refinement, HSeg's region intercomparisons among nonadjacent regions are limited to regions of a dynamically determined minimum size. We show that this refined version of HSeg can process moderately sized images in about the same amount of time as RHSeg incorporating the original HSeg. Nonetheless, RHSeg is still required for processing very large images due to its lower computer memory requirements and amenability to parallel processing. We then note a limitation of RHSeg with the original HSeg for high spatial resolution images and show how incorporating the refined HSeg into RHSeg overcomes this limitation. The quality of the image segmentations produced by the refined HSeg is then compared with other available best merge segmentation approaches. Finally, we comment on the unique nature of the hierarchical segmentations produced by HSeg.

In this paper, a new method for supervised hyperspectral data classification is proposed. In particular, the notion of stochastic minimum spanning forest (MSF) is introduced. For a given hyperspectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule in order to build the final classification map. The proposed method is tested on three different data sets of hyperspectral airborne images with different resolutions and contexts. The influences of the number of markers and of the number of realizations M on the results are investigated in experiments. The performance of the proposed method is compared to several classification techniques (both pixelwise and spectral-spatial) using standard quantitative criteria and visual qualitative evaluation.

The Hierarchical SEGmentation (HSEG) algorithm, which is a combination of hierarchical step-wise optimization and spectral clustering, has given good performances for hyperspectral image analysis. This technique produces at its output a hierarchical set of image segmentations. The automated selection of a single segmentation level is often necessary. We propose and investigate the use of automatically selected markers for this purpose. In this paper, a novel Marker-based HSEG (M-HSEG) method for spectral-spatial classification of hyperspectral images is proposed. First, a map of markers is constructed using classification results. Then, a novel constrained M-HSEG algorithm is applied. The experimental results show that the proposed approach yields accurate segmentation and classification maps, and thus is attractive for hyperspectral image analysis.

A new method for supervised hyperspectral data classification is proposed. In particular, the notion of Stochastic Minimum Spanning Forests (MSFs) is introduced. For a given hyper-spectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule, resulting in a final classification map. The experimental results presented on an AVIRIS image of the vegetation area show that the proposed approach yields accurate classification maps, and thus is attractive for hyperspectral data analysis.

A new method for spectral-spatial classification of hyperspectral images is proposed. The method is based on the integration of probabilistic classification within the hierarchical best merge region growing algorithm. For this purpose, preliminary probabilistic support vector machines classification is performed. Then, hierarchical step-wise optimization algorithm is applied, by iteratively merging regions with the smallest Dissimilarity Criterion (DC). The main novelty of this method consists in defining a DC between regions as a function of region statistical and geometrical features along with classification probabilities. Experimental results are presented on a 200-band AVIRIS image of the Northwestern Indiana's vegetation area and compared with those obtained by recently proposed spectral-spatial classification techniques. The proposed method improves classification accuracies when compared to other classification approaches.

A new multiple classifier method for spectral-spatial classification of hyperspectral images is proposed. Several classifiers are used independently to classify an image. For every pixel, if all the classifiers have assigned this pixel to the same class, the pixel is kept as a marker, i.e., a seed of the spatial region, with the corresponding class label. We propose to use spectral-spatial classifiers at the preliminary step of the marker selection procedure, each of them combining the results of a pixel-wise classification and a segmentation map. Different segmentation approaches lead to different classification results. Furthermore, a minimum spanning forest is built, where each tree is rooted on a classification-driven marker and forms a region in the spectral-spatial classification map. Experimental results are presented on a 103-band ROSIS image of the University of Pavia, Italy. The proposed method significantly improves classification accuracies, when compared to previously proposed classification techniques.

The high number of spectral bands acquired by hyperspectral sensors increases the capability to distinguish physical materials and objects, presenting new challenges to image analysis and classification. This letter presents a novel method for accurate spectral-spatial classification of hyperspectral images. The proposed technique consists of two steps. In the first step, a probabilistic support vector machine pixelwise classification of the hyperspectral image is applied. In the second step, spatial contextual information is used for refining the classification results obtained in the first step. This is achieved by means of a Markov random field regularization. Experimental results are presented for three hyperspectral airborne images and compared with those obtained by recently proposed advanced spectral-spatial classification techniques. The proposed method improves classification accuracies when compared to other classification approaches.

A new multiple-classifier approach for spectral–spatial classification of hyperspectral images is proposed. Several classifiers are used independently to classify an image. For every pixel, if all the classifiers have assigned this pixel to the same class, the pixel is kept as a marker, i.e., a seed of the spatial region with a corresponding class label. We propose to use spectral–spatial classifiers at the preliminary step of the marker-selection procedure, each of them combining the results of a pixelwise classification and a segmentation map. Different segmentation methods based on dissimilar principles lead to different classification results. Furthermore, a minimum spanning forest is built, where each tree is rooted on a classification-driven marker and forms a region in the spectral–spatial classification map. Experimental results are presented for two hyperspectral airborne images. The proposed method significantly improves classification accuracies when compared with previously proposed classification techniques.

A new method for segmentation and classification of hyperspectral images is proposed. The method is based on the construction of a minimum spanning forest (MSF) from region markers. Markers are defined automatically from classification results. For this purpose, pixelwise classification is performed, and the most reliable classified pixels are chosen as markers. Each classification-derived marker is associated with a class label. Each tree in the MSF grown from a marker forms a region in the segmentation map. By assigning a class of each marker to all the pixels within the region grown from this marker, a spectral-spatial classification map is obtained. Furthermore, the classification map is refined using the results of a pixelwise classification and a majority voting within the spatially connected regions. Experimental results are presented for three hyperspectral airborne images. The use of different dissimilarity measures for the construction of the MSF is investigated. The proposed scheme improves classification accuracies, when compared to previously proposed classification techniques, and provides accurate segmentation and classification maps.

A new method for segmentation and classification of hyper-spectral images is proposed. The method is based on a pixel-wise classification followed by selection of the most reliable classified pixels as markers for watershed segmentation. Furthermore, each marker defined from classification results is associated with a class label. By assigning the class label of each marker to all the pixels within the region grown from this marker, a spectral-spatial classification map is obtained. Experimental results are presented on a 200-band AVIRIS image of the Northwestern Indiana's Indian Pine site. The developed segmentation and classification scheme significantly decreases oversegmentation, improves classification accuracies and provides classification maps with more homogeneous regions, when compared to pixel-wise classification or previously proposed spectral-spatial classification techniques.

A new spectral-spatial classification scheme for hyperspectral images is proposed. The method combines the results of a pixel wise support vector machine classification and the segmentation map obtained by partitional clustering using majority voting. The ISODATA algorithm and Gaussian mixture resolving techniques are used for image clustering. Experimental results are presented for two hyperspectral airborne images. The developed classification scheme improves the classification accuracies and provides classification maps with more homogeneous regions, when compared to pixel wise classification. The proposed method performs particularly well for classification of images with large spatial structures and when different classes have dissimilar spectral responses and a comparable number of pixels.

The paper presents a new segmentation and classification scheme to analyze hyperspectral (HS) data. The Robust Color Morphological Gradient of the HS image is computed, and the watershed transformation is applied to the obtained gradient. After the pixel-wise Support Vector Machines classification, the majority voting within the watershed regions is performed. Experimental results are presented on a 103-airborne ROSIS image, of the University of Pavia, Italy. The integration of the spatial information from the watershed segmentation into the HS image classification improves the classification accuracies, when compared to the pixel-wise classification.

Multivariate normal mixture models, where a complex statistical distribution is represented by a weighted sum of several multivariate normal probability distributions, have many potential applications including anomaly detection (AD) in hyperspectral (HS) images. The high computational cost of mixture models requires hardware and/or algorithmic acceleration to make AD run in real time. In this paper we describe the concurrency present in the AD algorithm that includes a normal mixture estimation task. We explore the use of graphics processing units (GPUs) for parallel implementation of the algorithm. The GPU implementations provide a significant speedup compared to multi-core central processing unit (CPU) implementations, and enable the algorithm to execute in real time.

Lip reading relies on visible articulators to ease audiovisual speech understanding.However, lips and face alone provide very incomplete phonetic information: the tongue, that is generally not entirely seen, carries an important part of the articulatory information not accessible through lip reading.The question was thus whether the direct and full vision of the tongue allows tongue reading.We have therefore generated a set of audiovisual VCV stimuli by controlling an audiovisual talking head that can display all speech articulators, including tongue, in an augmented speech mode, from articulators movements tracked on a speaker.These stimuli have been played to subjects in a series of audiovisual perception tests in various presentation conditions (audio signal alone, audiovisual signal with profile cutaway display with or without tongue, complete face), at various Signal-to-Noise Ratios.The results show a given implicit effect of tongue reading learning, a preference for the more ecological rendering of the complete face in comparison with the cutaway presentation, a predominance of lip reading over tongue reading, but the capability of tongue reading to take over when the audio signal is strongly degraded or absent.We conclude that these tongue reading capabilities could be used for applications in the domain of speech therapy for speech retarded children, perception and production rehabilitation of hearing impaired children, and pronunciation training for second language learners.