Compared to standard deep convolutional neural networks (CNN) which include a global average pooling operator, second-order neural networks have a global covariance pooling operator which allows to capture richer statistics of CNN features. They have been shown to improve representation and generalization abilities. However, this covariance pooling is performed only on the deepest CNN feature maps. To benefit from different levels of abstraction, we propose to extend these models by using a multi-layer approach. In addition, to obtain better predictive performance, an end-to-end ensemble learning architecture is proposed. Experiments are conducted on four datasets and have confirmed the potential of the proposed model for various image processing applications such as remote sensing scene classification, indoor scene recognition and texture classification.

The pansharpening problem amounts to fusing a high-resolution panchromatic image with a low-resolution multispectral image so as to obtain a high-resolution multispectral image. Therefore, the preservation of the spatial resolution of the panchromatic image and the spectral resolution of the multispectral image is of key importance for the pansharpening problem. To cope with it, we propose a new method based on a bidiscriminator in a generative adversarial network (GAN) framework. The first discriminator is optimized to preserve textures of images by taking as input the luminance and the near-infrared band of images, and the second discriminator preserves the color by comparing the chroma components Cb and Cr. Thus, this method allows to train two discriminators, each one with a different and complementary task. Moreover, to enhance these aspects, the proposed method based on bidiscriminator, and called MDSSC-GAN SAM, considers a spatial and a spectral constraint in the loss function of the generator. We show the advantages of this new method on experiments carried out on Pléiades and World View 3 satellite images.

The launch of the last generation of Earth observation satellites has yield to a great improvement in the capabilities of acquiring Earth surface images, providing series of multitemporal images. To process these time series images, many machine learning algorithms have been proposed in the literature such as warping based methods and recurrent neural networks (LSTM,...). Recently, based on an ensemble learning approach, the time series cluster kernel (TCK) has been proposed and has shown competitive results compared to the state-of-the-art. Unfortunately, it does not model the spectral/spatial dependencies. To overcome this problem, this paper aims at extending the TCK approach by modeling the time series of second-order statistical features (SO-TCK). Experimental results are conducted on different benchmark datasets, and land cover classification with remote sensing satellite time series over the Reunion Island.

The pansharpening problem consists in fusing a high resolution panchromatic image with a low resolution multispectral image in order to obtain a high resolution multispectral image. In this paper, we adapt a Residual Dense architecture for the generator in a Generative Adversarial Network framework. Indeed, this type of architecture avoids the vanishing gradient problem faced when training a network by re-injecting previous information thanks to dense and residual connections. Moreover, an important point for the pansharpening problem is to preserve the geometry of the image. Hence, we propose to add a regularization term in the loss function of the generator: it preserves the geometry of the target image so that a better solution is obtained. In addition, we propose geometrical measures that illustrate the advantages of this new method.

This paper presents a novel framework for visual content classification using jointly local mean vectors and covariance matrices of pixel level input features. We consider local mean and covariance as realizations of a bivariate Riemannian Gaussian density lying on a product of submanifolds. We first introduce the generalized Mahalanobis distance and then we propose a formal definition of our product-spaces Gaussian distribution on Rm× SPD(m). This definition enables us to provide a mixture model from a mixture of a finite number of Riemannian Gaussian distributions to obtain a tractable descriptor. Mixture parameters are estimated from training data by exploiting an iterative Expectation-Maximization (EM) algorithm. Experiments in a texture classification task are conducted to evaluate this extended modeling on several color texture databases, namely popular Vistex, 167-Vistex and CUReT. These experiments show that our new mixture model competes with state-of-the-art on the experimented datasets.

Conventional feature extraction methods cannot fully exploit both the spectral and spatial information of hyperspectral imagery. In this paper, we propose an ensemble method of subspace independent component analysis (ICA) and edge-preserving filtering (EPF) for the classification of hyper-spectral data to achieve this task. First, several subsets are randomly selected from the original feature space. Second, ICA is used to extract spectral independent components followed by a recent and effective EPF method, rolling guidance filter (RGF), to produce spatial features. The spatial features are treated as the input of a random forest (RF) classifier. Finally, the classification results from each subset are integrated together to produce the final map. Experimental results on real hyperspectral data demonstrate the effectiveness of the proposed method. A sensitivity analysis of this new classifier is also performed.

This paper introduces a generalization of the Fisher vectors to the Riemannian manifold. The proposed descriptors, called Riemannian Fisher vectors, are defined first, based on the mixture model of Riemannian Gaussian distributions. Next, their expressions are derived and they are applied in the context of texture image classification. The results are compared to those given by the recently proposed algorithms, bag of Riemannian words and R-VLAD. In addition, the most discriminant Riemannian Fisher vectors are identified.

Images are usually represented by different groups of features, such as color, shape and texture attributes. In this paper, we propose a classification approach that integrates multiple features, such as spectral and spatial information. We refer this approach to multiple feature learning via rotation (MFL-R) strategy, which adopt a rotation-based ensemble method by using a data transformation approach. Five data transformation methods, including principal component analysis (PCA), neighborhood preserving embedding (NPE), linear local tangent space alignment (LLTSA), linearity preserving projection (LPP) and multiple feature combination via manifold learning and patch alignment (MLPA) are used in the MFL-R framework. Experimental results over two hyperspectral remote sensing images demonstrate that MFL-R with MLPA gains better performances and is not sensitive to the tuning parameters.

Rotation Forest (RoF) is a decision tree ensemble classifier, which uses random feature selection and data transformation techniques to improve both the diversity and accuracy of base classifiers. Traditional RoF only considers data transformation on spectral information. In order to further improve the performance of RoF, we introduce spectral-spatial data transformation into RoF and thus propose a spectral-spatial Rotation Forest (SSRoF). The proposed method is experimentally investigated on a hyperspectral remote sensing image collected by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. Experimental results indicate that the proposed methodology achieves excellent performance.

This letter introduces a new robust estimation method for the central value of a set of N covariance matrices. This estimator, called Huber's centroid, is described starting from the expression of two well-known methods that are the center of mass and the median. In addition, a computation algorithm based on the gradient descent is proposed. Moreover, Huber's centroid performances are analyzed on simulated data to identify the impact of outliers on the estimation process. In the end, the algorithm is applied to brain decoding, based on magnetoencephalography data. For both simulated and real data, the covariance matrices are considered as realizations of Riemannian Gaussian distributions and the results are compared to those given by the center of mass and the median.

In this paper, we explore the potentialities of using wavelet-based multivariate models for the classification of very high resolution optical images. A strategy is proposed to apply these models in a supervised classification framework. This strategy includes a content-based image retrieval analysis applied on a texture database prior to the classification in order to identify which multivariate model performs the best in the context of application. Once identified, the best models are further applied in a supervised classification procedure by extracting texture features from a learning database and from regions obtained by a presegmentation of the image to classify. The classification is then operated according to the decision rules of the chosen classifier. The use of the proposed strategy is illustrated in two real case applications using Pléiades panchromatic images: the detection of vineyards and the detection of cultivated oyster fields. In both cases, at least one of the tested multivariate models displays higher classification accuracies than gray-level cooccurrence matrix descriptors. Its high adaptability and the low number of parameters to be set are other advantages of the proposed approach.

To obtain accurate classification results of hyperspectral images, both spectral and spatial information should be fully exploited in the classification process. In this paper, we propose a novel method using independent component analysis (ICA) and edge-preserving filtering (EPF) via an ensemble strategy for the classification of hyperspectral data. First, several subsets are randomly selected from the original feature space. Second, ICA is used to extract spectrally independent components followed by an effective EPF method, to produce spatial features. Two strategies (i.e., parallel and concatenated) are presented to include the spatial features in the analysis. The spectral-spatial features are then classified with a random forest or a rotation forest classifier. Experimental results on two real hyperspectral data sets demonstrate the effectiveness of the proposed methods. A sensitivity analysis of the new classifiers is also performed.

This paper introduces a new statistical hypothesis test for robust image classification. First, we introduce the proposed statistical hypothesis test based on the geodesic distance and on the fixed point estimation algorithm. Next, we analyze its properties in the case of the zero-mean multivariate Gaussian distribution by studying its asymptotic distribution under the null hypothesis H0. Then, the performance of the proposed classifier is addressed by analyzing its noise robustness. Finally, the robust classification method is employed for the classification of simulated Polarimetric Synthetic Aperture Radar images of maritime pine forests.

This work investigates the discriminative power of wavelet decomposition based texture features in forest cover classification. Our texture features are used as inputs in a random forests classifier. The performances of this tree-based ensemble classifier are assessed by classification accuracy as well as classification confidence provided by an unsupervised version of ensemble margin. The effectiveness of the proposed texture based multiple classifier system is demonstrated in performing mapping of very high resolution forest imagery. Traditional grey level co-occurrence matrix derived texture features are also evaluated through our ensemble classification framework for comparison.

This paper introduces a new statistical hypothesis test for image classification based on the geodesic distance. We present how it can be used for the classification of texture image. The proposed method is then employed for the classification of Polarimetric Synthetic Aperture Radar images of maritime pine forests on both simulated data with the PolSARproSim software and real data acquired during the ONERA RAMSES campaign in 2004.

In this study, we propose to evaluate the potential of combining very high resolution optical and SAR images for the classification of oyster habitats in tidal flats. To describe the classes of interest in both data, features are extracted by using wavelet-based texture features and polarimetric inter-band dependencies. A multisensor fusion scheme is then applied by adopting a maximum probability rule based on the outputs of SVM classifiers. Classification results show higher accuracies of detection of cultivated and abandoned oyster fields in comparison to classifications obtained using only texture features. This demonstrate the benefit of using both optical and SAR data for oyster habitats mapping in tidal flats.

This letter evaluates the potential of wavelet-based texture modeling for the classification of stand age in a managed maritime pine forest using very high resolution panchromatic and multispectral PLEIADES data. A cross-validation approach based on stand age reference data is used to compare classification performances obtained from different multivariate models (multivariate Gaussian, spherically invariant random vector (SIRV)-based models, and Gaussian copulas) and from co-occurrence matrices. Results show that the multivariate modeling of the spatial dependence of wavelet coefficients (particularly when using the Gaussian SIRV model) outperforms the use of features derived from co-occurrence matrices. Simultaneously adding features representing the color dependence and leveling the dominant orientation in anisotropic forest stands enhances the classification performances. These results confirm the ability of such wavelet-based multivariate models to efficiently capture the textural properties of very high resolution forest data and open up perspectives for their use in the mapping of monospecific forest structure variables.

This paper proposes a two-stage texture synthesis algorithm. At the first stage, a structure tensor map carrying information about the local orientation is synthesized from the exemplar's data and used at the second stage to constrain the synthesis of the texture. Keeping in mind that the algorithm should be able to reproduce as faithfully as possible the visual aspect, statistics, and morphology of the input sample, the method is tested on various textures and compared objectively with existing methods, highlighting its strength in successfully synthesizing the output texture in many situations where traditional algorithms fail to reproduce the exemplar's patterns. The promising results pave the way towards the synthesis of accurately large and multi-scale patterns as it is the case for carbon material samples showing laminar structures, for example.

Exemplar-based texture synthesis schemes are promising for virtual material design. They provide impressive results in many cases, but fail in difficult situations with large and multi-scale patterns, or with long range directional variations. Since a prior synthesis of a geometric layer may help in the synthesis of the texture layer, a two-stage structure/texture synthesis algorithm is proposed. At the first stage, a structure tensor map carrying information about the local orientation is synthesized from the exemplar's data, and at the second stage, the synthesized tensor field is used to constrain the synthesis of the texture. Results show that the proposed approach not only yields better synthesized textures, but also successfully synthesizes the output texture in many situations where traditional algorithms fail to reproduce the exemplar's patterns, which paves the way towards the synthesis of accurately large and multi-scale patterns as it is the case for pyrolytic carbon samples showing laminar structures observed by Transmission Electronic Microscopy.

This study evaluates the potential of wavelet-based texture modeling for the classification of stand age in a managed maritime pine forest using very high resolution satellite data. A cross-validation approach based on stand age reference data shows that multivariate modeling of the spatial dependence of wavelet coefficients outperforms the use of features derived from co-occurrence matrices. Simultaneously adding features representing the color dependence and leveling the dominant orientation in anisotropic forest stands enhances the classification performances. These results obtained from panchromatic and multispectral PLEIADES data confirm the ability of such wavelet-based multivariate models to efficiently capture the textural properties of very high resolution forest data and opens up perspectives for their use in the mapping of mono-specific forest structure variables.

This study evaluates the potential of wavelet-based texture multivariate modeling for the detection of cultivated oyster fields and their differentiation from abandoned fields in Very High Resolution panchromatic PLEIADES data. The proposed models are tested in a supervised classification context using a training database composed of extracted image patches representative of the land covers of interest. The obtained classification results exhibit high detection rate for cultivated fields. Classification errors appear however in the detection of abandoned fields. These results demonstrate the interest of such model for the characterization of inter-tidal ecosystems and opens up perspectives for their use in the management of oyster farming activities.

Data on forest variables (e.g., biomass, trunk height, density) are necessary for environmental and forest management applications. It has been shown that texture can be used instead of the usual σo/age relationships at P-band to retrieve plantation forest parameters, but the analysis of σo spatial characteristics has not been fully explored. The aim of this letter is to investigate the relationships between stand age (which is correlated to forest variables) and texture descriptors calculated from statistics generated by the gray-level co-occurrence matrix for varying distance d, and orientation α, values used to calculate the matrix. Synthetic aperture radar images are P-band airborne data acquired by the ONERA RAMSES instrument over a controlled homogeneous test site located in the Landes region, France. It is found that texture descriptors contrast, inverse difference moment, homogeneity, and correlation are strongly influenced by the parameters (d, α) related to forest stand structure (forest rows, stand density) and image resolution. In contrast, energy and entropy are observed to be highly correlated to stand age and displayed a stable performance whatever the distance and orientation parameters (d, α), thus rendering them a good contender as an alternative to the usual σo based relationships applied to this type of forest.

We propose a genuine 3D texture synthesis algorithm based on a probabilistic 2D Markov random field conceptualization, capable of capturing the visual characteristics of a texture into a unique statistical texture model. We intend to reproduce, in the volumetric texture, the interactions between pixels learned in an input 2D image. The learning is done by nonparametric Parzen-windowing. Optimization is handled voxel by a relaxation algorithm, aiming at maximizing the likelihood of each voxel in terms of its local conditional probability function. Variants are proposed regarding the relaxation algorithm and the heuristic strategies used for the simultaneous handling of the orthogonal slices containing the voxel. The procedures are materialized on various textures through a comparative study and a sensitivity analysis, highlighting the variants strengths and weaknesses. Finally, the probabilistic model is compared objectively with a nonparametric neighborhood-search-based algorithm.

We propose a genuine 3D texture synthesis algorithm based on a probabilistic 2D Markov Random Field conceptualization. We intend to reproduce, in the volumetric texture, the interactions between pixels learned non-parametrically in an input 2D image. Optimization is handled voxel by voxel by a deterministic algorithm, deciding for each voxel the most probable grey level. Variants are proposed regarding the heuristic strategies used for the simultaneous handling of the orthogonal slices containing the voxel. Voxel relaxation and scale handling policies are al so investigated. The procedures are materialized on various textures to identify their strengths and weaknesses.

Textural analysis can bring valuable information in the classification or the segmentation process of land covers displaying regular patterns in very high resolution remotely sensed images. In this study, we investigate how features extracted by multivariate modeling of the local spatial dependence in the wavelet domain can efficiently capture the textural content of maritime pine forest images in comparison with a commonly used texture analysis approach, the GLCM. To evaluate the performances of the tested methods, we used a content based image retrieval framework and created a database of image patches representing different development stages of the forest stands. Results show that multivariate models display higher retrieval rates than GLCM-based methods with yet a higher sensitivity to the dominant orientation in anisotropic textures. These observations open up new perspectives in the use of multivariate modeling for textural features extraction in very high resolution image classification.

The goal of this paper is to evaluate non-parametric algorithms that achieve 3D texture synthesis from a single 2D sample. The algorithms under study are variants of the algorithm proposed by Wei and Levoy [1]. Several authors have proposed different algorithms that intend to better reproduce, in the output texture, the diversity learned in the input sample. Hence, we turn our attention to the improved algorithm proposed by Kopf et al. [2] and the particular histogram matching approach of Chen and Wang [3]. In addition, we propose to visit the voxels during synthesis according to the 3D extension of space filling curves. We investigate the algorithms capability to reproduce anisotropic textures. In particular we are interested in laminar textures, i.e. textures made of anisotropic sheets stacked along a given direction. Examples of such textures are snapshots of dense carbons observed by high resolution transmission electronic microscopy (HRTEM). Beyond the traditional subjective evaluation of the synthesized textures, we propose a genuine quantitative benchmark for the analysis of the synthesized textures which consists in comparing input and output gray level statistics and patterns morphology.

In this paper, we investigate both linear and circular stochastic models in the context of texture discrimination. These models aim at representing the magnitudes and orientations obtained by a complex wavelet decomposition, such as the steerable pyramid.The novelty consists in considering specific parametric models for circular data such as von Mises and psi- distributions to describe the distributions of orientations. Particular attention is paid to the choice of a metric and to its adequation to the models. Indexing experiments are conducted to quantitatively evaluate the performances of the proposed models and of the chosen matrices, i.e. the L1and Kullback-Leibler distances.

We introduce a new anisotropic diffusion based approach for image filtering and enhancement. The main novelty of our method consists in considering a modulo 2¿ based definition of the diffusion axis along the main directions of the processed image. Such an asymmetric formulation of the filter induces better junction and corner preservation properties coupled with a good noise removal capacity.

Given textured images considered as realizations of 2-D stochastic processes, a framework is proposed to evaluate the stationarity of their mean and variance. Existing strategies focus on the asymptotic behavior of the empirical mean and variance (respectively EM and EV), known for some types of nondeterministic processes. In this paper, the theoretical asymptotic behaviors of the EM and EV are studied for large classes of second-order stationary ergodic processes, in the sense of the Wold decomposition scheme, including harmonic and evanescent processes. Minimal rates of convergence for the EM and the EV are derived for these processes; they are used as criteria for assessing the stationarity of textures. The experimental estimation of the rate of convergence is achieved using a nonparametric block sub-sampling method. Our framework is evaluated on synthetic processes with stationary or nonstationary mean and variance and on real textures. It is shown that anomalies in the asymptotic behavior of the empirical estimators allow detecting nonstationarities of the mean and variance of the processes in an objective way.

This paper focuses on directional textures. It provides a new framework for the design of convolution masks dedicated to orientation estimation. We propose a new technique based on the combination of two complementary operators: a gradient-based operator which is adapted to sloped regions and a valleyness detector which fits the crests and valleys. On each operator, a double optimization procedure is carried out with respect to bias and noise sensitivity reduction. The procedure is generic and applies to any kind of underlying directional texture. Experiments on a synthetic sine wave texture and on natural textures are provided and show the efficiency and the relevance of our approach.

This paper proposes new estimators for the local orientation of elongated structures. Classical methods derive local orientation from the argument of gradient estimator. As they were originally designed for edge detection, such operators can lead to high bias in orientation estimation. We introduce the line orientation operator which applies an oriented model made with straight parallel lines on the local neighborhood. We present results either on synthetic and real images. We compare this new operator with other classical orientation operators.

Topographic approaches are often used in the framework of texture characterization. More particularly, level curves proved to be interesting features to describe textures containing elongated patterns. Here, we provide an algorithm for level curve tracking based on a step by step propagation of a level set from a pixel to its neighbors. Textural feature extraction is then considered; orientation and pattern length measurement techniques are proposed. Pattern length retrieval is then exercised on composite material images taken from transmission electron microscopy.

This paper deals with the characterization of textures. It focuses on textural properties based on the local differences of attributes, such as homogeneity and orientation. We introduce the scaled scattering indicator (SSI) in order to estimate such textural properties. SSI is computed at various observation scales to take into account micro-textures as well as macro-textures involved in pictures. We establish the theoretical behavior of SSI in the case of a pavement filled with a given textural pattern. We devote this indicator to the multiscale characterization of the anisotropy of pictures. Simulation results on synthetic and real pictures confirm the the theoretical approach we have provided: SSI provides an accurate estimation of textural properties like anisotropy, but also gives an estimation of the size of the textural patterns involved in the picture.

The estimation of the textural features of an object is a classical problem for image processing. When directional textures occur, texture anisotropy is one of the most relevant textural features that can be extracted in order to identify the physical properties of the studied object. Nevertheless, we show that this property is strongly linked with the scale of observation of the texture. For this purpose, in this paper, we introduce a method for the estimation of texture orientation at a given scale and the corresponding anisotropy feature. Some results are given, based both on natural and synthetic pictures.