Convolutional neural networks (CNNs) learn filters in order to capture local correlation patterns in feature space. We propose to learn these filters as combinations of preset spectral filters defined by the Discrete Cosine Transform (DCT). Our proposed DCT-based harmonic blocks replace conventional convolutional layers to produce partially or fully harmonic versions of new or existing CNN architectures. Using DCT energy compaction properties, we demonstrate how the harmonic networks can be efficiently compressed by truncating high-frequency information in harmonic blocks thanks to the redundancies in the spectral domain. We report extensive experimental validation demonstrating benefits of the introduction of harmonic blocks into state-of-the-art CNN models in image classification, object detection and semantic segmentation applications.

We show how parameter redundancy in Convolutional Neural Network (CNN) filters can be effectively reduced by pruning in spectral domain. Specifically, the representation extracted via Discrete Co-sine Transform (DCT) is more conducive for pruning than the original space. By relying on a combination of weight tensor reshaping and reordering we achieve high levels of layer compression with just minor accuracy loss. Our approach is applied to compress pre-trained CNNs and we show that minor additional fine-tuning allows our method to recover the original model performance after a significant parameter reduction. We validate our approach on ResNet-50 and MobileNet-V2 architectures for ImageNet classification task.

Road information extraction based purely on remote sensing can be affected by occlusions of the road surface caused by trees, shadows, and buildings. We propose a multimodal fusion method that addresses road extraction and road width estimation by combining aerial imagery, monocular images taken at ground level (street-level), and geospatial data (Open-StreetMap). The method combines semantic segmentation through convolutional neural networks, Voronoi diagram processing, and graph matching.

Abundant image and sensory data collected over the last decades represents an invaluable source of information for cataloging and monitoring of the environment. Fusion of heterogeneous data sources is a challenging but promising tool to efficiently leverage such information. In this work we propose a pipeline for automatic detection and geolocation of recurring stationary objects deployed on fusion scenario of street level imagery and LiDAR point cloud data. The objects are geolocated coherently using a fusion procedure formalized as a Markov random field problem. This allows us to efficiently combine information from object segmentation, triangulation, monocular depth estimation and position matching with LiDAR data. The proposed fusion approach produces object mappings robust to scenes reporting multiple object instances. We introduce a new challenging dataset of over 200 traffic lights in Dublin city centre and demonstrate high performance of the proposed methodology and its capacity to perform multi -sensor data fusion.

The availability of synthetic aperture radar (SAR) data with high spatial resolution offers great potential for environmental monitoring due to the insensitivity of SAR to atmospheric and sunlight-illumination conditions. In this paper, an unsupervised change detection method for SAR images at medium to high resolution is proposed. The image ratioing approach is adopted, and a Bayesian unsupervised minimum-error thresholding algorithm is extended by proposing a technique based on Generalized Gamma distributions (GΓD). GΓD was recently found to be an accurate model for the statistics of SAR amplitudes at moderate to high resolution. Here, a specific parametric modeling approach for the ratio of G D-distributed SAR images is proposed and endowed with a probability density function estimation algorithm based on the method of log-cumulants. Consistency of this estimator is proven. Experimental results confirm the accuracy of the method for medium and high resolutions X-band SAR images.

In this paper we address the problem of urban optical imagery classification by developing a convolutional neural network (CNN) approach. We design a custom CNN that operates on local patches in order to produce dense pixel-level classification map. In this work we focus on a comprehensive dataset of 2.5-meter SPOT-5 imagery acquired at different dates and sites. The performance of the proposed model is validated on a five target-class problem and compared with a benchmark random forest classifier with a set of hand-picked features.

The accurate statistical modeling of synthetic aperture radar (SAR) images is a crucial problem in the context of effective SAR image processing, interpretation, and application. In this paper, a semi-parametric approach is designed within the framework of finite mixture models based on the generalized Gamma distribution in view of its flexibility and compact form. Specifically, we develop a generalized Gamma mixture model to implement an effective statistical analysis of high-resolution SAR images and prove the identifiability of such mixtures. A low-complexity unsupervised estimation method is derived by combining the proposed histogram-based expectation-conditional maximization algorithm and the Figueiredo-Jain algorithm. This results in a numerical maximum-likelihood (ML) estimator that can simultaneously determine the ML estimates of component parameters and the optimal number of mixture components. Finally, the state-of-the-art performance of this proposed method is verified by experiments with a wide range of high-resolution SAR images.

In this paper, we address the problem of unsupervised change detection on two or more coregistered images of the same object or scene at several time instants. We propose a novel empirical-Bayesian approach that is based on a false discovery rate formulation for statistical inference on local patch-based samples. This alternative error metric allows to efficiently adjust the family-wise error rate in case of the considered large-scale testing problem. The designed change detector operates in an unsupervised manner under the assumption of the limited amount of changes in the analyzed imagery. The detection is based on the use of various statistical features, which enable the detector to address application-specific detection problems provided an appropriate ad hoc feature choice. In particular, we demonstrate the use of the rank-based statistics: Wilcoxon and Cramér-von Mises for image pairs, and multisample Levene statistic for short image sequences. The experiments with remotely sensed radar, dermatological, and still camera surveillance imagery demonstrate accurate performance and flexibility of the proposed method.

In this paper, we develop a novel classification approach for multiresolution, multisensor [optical and synthetic aperture radar (SAR)], and/or multiband images. This challenging image processing problem is of great importance for various remote sensing monitoring applications and has been scarcely addressed so far. To deal with this classification problem, we propose a two-step explicit statistical model. We first design a model for the multivariate joint class-conditional statistics of the coregistered input images at each resolution by resorting to multivariate copulas. Such copulas combine the class-conditional marginal probability density functions (pdfs) of each input channel that are estimated by finite mixtures of well-chosen parametric families. We consider different distribution families for the most common types of remote sensing imagery acquired by optical and SAR sensors. We then plug the estimated joint pdfs into a hierarchical Markovian model based on a quad-tree structure, where each tree-scale corresponds to the different input image resolutions and to corresponding multiscale decimated wavelet transforms, thus preventing a strong resampling of the initial images. To obtain the classification map, we resort to an exact estimator of the marginal posterior mode. We integrate a prior update in this model in order to improve the robustness of the developed classifier against noise and speckle. The resulting classification performance is illustrated on several remote sensing multiresolution data sets, including very high resolution and multisensor images acquired by COSMO-SkyMed and GeoEye-1.

The automatic extraction of elongated curvilinear structures (CLSs) is an important task in various image processing applications, including numerous remote sensing, and biometrical and medical problems. To address this task, we develop a stochastic approach that relies on a fixed-grid, localized Radon transform for line segment extraction and a conditional random field model to incorporate local interactions and refine the extracted CLSs. We propose several different energy data terms, the appropriate choice of which allows us to process images with different noise and geometry properties. The contribution of this paper is the design of a flexible and robust elongated CLS extraction framework that is comparatively fast due to the use of a fixed-grid configuration and fast deterministic Radon-based line detector. We present several different applications of the developed approach, namely: 1) CLS extraction in mammographic images; 2) road networks extraction from optical remotely sensed images; and 3) line extraction from palmprint images. The experimental results demonstrate that the method is fairly robust to CLS curvature and can accurately extract blurred and low-contrast elongated CLS.

In this paper we address the problem of unsupervised change detection on image pairs. We develop a novel patch-based hypothesis testing approach that employs the false discovery rate technique for statistical hypothesis testing. The designed approach can be adopted to specific detection applications via selection of appropriate statistical features. Experiments with still camera imagery demonstrate high performance and flexibility of the proposed method.

This letter addresses the problem of classifying synthetic aperture radar (SAR) images of urban areas by using a supervised Bayesian classification method via a contextual hierarchical approach. We develop a bivariate copula-based statistical model that combines amplitude SAR data and textural information, which is then plugged into a hierarchical Markov random field model. The contribution of this letter is thus the development of a novel hierarchical classification approach that uses a quad-tree model based on wavelet decomposition and an innovative statistical model. The performance of the developed approach is illustrated on a high-resolution satellite SAR image of urban areas.

Parameter estimation of probability density functions is one of the major steps in the area of statistical image and signal processing. In this paper we explore several properties and limitations of the recently proposed method of logarithmic cumulants (MoLC) parameter estimation approach which is an alternative to the classical maximum likelihood (ML) and method of moments (MoM) approaches. We derive the general sufficient condition for a strong consistency of the MoLC estimates which represents an important asymptotic property of any statistical estimator. This result enables the demonstration of the strong consistency of MoLC estimates for a selection of widely used distribution families originating from (but not restricted to) synthetic aperture radar image processing. We then derive the analytical conditions of applicability of MoLC to samples for the distribution families in our selection. Finally, we conduct various synthetic and real data experiments to assess the comparative properties, applicability and small sample performance of MoLC notably for the generalized gamma and K families of distributions. Supervised image classification experiments are considered for medical ultrasound and remote-sensing SAR imagery. The obtained results suggest that MoLC is a feasible and computationally fast yet not universally applicable alternative to MoM. MoLC becomes especially useful when the direct ML approach turns out to be unfeasible.

This paper considers change detection with multitemporal synthetic aperture radar (SAR) images. A novel statistical approach is developed that formulates the detection of changes between two coregistered SAR images acquired at different dates as a non-parametric local-window hypothesis testing problem. This approach takes into account the simultaneous testing of a large amount of similar hypotheses (i.e., one Wilcoxon test at each pixel of the image), and is based on a likelihood ratio test to obtain a detection map at the pixel level. Some encouraging detection results are obtained on XSAR and very high resolution COSMO-SkyMed images.

In the framework of the monitoring of structures and infrastructures from environmental disasters, the COSMO-SkyMed constellation has a huge potential, thanks to up to metric spatial resolution, short revisit time, and the day/night all-weather acquisition capability ensured by SAR. This paper focuses on the scientific results of the project “Development and validation of multitemporal image analysis methodologies for multirisk monitoring of critical structures and infrastructures,” funded by the Italian Space Agency. Several change-detection, data-fusion, and feature-extraction techniques, which were developed and experimentally validated in the project for COSMO-SkyMed imagery and for their integration with other data sources (including very high resolution optical data), are described and examples of processing results are discussed.

In this letter, we address the problem of estimating the amplitude probability density function (pdf) of single-channel synthetic aperture radar (SAR) images. A novel flexible method is developed to solve this problem, extending the recently proposed dictionary-based stochastic expectation maximization approach (developed for a medium-resolution SAR) to very high-resolution (VHR) satellite imagery, and enhanced by introduction of a novel procedure for estimating the number of mixture components, that permits to reduce appreciably its computational complexity. The specific interest is the estimation of heterogeneous statistics, and the developed method is validated in the case of the VHR SAR imagery, acquired by the last-generation satellite SAR systems, TerraSAR-X and COSMO-SkyMed. This VHR imagery allows the appreciation of various ground materials resulting in highly mixed distributions, thus posing a difficult estimation problem that has not been addressed so far. We also conduct an experimental study of the extended dictionary of state-of-the-art SAR-specific pdf models and consider the dictionary refinements.