Online distributed optimization is particularly useful for solving optimization problems with streaming data collected by multiple agents over a network. When the solutions lie on a Riemannian manifold, such problems become challenging to solve, particularly when efficiency and continuous adaptation are required. This work tackles these challenges and devises a diffusion adaptation strategy for decentralized optimization over general manifolds. A theoretical analysis shows that the proposed algorithm is able to approach network agreement after sufficient iterations, which allows a non-asymptotic convergence result to be derived. We apply the algorithm to the online decentralized principal component analysis problem and Gaussian mixture model inference. Experimental results with both synthetic and real data illustrate its performance.

Distributed adaptation and learning recently gained considerable attention in solving optimization problems with streaming data collected by multiple agents over a graph. This work focuses on such problems where the solutions lie on a Riemannian manifold. This research topic is of particular interest for many applications, e.g., principal component analysis (PCA). Although several incremental and consensus algorithms have been proposed, there is a lack of methods designed for general Riemannian manifolds with efficient diffusion strategies. In this paper, we devise two Riemannian diffusion adaptation strategies, namely, adaptation-then-combination (ATC) and combination-then-adaptation (CTA), for decentralized Riemannian optimization over graphs. In the adaptation step, a Riemannian stochastic gradient descent method (SGD) is used to estimate the local solution at each node. In the combination step, the local estimates at the different nodes are combined by computing the weighted Fréchet mean over the neighborhood of each node. We apply our algorithms to online distributed PCA and compare them to both non-cooperative and centralized solutions.

Non-parametric detection of change points in streaming time series data that belong to Euclidean spaces has been extensively studied in the literature. Nevertheless, when the data belongs to a Riemannian manifold, existing approaches are no longer applicable as they fail to account for the structure and geometry of the manifold. In this paper, we introduce a non-parametric algorithm for online change point detection in manifold-valued data streams. This algorithm monitors the generalized Karcher mean of the data, computed using stochastic Riemannian optimization. We provide theoretical bounds on the detection and false alarm rate performances of the algorithm, using a new result on the non-asymptotic convergence of the stochastic Riemannian gradient descent. We apply our algorithm to two different Riemannian manifolds. Experimental results with both synthetic and real data illustrate the performance of the proposed method.

Detecting change points in streaming time series data is a long standing problem in signal processing. A plethora of methods have been proposed to address it, depending on the hypotheses at hand. Non-parametric approaches are particularly interesting as they do not make any assumption on the distribution of data or on the nature of changes. Nevertheless, leveraging recent advances in deep learning to detect change points in time series data is still challenging. In this paper, we propose a change point detection method using an online approach based on neural networks to directly estimate the density-ratio between current and reference windows of the data stream. A variational continual learning framework is employed to train the neural network in an online manner while retaining information learned from past data. This leads to a statistically-principled fully nonparametric framework to detect change points from streaming data. Experimental results with synthetic and real data illustrate the effectiveness of the proposed approach.

Deconvolution is a widely used strategy to mitigate the blurring and noisy degradation of hyperspectral images (HSI) generated by the acquisition devices. This issue is usually addressed by solving an ill-posed inverse problem. While investigating proper image priors can enhance the deconvolution performance, it is not trivial to handcraft a powerful regularizer and to set the regularization parameters. To address these issues, in this paper we introduce a tuning-free Plug-and-Play (PnP) algorithm for HSI deconvolution. Specifically, we use the alternating direction method of multipliers (ADMM) to decompose the optimization problem into two iterative sub-problems. A flexible blind 3D denoising network (B3DDN) is designed to learn deep priors and to solve the denoising sub-problem with different noise levels. A measure of 3D residual whiteness is then investigated to adjust the penalty parameters when solving the quadratic sub-problems, as well as a stopping criterion. Experimental results on both simulated and real-world data with ground-truth demonstrate the superiority of the proposed method.

Hyperspectral image (HI) and multispectral image (MI) fusion allows us to overcome the hardware limitations of hyperspectral imaging systems inherent to their lower spatial resolution. Nevertheless, existing algorithms usually fail to consider realistic image acquisition conditions. This article presents a general imaging model that considers inter-image variability of data from heterogeneous sources and flexible image priors. The fusion problem is stated as an optimization problem in the maximum a posteriori framework. We introduce an original image fusion method that, on one hand, solves the optimization problem accounting for inter-image variability with an iteratively reweighted scheme and, on the other hand, that leverages lightweight convolutional neural network (CNN)-based networks to learn realistic image priors from data. In addition, we propose a zero-shot strategy to directly learn the image-specific prior of the latent images in an unsupervised manner. The performance of the algorithm is illustrated with real data subject to inter-image variability.

To overcome inherent hardware limitations of hyperspectral imaging systems with respect to their spatial resolution, fusion-based hyper-spectral image (HSI) super-resolution is attracting increasing attention. This technique aims to fuse a low-resolution (LR) HSI and a conventional high-resolution (HR) RGB image in order to obtain an HR HSI. Recently, deep learning architectures have been used to address the HSI super-resolution problem and have achieved remarkable performance. However, they ignore the degradation model even though this model has a clear physical interpretation and may contribute to improving the performance. We address this problem by proposing a method that, on the one hand, makes use of the linear degradation model in the data-fidelity term of the objective function and, on the other hand, utilizes the output of a convolutional neural network for designing a deep prior regularizer in spectral and spatial gradient domains. Experiments show the performance improvement achieved with this strategy.

Hyperspectral image (HSI) super-resolution is commonly used to overcome the hardware limitations of existing hyperspectral imaging systems on spatial resolution. It fuses a low-resolution (LR) HSI and a high-resolution (HR) conventional image of the same scene to obtain an HR HSI. In this work, we propose a method that integrates a physical model and deep prior information. Specifically, a novel, yet effective two-stream fusion network is designed to serve as a regularizer for the fusion problem. This fusion problem is formulated as an optimization problem whose solution can be obtained by solving a Sylvester equation. Furthermore, the regularization parameter is simultaneously estimated to automatically adjust contribution of the physical model and the learned prior to reconstruct the final HR HSI. Experimental results on both simulated and real data demonstrate the superiority of the proposed method over other state-of-the-art methods on both quantitative and qualitative comparisons.

Spectral unmixing is a widely used technique in hyperspectral image processing and analysis. It aims to separate mixed pixels into the component materials and their corresponding abundances. Early solutions to spectral unmixing are performed independently on each pixel. Nowadays, investigating proper priors into the unmixing problem has been popular as it can significantly enhance the unmixing performance. However, it is nontrivial to handcraft a powerful regularizer, and complex regularizers may introduce extra difficulties in solving optimization problems in which they are involved. To address this issue, we present a plug-and-play (PnP) priors framework for hyperspectral unmixing. More specifically, we use the alternating direction method of multipliers (ADMM) to decompose the optimization problem into two iterative subproblems. One is a regular optimization problem depending on the forward model, and the other is a proximity operator related to the prior model and can be regarded as an image denoising problem. Our framework is flexible and extendable which allows a wide range of denoisers to replace prior models and avoids handcrafting regularizers. Experiments conducted on both synthetic data and real airborne data illustrate the superiority of the proposed strategy compared with other state-of-the-art hyperspectral unmixing methods.

High spectral resolution of hyperspectral images allows the detection and classification of materials in the observed images. However, existing research on hyperspectral detection mainly focuses on pixel-level study, partially due to the low spatial resolution in typical earth observation applications. With the development of imaging techniques, high-spatial-resolution hyperspectral data can be obtained and object-level detection is necessary for many applications. In this work, the object-based hyperspectral detection problem is formulated, and a convolutional neural network is then designed based on the specific characteristics of this problem. Moreover, a hyperspectral dataset with over 400 high-quality images for object-level target detection is created. Experimental results validate the proposed framework and show its superior performance.

Hyperspectral image (HSI) deconvolution is an ill-posed problem aiming at recovering sharp images with tens or hundreds of spectral channels from blurred and noisy observations. In order to successfully conduct the deconvolution, proper priors are required to regularize the optimization problem. However, handcrafting a good regularizer may not be trivial and complex regularizers lead to difficulties in solving the optimization problem. In this paper, we use the alternating direction method of multipliers (ADMM) to decompose the optimization problem into iterative subproblems where the prior only appears in a denoising subproblem. Then a 3D denoising convolutional neural network (3DDnCNN) is designed and trained with data for solving this problem. In this way, the hyperspectral image deconvolution is then solved with a framework that integrates the optimization techniques and deep learning. Experimental results demonstrate the superiority of the proposed method with several blurring settings in both quantitative and qualitative comparisons.

Hyperspectral unmixing aims at separating a mixed pixel into a set of pure spectral signatures and their corresponding fractional abundances. Investigating prior spatial and spectral information to regularize the unmixing problem can effectively improve the estimation performance. However, handcrafting a powerful regularizer is a non-trivial task and complex regularizers introduce extra difficulties in solving the optimization problem. In this paper, we present a flexible spectral unmixing method using plug-and-play priors. This method benefits from the alternating direction method of multipliers (ADMM) to decompose the optimization problem into iterative subproblems and incorporates the image denoisers as prior models in a subproblem. In this form, we can plug in various image denoising operations to bypass handcrafting regularizers. We demonstrate the superiority of the proposed unmixing method comparing with other state-of-the-art methods both on synthetic data and real airborne data.

Near-infrared (NIR) images provide spectral information beyond the visible light spectrum and thus are useful in many applications. However, single-channel NIR images contain less information per pixel than RGB images and lack visibility for human perception. Transforming NIR images to RGB images is necessary for performing further analysis and computer vision tasks. In this work, we propose a novel NIR-to-RGB translation method. It contains two sub-networks and a fusion operator. Specifically, a U-net based neural network is used to learn the texture information while a CycleGAN based neural network is adopted to excavate the color information. Finally, a guided filter based fusion strategy is applied to fuse the outputs of these two neural networks. Experiment results show that our proposed method achieves superior NIR-to-RGB translation performance.

A 4-dimensional (4D) image can be viewed as a stack of volumetric images over channels of observation depth or temporal frames. This data contains rich information at the cost of high demands for storage and transmission resources due to its large volume. In this paper, we present a lossless 4D image compression algorithm by extending CCSDS-123.0-B-1 standard. Instead of separately compressing the volumetric image at each channel of 4D images, the proposed algorithm efficiently exploits redundancy across the fourth dimension of data. Experiments conducted on two types of 4D images demonstrate the effectiveness of the proposed lossless compression method.