Online distributed optimization is particularly useful for solving optimization problems with streaming data collected by multiple agents over a network. When the solutions lie on a Riemannian manifold, such problems become challenging to solve, particularly when efficiency and continuous adaptation are required. This work tackles these challenges and devises a diffusion adaptation strategy for decentralized optimization over general manifolds. A theoretical analysis shows that the proposed algorithm is able to approach network agreement after sufficient iterations, which allows a non-asymptotic convergence result to be derived. We apply the algorithm to the online decentralized principal component analysis problem and Gaussian mixture model inference. Experimental results with both synthetic and real data illustrate its performance.

Polynomial Neural Networks (PNNs) possess a rich algebraic and geometric structure. However, their identifiability-a key property for ensuring interpretability-remains poorly understood. In this work, we present a comprehensive analysis of the identifiability of deep PNNs, including architectures with and without bias terms. Our results reveal an intricate interplay between activation degrees and layer widths in achieving identifiability. As special cases, we show that architectures with non-increasing layer widths are generically identifiable under mild conditions, while encoder-decoder networks are identifiable when the decoder widths do not grow too rapidly compared to the activation degrees. Our proofs are constructive and center on a connection between deep PNNs and low-rank tensor decompositions, and Kruskal-type uniqueness theorems. We also settle an open conjecture on the dimension of PNN's neurovarieties, and provide new bounds on the activation degrees required for it to reach the expected dimension.

Distributed adaptation and learning recently gained considerable attention in solving optimization problems with streaming data collected by multiple agents over a graph. This work focuses on such problems where the solutions lie on a Riemannian manifold. This research topic is of particular interest for many applications, e.g., principal component analysis (PCA). Although several incremental and consensus algorithms have been proposed, there is a lack of methods designed for general Riemannian manifolds with efficient diffusion strategies. In this paper, we devise two Riemannian diffusion adaptation strategies, namely, adaptation-then-combination (ATC) and combination-then-adaptation (CTA), for decentralized Riemannian optimization over graphs. In the adaptation step, a Riemannian stochastic gradient descent method (SGD) is used to estimate the local solution at each node. In the combination step, the local estimates at the different nodes are combined by computing the weighted Fréchet mean over the neighborhood of each node. We apply our algorithms to online distributed PCA and compare them to both non-cooperative and centralized solutions.

Recent advances in the theory of Neural Operators (NOs) have enabled fast and accurate computation of the solutions to complex systems described by partial differential equations (PDEs). Despite their great success, current NO-based solutions face important challenges when dealing with spatio-temporal PDEs over long time scales. Specifically, the current theory of NOs does not present a systematic framework to perform data assimilation and efficiently correct the evolution of PDE solutions over time based on sparsely sampled noisy measurements. In this paper, we propose a learning-based state-space approach to compute the solution operators to infinite-dimensional semilinear PDEs. Exploiting the structure of semilinear PDEs and the theory of nonlinear observers in function spaces, we develop a flexible recursive method that allows for both prediction and data assimilation by combining prediction and correction operations. The proposed framework is capable of producing fast and accurate predictions over long time horizons, dealing with irregularly sampled noisy measurements to correct the solution, and benefits from the decoupling between the spatial and temporal dynamics of this class of PDEs. We show through experiments on the Kuramoto-Sivashinsky, Navier-Stokes and Korteweg-de Vries equations that the proposed model is robust to noise and can leverage arbitrary amounts of measurements to correct its prediction over a long time horizon with little computational overhead.

Non-parametric detection of change points in streaming time series data that belong to Euclidean spaces has been extensively studied in the literature. Nevertheless, when the data belongs to a Riemannian manifold, existing approaches are no longer applicable as they fail to account for the structure and geometry of the manifold. In this paper, we introduce a non-parametric algorithm for online change point detection in manifold-valued data streams. This algorithm monitors the generalized Karcher mean of the data, computed using stochastic Riemannian optimization. We provide theoretical bounds on the detection and false alarm rate performances of the algorithm, using a new result on the non-asymptotic convergence of the stochastic Riemannian gradient descent. We apply our algorithm to two different Riemannian manifolds. Experimental results with both synthetic and real data illustrate the performance of the proposed method.

In hyperspectral sparse unmixing, a successful approach employs spectral bundles to address the variability of the endmembers in the spatial domain. However, the regularization penalties usually employed aggregate substantial computational complexity, and the solutions are very noise-sensitive. We generalize a multiscale spatial regularization approach to solve the unmixing problem by incorporating group sparsity-inducing mixed norms. Then, we propose a noise-robust method that can take advantage of the bundle structure to deal with endmember variability while ensuring inter- and intra-class sparsity in abundance estimation with reasonable computational cost. We also present a general heuristic to select themost representativeabundance estimation over multiple runs of the unmixing process, yielding a solution that is robust and highly reproducible. Experiments illustrate the robustness and consistency of the results when compared to related methods.

Deep learning-based frameworks have been recently applied to hyperspectral umixing due to their flexibility and powerful representation capabilities. However, such techniques either use black-box models which are not physically interpretable, or fail to address the non-idealities of the unmixing problem. In this paper, we propose a physically interpretable deep learning method for hyperspectral unmixing accounting for nonlinearity and the variability of the endmembers. The proposed method is based on a probabilistic variational deep learning framework which employs semi-supervised disentanglement learning to properly separate the abundances and endmembers. A self-supervised strategy is used to generate labeled training data, and the model is learned end-to-end using stochastic backpropagation. Experimental results on both synthetic and real datasets illustrate the performance of the proposed method compared to state-of-the-art algorithms.

Discovering components that are shared in multiple datasets, next to dataset-specific features, has great potential for studying the relationships between different subjects or tasks in functional Magnetic Resonance Imaging (fMRI) data. Coupled matrix and tensor factorization approaches have been useful for flexible data fusion, or decomposition to extract features that can be used in multiple ways. However, existing methods do not directly recover shared and dataset-specific components, which requires post-processing steps involving additional hyperparameter selection. In this paper, we propose a tensor-based framework for multi-task fMRI data fusion, using a partially constrained canonical polyadic (CP) decomposition model. Differently from previous approaches, the proposed method directly recovers shared and dataset-specific components, leading to results that are directly interpretable. A strategy to select a highly reproducible solution to the decomposition is also proposed. We evaluate the proposed methodology on real fMRI data of three tasks, and show that the proposed method finds meaningful components that clearly identify group differences between patients with schizophrenia and healthy controls.

Detecting change points in streaming time series data is a long standing problem in signal processing. A plethora of methods have been proposed to address it, depending on the hypotheses at hand. Non-parametric approaches are particularly interesting as they do not make any assumption on the distribution of data or on the nature of changes. Nevertheless, leveraging recent advances in deep learning to detect change points in time series data is still challenging. In this paper, we propose a change point detection method using an online approach based on neural networks to directly estimate the density-ratio between current and reference windows of the data stream. A variational continual learning framework is employed to train the neural network in an online manner while retaining information learned from past data. This leads to a statistically-principled fully nonparametric framework to detect change points from streaming data. Experimental results with synthetic and real data illustrate the effectiveness of the proposed approach.

This article presents a multichannel Wiener filter (MWF) based noise reduction method with preservation of the interaural level difference (ILD). It minimizes the MWF cost function subject to two constraints for ILD preservation. Under this approach, the weighting coefficient that establishes the trade-off between noise reduction and binaural cue preservation takes a physical interpretation, facilitating its design. The proposed approach results in a convex optimization problem that admits a computationally efficient semi-analytical closed-form solution. Simulation experiments in hearing aid applications were performed considering practical acoustic scenarios. Considering an appropriate set of control parameters, the average performance of the proposed method preserves the ILD of a single interfering source in the same way as the conventional MWF-ILD, keeping the same level of noise reduction as the classic MWF, with approximately the same amount of speech ILD distortion. The proposed method is particularly interesting for implementing real-time noise reduction methods in binaural hearing aids.

Hyperspectral image (HI) and multispectral image (MI) fusion allows us to overcome the hardware limitations of hyperspectral imaging systems inherent to their lower spatial resolution. Nevertheless, existing algorithms usually fail to consider realistic image acquisition conditions. This article presents a general imaging model that considers inter-image variability of data from heterogeneous sources and flexible image priors. The fusion problem is stated as an optimization problem in the maximum a posteriori framework. We introduce an original image fusion method that, on one hand, solves the optimization problem accounting for inter-image variability with an iteratively reweighted scheme and, on the other hand, that leverages lightweight convolutional neural network (CNN)-based networks to learn realistic image priors from data. In addition, we propose a zero-shot strategy to directly learn the image-specific prior of the latent images in an unsupervised manner. The performance of the algorithm is illustrated with real data subject to inter-image variability.

Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupervised frameworks. In this work, we propose an unsupervised MTHU algorithm based on variational recurrent neural networks. First, a stochastic model is proposed to represent both the dynamical evolution of the endmembers and their abundances, as well as the mixing process. Moreover, a new model based on a low-dimensional parametrization is used to represent spatial and temporal endmember variability, significantly reducing the amount of variables to be estimated. We propose to formulate MTHU as a Bayesian inference problem. However, the solution to this problem does not have an analytical solution due to the nonlinearity and non-Gaussianity of the model. Thus, we propose a solution based on deep variational inference, in which the posterior distribution of the estimated abundances and endmembers is represented by using a combination of recurrent neural networks and a physically motivated model. The parameters of the model are learned using stochastic backpropagation. Experimental results show that the proposed method outperforms state of the art MTHU algorithms.

The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications press for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember (EME) representations to account for temporal spectral variability. By representing the multitemporal mixing process using a state-space formulation, we are able to exploit the Bayesian filtering machinery to estimate the EME variability coefficients. Moreover, by assuming that the temporal variability of the abundances is small over short intervals, an efficient implementation of the expectation–maximization (EM) algorithm is employed to estimate the abundances and the other model parameters. Simulation results indicate that the proposed strategy outperforms state-of-the-art multi-temporal SU (MTSU) algorithms.

Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model, and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to explore the fact that the encoder should invert the mixing process, which might reduce their robustness. In this letter, we propose a model-based AEC for nonlinear HU by considering the mixing model a nonlinear fluctuation over a linear mixture. Different from previous works, we show that this restriction naturally imposes a particular structure to both the encoder and decoder networks. This introduces prior information in the AEC without reducing the flexibility of the mixing model. Simulations with synthetic and real data indicate that the proposed strategy improves nonlinear HU.

In this paper, we propose to jointly solve the hyperspectral super-resolution problem and the unmixing problem of the underlying super-resolution image using a coupled LL1 block-tensor decomposition. We consider a spectral variability phenomenon occurring between the observed low-resolution images. Exact recovery conditions for the image and mixing factors are provided. We propose two algorithms, an unconstrained one and another one subject to nonnegativity constraints, to solve the problems at hand. We showcase performance of the proposed approach on synthetic and real images.

Several approaches have been proposed to solve the spectral unmixing problem in hyperspectral image analysis. Among them the use of sparse regression techniques aims to characterize the abundances in pixels based on a large library of spectral signatures known a priori. Recently, the integration of image spatial-contextual information significantly enhanced the performance of sparse unmixing. In this work, we propose a computationally efficient multiscale representation method for hyperspectral data adapted to the unmixing problem. The proposed method is based on a hierarchical extension of the SLIC oversegmentation algorithm constructed using a robust homogeneity testing. The image is subdivided into a set of spectrally homogeneous regions formed by pixels with similar characteristics (superpixels). This representation is then used to provide prior spatial regularity information for the abundances of materials present in the scene, improving the conditioning of the unmixing problem. Simulation results illustrate that the method is capable of estimating abundances with high quality and low computational cost, especially in noisy scenarios.

Identifying directed connectivity patterns from nodal measurements is an important problem in network analysis. Recent works proposed to leverage the performance and flexibility of strategies operating in reproducing kernel Hilbert spaces (RKHS) to model nonlinear interactions between network agents. Moreover, several applications require online and efficient solutions, which motivated the consideration of distributed adaptive learning strategies inspired by algorithms such as the kernel least mean square (KLMS). Despite showing good performance, a thorough theoretical understanding of the behavior of such algorithms is still missing. This makes applying them in practice challenging, especially because the set-up of adaptive algorithms involves additional parameters like the step size and a dictionary of kernel functions. In this paper, we present a convergence analysis of the graph-topology-inference KLMS algorithm. Monte Carlo simulations demonstrate the accuracy of the theoretical models.

Multiple endmember spectral mixture analysis (MESMA) is one of the leading approaches to perform spectral unmixing (SU) considering the variability of the endmembers (EMs). It represents each EM in the image using libraries of spectral signatures acquireda priori. However, existing spectral libraries are often small and unable to properly capture the variability of each EM in practical scenes, which compromises the performance of MESMA. In this letter, we propose a library augmentation strategy to increase the diversity of existing spectral libraries, thus improving their ability to represent the materials in real images. First, we leverage the power of deep generative models to learn the statistical distribution of the EMs based on the spectral signatures available in the existing libraries. Afterward, new samples can be drawn from the learned EM distributions and used to augment the spectral libraries, improving the overall quality of the SU process. Experimental results using synthetic and real data attest to the superior performance of the proposed method even under library mismatch conditions.

In graph signal processing, there are often settings where the graph topology is not known beforehand and has to be estimated from data. Moreover, some graphs can be dynamic, such as brain activity supported by neurons or brain regions. This paper focuses on estimating in an online and adaptive manner a network structure capturing the non-linear dependencies among streaming graph signals in the form of a possibly directed, adjacency matrix. By projecting data into a higher- or infinite-dimension space, we focus on capturing nonlinear relationships between agents. In order to mitigate the increasing number of data points, we employ kernel dictionaries. Finally, we run a series of tests in order to experimentally illustrate the usefulness of our kernel-based approach on biomedical data, on which we obtain results comparable to state-of-the-art methods.

Traditional hyperspectral unmixing methods neglect the underlying variability of spectral signatures often observed in typical hyperspectral images (HI), propagating these mismodeling errors throughout the whole unmixing process. Attempts to model material spectra as members of sets or as random variables tend to lead to severely ill-posed unmixing problems. Although parametric models have been proposed to overcome this drawback by handling endmember (EM) variability through generalizations of the mixing model, the success of these techniques depends on employing appropriate regularization strategies. Moreover, the existing approaches fail to adequately explore the natural multidimensinal representation of HIs. Recently, tensor-based strategies considered low-rank decompositions of HIs as an alternative to impose low-dimensional structures on the solutions of standard and multitemporal unmixing problems. These strategies, however, present two main drawbacks: 1) they confine the solutions to low-rank tensors, which often cannot represent the complexity of real-world scenarios and 2) they lack guarantees that EMs and abundances will be correctly factorized in their respective tensors. In this article, we propose a more flexible approach, called unmixing with low-rank tensor regularization algorithm accounting for EM variability (ULTRA-V), that imposes low-rank structures through regularizations whose strictness is controlled by scalar parameters. Simulations attest the superior accuracy of the method when compared with state-of-the-art unmixing algorithms that account for spectral variability.

Image fusion combines data from different heterogeneous sources to obtain more precise information about an underlying scene. Hyperspectral-multispectral (HS-MS) image fusion is currently attracting great interest in remote sensing since it allows the generation of high spatial resolution HS images and circumventing the main limitation of this imaging modality. Existing HS-MS fusion algorithms, however, neglect the spectral variability often existing between images acquired at different time instants. This time difference causes variations in spectral signatures of the underlying constituent materials due to the different acquisition and seasonal conditions. This paper introduces a novel HS-MS image fusion strategy that combines an unmixing-based formulation with an explicit parametric model for typical spectral variability between the two images. Simulations with synthetic and real data show that the proposed strategy leads to a significant performance improvement under spectral variability and state-of-the-art performance otherwise.

Spectral variability in hyperspectral images can result from factors including environmental, illumination, atmospheric and temporal changes. Its occurrence may lead to the propagation of significant estimation errors in the unmixing process. To address this issue, extended linear mixing models have been proposed which lead to large scale nonsmooth ill-posed inverse problems. Furthermore, the regularization strategies used to obtain meaningful results have introduced interdependencies among abundance solutions that further increase the complexity of the resulting optimization problem. In this paper we present a novel data dependent multiscale model for hyperspectral unmixing accounting for spectral variability. The new method incorporates spatial contextual information to the abundances in extended linear mixing models by using a multiscale transform based on superpixels. The proposed method results in a fast algorithm that solves the abundance estimation problem only once in each scale during each iteration. Simulation results using synthetic and real images compare the performances, both in accuracy and execution time, of the proposed algorithm and other state-of-the-art solutions.

Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might comprise intricate interactions between their fractional abundances and nonlinear contributions. In this paper, we consider a multiscale regularization strategy for nonlinear spectral unmixing with kernels. The proposed methodology splits the unmixing problem into two sub-problems at two different spatial scales: a coarse scale containing low-dimensional structures, and the original fine scale. The coarse spatial domain is defined using superpixels that result from a multiscale transformation. Spectral unmixing is then formulated as the solution of quadratically constrained optimization problems, which are solved efficiently by exploring their strong duality and a reformulation of their dual cost functions in the form of root-finding problems. Furthermore, we employ a theory-based statistical framework to devise a consistent strategy to estimate all required parameters, including both the regularization parameters of the algorithm and the number of superpixels of the transformation, resulting in a truly blind (from the parameters setting perspective) unmixing method. Experimental results attest the superior performance of the proposed method when comparing with other, state-of-the-art, related strategies.

Endmember (EM) variability has an important impact on the performance of hyperspectral image (HI) analysis algorithms. Recently, extended linear mixing models have been proposed to account for EM variability in the spectral unmixing (SU) problem. The direct use of these models has led to severely ill-posed optimization problems. Different regularization strategies have been considered to deal with this issue, but none so far has consistently exploited the information provided by the existence of multiple pure pixels often present in HIs. In this work, we propose to break the SU problem into a sequence of two problems. First, we use pure pixel information to estimate an interpolated tensor of scaling factors representing spectral variability. This is done by considering the spectral variability to be a smooth function over the HI and confining the energy of the scaling tensor to a low-rank structure. Afterwards, we solve a matrix-factorization problem to estimate the fractional abundances using the variability scaling factors estimated in the previous step, what leads to a significantly more well-posed problem. Simulations with synthetic and real data attest the effectiveness of the proposed strategy.

Sparse hyperspectral unmixing from large spectral libraries has been considered to circumvent the limitations of endmember extraction algorithms in many applications. This strategy often leads to ill-posed inverse problems, which can greatly benefit from spatial regularization strategies. However, existing spatial regularization strategies lead to large-scale nonsmooth optimization problems. Thus, efficiently introducing spatial context in the unmixing problem remains a challenge and a necessity for many real world applications. In this letter, a novel multiscale spatial regularization approach for sparse unmixing is proposed. The method uses a signal-adaptive spatial multiscale decomposition based on segmentation and oversegmentation algorithms to decompose the unmixing problem into two simpler problems: one in an approximation image domain and another in the original domain. Simulation results using both synthetic and real data indicate that the proposed method outperforms the state-of-the-art total variation-based algorithms with a computation time comparable to that of their unregularized counterparts.

In this paper, a new video super-resolution reconstruction (SRR) method with improved robustness to outliers is proposed. Although the regularized least mean squares (R-LMSs) are one of the SRR algorithms with the best reconstruction quality for its computational cost, and is naturally robust to registration inaccuracies, its performance is known to degrade severely in the presence of innovation outliers. By studying the proximal point cost function representation of the R-LMS iterative equation, a better understanding of its performance under different situations is attained. Using statistical properties of typical innovation outliers, a new cost function is then proposed and two new algorithms are derived, which present improved robustness to outliers while maintaining computational costs comparable with that of R-LMS. The Monte-Carlo simulation results illustrate that the proposed method outperforms the traditional and regularized versions of LMS, and is competitive with state-of-the-art SRR methods at a much smaller computational cost.

Endmember variability is an important factor for accurately unveiling vital information relating the pure materials and their distribution in hyperspectral images. Recently, the extended linear mixing model (ELMM) has been proposed as a modification of the linear mixing model (LMM) to consider endmember variability effects resulting mainly from illumination changes. In this paper, we further generalize the ELMM leading to a new model (GLMM) to account for more complex spectral distortions where different wavelength intervals can be affected unevenly. We also extend the existing methodology to jointly estimate the variability and the abundances for the GLMM. Simulations with real and synthetic data show that the unmixing process can benefit from the extra flexibility introduced by the GLMM.

In see-through systems an observer watches a (background) scene partially occluded by a display. In this display, usually positioned close to the observer, a region of the background scene is shown, yielding the sensation that the display is transparent. To achieve the transparency effect, it is very important to compensate the parallax error and other distortions caused by the image acquisition system. In this paper a detailed study of a video see-through methodology with parallax correction is performed. In a system composed by two cameras-one directed to the user and another to the background scene-and a display, the relative position between the user, the display and the scene is estimated using a feature detection algorithm and the parallax error is compensated assuming a planar scene model. The application of the proposed methodology on Driver Assistance Systems (DAS) is proposed. A theoretical assessment of the algorithm shows that although approximations are proposed to simplify the methodology and reduce the computational cost, such as the planar scene model and fixed working distance, on some practical situations their effects can be neglected without noticeable impact on the perceptual quality of the solution.