Mathematical models for anomaly detection (AD) in dynamic systems have demonstrated high performance, particularly diagnostic observer models.Deep learning (DL) models are also effective, but they often require complex architectures, large training datasets and costly finetuning to achieve generalization across diverse time series (TS) types.This paper introduces a DL AD model based on the algebraic design of a diagnostic observer, marking its first adaptation for TS data.Experiments on large TS benchmark datasets demonstrate its superiority over various recent DL models.

Class-Incremental Learning (Class-IL) is challenging as the model must adapt to new classes while retaining knowledge of old ones.To avoid catastrophic forgetting in knowledge distillation with a fixed-budget memory, exemplars from previously learned classes need to be stored.We propose a novel sample selection method based on the coherence measure to boost Class-IL performance.This is the first time the coherence is investigated in a deep model, specifically for Class-IL.We define the coherence between two samples as a normalized inner product between their deep feature extractor features.Theoretical results and extensive experiments demonstrate the relevance of our approach.

Universal Domain Adaptation (UniDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain, even when their classes are not fully shared. Few dedicated UniDA methods exist for Time Series (TS), which remains a challenging case. In general, UniDA approaches align common class samples and detect unknown target samples from emerging classes. Such detection often results from thresholding a discriminability metric. The threshold value is typically either a fine-tuned hyperparameter or a fixed value, which limits the ability of the model to adapt to new data. Furthermore, discriminability metrics exhibit overconfidence for unknown samples, leading to misclassifications. This paper introduces UniJDOT, an optimal-transport-based method that accounts for the unknown target samples in the transport cost. Our method also proposes a joint decision space to improve the discriminability of the detection module. In addition, we use an auto-thresholding algorithm to reduce the dependence on fixed or fine-tuned thresholds. Finally, we rely on a Fourier transform-based layer inspired by the Fourier Neural Operator for better TS representation. Experiments on TS benchmarks demonstrate the discriminability, robustness, and state-of-the-art performance of UniJDOT.

ABSTRACT This paper presents a novel approach to pollution assessment by investigating support vector machines (SVM) with an uncertainty option to overcome the limitations of traditional kriging. While kriging is a major tool for geostatistical modelling, allowing to estimate the distribution of contaminants in a region from a small set of samples, it does not allow to extract also the uncertainty map. An uncertainty map is of great interest, as it allows to identify regions of high uncertainty where one should sample in order to reduce high level of uncertainties. In this paper, we propose two variants of the SVM with an uncertainty option, each using a different hinge loss to improve the accuracy and efficiency. These losses allow to estimate different levels of contaminations, as well as uncertainty, such as the three levels: positive, uncertain and negative, namely for pollution estimation: high‐pollution, uncertain and low‐pollution. In addition to the exploration of SVM variants, we propose an innovative active sample selection strategy based on the uncertainty criterion. This strategy is designed to systematically reduce uncertainties in pollution assessment, thus providing adaptability to dynamic environmental changes. An incremental SVM with an uncertainty option is introduced to further optimise the sample selection process. Furthermore, the decision‐making process is refined through the introduction of a novel three‐hinge loss. The corresponding optimization problem and its resolution allow for a more nuanced contamination assessment with multiple levels of estimation, providing a valuable tool for characterising contamination levels with increased granularity. Extensive experiments on synthetic and real data validate the proposed methodology. Synthetic data simulations assess the quality of the approach, while real data from a two‐dimensional porosity measurement demonstrate practical applicability. This research contributes to the advancement of pollution assessment methodologies, providing an adaptable solution for environmental monitoring.

Contrastive learning has demonstrated great effectiveness in representation learning especially for image classification tasks. However, there is still a shortage in the studies targeting regression tasks, and more specifically applications on hyperspectral data. In this paper, we propose a contrastive learning framework for the regression tasks for hyperspectral data. To this end, we provide a collection of transformations relevant for augmenting hyperspectral data, and investigate contrastive learning for regression. Experiments on synthetic and real hyperspectral datasets show that the proposed framework and transformations significantly improve the performance of regression models, achieving better scores than other state-of-the-art transformations.

Since the Message Passing (Graph) Neural Networks (MPNNs) have a linear complexity with respect to the number of nodes when applied to sparse graphs, they have been widely implemented and still raise a lot of interest even though their theoretical expressive power is limited to the first order Weisfeiler-Lehman test (1-WL). In this paper, we show that if the graph convolution supports are designed in spectral-domain by a non-linear custom function of eigenvalues and masked with an arbitrary large receptive field, the MPNN is theoretically more powerful than the 1-WL test and experimentally as powerful as a 3-WL existing models, while remaining spatially localized. Moreover, by designing custom filter functions, outputs can have various frequency components that allow the convolution process to learn different relationships between a given input graph signal and its associated properties. So far, the best 3-WL equivalent graph neural networks have a computational complexity in $\mathcal{O}(n^3)$ with memory usage in $\mathcal{O}(n^2)$, consider non-local update mechanism and do not provide the spectral richness of output profile. The proposed method overcomes all these aforementioned problems and reaches state-of-the-art results in many downstream tasks.

Today, deep convolutional neural networks (CNNs) have demonstrated state of the art performance for supervised medical image segmentation, across various imaging modalities and tasks. Despite early success, segmentation networks may still generate anatomically aberrant segmentations, with holes or inaccuracies near the object boundaries. To mitigate this effect, recent research works have focused on incorporating spatial information or prior knowledge to enforce anatomically plausible segmentation. If the integration of prior knowledge in image segmentation is not a new topic in classical optimization approaches, it is today an increasing trend in CNN based image segmentation, as shown by the growing literature on the topic. In this survey, we focus on high level prior, embedded at the loss function level. We categorize the articles according to the nature of the prior: the object shape, size, topology, and the inter-regions constraints. We highlight strengths and limitations of current approaches, discuss the challenge related to the design and the integration of prior-based losses, and the optimization strategies, and draw future research directions.

Nonlinear spectral unmixing is a challenging and important task in hyperspectral image analysis. The kernel-based bi-objective non-negative matrix factorization (Bi-NMF) has shown its usefulness in nonlinear unmixing; However, it suffers several issues that prohibit its practical application. In this work, we propose an unsupervised nonlinear unmixing method that overcomes these weaknesses. Specifically, the new method introduces into each pixel a parameter that adjusts the nonlinearity therein. These parameters are jointly optimized with endmembers and abundances, using a carefully designed objective function by multiplicative update rules. Experiments on synthetic and real datasets confirm the effectiveness of the proposed method.

Despite the utility and benefits of omnidirectional images in robotics and automotive applications, there are no datasets of omnidirectional images available with semantic segmentation, depth map, and dynamic properties. This is due to the time cost and human effort required to annotate ground truth images. This paper presents a framework for generating omnidirectional images using images that are acquired from a virtual environment. For this purpose, we demonstrate the relevance of the proposed framework on two well-known simulators: CARLA Simulator, which is an open-source simulator for autonomous driving research, and Grand Theft Auto V (GTA V), which is a very high quality video game. We explain in details the generated OmniScape dataset, which includes stereo fisheye and catadioptric images acquired from the two front sides of a motorcycle, including semantic segmentation, depth map, intrinsic parameters of the cameras and the dynamic parameters of the motorcycle. It is worth noting that the case of two-wheeled vehicles is more challenging than cars due to the specific dynamic of these vehicles.

In hyperspectral images, some spectral bands suffer from low signal-to-noise ratio due to noisy acquisition and atmospheric effects, thus requiring robust techniques for the unmixing problem. This paper presents a robust supervised spectral unmixing approach for hyperspectral images. The robustness is achieved by writing the unmixing problem as the maximization of the correntropy criterion subject to the most commonly used constraints. Two unmixing problems are derived: the first problem considers the fully constrained unmixing, with both the nonnegativity and sum-to-one constraints, while the second one deals with the nonnegativity and the sparsity promoting of the abundances. The corresponding optimization problems are solved using an alternating direction method of multipliers (ADMM) approach. Experiments on synthetic and real hyperspectral images validate the performance of the proposed algorithms for different scenarios, demonstrating that the correntropy-based unmixing with ADMM is particularly robust against highly noisy outlier bands.

The correntropy provides a robust criterion for outlier-insensitive machine learning, and its maximisation has been increasingly investigated in signal and image processing. In this paper, we investigate the problem of unmixing hyperspectral images, namely decomposing each pixel/spectrum of a given image as a linear combination of other pixels/spectra called endmembers. The coefficients of the combination need to be estimated subject to the nonnegativity and the sum-to-one constraints. In practice, some spectral bands suffer from low signal-to-noise ratio due to acquisition noise and atmospheric effects, thus requiring robust techniques for the unmixing problem. In this work, we cast the unmixing problem as the maximization of a correntropy criterion, and provide a relevant solution using the alternating direction method of multipliers (ADMM) method. Finally, the relevance of the proposed approach is validated on synthetic and real hyperspectral images, demonstrating that the correntropy-based unmixing is robust to outlier bands.

Linear mixture models are commonly used to represent a hyperspectral data cube as linear combinations of endmember spectra. However, determining the number of endmembers for images embedded in noise is a crucial task. This paper proposes a fully automatic approach for estimating the number of endmembers in hyperspectral images. The estimation is based on recent results of random matrix theory related to the so-called spiked population model. More precisely, we study the gap between successive eigenvalues of the sample covariance matrix constructed from high-dimensional noisy samples. The resulting estimation strategy is fully automatic and robust to correlated noise owing to the consideration of a noise-whitening step. This strategy is validated on both synthetic and real images. The experimental results are very promising and show the accuracy of this algorithm with respect to state-of-the-art algorithms.

Nonnegative matrix factorization (NMF) is a powerful class of feature extraction techniques that has been successfully applied in many fields, particularly in signal and image processing. Current NMF techniques have been limited to a single-objective optimization problem, in either its linear or nonlinear kernel-based formulation. In this paper, we propose to revisit the NMF as a multiobjective problem, particularly a biobjective one, where the objective functions defined in both input and feature spaces are taken into account. By taking the advantage of the sum-weighted method from the literature of multiobjective optimization, the proposed biobjective NMF determines a set of nondominated, Pareto optimal, solutions. Moreover, the corresponding Pareto front is approximated and studied. Experimental results on unmixing synthetic and real hyperspectral images confirm the efficiency of the proposed biobjective NMF compared with the state-of-the-art methods.

This paper presents three hyperspectral mixture models jointly with Bayesian algorithms for supervised hyperspectral unmixing. Based on the residual component analysis model, the proposed general formulation assumes the linear model to be corrupted by an additive term whose expression can be adapted to account for nonlinearities (NLs), endmember variability (EV), or mismodeling effects (MEs). The NL effect is introduced by considering a polynomial expression that is related to bilinear models. The proposed new formulation of EV accounts for shape and scale endmember changes while enforcing a smooth spectral/spatial variation. The ME formulation considers the effect of outliers and copes with some types of EV and NL. The known constraints on the parameter of each observation model are modeled via suitable priors. The posterior distribution associated with each Bayesian model is optimized using a coordinate descent algorithm, which allows the computation of the maximum a posteriori estimator of the unknown model parameters. The proposed mixture and Bayesian models and their estimation algorithms are validated on both synthetic and real images showing competitive results regarding the quality of the inferences and the computational complexity, when compared with the state-of-the-art algorithms.

This paper presents an unsupervised Bayesian algorithm for hyperspectral image unmixing accounting for endmember variability. Each image pixel is modeled by a linear combination of random endmembers to take into account endmember variability in the image. The coefficients of this linear combination (referred to as abundances) allow the proportions of each material (endmembers) to be quantified in the image pixel. An additive noise is also considered in the proposed model generalizing the normal compositional model. The proposed Bayesian algorithm exploits spatial correlations between adjacent pixels of the image and provides spectral information by achieving a spectral unmixing. It estimates both the mean and the covariance matrix of each endmember in the image. A spatial classification is also obtained based on the estimated abundances. Simulations conducted with synthetic and real data show the potential of the proposed model and the unmixing performance for the analysis of hyperspectral images.

In this paper, we propose a semiparametric regression model that relates the received signal strength indicators (RSSIs) to the distances separating stationary sensors and moving sensors in a wireless sensor network. This model combines the well-known log-distance theoretical propagation model with a nonlinear fluctuation term, estimated within the framework of kernel-based machines. This leads to a more robust propagation model. A fully comprehensive study of the choices of parameters is provided, and a comparison to state-of-the-art models using real and simulated data is given as well.

Integrating spatial information into hyperspectral unmixing procedures has been shown to have a positive effect on the estimation of fractional abundances due to the inherent spatial-spectral duality in hyperspectral scenes. However, current research works that take spatial information into account are mainly focused on the linear mixing model. In this paper, we investigate how to incorporate spatial correlation into a nonlinear abundance estimation process. A nonlinear unmixing algorithm operating in reproducing kernel Hilbert spaces, coupled with a l1-type spatial regularization, is derived. Experiment results, with both synthetic and real hyperspectral images, illustrate the effectiveness of the proposed scheme.

The massive use of information and communication technologies in supervisory control and data acquisition (SCADA) systems opens new ways for carrying out cyberattacks against critical infrastructures relying on SCADA networks. The various vulnerabilities in these systems and the heterogeneity of cyberattacks make the task extremely difficult for traditional intrusion detection systems (IDS). Modeling cyberattacks has become nearly impossible and their potential consequences may be very severe. The primary objective of this work is to detect malicious intrusions once they have already bypassed traditional IDS and firewalls. This paper investigates the use of machine learning for intrusion detection in SCADA systems using one-class classification algorithms. Two approaches of one-class classification are investigated: 1) the support vector data description (SVDD); and 2) the kernel principle component analysis. The impact of the considered metric is examined in detail with the study of lp-norms in radial basis function (RBF) kernels. A heuristic is proposed to find an optimal choice of the bandwidth parameter in these kernels. Tests are conducted on real data with several types of cyberattacks.

In recent years, nonlinear unmixing of hyperspectral data has become an attractive topic in hyperspectral image analysis, because nonlinear models appear as more appropriate to represent photon interactions in real scenes. For this challenging problem, nonlinear methods operating in reproducing kernel Hilbert spaces have shown particular advantages. In this paper, we derive an efficient nonlinear unmixing algorithm based on a recently proposed linear mixture/ nonlinear fluctuation model. A multi-kernel learning support vector regressor is established to determine material abundances and nonlinear fluctuations. Moreover, a low complexity locally-spatial regularizer is incorporated to enhance the unmixing performance. Experiments with synthetic and real data illustrate the effectiveness of the proposed method.

Nonlinear system identification has always been a challenging problem. The use of kernel methods to solve such problems becomes more prevalent. However, the complexity of these methods increases with time which makes them unsuitable for online identification. This drawback can be solved with the introduction of the coherence criterion. Furthermore, dictionary adaptation using a stochastic gradient method proved its efficiency. Mostly, all approaches are used to identify Single Output models which form a particular case of real problems. In this letter we investigate online kernel adaptive algorithms to identify Multiple Inputs Multiple Outputs model as well as the possibility of dictionary adaptation for such models.

Granger causality considers the question of whether two time series exert causal influences on each other. Causality testing usually relies on prediction, i.e., if the prediction error of the first time series is reduced by taking measurements from the second one into account, then the latter is said to have a causal influence on the former. In this paper, a nonparametric framework based on functional estimation is proposed. Nonlinear prediction is performed via the Bayesian paradigm, using Gaussian processes. Some experiments illustrate the efficiency of the approach.

The autoregressive (AR) model is a well-known technique to analyze time series. The Yule-Walker equations provide a straightforward connection between the AR model parameters and the covariance function of the process. In this paper, we propose a nonlinear extension of the AR model using kernel machines. To this end, we explore the Yule-Walker equations in the feature space, and show that the model parameters can be estimated using the concept of expected kernels. Finally, in order to predict once the model identified, we solve a pre-image problem by getting back from the feature space to the input space. We also give new insights into the convexity of the pre-image problem. The relevance of the proposed method is evaluated on several time series.

Spectral model is simple and efficient for modeling the rain attenuation which occurs in satellite communication channels. The prediction of this attenuation series is a vital step for adaptive coding or adaptive power control, which can improve the efficiency of a communication system. In simulation tasks, the discretized spectral model is usually used for generating the attenuation sequence. Due to this reason, in this paper we derive the conditional probability distribution of the predicted attenuation based on the discretized spectral model. This predictor can be used as a bound for others linear or nonlinear predictor of this model.

In hyperspectral image analysis, pixels are mixtures of spectral components associated to pure materials. Although the linear mixture model is the mostly studied case, nonlinear techniques have been proposed to overcome its limitations. In this paper, a manifold learning approach is used as a dimensionality-reduction step to deal with non-linearities beforehand, or is integrated directly in the endmember extraction and abundance estimation steps using geodesic distances. Simulation results show that these methods improve the precision of estimation in severely nonlinear cases.

The one-class classification has been successfully applied in many communication, signal processing, and machine learning tasks. This problem, as defined by the one-class SVM approach, consists in identifying a sphere enclosing all (or the most) of the data. The classical strategy to solve the problem considers a simultaneous estimation of both the center and the radius of the sphere. In this paper, we study the impact of separating the estimation problem. It turns out that simple one-class classification methods can be easily derived, by considering a least-squares formulation. The proposed framework allows us to derive some theoretical results, such as an upper bound on the probability of false detection. The relevance of this work is illustrated on well-known datasets.

Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.

In hyperspectral imaging, spectral unmixing is one of the most challenging and fundamental problems. It consists of breaking down the spectrum of a mixed pixel into a set of pure spectra, called endmembers, and their contributions, called abundances. Many endmember extraction techniques have been proposed in literature, based on either a statistical or a geometrical formulation. However, most, if not all, of these techniques for estimating abundances use a least-squares solution. In this paper, we show that abundances can be estimated using a geometric formulation. To this end, we express abundances with the barycentric coordinates in the simplex defined by endmembers. We propose to write them in terms of a ratio of volumes or a ratio of distances, which are quantities that are often computed to identify endmembers. This property allows us to easily incorporate abundance estimation within conventional endmember extraction techniques, without incurring additional computational complexity. We use this key property with various endmember extraction techniques, such as N-Findr, vertex component analysis, simplex growing algorithm, and iterated constrained endmembers. The relevance of the method is illustrated with experimental results on real hyperspectral images.

An operational framework is developed for testing stationarity relatively to an observation scale. The proposed method makes use of a family of stationary surrogates for defining the null hypothesis of stationarity. As a further contribution to the field, we demonstrate the strict-sense stationarity of surrogate signals and we exploit this property to derive the asymptotic distributions of their spectrogram and power spectral density. A statistical hypothesis testing framework is then proposed to check signal stationarity. Finally, some results are shown on a typical model of signals that can be thought of as stationary or nonstationary, depending on the observation scale used.

In this paper, we study the problem of decomposing spectra in hyperspectral data into the sum of pure spectra, or endmembers. We propose to jointly extract the endmembers and estimate the corresponding fractions, or abundances. For this purpose, we show that these abundances can be easily computed using volume of simplices, from the same information used in the classical N-Findr algorithm. This results into a simple scheme for unmixing hyperspectral data, with low computational complexity. Experimental results show the efficiency of the proposed method.

In this paper, we propose a distributed learning strategy in wireless sensor networks. Taking advantage of recent developments on kernel-based machine learning, we consider a new sparsification criterion for online learning. As opposed to previously derived criteria, it is based on the estimated error and is therefore is well suited for tracking the evolution of systems over time. We also derive a gradient descent algorithm, and we demonstrate its relevance to estimate the dynamic evolution of temperature in a given region.

Over the past few years, wireless sensor networks received tremendous attention for monitoring physical phenomena, such as the temperature field in a given region. Applying conventional kernel regression methods for functional learning such as support vector machines is inappropriate for sensor networks, since the order of the resulting model and its computational complexity scales badly with the number of available sensors, which tends to be large. In order to circumvent this drawback, we propose in this paper a reduced-order model approach. To this end, we take advantage of recent developments in sparse representation literature, and show the natural link between reducing the model order and the topology of the deployed sensors. To learn this model, we derive a gradient descent scheme and show its efficiency for wireless sensor networks. We illustrate the proposed approach through simulations involving the estimation of a spatial temperature distribution.

This paper provides new insights into on-line nonlinear sparse approximation of functions based on the coherence criterion. We revisit previous work, and propose tighter bounds on the approximation error based on the coherence criterion. Moreover, we study the connections between the coherence criterion and both the approximate linear dependence criterion and the principal component analysis. Finally, we derive a kernel normalized LMS algorithm based on the coherence criterion, which has linear computational complexity on the model order. Initial experimental results are presented on the performance of the algorithm.

In this paper, we propose a method for selecting time-frequency distributions appropriate for given learning tasks. It is based on a criterion that has recently emerged from the machine learning literature: the kernel-target alignment. This criterion makes possible to find the optimal representation for a given classification problem without designing the classifier itself. Some possible applications of our framework are discussed. The first one provides a computationally attractive way of adjusting the free parameters of a distribution to improve classification performance. The second one is related to the selection, from a set of candidates, of the distribution that best facilitates a classification task. The last one addresses the problem of optimally combining several distributions