Graph Transformers (GTs) have emerged as a promising graph learning tool, leveraging their all-pair connected property to effectively capture global information. To address the over-smoothing problem in deep GNNs, global attention was initially introduced, eliminating the necessity for using deep GNNs. However, through empirical and theoretical analysis, we verify that the introduced global attention exhibits severe over-smoothing, causing node representations to become indistinguishable due to its inherent low-pass filtering. This effect is even stronger than that observed in GNNs. To mitigate this, we propose PageRank Transformer (ParaFormer), which features a PageRank-enhanced attention module designed to mimic the behavior of deep Transformers. We theoretically and empirically demonstrate that ParaFormer mitigates over-smoothing by functioning as an adaptive-pass filter. Experiments show that ParaFormer achieves consistent performance improvements across both node classification and graph classification tasks on 11 datasets ranging from thousands to millions of nodes, validating its efficacy. The supplementary material, including code and appendix, can be found in https://github.com/chaohaoyuan/ParaFormer.

Functional brain network (FBN) analysis aims to enhance the understanding of brain organization and support the diagnosis of neurological and psychiatric disorders. Prior studies have shown that FBNs exhibit small-world topology, where brain regions form functional clusters, and abnormalities in these clusters are strongly associated with disease. However, current learning-based methods either ignore this special topological structure or impose it as a post-hoc step outside the learning process, limiting both performance and interpretability. In this paper, we propose Learning Optimal Spectral Clustering (LOSC), a new framework that integrates the FBN generation, clustering, and classification with a novel graph theory grounded loss to fully exploit the small-world topology. Firstly, LOSC learns brain connectivity in a nonlinear spatio-spectral embedding space, guided by our proposed Rayleigh Quotient Loss (RQL), to preserve the small-world properties in generated FBNs. Then, the FBNs are partitioned into clusters of functionally synchronized regions, and both intra- and inter-cluster relations are utilized for brain network classification. Our contributions are threefold: (1) Improved brain network classification accuracy: by leveraging small-world functional clusters, LOSC achieves consistent gains of 2.0%, 3.6%, and 2.6% on the ABIDE, ADHD-200, and HCP datasets compared with state-of-the-art models, respectively; (2) Theoretical grounding: with our proposed RQL, LOSC bridges the gap between the graph theory and learning-based FBN analysis; and (3) Interpretability: the discovered functional clusters align with known neuropathology and contribute to the discovery of new functional community biomarkers.

In this paper, we propose a novel framework that leverages Large Language Models (LLMs) for predicting missing values in time-varying graph signals by exploiting spatial and temporal smoothness. We leverage the power of LLM to achieve a message-passing scheme. For each missing node, its neighbors and previous estimates are fed into and processed by LLM to infer the missing observations. Tested on the task of the online prediction of wind-speed graph signals, our model outperforms online graph filtering algorithms in terms of accuracy, demonstrating the potential of LLMs in effectively addressing partially observed signals in graphs.

Hyperspectral imaging offers extensive spectral and spatial information. However, effectively utilizing this data for accurate classification remains a challenge. This study introduced the CASSX-Net, a novel framework designed to capture both short- and long-range dependencies in HSI data for land cover classification. The network combined a dual spectral-spatial feature extraction mechanism with a multi-head cross-attention module to leverage local and global feature interactions. By combining convolutional layers for short-range feature extraction with cross-attention mechanisms for long-range dependencies, the CASSX-Net addressed the intricate spectral-spatial correlations often missed by traditional CNNs. In addition, the maximal correlation fusion strategy optimally integrated the features from various pathways, improving the ability of the model to distinguish between classes with similar spectral signatures. The rigorous evaluation of four benchmark HSI datasets, including Pavia University, Pavia Centre, Salinas, and Houston 2018, demonstrated that the proposed framework consistently achieved the state-of-the-art performance, surpassing the existing methods in terms of classification accuracy and advancing the HSI land cover classification.

Molecule graph generation plays a crucial role in the field of drug discovery, as it can significantly expedite the identification and development of potential drug candidates. Although many deep graph generative models have been developed, they often generate molecular graphs without optimizing the samples, leading to limited validity and uniqueness, which are key factors in drug discovery. In this work, we introduce a novel approach that incorporates graph statistics, specifically the Spectral Core Entropy, into the generative process. The Spectral Core Entropy, which measures the complexity and structural diversity of a graph, serves as a guiding metric to enhance the quality of generated molecular graphs. By incorporating entropy-based measures, our method generates more valid and unique molecular structures than baseline models. Empirical evaluations show significant improvements in validity and uniqueness, leading to a more diverse set of drug candidates. These results highlight the potential of integrating Spectral Core Entropy into molecular graph generation for advancing computational drug discovery.

Graph neural networks (GNNs) have achieved significant success in a variety of graph-related tasks, such as node classification, link prediction, and graph classification. However, GNNs generally lack the capability to quantify the uncertainty associated with their predictions. In contrast, Bayesian neural networks (BNNs) have been shown to provide robust uncertainty estimates, which are valuable in many applications. Quantifying model uncertainty, which entails assessing and measuring the uncertainty intrinsic to a model’s predictions, is crucial for evaluating its confidence and reliability. One widely adopted approach to improve the robustness of neural networks involves the introduction of randomness, often achieved through noise injection. In this study, we present a Bayesian graph convolutional network (GCN) that incorporates noise into the GCN weights. Specifically, we propose a Monte Carlo Noise Injection (MCNI) approach, where noise is applied to the model parameters during training. Additionally, multiple forward passes are performed during inference to quantify prediction uncertainty. Experimental results demonstrate that our approach outperforms the baseline model in terms of performance.

Autism Spectrum Disorder (ASD) is a complex neurological condition characterized by varied developmental impairments, especially in communication and social interaction. Accurate and early diagnosis of ASD is crucial for effective interventions. The brain functional connectome, which refers to the statistical relationships between different brain regions measured through neuroimaging, provides crucial insights into brain function. Traditional static brain analysis methods often fail to capture the dynamic nature of the brain activity. In contrast, dynamic brain analysis provides a more comprehensive view by capturing the temporal variations in the brain. This work proposes a novel graph-level dynamic brain network embedding approach using Temporal Random Walk with Transformer-based model (BrainTWT) that captures the temporal evolution of brain connectivity over time and considers also the dynamics between different temporal network snapshots. BrainTWT employs temporal random walks to capture dynamics across different temporal network snapshots and leverages the Transformer’s ability to model long-term dependencies in sequential data to learn discriminative embeddings from these temporal sequences using temporal structure prediction tasks. Experimental evaluation using the Autism Brain Imaging Data Exchange (ABIDE) dataset demonstrates that BrainTWT outperforms baseline methods in ASD classification.

Model uncertainty quantification involves measuring and evaluating the uncertainty linked to a model’s predictions, helping assess their reliability and confidence. Noise injection is a technique used to enhance the robustness of neural networks by introducing randomness. In this paper, we establish a connection between noise injection and uncertainty quantification from a Bayesian standpoint. We theoretically demonstrate that injecting noise into the weights of a neural network is equivalent to Bayesian inference on a deep Gaussian process. Consequently, we introduce a Monte Carlo Noise Injection (MCNI) method, which involves injecting noise into the parameters during training and performing multiple forward propagations during inference to estimate the uncertainty of the prediction. Through simulation and experiments on regression and classification tasks, our method demonstrates superior performance compared to the baseline model.

We formulate sequential maximum a posteriori inference as a recursion of loss functions and reduce the problem of continual learning to approximating the previous loss function. We then propose two coreset-free methods: autodiff quadratic consolidation, which uses an accurate and full quadratic approximation, and neural consolidation, which uses a neural network approximation. These methods are not scalable with respect to the neural network size, and we study them for classification tasks in combination with a fixed pre-trained feature extractor. We also introduce simple but challenging classical task sequences based on Iris and Wine datasets. We find that neural consolidation performs well on the classical task sequences, where the input dimension is small, while autodiff quadratic consolidation performs consistently well on image task sequences with a fixed pre-trained feature extractor, achieving comparable performance to joint maximum a posteriori training in many cases.

To enhance the generalization ability of graph neural networks (GNNs) in learning and simulation physical dynamics, a series of equivariant GNNs have been developed to incorporate the symmetric inductive bias. However, the existing methods do not take into account the non-stationarity nature of physical dynamics, where the joint distribution changes over time. Moreover, previous approaches for modeling non-stationary time series typically involve normalizing the data, which disrupts the symmetric assumption inherent in physical dynamics. To model the non-stationary physical dynamics while preserving the symmetric inductive bias, we introduce a Non-Stationary Equivariant Graph Neural Network (NS-EGNN) to capture the non-stationarity in physical dynamics while preserving the symmetric property of the model. Specifically, NS-EGNN employs Fourier Transform on segments of physical dynamics to extract time-varying frequency information from the trajectories. It then uses the first and second-order differences to mitigate non-stationarity, followed by pooling for future predictions. Through capturing varying frequency characteristics and alleviate the linear and quadric trend in the raw physical dynamics, NS-EGNN better models the temporal dependencies in the physical dynamics. NS-EGNN has been applied on various types of physical dynamics, including molecular, motion and protein dynamics. In various scenario, NS-EGNN consistently surpasses the performance of existing state-of-the-art algorithms, underscoring its effectiveness. The implementation of NS-EGNN is available at https://github.com/MaojiWEN/NS-EGNN.

Significant progress has been achieved using Vision Transformers (ViTs) in computer vision. However, challenges persist in modeling multi-scale spatial relationships, hindering effective integration of fine-grained local details and long-range global dependencies. To address this limitation, a Multi-Kernel Correlation-Attention Vision Transformer (MK-CAViT) grounded in the Hirschfeld-Gebelein-Rényi (HGR) theory was proposed, introducing three key innovations. A parallel multi-kernel architecture was utilized to extract multi-scale features through small, medium, and large kernels, overcoming the single-scale constraints of conventional ViTs. The cross-scale interactions were enhanced through the Fast-HGR attention mechanism, which models nonlinear dependencies and applies adaptive scaling to weigh connections and refine contextual reasoning. Additionally, a stable multi-scale fusion strategy was adopted, integrating dynamic normalization and staged learning to mitigate gradient variance, progressively fusing local and global contexts, and improving training stability. The experimental results on ImageNet, COCO, and ADE20K validated the superiority of MK-CAViT in classification, detection, and segmentation, surpassing state-of-the-art baselines in capturing complex spatial relationships while maintaining efficiency. These contributions can establish a theoretically grounded framework for visual representation learning and address the longstanding limitations of ViTs.

A common approach to learning Bayesian networks involves specifying an appropriately chosen family of parameterized probability density such as Gaussian. However, the distribution of most real-life data is leptokurtic and may not necessarily be best described by a Gaussian process. In this work we introduce Cauchy Graphical Models (CGM), a class of multivariate Cauchy densities that can be represented as directed acyclic graphs with arbitrary network topologies, the edges of which encode linear dependencies between random variables. We develop CGLearn, the resultant algorithm for learning the structure and Cauchy parameters based on Minimum Dispersion Criterion (MDC). Experiments using simulated datasets on benchmark network topologies demonstrate the efficacy of our approach when compared to Gaussian Graphical Models (GGM). Most Graph Convolutional Neural Networks (GCN) process input graphs as ground-truth representations of node relationships, yet these graphs are constructed based on modeling assumptions and noisy data and their use may lead to suboptimal performance on downstream prediction tasks. We propose Cauchy GCN which leverages CGM to infer graph topology that depicts latent relationships between nodes. We evaluate the effectiveness and quality of the structural graphs learned by CGM, and demonstrate that Cauchy-GCN achieves superior performance compared to widely used graph construction methods.

This paper proposes Graph Signal Adaptive Message Passing (GSAMP) by employing a distinct approach that utilizes localized computations at each node based on an adaptive solution obtained from an optimization problem designed to minimize the discrepancy between observed and estimated values. This localized approach distinguishes GSAMP from conventional graph methods derived from the entire graph. GSAMP efficiently handles real-world time-varying graph signals under Gaussian and impulsive noise, performing online prediction, missing data imputation, and noise removal simultaneously.

Quantitative markets are characterized by swift dynamics and abundant uncertainties, making the pursuit of profit-driven stock trading actions inherently challenging. Within this context, Reinforcement Learning (RL) — which operates on a reward-centric mechanism for optimal control — has surfaced as a potentially effective solution to the intricate financial decision-making conundrums presented. This paper delves into the fusion of two established financial trading strategies, namely the constant proportion portfolio insurance (CPPI) and the time-invariant portfolio protection (TIPP), with the multi-agent deep deterministic policy gradient (MADDPG) framework. As a result, we introduce two novel multi-agent RL (MARL) methods: CPPI-MADDPG and TIPP-MADDPG, tailored for probing strategic trading within quantitative markets. To validate these innovations, we implemented them on a diverse selection of 100 real-market shares. Our empirical findings reveal that the CPPI-MADDPG and TIPP-MADDPG strategies consistently outpace their traditional counterparts, affirming their efficacy in the realm of quantitative trading.

An increasingly important brain function analysis modality is functional connectivity analysis which regards connections as statistical codependency between the signals of different brain regions. Graph-based analysis of brain connectivity provides a new way of exploring the association between brain functional deficits and the structural disruption related to brain disorders, but the current implementations have limited capability due to the assumptions of noise-free data and stationary graph topology. We propose a new methodology based on the particle filtering algorithm, with proven success in tracking problems, which estimates the hidden states of a dynamic graph with only partial and noisy observations, without the assumptions of stationarity on connectivity. We enrich the particle filtering state equation with a graph Neural Network called Sequential Monte Carlo Graph Convolutional Network (SMC-GCN), which due to the nonlinear regression capability, can limit spurious connections in the graph. Experimental studies demonstrate that SMC-GCN achieves superior performance among several other methods in brain disorder classification.

The increasing complexity and density of graphs present significant challenges in the training of Graph Neural Networks (GNNs), often resulting in a substantial consumption of computational resources. In response to this challenge, it becomes imperative to consider techniques for graph sparsification, among which graph pruning emerges as an effective method. Since Simulated Annealing(SA) is particularly beneficial due to its ability to circumvent local optima and effectively navigate the solution space. In this work, after pruning the graph, we employed SA to incrementally explore and identify the optimal graph structure. Empirical results demonstrate that there exists sub-graphs derived from original graphs and some edges are redundant in fact and after removing them, our GNN can obtain better performance.

Performance degradation due to data heterogeneity and low output interpretability are the most significant challenges faced by federated learning in practical applications. Personalized federated learning diverges from traditional approaches, as it no longer seeks to train a single model, but instead tailors a unique personalized model for each client. However, previous work focused only on personalization from the perspective of neural network parameters and lack of robustness and interpretability. In this work, we establish a novel framework for personalized federated learning, incorporating Bayesian methodology which enhances the algorithm’s ability to quantify uncertainty. Furthermore, we introduce normalizing flow to achieve personalization from the parameter posterior perspective and theoretically analyze the impact of normalizing flow on out-of-distribution (OOD) detection for Bayesian neural networks. Finally, we evaluated our approach on heterogeneous datasets, and the experimental results indicate that the new algorithm not only improves accuracy but also outperforms the baseline significantly in OOD detection due to the reliable output of the Bayesian approach.

Abstract This paper investigates non‐stationary time series analysis and forecasting techniques for financial datasets. We focus on the use of a popular non‐stationary parametric model namely GARCH and neural network model LSTM, with an attention mechanism to capture the complex temporal dynamics and dependencies in the data. We propose a hybrid GARCH‐ATT‐LSTM model where the GARCH model is employed for volatility forecasting, attention mechanism is applied to capture the more important parts of the data sequence and enhance the interpretability of the model, and the LSTM model is used for price forecasting. Our experiments are conducted on real‐world financial datasets, that is, Apple stock price, Dow Jones index, and gold futures price. We compare the performance of GARCH‐ATT‐LSTM against the sole LSTM model, ATT‐LSTM model, and LSTM‐GARCH model. Our results show that GARCH‐ATT‐LSTM outperforms the baseline methods and achieves high accuracy in price forecasting. It implies the effectiveness of the attention mechanism in improving the interpretability and stability of the model and the success of combining parametric models with neural network models. The findings suggest that GARCH‐ATT‐LSTM can be a valuable tool for non‐stationary time series analysis and forecasting in financial applications.

This article introduces a novel probability distribution model, namely, complex isotropic$\alpha $-Stable-Rician (CI$\alpha $SR), for characterizing the data histogram of synthetic aperture radar (SAR) images. Having its foundation situated on the Lévy$\alpha $-stable distribution suggested by a generalized central limit theorem, the model promises great potential in accurately capturing SAR image features of extreme heterogeneity. A novel parameter estimation method based on the generalization of method of moments to expectations of Bessel functions is devised to resolve the model in a relatively compact and computationally efficient manner. Experimental results based on both simulated and empirical SAR data exhibit the CI$\alpha $SR model’s superior capacity in modeling scenes of a wide range of heterogeneity when compared with other state-of-the-art models as quantified by various performance metrics. Additional experiments are conducted utilizing large-swath SAR images, which encompass mixtures of several scenes to help interpret the CI$\alpha $SR model parameters and to demonstrate the model’s potential application in segmentation and target detection.

Although learning-based light field disparity estimation has achieved great progress in the most recent years, the performance of unsupervised light field learning is still hindered by occlusions and noises. By analyzing the overall strategy underlying the unsupervised methodology and the light field geometry implied in epipolar plane images (EPIs), we look beyond the photometric consistency assumption, and design an occlusion-aware unsupervised framework to deal with the situations of photometric consistency conflict. Specifically, we present a geometry-based light field occlusion modeling, which predicts a group of visibility masks and occlusion maps, respectively, by forward warping and backward EPI-line tracing. In order to learn better the noise- and occlusion-invariant representations of the light field, we propose two occlusion-aware unsupervised losses: occlusion-aware SSIM and statistics-based EPI loss. Experiment results demonstrate that our method can improve the estimation accuracy of light field depth over the occluded and noisy regions, and preserve the occlusion boundaries better.

Lithium-ion batteries are becoming increasingly omnipresent in energy supply. However, the durability of energy storage using lithium-ion batteries is threatened by their dropping capacity with the growing number of charging/discharging cycles. An accurate capacity prediction is the key to ensure system efficiency and reliability, where the exploitation of battery state information in each cycle has been largely undervalued. In this paper, we propose a semiparametric deep Gaussian process regression framework named SDG-L to give predictions based on the modeling of time series battery state data. By introducing an LSTM feature extractor, the SDG-L is specially designed to better utilize the auxiliary profiling information during charging/discharging process. In experimental studies based on NASA dataset, our proposed method obtains an average test MSE error of 1.2‰. We also show that SDG-L achieves better performance compared to existing works and validate the framework using ablation studies.

Graph signal processing refers to dealing with irregularly structured data. Compared with traditional signal processing, it can preserve the complex interactions within irregular data. In this work, we devise a robust algorithm to recover band-limited graph signals in the presence of impulsive noise. First, the observed data vector is recast, such that the noise component is divided into two vectors, representing the dense-noise component and sparse outliers, respectively. We then exploit ℓ0-norm to characterize the sparse vector as a regularization term. Alternating minimization is subsequently adopted as the solver for the resultant optimization problem. Besides, we suggest an approach to automatically update the penalty parameter of the ℓ0-norm term. In addition, we analyze the computational complexity and the steady-state convergence of our algorithm. Experimental results on synthetic and temperature data exhibit the superiority of the developed method over state-of-the-art algorithms in impulsive noise environments in terms of recovery accuracy and convergence speed.

This letter presents a new statistical model for urban scene synthetic aperture radar (SAR) images by combining the Cauchy distribution, which is heavy tailed, with the Rician backscattering. The literature spans various well-known models most of which are derived under the assumption that the scene consists of multitudes of random reflectors. This idea specifically fails for urban scenes since they accommodate a heterogeneous collection of strong scatterers such as buildings, cars, and wall corners. Moreover, when it comes to analyzing their statistical behavior, due to these strong reflectors, urban scenes include a high number of high amplitude samples, which implies that urban scenes are mostly heavy-tailed. The proposed Cauchy–Rician model contributes to the literature by leveraging nonzero location (Rician) heavy-tailed (Cauchy) signal components. In the experimental analysis, the Cauchy–Rician model is investigated in comparison to state-of-the-art statistical models that include$\mathcal {G}_{0}$, generalized gamma, and the lognormal distribution. The numerical analysis demonstrates the superior performance and flexibility of the proposed distribution for modeling urban scenes.

We propose multivariate skewed${t}$-distribution (MVSkt) for hyperspectral anomaly detection (AD). The proposed distribution model is able to increase the detection performance of autoencoder (AE)-based anomaly detectors. In the proposed method, the reconstruction error of a deep AE is modeled with a skewed${t}$-distribution. The deep AE network is trained based on adversarial learning strategy by feeding its input with the hyperspectral data cubes. The parameters of the${t}$-distribution model are estimated using variational Bayesian approach. We define an MVSkt-based detection rule for pixel-wise AD. We compare our proposed method with those based on the multivariate normal (MVN) distribution and the robust MVN variance–mean mixture distributions on real hyperspectral datasets. The experimental results show that the proposed approach outperforms other detectors in the benchmark.

We present a novel statistical model, the generalized-Gaussian–Rician (GG-Rician) distribution, for the characterization of synthetic aperture radar (SAR) images. Since accurate statistical models lead to better results in applications such as target tracking, classification, or despeckling, characterizing SAR images of various scenes including urban, sea surface, or agricultural is essential. The proposed statistical model is based on the Rician distribution to model the amplitude of a complex SAR signal, the in-phase and quadrature components of which are assumed to be generalized-Gaussian (GG) distributed. The proposed amplitude GG-Rician model is further extended to cover the intensity of SAR signals. In the experimental analysis, the GG-Rician model is investigated for amplitude and intensity SAR images of various frequency bands and scenes in comparison to state-of-the-art statistical models that include Weibull,$\mathcal {G}_{0}$, Generalized gamma, and the lognormal distribution. The statistical significance analysis and goodness-of-fit test results demonstrate the superior performance and flexibility of the proposed model for all frequency bands and scenes, and its applicability on both amplitude and intensity SAR images.

Over the past decades, the hybrid models which integrate parametric and non-parametric learning models have proven to be a viable method in time series forecasting. Several structures were proposed as a combination of parametric models such as autoregressive moving average (ARIMA) and non-parametric models such as artificial neural network (ANN). Although these models show superior performance than a single model, there is the scope of further improvement if the underlying feature laid in the original data is augmented before applying models. In this work, empirical wavelet transform (EWT) is implemented to decompose the original series into several sub-series which contain unique features at different frequency horizon. The sub-series are then used along with moving average filter (MA), ARIMA and ANN to perform forecasting tasks. The experiments were performed on four public real-world data sets and compared to the other six benchmark models. The results showed that the proposed model achieved considerably higher forecast accuracy.

This paper presents a novel statistical model for the characterisation of synthetic aperture radar (SAR) images of the sea surface. The analysis of ocean surface is widely performed using satellite imagery as it produces information for wide areas under various weather conditions. An accurate SAR amplitude distribution model enables better results in despeckling, ship detection/tracking and so forth. In this paper, we develop a new statistical model, namely the LaplaceRician distribution for modelling amplitude SAR images of the sea surface. The proposed statistical model is based on Rician distribution to model the amplitude of a complex SAR signal, the in-phase and quadrature components of which are assumed to be Laplace distributed. The Laplace-Rician model is investigated for SAR images of the sea surface from COSMO-SkyMed and Sentinel-1 in comparison to state-of-the-art statistical models such as K, lognormal and Weibull distributions. In order to decide on the most suitable model, statistical significance analysis via Kullback-Leibler divergence and Kolmogorov-Smirnov statistics is performed. The results show a superior modelling performance of the proposed model for all of the utilised images.

Counterfactual learning for dealing with missing-not-at-random data (MNAR) is an intriguing topic in the recommendation literature, since MNAR data are ubiquitous in modern recommender systems. Instead, missing-at-random (MAR) data, namely randomized controlled trials (RCTs), are usually required by most previous counterfactual learning methods. However, the execution of RCTs is extraordinarily expensive in practice. To circumvent the use of RCTs, we build an information theoretic counterfactual variational information bottleneck (CVIB), as an alternative for debiasing learning without RCTs. By separating the task-aware mutual information term in the original information bottleneck Lagrangian into factual and counterfactual parts, we derive a contrastive information loss and an additional output confidence penalty, which facilitates balanced learning between the factual and counterfactual domains. Empirical evaluation on real-world datasets shows that our CVIB significantly enhances both shallow and deep models, which sheds light on counterfactual learning in recommendation that goes beyond RCTs.

Synthetic aperture radar (SAR) and ultrasound (US) are two important active imaging techniques for remote sensing, both of which are subject to speckle noise caused by coherent summation of back-scattered waves and subsequent nonlinear envelope transformations. Estimating the characteristics of this multiplicative noise is crucial to develop denoising methods and to improve statistical inference from remote sensing images. In this paper, reversible jump Markov chain Monte Carlo (RJMCMC) algorithm has been used with a wider interpretation and a recently proposed RJMCMC-based Bayesian approach, trans-space RJMCMC, has been utilized. The proposed method provides an automatic model class selection mechanism for remote sensing images of SAR and US where the model class space consists of popular envelope distribution families. The proposed method estimates the correct distribution family, as well as the shape and the scale parameters, avoiding performing an exhaustive search. For the experimental analysis, different SAR images of urban, forest and agricultural scenes, and two different US images of a human heart have been used. Simulation results show the efficiency of the proposed method in finding statistical models for speckle.

Stable random variables are motivated by the central limit theorem for densities with (potentially) unbounded variance and can be thought of as natural generalizations of the Gaussian distribution to skewed and heavy-tailed phenomenon. In this paper, we introduce $\alpha$-stable graphical ($\alpha$-SG) models, a class of multivariate stable densities that can also be represented as Bayesian networks whose edges encode linear dependencies between random variables. One major hurdle to the extensive use of stable distributions is the lack of a closed- form analytical expression for their densities. This makes penalized maximum-likelihood based learning computationally demanding. We establish theoretically that the Bayesian information criterion (BIC) can asymptotically be reduced to the computationally more tractable minimum dispersion criterion (MDC) and develop StabLe, a structure learning algorithm based on MDC. We use simulated datasets for five benchmark network topologies to empirically demonstrate how StabLe improves upon ordinary least squares (OLS) regression. We also apply StabLe to microarray gene expression data for lymphoblastoid cells from 727 individuals belonging to eight global population groups. We establish that StabLe improves test set performance relative to OLS via ten-fold cross-validation. Finally, we develop SGEX, a method for quantifying differential expression of genes between different population groups.

Most existing methods used for gene regulatory network modeling are dedicated to inference of steady state networks, which are prevalent over all time instants. However, gene interactions evolve over time. Information about the gene interactions in different stages of the life cycle of a cell or an organism is of high importance for biology. In the statistical graphical models literature, one can find a number of methods for studying steady-state network structures while the study of time varying networks is rather recent. A sequential Monte Carlo method, namely particle filtering (PF), provides a powerful tool for dynamic time series analysis. In this work, the PF technique is proposed for dynamic network inference and its potentials in time varying gene expression data tracking are demonstrated. The data used for validation are synthetic time series data available from the DREAM4 challenge, generated from known network topologies and obtained from transcriptional regulatory networks of S. cerevisiae. We model the gene interactions over the course of time with multivariate linear regressions where the parameters of the regressive process are changing over time.

In this paper we review a maximum a posteriori (MAP) approach detection method in a Bayesian scheme which incorporates prior information about the source flux distribution, the locations and the number of sources of extragalactic point sources in images of the Cosmic Microwave Background. This new technique allows us to obtain fast solutions and to fix the number of detected sources in a non-arbitrary way. The performance of the method is superior to that of the standard frequentist approach based on the matched filter.

We propose to model the image differentials of astrophysical source maps by Student's t-distribution and to use them in the Bayesian source separation method as priors. We introduce an efficient Markov Chain Monte Carlo (MCMC) sampling scheme to unmix the astrophysical sources and describe the derivation details. In this scheme, we use the Langevin stochastic equation for transitions, which enables parallel drawing of random samples from the posterior, and reduces the computation time significantly (by two orders of magnitude). In addition, Student's t-distribution parameters are updated throughout the iterations. The results on astrophysical source separation are assessed with two performance criteria defined in the pixel and the frequency domains.

Standard hidden Markov model (HMM) and the more general dynamic Bayesian network (DBN) models assume stationarity of state transition distribution. However, this assumption does not hold for many real life events of interest. In this paper, we propose a new time sequence model that extends HMM to time varying scenario. The time varying property is realized in our model by explicitly allowing the change of state transition density as the time spent in a particular state passes by. Rather than keeping transition densities at different time spots independent of each other, we exploit their temporal correlation by applying a hierarchical Dirichlet prior. This leads to a more robust time varying model, especially when training data are scarce. We also employ Markov chain Monte Carlo (MCMC) sampling in learning the MAP estimate of time varying parameters, with a transition kernel incorporating linear optimization. The proposed model is applied to recognizing real video events, and is shown to outperform existing HMM-based methods.

We propose an adaptive Monte Carlo Markov Chain (MCMC) simulation for the Bayesian source separation problem and apply it to the unmixing of astrophysical components. In this method, we use the Langevin stochastic equation for transitions, which enables parallel drawing of random samples from the posterior, and which reduces the computation time significantly (by two orders of magnitude). In addition to this, the parameters of the Markov Random Field (MRF) model are updated via Maximum Likelihood (ML) throughout the iterations.

We investigate the source separation problem of random fields within a Bayesian framework. The Bayesian formulation enables the incorporation of prior image models in the estimation of sources. Due to the intractability of the analytical solution, we resort to numerical methods for the joint maximization of the a posteriori distribution of the unknown variables and parameters. We construct the prior densities of pixels using Markov random fields based on a statistical model of the gradient image, and we use a fully Bayesian method with modified-Gibbs sampling. We contrast our work to approximate Bayesian solutions such as Iterated Conditional Modes (ICM) and to non-Bayesian solutions of ICA variety. The performance of the method is tested on synthetic mixtures of texture images and astrophysical images under various noise scenarios. The proposed method is shown to outperform significantly both its approximate Bayesian and non-Bayesian competitors.

A novel evolutionary global optimization approach based on adaptive covariance estimation is proposed. The proposed method samples from a multivariate Levy Skew Alpha-Stable distribution with the estimated covariance matrix to realize a random walk and so to generate new solution candidates in the mutation step. The proposed method is compared to the popular Differential Evolution method, which is one of the best general evolutionary global optimizers available. Experimental results indicate that the proposed approach yields a general improvement in the required number of function evaluations to solve global optimization problems. Especially, as shown in experiments, the underlying heavy tailed alpha-stable distribution enables a considerably more effective global search in more complex problems.

In this paper, we consider online separation and detection of superimposed events by applying particle filtering. We concentrate on a model where a background process, represented by a ID-signal, is superimposed by an auto-regressive (AR) 'event signal', but the proposed approach is applicable in a more general setting. The activation and deactivation times of the event-signal are assumed to be unknown. We solve the online detection problem of this superpositional event by extending the state space dimension by one. The additional parameter of the state represents the AR-signal, which is zero when deactivated. Numerical experiments demonstrate the effectiveness of our approach.

In this work, we study the estimation of mixtures of symmetric alpha-stable distributions using Bayesian inference. We utilise numerical Bayesian sampling techniques such as Markov chain Monte Carlo (MCMC). Our estimation technique is capable of estimating also the number of alpha-stable components in the mixture in addition to the component parameters and mixing coefficients which is accomplished by using the reversible jump MCMC (RJMCMC) algorithm

Synthetic aperture radar (SAR) images are inherently affected by a signal dependent noise known as speckle, which is due to the radar wave coherence. In this paper, we propose a novel adaptive despeckling filter and derive a maximum a posteriori (MAP) estimator for the radar cross section (RCS). We first employ a logarithmic transformation to change the multiplicative speckle into additive noise. We model the RCS using the recently introduced heavy-tailed Rayleigh density function, which was derived based on the assumption that the real and imaginary parts of the received complex signal are best described using the alpha-stable family of distribution. We estimate model parameters from noisy observations by means of second-kind statistics theory, which relies on the Mellin transform. Finally, we compare the proposed algorithm with several classical speckle filters applied on actual SAR images. Experimental results show that the homomorphic MAP filter based on the heavy-tailed Rayleigh prior for the RCS is among the best for speckle removal.

Recently, the dual-tree complex wavelet transform has been proposed as an analysis tool featuring near shift-invariance and improved directional selectivity compared to the standard wavelet transform. Within this framework, we describe a novel technique for removing noise from digital images. We design a bivariate maximum a posteriori estimator, which relies on the family of isotropic α-stable distributions. Using this relatively new statistical model we are able to better capture the heavy-tailed nature of the data as well as the interscale dependencies of wavelet coefficients. We test our algorithm for the Cauchy case, in comparison with several recently published methods. The simulation results show that our proposed technique achieves state-of-the-art performance in terms of root mean squared (RMS) error.

Within the framework of wavelet analysis, we describe a novel technique for removing noise from astrophysical images. We design a Bayesian estimator, which relies on a particular member of the family of isotropic /spl alpha/-stable distributions, namely the bivariate Cauchy density. Using the bivariate Cauchy model we develop a noise-removal processor that takes into account the interscale dependencies of wavelet coefficients. We show through simulations that our proposed technique outperforms existing methods both visually and in terms of root mean squared error.

Synthetic aperture radar (SAR) imagery has found important applications due to its clear advantages over optical satellite imagery one of them being able to operate in various weather conditions. However, due to the physics of the radar imaging process, SAR images contain unwanted artifacts in the form of a granular look which is called speckle. The assumptions of the classical SAR image generation model lead to a Rayleigh distribution model for the histogram of the SAR image. However, some experimental data such as images of urban areas show impulsive characteristics that correspond to underlying heavy-tailed distributions, which are clearly non-Rayleigh. Some alternative distributions have been suggested such as the Weibull, log-normal, and the k-distribution which had success in varying degrees depending on the application. Recently, an alternative model namely the alpha-stable distribution has been suggested for modeling radar clutter. In this paper, we show that the amplitude distribution of the complex wave, the real and the imaginery components of which are assumed to be distributed by the alpha-stable distribution, is a generalization of the Rayleigh distribution. We demonstrate that the amplitude distribution is a mixture of Rayleighs as is the k-distribution in accordance with earlier work on modeling SAR images which showed that almost all successful SAR image models could be expressed as mixtures of Rayleighs. We also present parameter estimation techniques based on negative order moments for the new model. Finally, we test the performance of the model on urban images and compare with other models such as Rayleigh, Weibull, and the k-distribution.

We present an analytical approximation to positive /spl alpha/-stable probability distribution functions, which in general do not possess a compact analytical form. Our approximation is based on decomposing a positive /spl alpha/-stable random variable into a product of a Pearson and another positive stable random variable. This decomposition allows one to approximate any positive stable pdf as a mixture of Pearson densities, hence providing an analytical representation. This representation allows one to employ maximum likelihood estimation and Bayesian techniques in the presence of positive /spl alpha/-stable noise or signals. The efficiency of the decomposition is demonstrated by simulation studies.

A microwave sky map results from a combination of signals from various astrophysical sources, such as cosmic microwave background radiation, synchrotron radiation and galactic dust radiation. To derive information about these sources, one needs to separate them from the measured maps on different frequency channels. Our insufficient knowledge of the weights to be given to the individual signals at different frequencies makes this a difficult task. Recent work on the problem led to only limited success due to ignoring the noise and to the lack of a suitable statistical model for the sources. In this paper, we derive the statistical distribution of some source realizations, and check the appropriateness of a Gaussian mixture model for them. A source separation technique, namely, independent factor analysis, has been suggested recently in the literature for Gaussian mixture sources in the presence of noise. This technique employs a three layered neural network architecture which allows a simple, hierarchical treatment of the problem. We modify the algorithm proposed in the literature to accommodate for space-varying noise and test its performance on simulated astrophysical maps. We also compare the performances of an expectation-maximization and a simulated annealing learning algorithm in estimating the mixture matrix and the source model parameters. The problem with expectation-maximization is that it does not ensure global optimization, and thus the choice of the starting point is a critical task. Indeed, we did not succeed to reach good solutions for random initializations of the algorithm. Conversely, our experiments with simulated annealing yielded initialization-independent results. The mixing matrix and the means and coefficients in the source model were estimated with a good accuracy while some of the variances of the components in the mixture model were not estimated satisfactorily.

Many classes of noise encountered in real-life exhibit outliers that will not fit into a Gaussian noise model. /spl alpha/-stable distributions are among the most important non-Gaussian models that can be used to accurately model impulsive noise environments. We introduce an algorithm that can be used to jointly estimate DOA (direction of arrival), frequency and model order in /spl alpha/-stable noise. Approximating /spl alpha/-stable noise by a Gaussian mixture, we use Bayesian principles to define a posterior density on the signal and noise parameter space. We describe an efficient stochastic algorithm called reversible jump RCMC (Markov chain Monte Carlo) that is used to evaluate our posterior density.

For the estimation of the model coefficients of a polynomial autoregressive process with non-Gaussian innovations least l/sub p/-norm estimation (LLPN) is suggested. Simulations showed that LLPN estimation leads to better estimates than the least squares estimation in terms of the mean and the standard deviations of the estimates. The algorithm is also employed in modeling audio data in non-Gaussian noise with the objective of separating signal from noise and superior results have been obtained when compared to the linear autoregressive modeling. Directions of future research are also addressed.

Most of the existing coefficient estimation techniques in the literature for autoregressive (AR) symmetric /spl alpha/-stable (S/spl alpha/S) processes require large amounts of data for efficient estimation. However, in many practical cases, either only a short length of data is available or the data is nonstationary. Motivated by the norm of /spl alpha/-stable variables, the AR model coefficient estimation problem is formulated as an l/sub p/-norm minimization problem, and the interactively reweighted least squares (IRLS) is suggested for the solution. The simulation results indicate superior performance when compared to existing methods, especially when only short length data are available.