Causal discovery in time-series data is critical for analyzing dynamic systems across neuroscience, economics, and biomedical signal processing. Traditional methods, such as Vector Auto-regression (VAR) and constraint-based approaches, struggle with high-dimensional dependencies, nonlinear relationships, and non-stationary dynamics. Deep learning-based models, including cMLP, cLSTM, and VAE-based approaches, aim to address these challenges but suffer from instability, over-pruning, and reliance on sparsity constraints. While cMLP provides lag-specific causal inference, its accuracy is limited, and other methods fail to explicitly capture lag-wise dependencies. This paper introduces DVAE-GC, a structured deep learning framework integrating dynamic variational inference with lag-structured recurrent MLPs (lsrMLP) to explicitly model time-lagged causal dependencies. Unlike prior methods that infer causality via weight sparsity, DVAE-GC progressively refines causal estimation, leveraging a bidirectional recurrent encoder and structured decoder. Additionally, Noise Invalidation Soft Thresholding (NIST) eliminates spurious connections, enhancing interpretability and robustness. Empirically, DVAE-GC outperforms the best baseline (CUTS) on VAR(9) by +18.3 absolute F1 points averaged over multiple noise levels, and on NetSim fMRI-20 by +8.1 absolute F1 points averaged over sequence multiple lengths; in simulated atrial rotor detection, it improves Rotational Activity Estimation Precision (RAEP) by +22.4 % over the best alternative (VAR). These are absolute-point gains, and also precision, recall, and false discovery rate (FDR) has been reported. Although evaluated in biomedical simulations, DVAE-GC applies broadly to time-series domains, including neuroscience, climate science, and financial modeling.

Ad hoc retrieval, a cornerstone task in Information Retrieval (IR) , aims to rank documents in response to a user’s query, often without prior knowledge of the user’s specific information need. While transformer-based neural rankers have achieved state-of-the-art performance in ad hoc retrieval, their effectiveness varies significantly across queries. Certain queries—commonly referred to as hard queries —remain particularly challenging, highlighting critical gaps in retrieval models. Identifying these hard queries is essential for improving retrieval systems, motivating the task of Query Performance Prediction (QPP) , which aims to estimate the effectiveness of a query without requiring access to relevance judgments. In this article, we propose Context-aware Query Performance Prediction ( CA-QPP ) , a novel post-retrieval QPP method, which builds on the foundations of perturbation-based QPP methods that hypothesize a relationship between query sensitivity to small perturbations and query retrieval effectiveness. Building on this foundation, our approach exposes the given query to perturbations by constructing two query variations: an effective variation emphasizing terms that enhance retrieval and an ineffective variation accentuating terms that hinder it. By contrasting the retrieval outcomes of these variations using a cross-encoder model, CA-QPP captures the interplay of term contributions and predicts the performance for the given query. We evaluate CA-QPP on the widely used MS MARCO datasets and their associated query sets, including TREC DL 2019 , TREC DL 2020 , DL-Hard , TREC DL 2021 , and TREC DL 2022 , which feature extensive human-labeled relevance judgments. Our experiments demonstrate that CA-QPP consistently outperforms traditional and neural-based QPP baselines across standard correlation metrics, including Pearson’s \(\rho\) , Kendall’s \(\tau\) , and Spearman’s \(\rho\) . Through a detailed case study, we further illustrate the mechanics of CA-QPP and provide empirical evidence for its ability to model the contextual impact of individual query terms, making it a robust framework for query performance prediction.

The Generalized Adaptive Weighted Recursive Least Squares (GAWRLS) dictionary learning method has shown potential for unsupervised dictionary learning. This paper advances GAWRLS by incorporating classification error as an additional cost to enable supervised learning tasks and introduces the Label Consistency for online supervised dictionary learning in classification tasks. The new method is denoted as Label Consistent Generalized Adaptive Weighted Recursive Least Squares Dictionary Learning (LC-GAWRLS). By incorporating both sparse representation error and classification error into the cost function, LC-GAWRLS enables simultaneous learning of the dictionary and classifier parameters. Particularly, to ensure label consistency, the proposed algorithm introduces a correction weight to adaptively regulate the impact of each training data during the model update, enhancing robustness against variations in training data compared to previous dictionary learning methods. Simulation results on real datasets demonstrate that LC-GAWRLS achieves higher classification accuracy compared to existing state-of-the-art supervised dictionary learning methods, particularly in scenarios with limited training samples per class.

This work presents a novel semi-supervised dictionary learning framework that updates the dictionary by online learning and is efficient in utilizing the training data. The method employs a two-stage process to train the dictionary: initial training with limited labeled data, followed by online refinement using abundant unlabeled data. We introduce an adaptive correction weight to control the influence of new unlabeled data on the dictionary update based on its consistency with the current model estimate. This approach enables efficient use of the training data set. Moreover, results in faster dictionary convergence and improves data representation accuracy, especially in scenarios with limited training data. Experimental results demonstrate significant enhancement in the classification accuracy of the proposed method compared to the state-of-the-art semi-supervised dictionary learning methods, particularly when dealing with a limited number of training samples.

This paper investigates the order selection problem for autoregressive models from a new perspective. It is known that the modeling error is a decreasing function of the model complexity and cannot directly be used for order selection. In the proposed approach, denoted by minimum mismatch modeling error (3ME), the modeling error is used to estimate the 3ME which is the true representation of the optimum order. The proposed approach provides probabilistic upper-bounds on the mismatch modeling error using a statistical learning approach. Simulation results on generated synthetic data shows advantages of the 3ME method compared to existing order selection methods such as AIC and BIC as it avoids model overparametrizing or underparametrizing and improves the accuracy. 3ME can automate AR order selection which is a valuable feature. As shown in the simulation results for sleep-stage classification, the automated estimated order can be used as an additional feature in the classification process to increase accuracy.

Graph clustering algorithms require the number of clusters as an input. However, in many real-world practical applications, the correct number of clusters is unknown. Determining the optimal number of clusters for graph clustering algorithms is an essential and challenging task, which is a form of model order selection. Here, we propose a new algorithm for estimating the number of clusters in a graph using the centrality measure. In graph theory, the centrality measure is used for determining the most important and most influential nodes within a graph. The proposed centrality based number of cluster estimation (CB-NCE) method considers minimizing the probabilistic bounds on the average central error of centrality. The desired criterion represents an information theoretic distance measure in the form of description length of centrality. The simulation results show the superior performance of the proposed algorithm among other existing methods, in terms of clustering performance metrics such as normalized mutual information, Rand index, and F-measure.

Multi‐lead ECG compression (M‐lEC) has attracted tremendous attention in long‐term monitoring of the patient's heart behaviour. This study proposes a method denoted by block sparse M‐lEC (BlS M‐lEC) in order to exploit between‐lead correlations to compress the signals in a more efficient way. This is due to the fact that multi‐lead electrocardiography signals are multiple observations of the same source (heart) from different locations. Consequently, they have a high correlation in terms of the support set of their sparse models which leads them to share dominant common structure. In order to obtain the block sparse model, the collaborative version of lasso estimator is applied. In addition, it is shown that raised cosine kernel has advantages over conventional Gaussian and wavelet (Daubechies family) due to its specific properties. It is demonstrated that using raised cosine kernel in constructing the sparsifying basis matrix gives a sparser model which results in higher compression ratio and lower reconstruction error. The simulation results show the average improvement of 37, 88 and 90–97% for BlS M‐lEC compared to the non‐collaborative case with raised cosine kernel, Gaussian kernel and collaborative case with Daubechies wavelet kernels, respectively, in terms of reconstruction error while the compression ratio is considered fixed.

Blind source separation (BSS), i.e. extracting unknown sources from mixtures of them, has attracted great interest in various fields of signal processing, for instance in neurophysiological data analysis. By increasing the availability of multiple and complementary data, associated with a given case, many joint BSS (JBSS) algorithms have been developed, which attempt to jointly analyze all datasets and meaningfully integrate information from them. In this letter, we study the special case of multiple datasets analysis, where all datasets share the same set of underlying sources which contribute to different datasets through different mixing matrices. With the aim of efficient dimension reduction, we propose a novel joint preprocessing method to accurate and robust source separation from multipldatasets. We sequentially separate the source with the most significant impact on all datasets on average. To this end, the magnitudes of source projections on the subspaces spanned by each dataset are obtained and sum of their pth power is maximized. Dependency between datasets can be efficiently treated using p. Simulation results show the considerable improvement of the proposed method over existing JBSS algorithms.

Nonlocal means is one of the well known and mostly used image denoising methods. The conventional nonlocal means approach uses weighted version of all patches in a search neighbourhood to denoise the center patch. However, this search neighbourhood can include some dissimilar patches. In this letter, we propose a pre-processing hard thresholding algorithm that eliminates those dissimilar patches. Consequently, the method improves the performance of nonlocal means. The threshold is calculated based on the distribution of distances of noisy similar patches. The method denoted by Similarity Validation Based Nonlocal Means (NLM-SVB) shows improvement in terms of PSNR and SSIM of the retrieved image in comparison with nonlocal means and some recent variations of nonlocal means.

Block-matching and 3D filtering (BM3D) has shown a great success in image denoising. In this work we propose a new denoising approach for Magnetic Resonance Imaging (MRI) based on a modified BM3D algorithm. BM3D is a combination of nonlocal approach, 3D wavelet shrinkage, and 3D Wiener filtering. We improve the wavelet thresholding stage of BM3D using Noise Invalidation Denoising (NIDe) technique. The new approach provides the optimum wavelet threshold automatically and adaptive to the statistical characteristics of the available data. This is an advantage over the existing denoising stage of BM3D that currently uses an adhoc thresholding value. Combining the proposed BM3D approach with Variance Stabilization Transformation (VST) enables the use of the proposed method for Magnetic Resonance (MR) Image denoising. In our Simulations, the proposed method outperforms the state of the art BM3D based MRI denoising methods in the sense of PSNR and SSIM for T1, T2 and PD weighted MR images.

This paper provides a new unimodality test with application in hierarchical clustering methods. The proposed method denoted by signature test (Sigtest), transforms the data based on its statistics. The transformed data has much smaller variation compared to the original data and can be evaluated in a simple proposed unimodality test. Compared with the existing unimodality tests, Sigtest is more accurate in detecting the overlapped clusters and has a much less computational complexity. Simulation results demonstrate the efficiency of this statistic test for both real and synthetic data sets.

Low dose X-ray Computed Tomography (CT) is clinically desired to reduce the risk of cancer caused by X-ray radiation. Compressed Sensing (CS), which allows images to be formed from incomplete data, enables large dose reduction to be achieved. Though this remains to be clinically unrealized due to excessive computation times. In this paper we demonstrate a fast, complete CS-based ℓ2-TV minimizing CT reconstruction method applicable to both parallel and fan beam geometries to recover high quality images from highly undersampled (thus low-dose) data. We apply the fast pseudo-polar Fourier algorithm and the Central Slice Theorem to reduce the computation time of CS recovery. On a typical desktop computer, we are able to reconstruct a 512×512 CT image in approximately 30 seconds: a clinically-significant speedup compared to the many hours required by previous CS methods.

Compressed Sensing is the new trend in the signal processing context which aims to sample a compressible signal with a rate less than the Nyquist lower bound sampling rate. The main challenge arises due to the non‐convex optimisation problem to be solved in the reconstruction stage. This paper introduces a suitable objective function in order to simultaneously recover the true support of the underlying sparse signal while achieving an acceptable estimation error. Inspired by the well‐known Lasso objective function, we have developed an objective function based on a new penalty denoted by the Linearised Exponentially Decaying (LED) penalty. The comprehensive analysis of the LED based objective function shows that the new approach satisfies the oracle properties, as opposed to the conventional Lasso objective function. Furthermore, we have developed a Sequential Adaptive Coordinate‐wise (SAC) solution for the proposed objective function. The simulation results for the proposed LED‐SAC reconstruction algorithm are given and compared with other state of the art methods. It is shown that LED‐SAC approaches the least mean squared error criterion. Moreover, compared to the other methods, LED‐SAC has much more adaptation rate in terms of tracking the variations in the support of the underlying sparse signal.

Hyperspectral imaging analysis aims at the estimation of the number of constituent substances, known as endmembers, their spectral signatures as well as their abundance fractions . Due to the nature of hyperspectral sensors, output data is mostly associated with correlated noise rather than with the white Gaussian noise considered in most of the analysis. In the presence of correlated noise, estimation of dimensionality with the assumption of white noise is associated with consider able error. This error in the very first step will be propagated to the next steps and fully invalidate the unmixing process. On the other hand, existing methods which consider a correlated noise are lacking in robustness to noise. A Whitened Noiseless Code-length method (WNCLM) is presented for hyperspectral signals dimension estimation and unmixing in the presence of spectrally or spatially correlated noise. Variance and correlation coefficients are calculated to estimate the noise correlation matrix. This matrix is further used to whiten the noise. New processed hyperspectral data then goes through a simultaneous denoising and Least Square Error (LSE) based unmixing process that leads to the estimation of data dimensionality. Some numerical simulations are provided to illustrate the effectiveness of our proposed method.

We present a novel approach to estimating the mean square error (MSE) associated with any given threshold level in both hard and soft thresholding. The estimate is provided by using only the data that is being thresholded. This adaptive approach provides probabilistic confidence bounds on the MSE. The MSE bounds can be used to evaluate the denoising method. Our simulation results confirm that not only does the method provide an accurate estimate of the MSE for any given thresohlding method, but the proposed method can also search and find an optimum threshold for any noisy data with regard to MSE.

In this paper, we address the problem of order selection in noisy hyperspectral applications. In conventional unmixing methods, this problem has been divided into two separate processes of order selection and unmixing. Order selection methods generally use a denoising approach at the beginning stage. The data in this case pass through three stages: denoising, order selection, and unmixing. Each of these steps mainly aims to optimize a different criterion independently. In addition, any error created in the denoising process will be propagated not only to the order selection stage but also consequently to the unmixing results. Commonly used denoising methods such as eigenvalue-decomposition-based methods, e.g., singular-value-decomposition-based methods, provide a threshold value to separate the noise from the signal. These approaches are heavily sensitive to the threshold value and signal-to-noise ratio (SNR). Moreover, these methods tend to lose their efficiency rapidly for lower SNRs. Note that both the denoising step and the dimension estimation step aim to provide the optimum estimate of the same noiseless data. Consequently, adopting a simultaneous denoising and dimension estimation method with a goal to provide the optimum estimate of the desired noiseless data is rational. This process not only avoids possible error propagations from the denoising stage to the dimension estimation stage but also unifies the optimization criteria that were used in each of these steps. In this paper, a simultaneous denoising and dimension estimation method is introduced. The approach is based on minimizing the estimated mean square error. Minimization is done by comparing the estimated data in a range of subspaces dictated by a simultaneous process. Minimizing the error at once, the proposed method denoises the data and provides the optimum dimension simultaneously. Owing to the parallel processing of denoising and dimension estimation, the simulation results show the advantages of the proposed method over some of the state-of-the-art approaches and illustrate a substantial performance, particularly for cases with a lower SNR.

A method of noise variance estimation in BayesShrink image denoising is presented. The proposed approach competes with the well known MAD-based method and outperforms this method in more than 99% of our experimental results. The approach, called Residual Autocorrelation Power (RAP), provides a more accurate noise variance estimate and results in a smaller MSE.

Hyperspectral image analysis has been subjected to many improvements made in past decade. Yet the accurate estimation of dimensionality is still a challenge. Since dimension estimation of the hyperspectral data is the first step in analysis of an image, the accuracy of analysis results highly depends on the accuracy of the dimension estimation step. Mostly, existing methods isolate the process of dimension estimation and process of denoising which leads to an inaccurate estimation of constituent components in the signal. In this paper, the problem of estimating the dimensionality of hyperspectral data using the concept of ldquonoiseless code lengthrdquo is addressed. In our proposed method, NCLM, a set of nested subsets including the hyperspectral data is generated first and then an error comparison approach is utilized by estimating the noiseless data error rather than noisy data error used by the existing methods to find the optimum subset. It has been shown that the estimated noiseless error has a minimum that represents the accurate estimation of the dimensionality of hyperspectral data. The comparison of NCLM to other methods shows a substantial improvement in estimation of dimensionality in hyperspectral imagery.

This paper addresses problem of data compression when partial information on data structure is available and optimum code is known to be among a set of given parametric codes. The goal of the proposed method is to choose the optimum parametric code by using an observed finite length data that is generated by an unknown parameter. We provide a new approach that compares estimates of different order among the given parametric codes and chooses the one with minimum probabilistic worst-case average codelength (ACL).

This paper addresses the estimation of the Kulback-Leibler (KL) distance in data-driven modeling of parametric probability distributions. Given a finite observation of a parametric probability density function (pdf), the goal is to provide the best representative of the true parameter, which is known to belong to a given parametric model set. The first step in this problem setting is to estimate the true parameter in available nested model sets of different orders. The proposed method calculates the KL distance between these estimates and the unknown true parameter. By using only the observed data, we provide probabilistic worst case bounds on these KL distances. The best candidate among the available estimates is the solution of a resulting probabilistic min-max problem. A comparison of this approach with existing methods that estimate the KL distance is provided.

One important challenge in parametric spectrum estimation is the choice of an optimum number of parameters. In this paper, we provide a new method of order selection for the spectrum density estimation denoted by data model error whiteness (DMEW) criterion. Unlike the existing methods, this approach estimates the parameters and selects the optimum order simultaneously. We demonstrate the advantages of the new method over the existing approaches

In the thresholding method of denoising the optimum threshold is obtained as a function of additive noise variance. In practical problems, where the variance of the noise is unknown, the first step is to estimate the noise variance. The estimated noise variance is then implemented in calculation of the optimum threshold. The current available methods of variance estimation are heuristic. Here, we provide a new method for estimation of the additive noise variance. The method is derived from a new denoising method which is proposed in Beheshti et al. (2002). Unlike thresholding approaches the denoising method in Beheshti is based on comparison of subspaces of the basis. It compares a defined description length (DL) of the noisy data in the subspaces. We show how the estimation of the noise variance and the denoising process can be done simultaneously.