Autonomous agents are increasingly deployed in dynamic environments where their ability to perform a given task depends on both individual and collective proficiency. While PSA has been studied for single agents, its extension to a team of agents remains underexplored. This letter addresses this gap by introducing a framework for team PSA in centralized settings. Specifically, we investigate two metrics for centralized team PSA: the MPB and the KS statistic. These metrics quantify the in-situ discrepancy between predicted and actual measurements. Then, we use the KL divergence as a reference metric. Simulations in a target tracking scenario demonstrate that both MPB and KS metrics accurately capture model mismatches, align with the KL divergence reference, and enable real-time proficiency assessment.

Flexible and scalable decentralized learning solutions are fundamentally important in the application of multi-agent systems. While several recent approaches introduce (ensembles of) kernel machines in the distributed setting, Bayesian solutions are much more limited. We introduce a fully decentralized, asymptotically exact solution to computing the random feature approximation of Gaussian processes. We further address the choice of hyperparameters by introducing an ensembling scheme for Bayesian multiple kernel learning based on online Bayesian model averaging. The resulting algorithm is tested against Bayesian and frequentist methods on simulated and real-world datasets.

In this paper, we propose Online LASSO - a version of LASSO that is configured for streaming data. In standard LASSO, the penalty parameter is typically chosen by cross-validation, a procedure which requires the entire dataset upfront and repeated fitting. The main contribution of this work is in finding an easy and principled choice for the penalty parameter for every incoming data point, in cases where the input features are uncorrelated. The proposed Online LASSO has several benefits: i) it is memory and time efficient ii) it is easy to implement, iii) it does not require an initial batch of data to start, iv) it does not require any tuning (e.g., step size or tolerance), and finally v) it converges to the performance of the optimal predictor and correct selection of features. We demonstrate these capabilities and compare Online LASSO with standard LASSO as well as other adaptive LASSO variations and provide a discussion on their performances.

Bayesian Optimization (BO) is a popular black-box optimization method consisting of a surrogate model, typically a probabilistic model such as a Gaussian Process (GP) and an Acquisition function (AF). Effective selection of these functions has a strong impact on the optimization process. Existing ensemble-based methods for AF selection operate under the assumption that an "optimal" AF exists in the pool of considered AFs, and the method attempts to find the best AF. In this work, we operate in an agnostic setting and consider the optimal AF to be one which minimizes joint risk over all AFs considered. This allows us to treat the joint risk as a random process and perform Bayesian inference on the posterior. We empirically demonstrate the effectiveness of this method and provide theoretical bounds on the regret.

This paper introduces a novel structure for deep Gaussian processes (DGPs) and a method for determining their depths. The proposed framework enables faster convergence of their parameters and reduces computational cost to optimize them while maintaining performance comparable to that of conventional DGP models. Furthermore, our approach presents a feasible solution to reduce the risk that the model becomes trapped in local minima during the simultaneous training of multiple layers, which ensures more efficient and reliable model training. Through experimental evaluation, we demonstrate the effectiveness of these models and the advantages of integrating transfer learning to reduce retraining costs.

A backscatter tag-to-tag network enables battery-less communication by harvesting energy and reflecting wireless signals between tags, making it ideal for energy-efficient IoT applications such as asset tracking, structural health monitoring, and environmental sensing. Accurate localization is crucial for these applications. While RSSI-based (Received Signal Strength Indicator) localization is the most common method for RF localization—estimating distance based on the received signal strength—it is often dependent on the position and power of the excitation source. We present a novel distance estimation method based on the estimation of the channel path loss and phase between tags, which is independent of the excitation source’s position and power. The experimental results demonstrate millimeter-level accuracy in 67% of cases and 99% accuracy within 17 cm for tag-to-tag distances up to 2.4 meters at 915 MHz.

Random feature latent variable models (RFLVMs) are state-of-the-art tools for uncovering structure in high-dimensional, non-Gaussian data. However, their reliance on Monte Carlo sampling significantly limits scalability, posing challenges for large-scale applications. To overcome these limitations, we develop a scalable RFLVM framework based on variational Bayesian inference (VBI), a deterministic and optimization-based alternative to sampling methods. Applying VBI to RFLVMs is nontrivial due to two key challenges: (i) the lack of an explicit probability density function (PDF) for Dirichlet process (DP) mixing weights, and (ii) the inefficiency of existing VBI approaches when handling the high-dimensional variational parameters of RFLVMs. To address these issues, we adopt the stick-breaking construction for the DP, which provides an explicit and tractable PDF over mixing weights, and propose a novel inference algorithm, block coordinate descent variational inference (BCD-VI), which partitions variational parameters into blocks and applies tailored solvers to optimize them efficiently. The resulting scalable model, referred to as SRFLVM, supports various likelihoods; we demonstrate its effectiveness under Gaussian and logistic settings. Extensive experiments on diverse benchmark datasets show that SRFLVM achieves superior scalability, computational efficiency, and performance in latent representation learning and missing data imputation, consistently outperforming state-of-the-art latent variable models, including deep generative approaches.

Online prediction of time series under regime switching is a widely studied problem in the literature, with many celebrated approaches. Using the non-parametric flexibility of Gaussian processes, the recently proposed INTEL algorithm provides a product of experts approach to online prediction of time series under possible regime switching, including the special case of outliers. This is achieved by adaptively combining several candidate models, each reporting their predictive distribution at time t. However, the INTEL algorithm uses a finite context window approximation to the predictive distribution, the computation of which scales cubically with the maximum lag, or otherwise scales quartically with exact predictive distributions. We introduce LINTEL, which uses the exact filtering distribution at time t with constant-time updates, making the time complexity of the streaming algorithm optimal. We additionally note that the weighting mechanism of INTEL is better suited to a mixture of experts approach, and propose a fusion policy based on arithmetic averaging for LINTEL. We show experimentally that our proposed approach is over five times faster than INTEL under reasonable settings with better quality predictions.

In many science and engineering problems, we observe high-dimensional data acquired sequentially. At each time instant, these data correspond to one of a predefined number of classes. The sequence of classes follows a certain pattern, with the transition probabilities of the classes being unknown. Our hypothesized generative model of the observed data involves two latent processes. The first is a root process representing the sequence of classes, while the second is a low-dimensional process generated as a Markovian process, depending on the current class and the previous value of the low-dimensional process. The observed high-dimensional process is generated from the low-dimensional state process. Our objective is to infer the posterior distributions of the classes as they evolve over time based on the observed data and the adopted model. To achieve this, we propose a method for estimating the latent processes. We demonstrate the effectiveness of our approach on synthesized data.

Robotic swarms or portable sensor networks are emerging technologies for sensing physical processes that are spatially distributed- and temporally dynamic, both on Earth and in future Moon/Mars exploration missions. We develop a portable network composed of a multitude of self-organized “sensor eggs”. These eggs are equipped with ultra-wideband (UWB) transceivers, providing precise time and position information without additional infrastructures like Global Navigation Satellite Systems (GNSSs). Each egg is additionally equipped with environmental sensors, for example, a Sulfur dioxide gas sensor to explore volcanic activity. We use a real time decentralized particle filter (DPF) to estimate the a-posteriori probability density functions (PDFs) of the egg positions. These PDFs are then used in a static state binary Bayes filter for estimating the gas sources with potentially complex structures such as cracks on the volcano surface. The proposed sensor network is verified with an in-field experiment at La Fossa volcano on the island of Vulcano, Italy, in 2023.

Bayesian optimization (BO) is a popular approach to optimizing costly, black-box functions that rely on a statistical surrogate model of the function to select new query points, balancing exploration and exploitation of the parameter space. Most of the work on BO has focused on the time-invariant setting where the function does not change over time. Recently, the time-varying BO (TV-BO) framework has been introduced to handle non-stationary functions. In this work, we explore TV-BO with the use of dynamic random feature-based Gaussian processes (DRF-GPs). These processes capture the nonstationarity of the unknown functions by evolving the parameter vector of a linear model. We propose an evolution mechanism that results in an acquisition function with sensible exploitation-exploration trade-offs over time. We compare the resulting algorithm with the TV-BO baseline algorithms on a toy example and a localization problem with synthetic data.

Conventionally, it is often assumed that the training and testing data distributions are the same and that all classes in the test set are observed in the training set. However, this assumption may not be true in real-world tasks. In practice, there may be test samples from classes that were unknown during training. In such cases, one would like the adopted model to have the capacity to classify such samples into a "none of the above" class. This task is known as open-set recognition, and it has gained significant attention in recent years. In this paper, we propose a novel distance-based open-set recognition approach by constructing task-specific distance metrics with Gaussian processes. Our experimental results demonstrate that the proposed approach outperforms state-of-the-art methods on both synthetic and real-world datasets.

In this paper, we explore the estimation of directed time-varying graph topologies, which represent evolving relationships among nodes in high-dimensional, interdependent data. We introduce a novel fully Bayesian method based on Gaussian processes, employing random walks to model the time-varying edge weights with time. This approach accommodates nonlinear and time-varying lagged relationships among time series. We implement the proposed method using the Hamiltonian Monte Carlo method. Numerical tests reveal that our method performs very well and is comparable to state-of-the-art methods, making it a promising tool for unveiling dynamic graph structures and causality.

The inherent adaptability and flexibility of Gaussian processes lie in the capability of their kernel functions to capture diverse data characteristics. Thus, selecting an appropriate kernel function is crucial because an improper choice can detrimentally affect the model’s performance. One way to enhance the capability of Gaussian processes in capturing intricate data features is by stacking multiple Gaussian processes, thus constructing deep Gaussian processes. In this paper, we propose a novel structure for deep Gaussian processes that uses an ensemble of diverse kernels. We demonstrate that this structure leads to superior results compared to those of a single-kernel deep Gaussian processes.

Detection of anomalies is a common and important problem, especially when anomalies are rare and labels are difficult to acquire. Here we sequentially detect outliers in the outputs of an unknown function, which have been distorted by noise. We model the sequence of outputs by using Yule-Simon processes and provide an iterative algorithm for learning the function from input and output data using Gaussian processes. We tested our method by using both synthetic and real-world data. The experimental results indicate excellent performance of the proposed method.

Causal discovery with time series data remains a challenging yet increasingly important task across many scientific domains. Convergent cross mapping (CCM) and related methods have been proposed to study time series that are generated by dynamical systems, where traditional approaches like Granger causality are unreliable. However, CCM often yields inaccurate results depending upon the quality of the data. We propose the Tangent Space Causal Inference (TSCI) method for detecting causalities in dynamical systems. TSCI works by considering vector fields as explicit representations of the systems' dynamics and checks for the degree of synchronization between the learned vector fields. The TSCI approach is model-agnostic and can be used as a drop-in replacement for CCM and its generalizations. We first present a basic version of the TSCI algorithm, which is shown to be more effective than the basic CCM algorithm with very little additional computation. We additionally present augmented versions of TSCI that leverage the expressive power of latent variable models and deep learning. We validate our theory on standard systems, and we demonstrate improved causal inference performance across a number of benchmark tasks.

In this article, we propose novel Gaussian process-gated hierarchical mixtures of experts (GPHMEs). Unlike other mixtures of experts with gating models linear in the input, our model employs gating functions built with Gaussian processes (GPs). These processes are based on random features that are non-linear functions of the inputs. Furthermore, the experts in our model are also constructed with GPs. The optimization of the GPHMEs is performed by variational inference. The proposed GPHMEs have several advantages. They outperform tree-based HME benchmarks that partition the data in the input space, and they achieve good performance with reduced complexity. Another advantage is the interpretability they provide for deep GPs, and more generally, for deep Bayesian neural networks. Our GPHMEs demonstrate excellent performance for large-scale data sets, even with quite modest sizes.

In many machine learning problems, one has to work with data of different types, including continuous, discrete, and categorical data. Further, it is often the case that many of these data are missing from the database. This paper proposes a Gaussian process framework that efficiently captures the information from mixed numerical and categorical data that effectively incorporates missing variables. First, we propose a generative model for the mixed-type data. The generative model exploits Gaussian processes with kernels constructed from the latent vectors. We also propose a method for inference of the unknowns, and in its implementation, we rely on a sparse spectrum approximation of the Gaussian processes and variational inference. We demonstrate the performance of the method for both supervised and unsupervised tasks. First, we investigate the imputation of missing variables in an unsupervised setting, and then we show the results of joint imputation and classification on IBM employee data.

In science and engineering, we often deal with signals that are acquired from time-varying systems represented by dynamic graphs. We observe these signals, and the interest is in finding the time-varying topology of the graphs. We propose two Bayesian methods for estimating these topologies without assuming any specific functional relationships among the signals on the graphs. The two methods exploit Gaussian processes, where the first method uses the length scale of the kernel and relies on variational inference for optimization, and the second method is based on derivatives of the functions and Monte Carlo sampling. Both methods estimate the time-varying topologies of the graphs sequentially. We provide numerical tests that show the performance of the methods in two settings.

During the process of childbirth, fetal distress caused by hypoxia can lead to various abnormalities. Cardiotocography (CTG), which consists of continuous recording of the fetal heart rate (FHR) and uterine contractions (UC), is routinely used for classifying the fetuses as hypoxic or non-hypoxic. In practice, we face highly imbalanced data, where the hypoxic fetuses are significantly underrepresented. We propose to address this problem by boost ensemble learning, where for learning, we use the distribution of classification error over the dataset. We then iteratively select the most informative majority data samples according to this distribution. In our work, in addition to addressing the imbalanced problem, we also experimented with features that are not commonly used in obstetrics. We extracted a large number of statistical features of fetal heart tracings and uterine activity signals and used only the most informative ones. For classification, we implemented several methods: Random Forest, AdaBoost, k-Nearest Neighbors, Support Vector Machine, and Decision Trees. The paper provides a comparison in the performance of these methods on fetal heart rate tracings available from a public database. Our results on the publicly available Czech database show that most applied methods improved their performances considerably when boost ensemble was used.

Low umbilical artery pH is a marker for neonatal acidosis and is associated with an increased risk for neonatal complications. The phase-rectified signal averaging (PRSA) features have demonstrated superior discriminatory or diagnostic ability and good interpretability in many biomedical applications including fetal heart rate analysis. However, the performance of PRSA method is sensitive to values of the selected parameters which are usually either chosen based on a grid search or empirically in the literature. In this paper, we examine PRSA method through the lens of dynamical systems theory and reveal the intrinsic connection between state space reconstruction and PRSA. From this perspective, we then introduce a new feature that can better characterize dynamical systems comparing with PRSA. Our experimental results on an open-access intrapartum Cardiotocography database demonstrate that the proposed feature outperforms state-of-the-art PRSA features in pH-based fetal heart rate analysis.

Yule-Simon processes are one of the most commonly occurring processes in Nature. These processes generate power laws using a preferential attachment mechanism which can describe a variety of data distributions such as word frequencies, scientific citations, journal publications, income, node connections in complex networks, biological genera, and bosons in quantum states. Much of the work in this area has focused on modeling the properties of observable quantities such as these. In this work we focus on learning the properties of unobservable Yule-Simon processes which control the dynamics of sequential sensor measurements. This is motivated by the fact that Yule-Simon processes have a varying memory length which offer a more general framework for data modeling than hidden Markov models. In this paper we present an approximate online learning procedure based on multiple hypothesis pruning which is shown to reach 0.5dB of the posterior Cramer-Rao lower bound.

Determining the number of latent variables, or the dimensions of latent states, is a ubiquitous problem in dimension reduction. In this paper, we introduce a novel sequential method that relies on the Bayesian approach to estimate the dimension of a latent space of a Gaussian process latent variable model. The proposed method also considers settings where the number of latent variables varies with time. To evaluate our methodology, we compared the estimated dimensions with the true dimensions as they vary with time. Results on synthetic data demonstrate that our method has a very good performance.

The computer-aided interpretation of fetal heart rate (FHR) and uterine contraction (UC) has not been developed well enough for wide use in delivery rooms. The main challenges still lie in the lack of unclear and nonstandard labels for cardiotocography (CTG) recordings, and the timely prediction of fetal state during monitoring. Rather than traditional supervised approaches to FHR classification, this paper demonstrates a way to understand the UC-dependent FHR responses in an unsupervised manner. In this work, we provide a complete method for FHR-UC segment clustering and analysis via the Gaussian process latent variable model, and density-based spatial clustering. We map the UC-dependent FHR segments into a space with a visual dimension and propose a trajectory-based FHR interpretation method. Three metrics of FHR trajectory are defined and an open-access CTG database is used for testing the proposed method.

Large scale networks of intelligent sensors that can function without any batteries will have enormous implications in applications that range from smart spaces to structural and environmental monitoring. RF tags present an amenable platform for sensor integration as the backscatter communication offers low energy cost of communication. Current RF tags either use extremely low-power sensors or perform tasks of tag localization and identification based on the strength of the backscatter signal. We present a technique for estimation of amplitude and phase of the tag-to-tag channel that can be performed with very limited computational and energy resources. This enables monitoring of the interactions between tagged objects and activities around tags, as well as assessment of a variety of engineering structures. Experimental results demonstrate high resolution in the amplitude and phase channel measurement at a distances ranging from 22 cm to 1.34 m.

Fusing probabilistic information is a fundamental task in signal and data processing with relevance to many fields of technology and science. In this work, we investigate the fusion of multiple probability density functions (pdfs) of a continuous random variable or vector. Although the case of continuous random variables and the problem of pdf fusion frequently arise in multisensor signal processing, statistical inference, and machine learning, a universally accepted method for pdf fusion does not exist. The diversity of approaches, perspectives, and solutions related to pdf fusion motivates a unified presentation of the theory and methodology of the field. We discuss three different approaches to fusing pdfs. In the axiomatic approach, the fusion rule is defined indirectly by a set of properties (axioms). In the optimization approach, it is the result of minimizing an objective function that involves an information-theoretic divergence or a distance measure. In the supra-Bayesian approach, the fusion center interprets the pdfs to be fused as random observations. Our work is partly a survey, reviewing in a structured and coherent fashion many of the concepts and methods that have been developed in the literature. In addition, we present new results for each of the three approaches. Our original contributions include new fusion rules, axioms, and axiomatic and optimization-based characterizations; a new formulation of supra-Bayesian fusion in terms of finite-dimensional parametrizations; and a study of supra-Bayesian fusion of posterior pdfs for linear Gaussian models.

We present the problem of measuring the strength of a causal interaction, starting from the linear perspective and generalizing to a nonlinear measure of causal influence. The proposed measure of causal strength is interpretable and we demonstrate that it may be estimated efficiently using Gaussian process regression. We validate our results on several examples and connect our results to the existing causal inference literature.

We address the problem of localizing an (unauthorized) transmitter using a distributed set of sensors. Our focus is on developing techniques that perform the transmitter localization in an efficient manner, wherein the efficiency is defined in terms of the number of sensors used to localize. Localization of unauthorized transmitters is an important problem which arises in many important applications, e.g., in patrolling of shared spectrum systems for any unauthorized users. Localization of transmitters is generally done based on observations from a deployed set of sensors with limited resources, thus it is imperative to design techniques that minimize the sensors’ energy resources. In this paper, we design greedy approximation algorithms for the optimization problem of selecting a given number of sensors in order to maximize an appropriately defined objective function of localization accuracy. The obvious greedy algorithm delivers a constant-factor approximation only for the special case of two hypotheses (potential locations). For the general case of multiple hypotheses, we design a greedy algorithm based on an appropriate auxiliary objective function—and show that it delivers a provably approximate solution for the general case. We develop techniques to significantly reduce the time complexity of the designed algorithms by incorporating certain observations and reasonable assumptions. We evaluate our techniques over multiple simulation platforms, including an indoor as well as an outdoor testbed, and demonstrate the effectiveness of our designed techniques—our techniques easily outperform prior and other approaches by up to 50-60% in large-scale simulations and up to 16% in small-scale testbeds.

Regime-switching state-space models (RS-SSMs) are an important class of statistical models that can be used to represent real-world phenomena. Unlike regular state-space models, RS-SSMs allow for dynamic uncertainty in the state transition and observations distributions, making them much more expressive. Unfortunately, there are no existing Bayesian inference techniques for joint estimation of regimes, states, and model parameters in generic RS-SSMs. In this work, we develop a particle Gibbs sampling algorithm for Bayesian learning in RS-SSMs. We demonstrate the proposed inference approach on a synthetic data experiment related to an ecological application, where the goal is in estimating the abundance and demographic rates of penguins in the Antarctic.

In this paper, we develop a new method for estimating the tail-index found in extreme value statistics. Using a fixed quantile, model-selection approach, we derive the posterior distribution of the tail-index marginalizing out the unknown threshold and nuisance parameters. Our marginalized threshold method relies on a spliced likelihood density for the bulk and extreme tail of the underlying distribution where the switch-point is specified as a fixed quantile. We derive a closed form expression for the posterior of the tail-index and illustrate its application to quantile, or value-at-risk, estimation. Our simulation results show that the marginalized threshold outperforms the maximum likelihood method, or the Hill estimate, for both tail-index and quantile estimation. We also illustrate our method using returns for the S&P 500 stock market index from 1928 - 2020.

The quality of importance distribution is vital to adaptive importance sampling, especially in high dimensional sampling spaces where the target distributions are sparse and hard to approximate. This requires that the proposal distributions are expressive and easily adaptable. Because of the need for weight calculation, point evaluation of the proposal distributions is also needed. The Gaussian process has been proven to be a highly expressive non-parametric model for conditional density estimation whose training process is also straightforward. In this paper, we introduce a class of adaptive importance sampling methods where the proposal distribution is constructed in a way that Gaussian processes are combined autoregressively. By numerical experiments of sampling from a high dimensional target distribution, we demonstrate that the method is accurate and efficient compared to existing methods.

Identifying uterine contractions with the aid of machine learning methods is necessary vis-á-vis their use in combination with fetal heart rates and other clinical data for the assessment of a fetus wellbeing. In this paper, we study contraction identification by processing noisy signals due to uterine activities. We propose a complete four-step method where we address the imbalanced classification problem with an ensemble Gaussian process classifier, where the Gaussian process latent variable model is used as a decision-maker. The results of both simulation and real data show promising performance compared to existing methods.

Classification with imbalanced data is a common and challenging problem in many practical machine learning problems. Ensemble learning is a popular solution where the results from multiple base classifiers are synthesized to reduce the effect of a possibly skewed distribution of the training set. In this paper, binary classifiers based on Gaussian processes are chosen as bases for inferring the predictive distributions of test latent variables. We apply a Gaussian process latent variable model where the outputs of the Gaussian processes are used for making the final decision. The tests of the new method in both synthetic and real data sets show improved performance over standard approaches.

We present RF energy harvester and management strategy tailored for the passive near-zero power devices. Radio- less RF-powered backscattering tags that have the ability to recognize and localize activities in the surrounding environment are example of such devices. We propose a management strategy that determines the operation regime of the harvester based on the input power level at which harvester provides the instantaneous supply voltage for device operation. As the input power exceeds this level, the storage of the excess energy is managed by an adaptive capacitor charging circuit that keeps the voltage at the input of voltage regulator constant. We demonstrate that backscatter-based RF tag in the listening mode of operation can instantaneously operate with an input power of -34.4 dBm. Due to the adaptive capacitor charging circuit, the power efficiency of the energy harvester is higher than 50% over a range of input powers from -25 dBm up to -5 dBm.

Backscattering tags transmit passively without an on-board active radio transmitter. Almost all present-day backscatter systems, however, rely on active radio receivers. This presents a significant scalability, power and cost challenge for backscatter systems. To overcome this barrier, recent research has empowered these passive tags with the ability to reliably receive backscatter signals from other tags. This forms the building block of passive networks wherein tags talk to each other without an active radio on either the transmit or receive side. For wider functionality, accurate localization of such tags is critical. All known backscatter tag localization techniques rely on active receivers for measuring and characterizing the received signal. As a result, they cannot be directly applied to passive tag-to-tag networks. This paper overcomes the gap by developing a localization technique for such passive networks based on a novel method for phase-based ranging in passive receivers. This method allows pairs of passive tags to collaboratively determine the inter-tag channel phase while effectively minimizing the effects of multipath and noise in the surrounding environment. Building on this, we develop a localization technique that benefits from large link diversity uniquely available in a passive tag-to-tag network. We evaluate the performance of our techniques with extensive micro-benchmarking experiments in an indoor environment using fabricated prototypes of tag hardware. We show that our phase-based ranging performs similar to active receivers, providing median 1D ranging error <1 cm and median localization error also <1 cm. Benefiting from the large-scale link diversity our localization technique outperforms several state-of-the-art techniques that use active receivers.

Estimating and surveillance volumes of patients are of great importance for public health and resource allocation. In many situations, the change of these volumes is correlated with many factors, e.g., seasonal environmental variables, medicine sales, and patient medical claims. It is often of interest to predict patient volumes and to that end, discovering causalities can improve the prediction accuracy. Correlations do not imply causations and they can be spurious, which in turn may entail deterioration of prediction performance if the prediction is based on them. By contrast, in this paper, we propose an approach for prediction based on causalities discovered by Gaussian processes. Our interest is in estimating volumes of patients that suffer from allergy and where the model and the results are highly interpretable. In selecting features, instead of only using correlation, we take causal information into account. Specifically, we adopt the Gaussian processes-based convergent cross mapping framework for causal discovery which is proven to be more reliable than the Granger causality when time series are coupled. Moreover, we introduce a novel method for selecting the history or look-back length of features from the perspective of a dynamical system in a principled manner. The quasi-periodicities that commonly exist in observations of volumes of patients and environment variables can readily be accommodated. Further, the proposed method performs well even in cases when the data are scarce. Also, the approach can be modified without much difficulty to forecast other types of patient volumes. We validate the method with synthetic and real-world datasets.

In this paper, we present a technique whereby passive tags can track each other in a backscattering tag-to-tag network (BTTN). In such a network, passive tags without any on-board radio transceivers communicate directly with each other by backscattering an external excitation signal. First, we explain how the tags determine their distances to other communicating tags in their proximity and then how they can track nearby tags. Our technique is based on multiphase backscattering, more specifically, on the ability of backscattering tags to systematically change the phase offset of the signal that is being backscattered. A passive receiving tag with an envelope detector can then examine the received signal amplitude over the multiple backscattering phases and can draw inferences about the inter-tag distance. We demonstrate our method and show its accuracy on tags that we have built in our lab. Experiments show that our passive tags can measure Doppler shifts with approximately the same accuracy as that achieved by active conventional RFID readers. Our median tracking error based on data from two tags is only about 2.5 cm.

The population Monte Carlo (PMC) algorithm is a popular adaptive importance sampling (AIS) method used for approximate computation of intractable integrals. Over the years, many advances have been made in the theory and implementation of PMC schemes. The mixture PMC (M-PMC) algorithm, for instance, optimizes the parameters of a mixture proposal distribution in a way that minimizes that Kullback-Leibler divergence to the target distribution. The parameters in M-PMC are updated using a single step of expectation maximization (EM), which limits its accuracy. In this work, we introduce a novel M-PMC algorithm that optimizes the parameters of a mixture proposal distribution, where parameter updates are resolved via stochastic optimization instead of EM. The stochastic gradients w.r.t. each of the mixture parameters are approximated using a population of Markov chain Monte Carlo samplers. We validate the proposed scheme via numerical simulations on an example where the considered target distribution is multimodal.

Convergent cross mapping (CCM) is designed for causal discovery in coupled time series, where Granger causality may not be applicable because of a separability assumption. However, CCM is not robust to observation noise which limits its applicability on signals that are known to be noisy. Moreover, the parameters for state space reconstruction need to be selected using grid search methods. In this paper, we propose a novel improved version of CCM using Gaussian processes for discovery of causality from noisy time series. Specifically, we adopt the concept of CCM and carry out the key steps using Gaussian processes within a non-parametric Bayesian probabilistic framework in a principled manner. The proposed approach is first validated on simulated data, and then used for understanding the interaction between fetal heart rate and uterine activity in the last two hours before delivery and of interest in obstetrics. Our results indicate that uterine activity affects the fetal heart rate, which agrees with recent clinical studies.

Convergent cross mapping (CCM) is designed for causal discovery between coupled time series for which Granger's method for detecting causality is shown to be unreliable. The theoretical foundation of CCM is based on state space reconstruction, and therefore, for the accuracy of its results, the quality of the reconstruction is crucial. However, in the CCM framework, the reconstruction of an attractor manifold is usually implemented by direct delay embedding, where the reconstruction parameters are often selected by grid search methods. In this paper, we propose a more reliable and principled approach, which is based on Gaussian processes (GPs) that improves the attractor reconstruction. We validated the approach with the well-studied Lorenz attractor with and without observation noise. The experimental results indicate that our method is more robust to noise and that it consistently provides a reliable attractor manifold reconstruction. The proposed method was then tested on a real-world dataset, and the results suggested that the CCM equipped with an improved attractor manifold not only determined correctly the causal relationship but also improved the convergence, which is critical for causal discovery.

In this paper, we propose a new approach for analyzing extreme values that are witnessed in financial markets. Our goal is to compute the predictive distribution of extreme events that are clustered in time and, as opposed to modeling just the maximum of a block of observations, we model the conditional tail for the underlying random process. We apply a stochastic parameterization of the generalized Pareto distribution to model the asymptotic behavior of this conditional tail, or excess distribution. We utilize a Rao-Blackwellized particle filter, which reduces the parameter space, and we derive a concise, recursive solution for the parameters of the distribution. Using the filter, the predictive distribution of the parameters, conditioned on the past data, is computed at each sample-time. We test our model on simulated data which show an improvement over the block-maximum and the maximum likelihood approaches both in parameter estimation and predictive performance.

We address the problem of localizing an (illegal) transmitter using a distributed set of sensors. Our focus is on developing techniques that perform the transmitter localization in an efficient manner, wherein the efficiency is defined in terms of the number of sensors used to localize. Localization of illegal transmitters is an important problem which arises in many important applications, e.g., in patrolling of shared spectrum systems for any unauthorized users. Localization of transmitters is generally done based on observations from a deployed set of sensors with limited resources, thus it is imperative to design techniques that minimize the sensors' energy resources. In this paper, we design greedy approximation algorithms for the optimization problem of selecting a given number of sensors in order to maximize an appropriately defined objective function of localization accuracy. The obvious greedy algorithm delivers a constant-factor approximation only for the special case of two hypotheses (potential locations). For the general case of multiple hypotheses, we design a greedy algorithm based on an appropriate auxiliary objective function - and show that it delivers a provably approximate solution for the general case. We develop techniques to significantly reduce the time complexity of the designed algorithms, by incorporating certain observations and reasonable assumptions. We evaluate our techniques over multiple simulation platforms, including an indoor as well as an outdoor testbed, and demonstrate the effectiveness of our designed techniques - our techniques easily outperform prior and other approaches by up to 50-60% in large-scale simulations.

Free floating sub-mm and mm sized brain implants can communicate through a backscatter-based link in a presence of the EM field generated by the external coil. This link reduces the bandwidth requirement in the uplink communication of these implants to the external coil and enables a close-loop operation of the distributed implant system through reduced latency. The critical challenge in the link design stems from the low modulation index in the incident signal at the receiving coil. This calls for the design of the ASK demodulator that can resolve signals with low modulation index. We propose a demodulator design comprising a self-biased common-source based envelope detector that provides sufficient conversion gain and at the same time operates with a low power consumption. With 90 MHz carrier frequency and 50-kbps data rate, the ASK demodulator, implemented in 65 nm CMOS technology, resolves input RF signal with 1% modulation index consuming less than 100 nW when amplitude of the input RF signal is 200 mV.

In this paper we propose a probabilistic framework for proficiency self-assessment of autonomous systems. We define proficiency as a mathematical concept, i.e., as a metric that depends on a variety of factors. This concept allows for assessment of the degree of completion of a given task by a system. We provide the rationale behind the proposed concept and its forms for various settings. Further, we present motivating examples with details of evaluation of the proficiency. We anticipate that our definition of proficiency is a step forward toward achieving "self-awareness" of autonomous systems.

Adaptive importance sampling (AIS) methods are a family of algorithms which can be used to approximate Bayesian posterior distributions. Many AIS algorithms exist in the literature, where the differences arise in the manner by which the proposal distribution is adapted at each iteration. The adaptive population importance sampler (APIS), for example, deterministically samples from a mixture distribution and uses the local information given by the samples and weights to adapt the location parameter of each proposal. The update rules by nature are heuristic, but effective, especially in the case that the target posterior is multimodal. In this work, we introduce a novel AIS scheme which incorporates modern techniques in stochastic optimization to improve the methodology for higher-dimensional posterior inference. More specifically, we derive update rules for the parameters of each proposal by means of deterministic mixture sampling and show that the method outperforms other state-of-the-art approaches in high-dimensional scenarios.

In this paper, we propose a novel and simple method for discovery of Granger causality from noisy time series using Gaussian processes. More specifically, we adopt the concept of Granger causality, but instead of using autoregressive models for establishing it, we work with Gaussian processes. We show that information about the Granger causality is encoded in the hyper-parameters of the used Gaussian processes. The proposed approach is first validated on simulated data, and then used for understanding the interaction between fetal heart rate and uterine activity in the last two hours before delivery and of interest in obstetrics. Our results indicate that uterine activity affects fetal heart rate, which agrees with recent clinical studies.

Yule-Simon processes are preferential attachment processes that can be represented as urn processes where balls are added to a growing number of urns and where a new ball is placed in an urn with probability that is proportional to the number of balls in the urn. In this paper, we consider Yule-Simon processes that interact with each other. We propose two models of interaction, a deterministic and a probabilistic model. Based on observed two processes, we want to determine if the two processes interact and the direction of the interaction. In the case of the probabilistic model, the objective is also to estimate the strength of the interaction. We present detection/estimation schemes for each model. We also provide simulation results that demonstrate the performance of our schemes.

In this paper, we propose a new approach for analyzing extreme values such as large losses in financial markets. Our goal is to compute the predictive distribution of extreme events that are clustered in time. We apply a stochastic parametrization of the generalized extreme value distribution to model the asymptotic behavior of the block-maximum and derive a Rao-Blackwellized particle filter. This reduces the parameter space, and we derive a concise, recursive solution. Using the filter, the predictive distribution, conditioned on the past data, is computed at each sample-time. We introduce a new risk-measure, pVaRα, that is a more robust estimate of the true nature of value-at-risk, and illustrate our results using both simulated data and actual stock market returns from 1928-2017.

We have developed a type of RFID tags that can communicate with each other directly if there is an RF signal in their environment to support backscattering. These tags are passive and they can form a tag-to-tag network. Our tags communicate by what we refer to as multiphase probing. With this technique, we basically explore the backscatter channel by reflecting the incident RF signal with different changes in the phase. We define a measure of the backscatter channel, which we call backscatter channel state information (BCSI). The BCSI is composed of backscatter channel phase, backscatter amplitude, and change in baseline excitation level. When acquired over time, this measure provides rich RF analytics that can be used to extract various types of information from the environment of the tags by signal processing/machine learning methods. We show in the paper that this analytics is invariant w.r.t. to some variables including the deployment environment. We provide results from experiments with our tags that demonstrate the invariance of the BCSI.

We envision a future where every object in our living and working environment will carry one or more RF tags. Based on the backscattering tag-to-tag communication link, these RF tags will be connected in a network without the need for the central interrogating device. We present a novel tag architecture that enables estimation of the parameters of wireless tag-to-tag channel by a passive receiver. Sampling the received baseband signal at different reflecting phases at the backscattering tag enables estimation of amplitude and phase of the tag-to-tag channel. The low-power implementation of the channel estimator, after envelope detection, integrates amplification and filtering of the baseband signal that is followed by analog-to-digital conversion. The channel estimator, implemented in 65 nm CMOS technology, has sensitivity of -45 dBm at 2.5% modulation index and consumes 122 nW.

We present a distributed probability hypothesis density (PHD) filter for multitarget tracking in decentralized sensor networks with severely constrained communication. The proposed “cardinality consensus” (CC) scheme uses communication only to estimate the number of targets (or, the cardinality of the target set) in a distributed way. The CC scheme allows for different implementations-e.g., using Gaussian mixtures or particles-of the local PHD filters. Although the CC scheme requires only a small amount of communication and of fusion computation, our simulation results demonstrate large performance gains compared with noncooperative local PHD filters.

A novel framework and related methodologies are described to leverage RF power for building intelligent and battery-free devices with communication and computation capabilities. These passive devices are envisioned to make significant impact for the popular vision of smart dust due to extreme low power operation. The communication framework relies on tag-to-tag backscattering with very limited energy resources. The computing framework relies on a novel AC computing methodology that facilitates local data processing with an order of magnitude less power consumption. These enabling technologies, as described in this paper, revitalize the concept of smart dust with significant impact on various application domains such as smart spaces, implantable devices, and environmental/structural monitoring.

We use a crowdsourcing approach for RF spectrum patrolling, where heterogeneous, low-cost spectrum sensors are deployed widely and are tasked with detecting unauthorized transmissions in a collaborative fashion while consuming only a limited amount of resources. We pose this as a collaborative signal detection problem where the individual sensor's detection performance may vary widely based on their respective hardware or software configurations, but are hard to model using traditional approaches. Still an optimal subset of sensors and their configurations must be chosen to maximize the overall detection performance subject to given resource (cost) limitations. We present the challenges of this problem in crowdsourced settings and present a set of methods to address them. The proposed methods use data-driven approaches to model individual sensors and develops mechanisms for sensor selection and fusion while accounting for their correlated nature. We present performance results using examples of commodity-based spectrum sensors and show significant improvements relative to baseline approaches.

We present the vision of BARNET (Backscattering Activity Recognition NEtwork of Tags), a network of passive RF tags that use RF backscatter for tag-to-tag communication. BARNET not only provides identification of tagged objects but also can serve as a 'device-free' activity recognition system. BARNET's key innovation is the concept of backscatter channel state information (BCSI) which can be measured via systematic multiphase probing of the backscatter tag-to-tag channel using innovative processing on the passive tags. So far such measurements were only possible using active radio receivers that consume much higher power. Changes in BCSI provide signatures for different activities in the environment that can be learned using suitable machine learning tools. We develop the BARNET tag architecture which shows that an ASIC implementation can run on harvested RF power. We develop a printed circuit board (PCB) prototype using discrete components to evaluate activity recognition performance. We show that the prototype can recognize human daily activities with an average error around 6%. Overall, BARNET uses passive tags to achieve the same level of performance as systems that use powered, active radios.

In localization tasks, one typically assumes a statistical model of the observations, where the model quantifies the observations by exploiting interrelationships based on geometry. These models might incorporate unknown parameters that, in general, are functions of space. In this article, we propose a crowd sensing method for estimating a spatial field of a quantity (e.g., ranging biases due to line-of-sight/non-line-of-sight or path-loss parameter) allowing for improved indoor localization. Our method takes advantage of the information provided by various users that navigate the area of interest. The proposed learning approach is based on Gaussian processes and its computational cost does not increase with the number of measurements. We present numerical results that show how the proposed method estimates a spatial field of biases and how these estimates lead to much improved performance in estimation of user positions.

Opinion dynamics and its understanding in social networks is an emerging field of research in recent years. Existing work mainly considers direct exchanges of opinions among agents under certain conditions. This paper addresses a problem where the agents of a network make and exchange decisions repeatedly in a multi-hypothesis scenario and learn from the neighbors' decisions. Two models are proposed where the agents of the network use quasi-Bayesian learning to extract information about the true hypothesis from the neighbors' decisions. Theoretical analysis is provided about the conditions of a setting when agents become stubborn, that is, when they do not change their opinions anymore. We have run computer simulations to demonstrate the asymptotical properties of the proposed models. With our simulations we also show that, under one of the models, the agents of the network reach a consensus, and under the other, they form clusters.

We address the learning problem for infinite state Hidden Markov Models (HMMs) with preferential attachment dynamics. Preferential attachment describes a “rich get richer” process causing the HMM self transition probabilities to be proportional to the number of previous self transitions. Furthermore, the length of stay of the process in a particular state follows the Yule-Simon distribution. In describing the generative model of the hidden state processes, we use non-parametric models. We also establish the relationship of the proposed model with the Polya urn scheme and the Chinese restaurant process. The class of HMMs from this paper are applicable to data sets where the time spent in each state follows a power law. Our objective is to estimate the state sequence and the model parameters of the HMM. To that end, we propose a Gibbs sampling procedure. We evaluate the proposed procedure through computer simulations.

We present a novel Rao-Blackwellized multiple particle filtering method for inference of correlated latent states observed via nonlinear functions. We adopt a state-space framework and model the dynamic correlated states using a mixing matrix, embedded in white Gaussian noise. The critical challenges in practice are the lack of knowledge about the mixing parameters and the possibly large dimensionality of the state. We address these issues by implementing Rao-Blackwellization of the unknown parameters and adopting a divide-and-conquer approach. The former strategy amounts to marginalizing out some of the variables; the latter breaks the space of the system in subsystems, and runs a separate particle filter for each of them. The resulting Rao-Blackwellized multiple particle filtering accurately estimates the correlated latent states, as shown by the provided simulation results.

In this paper, we propose an application of non-parametric Bayesian (NPB) models to classification of fetal heart rate recordings. More specifically, the models are used to discriminate between fetal heart rate recordings that belong to fetuses that may have adverse asphyxia outcomes and those that are considered normal. In our work we rely on models based on hierarchical Dirichlet processes. Two mixture models were inferred from recordings that represent healthy and unhealthy fetuses, respectively. The models were then used to classify new recordings. We compared the classification performance of the NPB models with that of support vector machines on real data and concluded that the NPB models achieved better performance.

Particle filtering is a widely used sequential methodology that approximates probability distributions by using discrete random measures composed of weighted particles. A large number of particles improves the quality of the approximation but increases the computational requirements. Although there exists an abundant variety of particle filtering algorithms in the literature, there is lack of work devoted to selecting or adapting the number of particles systematically. In this paper we propose a novel methodology for online assessment of convergence of particle filtering. Based on theoretical analysis of the assessment, we propose an algorithm for the adaptation of the number of particles in online manner. The performance of the proposed algorithm is demonstrated for two state-space models.

In this paper, we consider state-space models where the latent processes represent correlated mixtures of fractional Gaussian processes embedded in white Gaussian noises. The observed data are nonlinear functions of the latent states. The fractional Gaussian processes have interesting properties including long-memory, self-similarity and scale-invariance, and thus, are of interest for building models in finance and econometrics. We propose sequential Monte Carlo (SMC) methods for inference of the latent processes where each method is based on different assumptions about the parameters of the state-space model. The methods are extensively evaluated via simulations of the popular stochastic volatility model.

Opinion dynamics in social networks has been widely studied in recent years, mostly by considering exchanges of opinions among neighboring agents. This paper addresses a scenario where the agents make decisions repeatedly on two hypotheses and where agents only exchange decisions. Motivated by the Bayesian models in the literature of human cognition, we model this learning procedure by the Bayes' rule. The social belief of each agent is defined to be the posterior of one of the hypotheses conditioned on the information it obtained from the society. We show that under certain conditions, once the social belief evolves to some region, the agent will refuse to change its belief. We demonstrate the asymptotical properties of the proposed model by computer simulations.

In many problems of signal processing, an important task is the classification of data. A group of methods that has attracted much interest for this purpose are the nonparametric Bayesian methods, and in particular, those based on the Dirichlet process. A useful metaphor for various generalizations of the Dirichlet process has been the Chinese restaurant process. Often the task of classification must be carried out in a sequential manner, and to that end the concepts from Bayesian non-parametrics cannot be applied straightforwardly. Recently, we introduced the notion of Chinese restaurant process with finite capacity to allow for classification of data on a time-varying basis. In this paper, we introduce the hierarchical Chinese restaurant process with finite capacity to provide further flexibilities to the process of classification. We show a generative model based on the process and then describe how to make online inference using the model. We demonstrate the approach with computer simulations.

In this paper, we propose to analyze fetal heart rate (FHR) signals by hierarchical Dirichlet process (HDP) mixture models. We investigate whether the clustering results of real-world FHR time series obtained by these models are informative in terms of determining the health status of a fetus. The FHR signals are divided into two groups, healthy and unhealthy, according to the umbilical arterial blood pH values of the fetuses. We computed the frequencies of clusters appearing in each of the groups, and applied the MannWhitney U test to compare the frequencies. The results showed that the frequencies of appearance of certain clusters are statistically significantly different across the two groups. This indicates that certain clusters may relate to pathological fetal heart rate patterns.

Backscatter-based tag-to-tag communication (BBTT) is a paradigm wherein radio-less devices communicate with each other by using purely passive backscatter modulation. This allows for highly inexpensive and low power devices. Traditional backscattering devices like RFID tags are designed to communicate directly with an active reader leading to a centralized framework centered on the reader. Under a BBTT network, the tags talk to each other using backscattering in the presence of an external excitation signal, which can come from multiple sources (e.g., dedicated exciters, WiFi access points, TV towers, or cell phone towers). The two main components that determine the range and robustness of a passive tag-to-tag link are the power harvesting and demodulation circuit blocks in the analog front end (AFE). In this paper, we investigate the design constraints, optimization goals, and tradeoffs in the design of the AFE for BBTT tags. We first analyze the BBTT link theoretically and then verify the predicted optimal AFE parameters by simulations.

In this paper, we investigate a unique phase cancellation problem that occurs in backscatter-based tag-to-tag (BBTT) communication systems. These are systems wherein two or more radio-less devices (tags) communicate with each other purely by reflecting (backscattering) an external signal (whether ambient or intentionally generated). A transmitting tag modulates baseband information onto the reflected signal using backscatter modulation. At the receiving tag, the backscattered signal is superimposed to the external excitation and the resulting signal is demodulated using envelope detection techniques. The relative phase difference between the backscatter signal and the external excitation signal at the receiving tag has a large impact on the envelope of the resulting signal. This often causes a complete cancellation of the baseband information contained in the envelope, and it results in a loss of communication between the two tags. This problem is ubiquitous in all BBTT systems and greatly impacts the reliability, robustness, and communication range of such systems. We theoretically analyze and experimentally demonstrate this problem for devices that use both ASK and PSK backscattering. We then present a solution to the problem based on the design of a new backscatter modulator for tags that enables multiphase backscattering. We also propose a new combination method that can further enhance the detection performance of BBTT systems. We examine the performance of the proposed techniques through theoretical analysis, computer simulations, and laboratory experiments with a prototype tag that we have developed.

This paper presents an analysis on optimal mobile sensor configuration for multiple-target-tracking (MTT) with Received-Signal-Strength-Indicator (RSSI) based measurements. The analysis is based on the underlying assumption that the complexity of this inherently high-dimensional problem is reduced by employing a multi-agent distributed tracking system. The assumed system assigns a single target of interest (TOI) to each agent and treats the remaining targets as interference sources. The measurement interference due to these sources is effectively compensated for by exchanging TOI information between agents, and fusing this information during the estimation process. Proper sensor placement within such an environment represents a unique challenge and optimal solutions are fundamentally different from a conventional MTT scenario. The main results of this paper include formulation and subsequent simplification of the optimality criterion along with a suboptimal solution yielding competitive performance and superior efficiency. Simulation results are presented demonstrating the performance of the proposed solution and comparisons are made to existing techniques.

Distributed filtration of state-space models with sensor networks assumes knowledge of a model of the data-generating process. However, this assumption is often violated in practice, as the conditions vary from node to node and are usually only partially known. In addition, the model may generally be too complicated, computationally demanding or even completely intractable. In this contribution, we propose a distributed filtration framework based on the novel approximate Bayesian computation (ABC) methods, which is able to overcome these issues. In particular, we focus on filtration in diffusion networks, where neighboring nodes share their observations and posterior distributions.

Particle filtering has been widely accepted as an important methodology for processing data represented by state-space models characterized by nonlinearities and/or non-Gaussianities. It is also well documented that particle filtering deteriorates quickly in performance when the dimension of the tracked state becomes large. This limits its application in many science/engineering problems. Previously we have proposed a way of alleviating this deficiency based on the use of multiple particle filtering. According to the approach, a number of particle filters are assigned to track different subsets of the state with time. In this paper, we propose a new method for accurate and efficient implementation of multiple particle filtering. We provide simulation results that demonstrate the performance of the new method.

We investigate the problem of real-time self-tracking of tagged objects in a new system with low-cost “smart” tags. These tiny and battery-less devices will play a pivotal role in the infrastructure of the Internet of Things (IoT). With capabilities of low-power computation and tag-to-tag backscattered communication, no readers will be needed for running the Radio Frequency Identification (RFID) system. In order to allow for low-cost tags, self-tracking has to be performed with simple algorithms while still exhibiting high accuracy. In this paper we propose a linear observation model for which Kalman filtering (KF) is the optimal method. We also consider a nonlinear model for which we apply particle filtering (PF) of reduced complexity as the tracking method. The performance and computational complexity of the different methods are compared by computer simulations.

This paper considers inference on the widely used state-space models described by hidden ARMA state processes of unknown order observed via non-linear functions of the states. We propose a particle filtering method for sequentially inferring the unknown ARMA time-series by Rao-Blackwellization of all the static unknowns. Our method does not rely either on any assumption on the model order or on the static ARMA and state innovation parameters. Consequently, when the ARMA model order is unknown, it can be used without a follow-up model selection procedure. Extensive simulation results validate the proposed method across different ARMA models.

In this paper, we consider the problem of social learning in a network of agents where the agents make decisions onK hypotheses sequentially and broadcast their decisions to others. Each agent in the system has a private observation that is generated by one of the hypotheses. All the observations are independently generated from the same hypothesis. We study a setting where the agents randomly choose to make decisions prudently or non-prudently. A prudent decision is based on the private observation of the agent and all the previous decisions, whereas a non-prudent decision relies only on the private observation of the agent. We present a Bayesian learning method for the agents that exploits the information from other decisions. We analyze the asymptotical property of this system. A proof is presented that with the proposed decision policy, the posterior probability of the true hypothesis converges to one in probability. Simulation results are also provided.

The letter studies the problem of sequential estimation of mixtures in diffusion networks whose nodes communicate only with their adjacent neighbors. The adopted quasi-Bayesian approach yields a probabilistically consistent and computationally non-intensive and fast method, applicable to a wide class of mixture models with unknown component parameters and weights. Moreover, if conjugate priors are used for inferring the component parameters, the solution attains a closed analytic form.

We propose a distributed sequential estimation scheme for wireless sensor networks with asynchronous measurements. Our scheme combines the prediction and update steps of a Bayesian filter (for time alignment and recursive state estimation) with a fusion rule (for intersensor fusion using local communication). We also propose a reduced-complexity implementation using particle filtering and Gaussian mixture approximations, and an estimator of the delays resulting from processing and communication. Simulations for a target tracking problem demonstrate the good performance of our scheme.

In this paper an analysis is conducted regarding the likelihood function of an RSSI-based sensor measurement that is affected by a target of interest (TOI) and an interfering target source. The interferer's true location is unknown but is assumed to be Gaussian distributed with known parameters. This analysis is motivated by its potential application within a multi-agent distributed tracking system, where each agent is tasked with tracking a single TOI while treating others as sources of interference. By exchanging TOI information, each agent can use the results established here to effectively compensate for “out-of-scope” target interference by fusing this external information. An exact analytical form is established for the aforementioned likelihood and a Gaussian approximation is analytically developed. An application of these results is presented through an example scenario, with computer simulation results demonstrating performance.

In this paper we introduce a new multiple particle filtering approach for problems where the state-space of the system is of high-dimension. We propose to break the space into subspaces and to perform separate particle filtering in each of them. The two critical operations of particle filtering, the particle propagation and weight computation of each particle filter are performed wherever necessary with the aid of parametric distributions received from other subspaces. The proposed method is demonstrated by computer simulations and the results show an excellent performance when compared to other implementations of multiple particle filtering.

In this paper, we study multi-agent systems where the agents learn not only from their own private observations, but also from the ones of other agents. We build on a recent work, where a Bayesian learning method proposed for a linear Gaussian model was studied. According to the method, the agents iteratively exchange information with their neighbors, and they update the summary of their information using the signals received from the neighbors. The agents aim at obtaining the global posterior distribution of the unknown parameters in as short time as possible in a distributed way. In this paper, the posteriors are modeled by Beta distributions. We address two settings, one where the private signals are observed without errors and another where they are contaminated with errors. Finally, we provide and discuss an example and show results from computer simulations.

There are many practical signal processing settings where a state-space model consists of a state described by an ARMA process that is observed via non-linear functions of the state. In this paper, we propose a particle filtering method for sequentially estimating the ARMA process in the presence of unknown parameters. In the considered problem, we have static and dynamic unknowns, and we show how to handle the static parameters so that the estimation of the state process does not degrade with time. We propose a new particle filter that approximates the posterior of all the unknowns by a Gaussian distribution, in combination with a Monte Carlo approach to the Rao-Blackwellization of the static parameters. We demonstrate the performance of the proposed method by extensive computer simulations.

In this paper, we consider the problem of social learning when decisions by agents in a network are made randomly. The agents receive private signals and use them for decision making on binary hypotheses under which the signals are generated. The agents make the decisions sequentially one at a time. All the agents know the decisions of the previous agents. We study a setting where the agents instead of making deterministic decisions by maximizing personal expected utility, they act randomly according to their private beliefs. We propose a method by which the agents learn from the previous agents' random decisions using the Bayesian theory. We define the concept of social belief about the truthfulness of the two hypotheses and analyze its convergence. We provide performance and convergence analysis of the proposed method as well as simulation results that include comparisons with a deterministic decision making system.

In a network, each agent communicates with its neighbors. All the agents have initial observations, and they update their beliefs with the average of the beliefs in their neighborhoods. It is well known that in the long run, the network will reach consensus. However, the agents do not necessarily converge to the global average of the initial observations of all the agents in the network. Instead, the result is always a weighted average. Moreover, it takes infinite time for the process to converge. In this paper, we address regular networks of agents, where each agent (node) has the same number of agents. We propose a method that allows agents in these networks to learn the global average using the history of its local average in finite time.

We investigate the performance of distributed estimation of static parameters in networks whose nodes make correlated measurements. The measurement models are linear with node specific time-varying observation matrices. The nodes cooperate with their neighbors by exchanging information that allow for approximation of the sufficient statistics in the estimation. We prove that the proposed estimation method is efficient. Specifically, we show that the performances of the distributed estimator and the centralized one are asymptotically the same, i.e., the limit of the ratio of their variances is 1.

Nonparametric Bayesian models are often preferred over parametric models due to their superior flexibility in interpreting data. A strong motivation for the use of these models is the desire of avoiding the assumptions that are necessary for parametric models. A prominent place in Bayesian nonparametrics is played by the Dirichlet process, which is defined by a base measure and a concentration parameter. In this paper, we propose the construction of models based on products of Dirichlet processes and corresponding mixture models. We show how these processes can be used for classification of data with shared features. The proposed processes are different from the recently introduced hierarchical Dirichlet processes. We show the use of the proposed model on classification of multivariate time series and demonstrate its performance with computer simulations.

This paper deals with the problem of real-time indoor tracking of tagged objects in Ultra High Frequency Radio Frequency Identification systems with asynchronous measurements. A new and more realistic model of the system is proposed, where the probability of detecting a tag by a reader is described by a function of both the distance and the angle between the tag and the reader's antenna. The model also accounts for the possibility of a tag being in a dead-zone where the tag cannot be detected. For tracking, we propose the use of the particle filtering methodology that takes into account the asynchronous nature of the measurements. The parameters for modeling the resulting system are obtained from real-world experiments and the performance of the algorithm is shown by extensive computer simulations.

In this paper, we propose a novel approach for decomposing hedge fund returns onto observable risk factors. We utilize a vector stochastic-volatility model to extract the time-varying exposure of low frequency hedge fund returns on high frequency market data. We implement the estimation by using particle filtering and the concept of Rao-Blackwellization. With the latter, we remove all the static parameters of the model and thereby reduce the dimension of the parameter space for particle generation. Thus, we are able to obtain accurate estimates of the posterior distributions of the model states. For our model, this reduction is significant because the number of static parameters is large. We use the proposed model to analyze hedge fund performance and to optimally replicate hedge fund strategies economically. We demonstrate the validity and effectiveness of the method by computer simulations.

A distributed fusion problem is addressed where cross-covariance matrices of estimated variables are unknown. We first try to estimate the cross-covariances, and then calculate the weighting coefficients to combine the estimates linearly. We consider two approaches, one where we do not use priors for the covariance matrices of the model and another, where we use priors and engage the Bayesian machinery. For the former, we exploit the maximum-entropy principle in finding the optimal cross-covariance estimate and for the latter, we employ Wishart distributions as priors and search for the maximum a posteriori estimate. Both problems turn out to require convex optimization which can be solved by existing techniques. When the cross-covariance estimates are obtained, the weighting coefficients can easily be calculated so that fusion can take place. Simulation results that demonstrate the performance of the proposed methods are provided.

In this paper we consider the problem of target tracking in a network of mobile agents that receive asynchronous measurements. The agents measure received signal strengths from the target and broadcast the information to the remaining agents engaged in the tracking. We propose several non-centralized schemes based on particle filtering that account for the lack of synchronization. We demonstrate the proposed methods by computer simulations and compare their performance to the synchronous scenario. The obtained results reveal that the proposed strategies efficiently compensate for the asynchronism of the measurements.

We study the problem of accurate automatic classification of fetal heart rate (FHR) signals using three different classification methods. FHR time series data are segmented into short (15s) spans of data, and features are extracted from them. These features include some established metrics of FHR trends such as acceleration and deceleration durations as well as a new set of features derived from the sequence of beat-to-beat percentage changes of the FHR signals. In total, we use 10 different features and demonstrate the feasibility of using them for classifying short segments into one of two suitably defined classes denoted as normal or abnormal. Classification is achieved using three different methods: support vector machine, a parametric Bayesian method and a non-parametric Bayesian method utilizing a neighbour-counting procedure for class-conditional density estimation. The performances of these methods are demonstrated on a database of physician-annotated recordings from which 580 short epochs of FHR patterns were extracted.

We present a consensus-based distributed particle filter (PF) for wireless sensor networks. Each sensor runs a local PF to compute a global state estimate that takes into account the measurements of all sensors. The local PFs use the joint (all-sensors) likelihood function, which is calculated in a distributed way by a novel generalization of the likelihood consensus scheme. A performance improvement (or a reduction of the required number of particles) is achieved by a novel distributed, consensus-based method for adapting the proposal densities of the local PFs. The performance of the proposed distributed PF is demonstrated for a target tracking problem.

Chirp signals are usually encountered in target tracking problems including radar and sonar systems. The multimodality characterizing the distribution of the chirp signal parameters makes their estimation very challenging. In this paper we apply marginalized population Monte Carlo (MPMC) sampling to the problem of parameter estimation of chirp signals in noise. MPMC reduces the dimension of the vector of unknowns by marginalizing the complex amplitudes, which are conditionally linear on the chirp rates and frequencies. A Gibbs sampling scheme is combined with the MPMC method to further improve the performance. Computer simulations illustrate the validity of the proposed approach.

In this paper, we study the problem of distributed hypothesis testing in cooperative networks of agents. All agents are trying to reach consensus on the state of nature by their private signals and the binary actions of their neighbors. This is a challenging problem because the exchanged information of the agents is highly compressed. We propose a gossip-type method where every agent's decision converges in probability to the optimal decision held by a fictitious fusion center. We prove the asymptotical property of the proposed method and provide simulation results that demonstrate the communication cost and convergence time of the method.

In this paper we present a novel Radio Frequency Identification (RFID) system for accurate indoor localization. The system is composed of a standard Ultra High Frequency (UHF), ISO-18006C compliant RFID reader, a large set of standard passive RFID tags whose locations are known, and a newly developed tag-like RFID component that is attached to the items that need to be localized. The new semi-passive component, referred to as sensatag (sense-a-tag), has a dual functionality wherein it can sense the communication between the reader and standard tags which are in its proximity, and also communicate with the reader like standard tags using backscatter modulation. Based on the information conveyed by the sensatags to the reader, localization algorithms based on binary sensor principles can be developed. We present results from real measurements that show the accuracy of the proposed system.

In particle filtering, resampling is the only step that cannot be fully parallelized. Recently, we have proposed algorithms for distributed resampling implemented on architectures with concurrent processing elements (PEs). The objective of distributed resampling is to reduce the communication among the PEs while not compromising the performance of the particle filter. An additional objective for implementation is to reduce the communication among the PEs. In this paper, we report an improved version of the distributed resampling algorithm that optimally selects the particles for communication between the PEs of the distributed scheme. Computer simulations are provided that demonstrate the improved performance of the proposed algorithm.

In this paper we model and simulate a biological system describing the evolution of cancer stem cells into tumors. Starting from some basic hypotheses about the behavior of these cells, we develop a model that mimics the evolution of a system of cancer stem cells and show how random-set-theory naturally leads to a generation algorithm. Computer simulations demonstrate the potential of our approach by using simple random sampling rules and a lattice environment.

In this paper we consider the problem of target tracking in a network of mobile agents. We propose a scheme with agents that are endowed with processing and decision-making capabilities and without a central unit that controls them and/or fuses information. The agents measure received signal strengths from the targets and communicate it to the remaining agents engaged in the tracking. Each agent applies particle filtering for tracking and uses an algorithm for optimal agent deployment for the next time instant. We describe the details of the tracking from its initialization to its completion. We demonstrate the proposed method by computer simulations.

We propose a distributed implementation of the Gaussian particle filter (GPF) for use in a wireless sensor network. Each sensor runs a local GPF that computes a global state estimate. The updating of the particle weights at each sensor uses the joint likelihood function, which is calculated in a distributed way, using only local communications, via the recently proposed likelihood consensus scheme. A significant reduction of the number of particles can be achieved by means of another consensus algorithm. The performance of the proposed distributed GPF is demonstrated for a target tracking problem.

In this paper, we propose a model for proliferation of cancer stem cells and a procedure for estimating the unknowns of the model. Understanding the proliferation of cancer stem cells is critical for the development of anti-cancer therapies. We propose to use a nonlinear and non-Gaussian state-space model for studying the proliferation process. For estimation of the unknowns we apply particle filtering, which is particularly appropriate given the nature of the model. In addition, we deal with a very large dimension of the state-space and very sparse time series of measurements. Computer simulations show promising results in a simple scenario generated with synthetic data.

In signal processing, it is typical to develop or use a method based on a given model. In practice, however, we almost never know the actual model and we hope that the assumed model is in the neighborhood of the true one. If deviations exist, the method may be more or less sensitive to them. Therefore, it is important to know more about this sensitivity, or in other words, how robust the method is to model deviations. To that end, it is useful to have a metric that can quantify the robustness of the method. In this paper we propose a procedure for developing a variety of metrics for measuring robustness. They are based on a discrete random variable that is generated from observed data and data generated according to past data and the adopted model. This random variable is uniform if the model is correct. When the model deviates from the true one, the distribution of the random variable deviates from the uniform distribution. One can then employ measures for differences between distributions in order to quantify robustness. In this paper we describe the proposed methodology and demonstrate it with simulated data.

Population Monte Carlo is a statistical method that is used for generation of samples approximately from a target distribution. The method is iterative in nature and is based on the principle of importance sampling. In this paper, we show that in problems where some of the parameters are conditionally linear on the remaining parameters, we can improve the computational efficiency of population Monte Carlo by generating samples of the nonlinear parameters only and marginalizing the linear parameters. We demonstrate the marginalized population Monte Carlo on the problem of frequency estimation of closely spaced sinusoids.

One of the most criticized aspects of particle filtering algorithms is their dependence on model assumptions. However, a rigorous study of the effect of modeling errors on the performance of such algorithms is still missing. In this paper, the problem of using an inaccurate discrete state-space model is considered and a systematic methodology for studying the effects on its performance is proposed. The methodology is based on the use of the Kolmogorov-Smirnov statistic, which in this case is a distance metric between the posterior characterization when respectively correct and incorrect model assumptions are made. An example with functional and distributional inaccuracies is studied.

One of the most basic problems in science and engineering is the assessment of a considered model. The model should describe a set of observed data and the objective is to find ways of deciding if the model should be rejected. It seems that this is an ill-conditioned problem because we have to test the model against all the possible alternative models. In this paper we use the Kolmogorov-Smirnov statistic to develop a test that shows if the model should be kept or it should be rejected. We explain how this testing can be implemented in the context of particle filtering. We demonstrate the performance of the proposed method by computer simulations.

In this paper, a data-driven extension of the variational algorithm is proposed. Based on a few selected sensors, target tracking is performed distributively without any information about the observation model. Tracking under such conditions is possible if one exploits the information collected from extra inter-sensor RSSI measurements. The target tracking problem is formulated as a kernel matrix completion problem. A probabilistic kernel regression is then proposed that yields a Gaussian likelihood function. The likelihood is used to derive an efficient and accelerated version of the variational filter without resorting to Monte Carlo integration. The proposed data-driven algorithm is, by construction, robust to observation model deviations and adapted to non-stationary environments.

Advances in the development of models that can satisfactorily describe biochemical networks are extremely valuable for understanding life processes. In order to get full description of such networks, one has to solve the inverse problem, that is, estimate unknowns (rates and populations of various species) or choose models from a set of hypothesized models using experimental data. In this paper we discuss signal processing techniques for resolving the inverse problem of biochemical networks using the stochastic approach based on Bayesian theory. The proposed methods are tested in simple scenarios and the results are promising and suggest application of these methods to more complex networks.

Sequential Monte Carlo (SMC) methods, also referred to as particle filters, have been successfully applied to a variety of highly nonlinear problems such as target tracking with sensor networks. In this paper, we propose the application of a new class of SMC methods named cost-reference particle filters (CRPFs) to target tracking with mobile sensors. CRPF techniques have been shown to be a flexible and robust alternative when there is no knowledge about the probability distributions of the noise in the system. The sensors positioning during tracking is determined by the predicted target's location as obtained by the CRPF. The performance of the method is investigated by simulations and compared to tracking with standard particle filters (SPFs).

Cost-reference particle filtering (CRPF) allows for tracking of nonlinear dynamic states without a prior knowledge of the probability distributions of the noises in the state-space representation of the system. In this paper we consider a setup where the system unknowns consist of linear and nonlinear states. We propose an efficient scheme for estimation of the states by combining CRPF with the recursive least square (RLS) algorithm. We applied the method to the problem of target tracking using biased bearing measurements. Simulation results show a very accurate performance of the proposed approach.

In this paper, we present a new method for stochastic simulation of coupled chemical reactions. In this method we obtain recursive expressions for propagating the first two moments of the probability distributions over time. Its advantage over other simulation methods is that it does not require Monte Carlo simulations, and hence it performs several orders of magnitude faster than existing Monte Carlo methods. Simulation results are presented for some examples of coupled first-order reactions.

There are two distinct problems in the stochastic analysis of biochemical networks, and they are known as the forward and inverse problems. Solutions of the former problem are used for simulating a system of molecular species in time according to the random laws that govern the reactions in which the species participate. Solutions of the latter problem provide estimates of the unknowns in the system that is represented by the biochemical network. The estimates are obtained from measurements that are functions of the number of molecules of some of the species. In the two problems, we have underlying assumptions about the probabilistic models of the studied network. In this paper we present two new methods for addressing these problems. For solving the forward problem we propose a method that does not employ Monte Carlo simulations, whereas for solving the inverse problem we use particle filtering.

Particle filtering is a sequential signal processing methodology that uses discrete random measures composed of particles and weights to approximate probability distributions of interest. The quality of approximation depends on many factors including the number of particles used for filtering and the way new particles are generated by the filter. The problem of good approximation becomes increasingly challenging as the dimension of the state space increases. In this paper, we address a possible solution for improved particle filtering in high dimensional cases by using a set of particle filters operating on partitioned subspaces of the complete state space. We provide simulation results that show the feasibility of the proposed approach.

Recent progress in distributed robotics and low power embedded systems has led to development of mobile sensor networks. Controlled mobility, moving sensors intentionally, enables a new set of possibilities in wireless sensor networks and facilitates many applications in signal processing areas such as target tracking. In this paper we consider the problem of tracking a target using three mobile sensors that measure the received signal strength (RSS) from the target. We propose the use of particle filtering where the positioning of the mobile sensor is based on the predicted target's positions. In deciding how to deploy the sensors, we have used the Cramer-Rao lower bound (CRLB) that we have derived for our scheme. The performance of the method is investigated by simulations and compared to tracking by traditional static sensor network.

Sensor localization methods based on Monte Carlo sampling approximate the sensor position distributions by a weighted set of samples. These approaches traditionally require complete knowledge of the probabilistic distributions of the uncertainties in the sensor system. In this paper, we propose alternative sampling-based methods which do not require complete knowledge of the probabilistic distributions. The sensor position distributions are represented by a set of samples and costs which are described by spatial parametric regions. Few parameters are needed to characterize these regions, and therefore the amount of information to be transmitted to the rest of the sensors to self-localize is simplified. Computer simulations show that the proposed methods are more robust and less computationally intensive than standard sampling approaches.

Cost-reference particle filtering (CRPF) is a methodology for recursive estimation of hidden states of dynamic systems. It is used for tracking nonlinear states when probabilistic assumptions about the state and observations noises are not made. Recently, we have proposed a CRPF algorithm for systems with conditionally linear states that combines the use of Kalman filtering for the linear states and CRPF for the nonlinear states. We have shown that this combined method yields improved results over the standard CRPF. In this paper, we further extend that approach by relaxing some of the assumptions about the noises in the system. As a result, the only statistical assumption that remains is that the noises are stationary and zero mean. We demonstrate the performance of the proposed method by computer simulations and compare it with standard CRPF, standard particle filtering (SPF), and marginalized particle filtering (MPF).

In many situations, solutions to nonlinear discrete- time filtering problems are available through approximations. Many of these solutions are based on approximating the posterior distributions of the states with Gaussian distributions. In this letter, we compare the performance of Gaussian-based filters including the extended Kalman filter, the unscented Kalman filter, and the Gaussian particle filter. To that end, we measure the distance between the posteriors obtained by these filters and the one estimated by a sequential Monte Carlo (particle filtering) method. As a distance metric, we apply the Kullback-Leibler and X2information measures. Through computer simulations, we rank the performance of the three filters.

We propose a distributed algorithm for sensor localization using beacon nodes. In this algorithm, beacon nodes broadcast distributions which contain information about their location. Nearby sensor nodes with unknown location information use this transmitted information and received beacon signal characteristics to estimate their positions. Sensors that estimate their positions become new beacons. A Monte Carlo method known as Importance Sampling is used for fusing these distributions and for obtaining approximations of the posterior distributions of the sensor locations. We also compute the Bayesian Cramér-Rao bounds for self-localization of sensors and study the impact of the beacons' prior location information and other system parameters. We analyze the performance of the proposed algorithm through computer simulations and compare it with numerically obtained bounds.

In this paper, we present a hardware architecture for a Sampling Importance Resampling Filter (SIRF) applied to systems with multiple interacting models. This filter outperforms traditional filters in practical scenarios due to superior abilities of the SIRFs in dealing with nonlinear and/or non-Gaussian models. Compared to existing approaches, our method does not require knowledge of model transition probabilities and keeps a constant number of particles per model at all times. This allows for a regular hardware structure with deterministic execution time. A highly scalable, parallel architecture consisting of distributed processing elements and a central unit is described. We propose an interconnection scheme and data exchange protocol using the concept of distributed resampling that greatly speeds up filter execution and drastically reduces the required interconnect to a single bus without causing any communication bottleneck. The proposed architecture is evaluated on a Xilinx FPGA platform for a multiple model target tracking application and its efficiency and scalability is shown.

In this paper, we consider tracking time-varying number of targets which move along a two-dimensional area monitored by a network of wireless sensors. We propose a novel fusion algorithm based on particle filtering that accounts for both detection of the number of active targets in the field and estimation of their positions and velocities. The method uses measurements collected by acoustic sensors, where the measurements represent superposition of received powers of signals transmitted by the targets. Computer simulations are provided to illustrate the feasibility of the proposed method in scenarios with zero, one, and two targets.

An important application of sensor networks is target tracking and localization. To deal with sensor nodes with limited energy supply and communication band width we propose energy-efficient hierarchical architectures for solving the target tracking problem. In these networks, sensors form clusters and transmit minimal quantized information about a sensed event to a specialized node, known as a cluster head. Cluster heads are equipped with capability of communicating over large distances with a fusion center or a base station. We consider two different hierarchical architectures : (a) the target dynamics are probabilistically estimated at the cluster heads and their statistics combined at the fusion center, and (b) the cluster heads perform simple compression rules on the quantized sensor data and the fusion center estimates the target dynamics using these severely compressed data. Sequential Monte Carlo algorithms for estimation of the target dynamics are used. Through computer simulations the performances of these two architectures are studied.

In this paper, we investigate different variants of the recently proposed cost-reference particle filters (CRPFs) and study their application to the problem of tracking of a high-speed maneuvering target in the two-dimensional space. CRPFs drop all probabilistic assumptions required by conventional particle filters and, as a consequence, lead to practically more robust algorithms. We introduce some suitable and natural modifications of CRPFs in order to increase their efficiency and reduce their computational complexity. Computer simulations are provided to illustrate the performance of the new alternatives.

When power utility asset managers are facing the task of resource planning for future deployment, often times, only partial information is available: installation dates and amounts, as well as failure and replacement rates. By combining records on yearly populations of the components, estimation of failure model parameters may be possible. Parametric models may then be used for forecasting of the system's short term future failure rates and for formulation of replacement strategies. We employ the Weibull distribution and show how we estimate its parameters from past failure data. With the obtained estimates, we forecast future failures and keep on improving the estimates as new data become available

Recent advances of wireless sensor networks have presented some very interesting problems for signal processing. For practical reasons, many networks are composed of simple sensors that use very little power and do not consume much communication bandwidth. A class of sensors that satisfy these requirements are the tertiary sensors. They report an approaching event with one signal and a receding event with another signal. When the event is out of their range, they do not report anything. In this paper, we apply particle filtering for processing signals from tertiary sensor networks with the purpose of tracking events (targets) within the field of the sensor network. We present an algorithm for tracking and demonstrate its performance by computer simulations.

The recently introduced cost-reference particle filter (CRPF) methodology allows for recursive estimation of unobserved states of dynamic systems without a priori knowledge of probability distributions of the noise in the system. We use CRPFs in problems where we eliminate one more strong assumption about the state space model, the one of knowing the function governing the state evolution. We replace this function by a linearly combined set of basis functions where the linear combination coefficients are unknown. We show how CRPFs can be modified to cope with this scenario and demonstrate their performance for positioning a moving vehicle in a two-dimensional space.

A new approach for the computation of probability distributions for coupled first order chemical reactions is introduced. The approach is based on system theory, where the system states are chemical species and the signals are probabilities. We derive the transfer functions of the so defined systems and show that they can be applied to various reaction environments. The use of block diagrams offers a clear, visual, and convenient way to decompose a complicated reaction system into simpler sub-systems and vice versa. Since the state of the system is defined as a molecule species instead of molecule population, with this method, one can study chemical reactions involving any number of molecules.

In this paper, we propose a method for blind multiuser detection (MUD) in synchronous systems over flat and fast Rayleigh fading channels. We adopt an autoregressive-moving-average (ARMA) process to model the temporal correlation of the channels. Based on the ARMA process, we propose a novel time-observation state space model (TOSSM) that describes the dynamics of the addressed multiuser system. The TOSSM allows an MUD with natural blending of low complexity particle filtering (PF) and mixture Kalman filtering (for channel estimation). We further propose to use a more efficient PF algorithm known as the stochastic M-algorithm (SMA), which, although having lower complexity than the generic PF implementation, maintains comparable performance.

Target tracking is a highly nonlinear problem that has been successfully addressed in recent years using sequential Monte Carlo (SMC) methods, usually called particle filters. We investigate the application of a new class of SMC techniques, termed cost reference particle filters (CRPFs), to the tracking of a high-speed maneuvering target. The new CRPF methodology drops all probabilistic assumptions (i.e., prior probabilities, knowledge of noise distributions and likelihood functions) that are common to conventional particle filters and, as a consequence, leads to practically more robust algorithms. The advantage of the proposed CRPF over the standard SMC filter in the context of maneuvering target tracking is illustrated through computer simulations.

In recent years the theory of particle filtering has continued to advance, and it has found increasing use in sequential signal processing. A weakness of particle filtering is that it is inadequate for problems that besides tracking of evolving states require the estimation of constant parameters. In this paper, we propose particle filters that do not have this limitation. We call these filters density assisted particle filters, of which special cases are the recently introduced Gaussian particle filters and Gaussian sum particle filters. An implementation of a density particle filter is shown on a relatively simple but important nonlinear model. Simulations are included that show the performance of this filter.

Accurate estimation of synchronization parameters is a fundamental issue in digital transmission. In this paper, we investigate a novel approach to joint synchronization and blind data detection in frequency non-selective fast fading channels, based on the application of sequential Monte Carlo (SMC) techniques. The algorithm is derived by modeling the transmission process as a dynamic system where the channel parameters and the transmitted symbols are unobserved state variables. The performance of the proposed technique is studied through computer simulations that illustrate the accuracy of timing recovery and the overall performance of the resulting receiver in terms of its symbol error rate (SER).

A novel low-complexity residual resampling scheme for particle filters is presented. The proposed scheme uses a simple but effective "particle-tagging" method to compensate for a possible error that can be caused by finite-precision quantization in the resampling step of particle filtering. The scheme guarantees that the number of particles after resampling is always equal to the number of particles before resampling. The resulting scheme is suitable for high-speed physical realization when the number of particles is a power of two.

Particle filtering has drawn much attention in recent years due to its capacity to handle nonlinear and non-Gaussian dynamic problems. One crucial issue in particle filtering is the selection of the importance function that generates the particles. In this letter, we propose a new type of importance function that possesses the advantages of the posterior and the prior importance functions. We demonstrate its use on the problem of blind detection in flat fading channels and provide simulation results that show its efficiency and performance.

The paper addresses the problem of joint velocity estimation and data detection in a realistic scenario where mobile velocity changes continuously, resulting in non-stationary fast fading channels. A time-varying AR model and a Gauss-Markov model are used to describe the respective fading channel and variation of velocity. A connection is shown between the coefficients of the TVAR model and mobile velocity which makes the joint estimation and detection possible. A hierarchical dynamic state space model is formed for the problem, and a particle filtering algorithm is proposed. In particular, a hybrid importance function and the mixture Kalman filter are used to achieve efficient implementation of particle filtering. Simulation results are provided that show the performance of the particle filtering algorithm.

Resampling is a critically important operation in the implementation of particle filtering. In parallel hardware implementations, resampling becomes a bottleneck due to its sequential nature and the increased complexity it imposes on the traffic of the designed interconnection network. To circumvent some of these difficulties, we propose two new resampling algorithms. The first one, called residual-systematic resampling, combines the merits of both systematic and residual resampling and is suitable for pipelined implementation. It also guarantees the fixed duration of the resampling procedure irrespective of the weight distribution of the particles. The second algorithm, referred to as partial resampling, has low complexity and reduces traffic load through the hardware network. These two algorithms should also be considered as resampling methods in simulations on standard computers.

Subband decomposition has already been shown to increase the performance of spectral estimators, but induced frequency overlapping may be troublesome, bringing edge effects at subband borders. A recent paper (Bonacci, D. et al., EUSIPCO, 2002) proposed a method (SDFW - subband decomposition and frequency warping) allowing subband decomposition to be performed without aliasing. We modify this subband decomposition in order to improve frequency resolution for any correlation based spectral estimator when applied to the subband outputs. Three main improvements are proposed: the subband decomposition is based on comb filters; the SDFW method warping operation is performed using a complex frequency modulation; the autocorrelation is estimated using all sub-series from each subband. Simulation results demonstrate the anticipated performance of the proposed method.

The paper addresses joint estimation of the timing epoch and detection of the transmitted symbols in a digital communication system. Most timing recovery techniques found in the literature are either approximately or heuristically derived, since optimal estimators are analytically intractable. Our approach to the problem relies on modeling the symbol timing as an autoregressive process. In this way, the digital communication system can be mathematically represented by a dynamic system in state-space form and the sequential Monte Carlo (SMC) methodology can be applied. SMC algorithms are powerful tools for Bayesian estimation that are based on representing the posterior distribution of the system state by a discrete measure with random support. This representation can be updated recursively, as new information becomes available, allowing for optimal estimation of both the transmitted symbols and their timing.

This work demonstrates the use of particle filtering for detection in BLAST systems. A novel dynamic state-space model (DSSM) is constructed for BLAST systems that are crucial for development of particle filtering algorithms. The proposed DSSM is based on QR decomposition and the output of the feedforward filter, and it evolves in space. The particle filtering solution does not suffer from error propagation, and our simulation show that it greatly outperforms the V-BLAST and provides near optimum performance.

In sequential signal processing, the main objective is to estimate evolving states. Often, however, the models under consideration contain additional unknowns, which are time invariant. When the state estimation is carried out by sequential importance sampling methods, the presence of fixed unknowns can present a nontrivial problem. In this paper, we provide a solution to this problem when the fixed unknowns are the covariance matrices of the additive Gaussian noise vectors in the state and observation equations. These matrices are first marginalized, and then the sequential processing carried out as usual. In the implementation of this approach, besides the assignment of a weight to every particle, two additional evolving quantities are required. Simulation results are provided that show the performance of the method.

Particle filtering has drawn much attention in recent years due to its capacity to handle nonlinear and non-Gaussian problems. One crucial issue in particle filtering is-the selection of importance function. In this paper, we propose a new type of importance function, which possess advantages over both the posterior and the prior importance functions. In addition, we demonstrate the use of the proposed importance function in blind detection in flat fading channels. Simulation results show its efficiency and performance.

Sequential detection over a bandlimited nonlinear channel is considered and a particle filtering algorithm is developed. The nonlinearity of the problem motivates the use of sequential Monte Carlo methods. Since the channel is bandlimited, the resulting memory of the channel allows for modeling the problem as a dynamic state space model. A particular application to nonlinear satellite communication is illustrated, where the channel is a cascade of linear filters and the nonlinear traveling wave tube amplifier at the satellite repeater; The approach results in very simple detectors with good performance characteristics and general applicability.

In adaptive signal processing the principle of exponentially weighted recursive least-squares plays a major role in developing various estimation algorithms. It is based on the concept of discounting of old measurements and allows for better performance in problems with time-varying signals and signals in nonstationary noise. We show how this concept can be combined with the Bayesian methodology. We propose that the discounting of old measurements within the Bayesian framework be implemented by employing particle filters. The main idea is presented by way of a simple example. The methodology is very attractive and can be used in a very wide range of scenarios including ones that involve highly nonlinear models and non-Gaussian noise.

Code-division multiple-access (CDMA) is a multiplexing technique which has become a driving force behind the rapidly advancing communications industry. In order to recover transmitted signals at the receiver when CDMA is used, multiuser detection techniques are engaged. In the past, various techniques have been developed to approach the performance of the optimum multiuser detector, and we have the same objective. We propose a new multiuser detector which is developed under the Bayesian framework and implemented by a novel efficient perfect sampling algorithm called the Gibbs coupler. The simulation results demonstrate the excellent performance of the proposed detector.

For dynamic systems, sequential Bayesian estimation requires updating of the filtering and predictive densities. For nonlinear and non-Gaussian models, sequential updating is not as straightforward as in the linear Gaussian model. Densities are approximated as finite mixture models as is done in the Gaussian sum filter. A novel method is presented whereby sequential updating of the filtering and posterior densities is performed by particle-based sampling methods. The filtering method has the combined advantages of Gaussian sum and particle-based filters and simulations show that the presented filter can outperform both methods.

This paper addresses the problem of classification of digital modulations. The proposed solution uses the Bayes classifier, which is implemented by the Markov chain Monte Carlo scheme. The implementation considers classifications in the presence of phase and frequency offsets as well as residual filtering effects coming from imperfect channel equalization. The proposed approach has been tested for many scenarios and its performance has been compared with the maximum likelihood classifier and the 4/sup th/ order cumulant-based method. The obtained results show that our classifier outperforms the other methods considerably.

We address the problem of parameter estimation of chirplets which are chirp signals with Gaussian shaped envelopes. The procedure we propose is an extension of our previous work on estimation of chirp signals (Lin and Djuric, 2000), and it is based on MCMC sampling. For fast convergence of the Markov chain Monte Carlo (MCMC) sampling based method, a critical step is the initialization of the method Since the chirplets have finite durations and may or may not overlap in time, we propose initialization procedures for each of these cases. We have tested the method by extensive simulations and compared it with Cramer-Rao bounds. The obtained results have been excellent.

The paper proposes a method for on-line model selection of nonstationary time series. The method is based on computation of the covariance matrix of the data, transformation of the matrix by Housholder's tridiagonalization, and application of a clustering algorithm that can separate the Gerschgorin disks of the transformed covariance matrix into disks that correspond to the signals and noise, respectively. The method is applied to on-line estimation of the the number of harmonic signals in noise. Simulation results are presented that show the performance of the proposed method.

In many signal processing problems, the estimation of random parameters must be carried out sequentially and under model uncertainty. In the paper, a Bayesian approach is proposed for solving this problem, which is based on sequential updating of the posterior distribution of the desired parameters. It is shown that under a certain general set of conditions, the posterior of the unknown parameters is a mixture density. Since the computation of the solution becomes very intensive as the number of data (records) grows, a numerical procedure is proposed based on the sequential importance sampling scheme. Its number of computations per new data record is constant, and the procedure can easily be implemented in parallel.

The problem of detecting transient signals in colored noise is addressed. The generalized likelihood ratio test fails to provide good performance, and as an alternative, a Bayesian approach is proposed. Its implementation is simplified by adopting an invariant transformation, and a sequential Monte Carlo sampling procedure is provided to compute multidimensional integrations. The simulation results demonstrate the ability of this approach to detect weak and fast decaying signals in colored noise.

Detection of symbols transmitted over a frequency flat Rayleigh fast-fading channel is considered. This problem can be modeled as a dynamic state space model. A novel method for channel estimation and detection of transmitted data is presented based on the Monte Carlo sampling filter methodology. The channel fading coefficients and transmitted variables are treated as hidden variables. The channel coefficients are modeled as an autoregressive (AR) process. Particles (samples) of hidden variables are sequentially generated from the so called importance sampling density based on past observations. These are then propagated and weighted according to the required conditional posterior distribution. The particles along with their weights provide an estimate of the hidden variables. It can be seen through the simulations that the performance of this detector is comparable to the matched filter with known channel fading coefficients. Moreover, the Gaussian noise assumption in the noisy channel can be easily relaxed and a solution provided by the same methodology.

This paper considers the problem of parameter estimation of chirp signals by using the Bayesian methodology. The concept of "mirror points" for constant-amplitude chirp signals is introduced, and its effect on the overall multicomponent chirp parameter estimation performance assessed. By combining the chirpogram with a Markov chain Monte Carlo (MCMC) technique, it is shown that accurate estimates can be obtained for signals comprising many chirps. Simulation results demonstrate that the parameter estimates are in agreement with the CRLB for SNRs as low as 2 dB.

Imperfect Bernoulli trials arise when the outcome of a Bernoulli experiment is not known with certainty. In signal processing, we often need to estimate a probability of occurrence p of an event from imperfect Bernoulli trials. A typical example is the estimation of the probability of a signal being present in noisy data. In his famous essay, Bayes solved the same problem but for perfect trials. In this letter, a solution is provided for imperfect trials. It is shown that it includes Bayes' solution as a special case.

Hidden Markov models are very important for analysis of signals and systems. They have been attracting the attention of the speech processing community, and they have become the favorite models of biologists. A major weakness of conventional hidden Markov models is their inflexibility in modeling state duration. In this paper, we analyze nonstationary hidden Markov models whose state transition probabilities are functions of time, thereby indirectly modeling state durations by a given probability mass function. The objective of our work is to estimate all the unknowns of the nonstationary hidden Markov model, its parameters and state sequence. To this end, we construct a Markov chain Monte Carlo sampling scheme in which all the posterior probability distributions of the unknowns are easy to sample from. Extensive simulation results show that the estimation procedure yields excellent results.

In many Monte Carlo simulations, it is important to generate samples from given densities. Researchers in statistical signal processing and related disciplines have shown increased interest for a generator of random vectors with truncated multivariate normal probability density functions (PDFs). A straightforward method for their generation is to draw samples from the multivariate normal density and reject the ones that are outside the acceptance region. This method, which is known as rejection sampling, can be very inefficient, especially for high dimensions and/or relatively small supports of the random vectors. We propose an approach for generation of vectors with truncated Gaussian densities based on Gibbs sampling, which is simple to use and does not reject any of the generated vectors.

Multibeam cellular communication systems in which multiple services are of interest are considered. An architecture of channel sharing and medium access which can support multiple services with different CIR requirements is proposed. Dynamic channel assignment (DCA) is used to improve channel reuse and to allocate different CIR quality channels within a channel layout which incorporates several frequency reuse patterns. Channels with higher CIR levels are primarily for services that require higher quality communication links, such as data calls, and channels with lower CIR levels are only for services that require lower quality links, such as voice calls. Services that require lower quality links can also access channels with higher CIR levels if channels of lower quality groups are not available. We develop a tractable analytical model for the system using multidimensional birth-death processes with an appropriate state characterization. Theoretical traffic performance characteristics such as blocking probability, forced termination probability and carried traffic are determined.

Markov chain Monte Carlo (MCMC) samplers have been a very powerful methodology for estimating signal parameters. With the introduction of the reversible jump MCMC sampler, which is a Metropolis-Hastings method adapted to general state spaces, the potential of the MCMC methods has risen to a new level. Consequently, the MCMC methods currently play a major role in many research activities. In this paper we propose a reversible jump MCMC sampler based on predictive densities obtained by integrating out unwanted parameters. The proposal densities are approximations of the posterior distributions of the remaining parameters obtained by sampling importance resampling (SIR). We apply the method to the problem of signal detection and parameter estimation of signals. To illustrate the proposed procedure, we present an example of sinusoids embedded in noise.

It is proposed to jointly estimate the parameters of non-Gaussian autoregressive (AR) processes in a Bayesian context using the Gibbs sampler. Using the Markov chains produced by the sampler an approximation to the vector MAP estimator is implemented. The results reported here used AR(4) models driven by noise sequences where each sample is i.i.d. as a two component Gaussian sum mixture. The results indicate that using the Gibbs sampler to approximate the vector MAP estimator provides estimates with precision that compares favorably with the CRLBs. Also discussed are issues regarding the implementation of the Gibbs sampler for AR mixture models.

The problem of model selection is addressed by the Bayesian methodology and the bootstrap technique. As a rule for choosing the best model from a set of proposed models, the maximum a posteriori principle is used. The evaluation of the maximum a posteriori probability (MAP) of each model amounts to computation of integrals whose integrands may be very peaked functions. We carry out the integration by importance sampling, where the importance function is a multivariate Gaussian whose samples are obtained by the bootstrap technique. The performance of the MAP rule is examined by computer simulations, and comparisons with the widely used AIC (Akaike information criterion) and MDL (minimum description length) rules are made.

In this paper cluster validation criteria for piecewise constant image segmentation are proposed. All the criteria are based on the maximum a posteriori (MAP) principle and derived and implemented by four different, but related approaches. They are obtained by using Taylor expansions and three of them are derived by Bayesian predictive densities. The third and fourth criteria are implemented by the bootstrap technique, and their evaluations are, therefore, computationally more intensive than the evaluations of the first two. The proposed rules are compared by computer simulations with the widely used Akaike's information criterion (AIC) and the minimum description length (MDL) criteria.

An algorithm to randomly generate the parameters of stable invertible autoregressive moving average processes of order (p,q)-ARMA(p,q)-is presented. The AR and MA portions are independent of each other, and their respective parameters have jointly uniform distributions with support defined by stability and invertibility considerations. The uniform density insures that each possible model is equally likely. The algorithm uses the Levinson-Durbin recursion to guarantee the poles and zeros are inside the unit circle, thus avoiding coefficient resampling typical of "generate and test" methods. To initialize the Levinson-Durbin recursion for each model order, the reflection coefficients are generated using a rejection sampling technique.

A novel method for edge detection in vector images is proposed that does not require any prior knowledge of the imaged scenes. In the derivation, it is assumed that the observed vector images are realizations of spatially quasistationary processes, and that the vector observations are generated by parametric probability distribution functions of known form whose parameters are in general unknown. The method detects and estimates the edge locations using a criterion derived by Bayesian theory. It chooses the number of edges and their locations according to the maximum a posteriori probability (MAP) principle. We provide results that demonstrate its performance on synthesized and real images.

In this correspondence, the objective is to segment vector images, which are modeled as multivariate finite mixtures. The underlying images are characterized by Markov random fields (MRFs), and the applied segmentation procedure is based on the expectation-maximization (EM) technique. We propose an initialization procedure that does not require any prior information and yet provides excellent initial estimates for the EM method. The performance of the overall segmentation is demonstrated by segmentation of simulated one-dimensional (1D) and multidimensional magnetic resonance (MR) brain images.

A new approach for the processing of piecewise-constant signals is proposed. It is based on modeling the observed data as a sum of a random signal and noise. The random signal has a Gibbs distribution, and the noise is Gaussian. A MAP criterion is derived for joint estimation of the number of signal levels and reconstruction of the signal. The criterion comprises of three terms, one corresponding to the likelihood of the data and two to penalties. One penalty term penalizes for unnecessary transitions, and the other, for unnecessary levels. The method has been tested on synthesized data and applied to single ion channel recordings.

We propose an unsupervised vector image segmentation technique that combines the iterated conditional modes (ICM) procedure with an initialization scheme that requires minimal prior knowledge. As is well known, every iterative segmentation procedure needs initialization parameters, which are usually obtained from training data. In the absence of such data, the initialization becomes a critical step towards accurate segmentation because bad initializations can lead to poor performance. Our initialization scheme, referred to as tree structure (TS) initialization, represents a sequence of binary searches and is similar to a method for data compression in coding theory. The scheme does not require any a priori information or initial parameters, except for the number of classes, and therefore is completely data-driven. Computer simulations on multidimensional magnetic resonance (MR) brain images are provided to demonstrate the overall excellent performance of the proposed TS-ICM method.

We propose an iterative algorithm for minimum mean square error (MMSE) estimation of the parameters of multiple superimposed linear chirp signals in white Gaussian noise. The parameter estimation of each chirp component is carried out by two dimensional integration. The integrals are derived with the assumption that the remaining chirp signals have parameters whose values are fixed at their current estimates. The necessary parameter initializations are obtained by tracking the Choi-Williams (1989) time-frequency distribution and applying the least-squares method. Computer simulations provide a comparison between our scheme and the alternating projection (AP) method.

In recent years, many image segmentation approaches have been based on Markov random fields (MRFs). The main assumption of the MRF approaches is that the class parameters are known or can be obtained from training data. In this paper the authors propose a novel method that relaxes this assumption and allows for simultaneous parameter estimation and vector image segmentation. The method is based on a tree structure (TS) algorithm which is combined with Besag's iterated conditional modes (ICM) procedure. The TS algorithm provides a mechanism for choosing initial cluster centers needed for initialization of the ICM. The authors' method has been tested on various one-dimensional (1-D) and multidimensional medical images and shows excellent performance. In this paper the authors also address the problem of cluster validation. They propose a new maximum a posteriori (MAP) criterion for determination of the number of classes and compare its performance to other approaches by computer simulations.

Investigates the problem of model order selection of damped sinusoids from a Bayesian perspective. The authors derive a maximum a posteriori (MAP) criterion through a combination of Bayesian inference and predictive densities. The MAP criterion is more appropriate for damped sinusoidal models (and transient data models in general) than are the SVD based information theoretic criteria in V. Umpathy Reddy and L.S. Biradar (1993). Simulation results are provided that display the breakdown of the AIC and MDL when the data record length is not properly coupled with the information bearing portion of the data model. This deterioration in performance is related to both the underlying asymptotics upon which the AIC and MDL rules were originally based, and to their invalid penalty terms. Conversely, the MAP criterion is not based on asymptotics, and proves to be more reliable and consistent when the observation length is varied.

The model order selection in signal processing problems has often been addressed by employing the Akaike information criterion (AIC) and the minimum description length principle (MDL). The popularity of these criteria partly stems from the intrinsically simple means by which they can be implemented. They can, however, produce misleading results if they are indiscriminately utilized. A case in point is the problem of model order selection of sinusoidal signals embedded in Gaussian noise. Following the Bayesian methodology, for these signals we derive a model order selection criterion whose general form is similar to the AIC and MDL. It contains both, the log-likelihood and the penalty terms, the latter of which is modified and more appropriate for the selection of sinusoidal-signals. Simulation results are provided, and they disclose remarkable improvement in our selection rule over the MDL and AIC.

A novel method for detection of closely spaced sinusoids in noise is proposed. It is based on the notch periodogram and a simple detection criterion. Compared with other well-known approaches, this method does not require precise estimation of the signal parameters and is not computationally intensive. Simulation results are included, which confirm the excellent performance of the method.

A Bayesian spectrum estimator of harmonic signals in Gaussian noise is derived. It is based on the expected value of the theoretical signal spectrum over the joint posterior density function of the signal and noise parameters. Simulation results are provided that show its performance and comparison with MUSIC.>

A new criterion based on Bayesian predictive densities and subspace decomposition is proposed to estimate the number and the frequencies of close cisoids in colored noise. The colored noise is modeled by an autoregression whose order has also to be estimated. The proposed criterion significantly outperforms the MDL and AIC in correctly determining the number of cisoids and the order of the autoregressive process. Furthermore, an algorithm for frequency estimation is proposed that considerably reduces the computational complexity of the criterion.>

A new criterion for off-line segmentation of signals is proposed. The derivation is general in the sense that it is valid for signals that are parameterized by linear or nonlinear functions embedded in additive noise, be it non-white or non-Gaussian. In addition, a penalty function is developed whose terms are easily justified and interpreted. As a special case, a criterion for segmentation of polynomial signals in white Gaussian noise is analyzed and compared with the AIC and MDL. The simulation results show that our criterion markedly outperforms its popular counterparts.>

A Bayesian approach to autoregressive (AR) signal modelling is proposed. In contrast to previous research, the exact posterior density of the model parameters is utilized and minimum mean square estimates (MMSE) are evaluated. To compute the estimates, a numerically efficient procedure is presented which can be viewed as an alternative to multidimensional optimization. Our approach can be used to investigate many signal characteristics such as the signal's spectrum, marginal densities for prediction or even model selection. Simulation results confirm our expectations and illustrate the improvement over the classic, maximum conditional likelihood (MCL) approach to AR signal modelling.>

Multiple hypotheses testing arises in many signal processing applications. It can be viewed as a model selection problem, and as such, is commonly resolved by invoking the popular MDL or AIC rules. These rules are very often inappropriately applied however, particularly when the signal models violate the underlying conditions on which the rules are based. The tools of Bayesian inference provide a mechanism for the specification of more accurate criteria for model selection. Through appropriate approximations of the prior predictive densities, one can develop rules similar in form to the AIC and MDL, but with a more complete penalty term. The derived rules are approximations of the maximum a posteriori criterion (MAP), which for a uniform cost function is known to be optimal. We present a general solution to the problem followed by a consideration of the special case of damped signals in white Gaussian noise. In particular, we investigate models whose signal components are comprised of damped sinusoids. Monte Carlo simulations are performed, the results of which indicate a marked improvement over both, the AIC and MDL.>

A novel solution for the problem of detecting the number of complex exponentials embedded in Gaussian noise and estimating their frequencies is proposed. In contrast to standard techniques, the marginalized posterior density is utilized to evaluate a model selection criterion and compute the MMSE estimates. To compute the required integrals, a numerically efficient procedure, termed adaptive importance sampling (AIS), is introduced. This procedure can naturally handle parameter constraints and it greatly improves convergence as compared to standard Monte Carlo approaches. Our method has the benefit of not only outperforming the standard techniques, but it also sidesteps the pitfalls associated with multidimensional optimization.>

A very useful and not too restrictive class of models of nonstationary signals is based upon the assumptions that the signals are composed of independent and stationary segments that can be represented by autoregressive models. A usual task is then to find the number of segments of the observed signal, their boundaries, and the best model for each segment. A Bayesian solution to this task is proposed which does not require setting of any thresholds. The technical implementation of the solution is carried out via dynamic programming. The Monte Carlo simulations show excellent results.>

The work on model order estimation by Bayesian predictive densities of 1-D real autoregressive processes is extended to 2-D complex autoregressive processes. According to the procedure, the best model is the one which most accurately predicts the data yet to be observed and whose parameters are estimated from the data already observed. The derivation steps of the algorithm are demonstrated and verified by computer simulations. The computer simulations show that the algorithm based on this approach yields good results.>

A model selection criterion based on Bayesian predictive densities is derived. Starting with an improper prior distribution of the model parameters and using one portion of the data, a proper distribution is obtained which is further used as a prior for obtaining predictive densities according to the model and the first portion of the data. The remaining portion is used to validate the model through the obtained predictive densities. The procedure is applied to the set of linear regression models. The performance of the criterion is illustrated by simulation results.>

Frequency-rate estimation of a linearly frequency modulated signal is addressed. A simple estimation procedure that yields accurate estimates for moderately high signal-to-noise ratios is proposed. By transforming the original sequence into a sequence that is the phase data differenced twice, the problem becomes equivalent to estimating a constant in colored noise. Computer simulations verify the expected performance and show that the estimator achieves the Cramer-Rao bound for signal-to-noise ratios above 8 dB.>

An approximate maximum-likelihood estimator is derived for ARMA (autoregressive moving-average) processes and is shown to correspond to least-squares fitting of the estimated cepstrum of the process by the model cepstrum. Experiments with several simple ARMA