This paper introduces the Quotient Bayesian Learning Rule, an extension of natural-gradient Bayesian updates to probability models that fall outside the exponential family. Building on the observation that many heavy-tailed and otherwise non-exponential distributions arise as marginals of minimal exponential families, we prove that such marginals inherit a unique Fisher–Rao information geometry via the quotient-manifold construction. Exploiting this geometry, we derive the Quotient Natural Gradient algorithm, which takes steepest-descent steps in the well-structured covering space, thereby guaranteeing parameterization-invariant optimization in the target space. Empirical results on the Student-$t$ distribution confirm that our method converges more rapidly and attains higher-quality solutions than previous variants of the Bayesian Learning Rule. These findings position quotient geometry as a unifying tool for efficient and principled inference across a broad class of latent-variable models.

Ship collision accidents frequently cause casualties and significant property losses. These collisions mainly occur by incorrectly interpreting the intents of other ships’ navigators. However, inferring the avoidance intents of other ships is challenging, because the uncertain motions and the long-lasting dynamic interactions during encounters usually obscure the true intents. To address this, we propose a probabilistic graph model to infer the hidden avoidance intent of ships. Specifically, the dynamic encounter evolution is expressed in a model represented by a factor graph, where the intent beliefs are accumulated and propagated over time through message passing in the graph. In the inference procedure, we develop a hybrid Bayesian inference approach, integrating a data-driven component derived from empirical priors fitted to historical data, and a model-driven component capturing the ship’s control process. We derive 9 types of intents from naturalistic ship encounters and evaluate the model on a validation split. The quantitative metrics demonstrate that, on average, the proposed procedure can accurately infer the intents 14.04 seconds in advance and outperforms the baseline in macro-averaged recall rate (0.2919) and F1-score (0.2843).Note to Practitioners—This work aims to enhance safety in ship navigation by introducing an intelligent navigation support system to prevent collisions. A significant cause of ship collision accidents is the misinterpretation of other navigators’ intents. Estimating the intents of marine traffic participants can significantly reduce collision rates. However, real-time intent inference in ship encounters poses challenges due to complex dynamics and uncertain motions. We propose a novel probabilistic graphical model that allows us to infer avoidance intents, evaluated on a naturalistic Automatic Identification System (AIS) dataset, demonstrating effectiveness in real-world scenarios. The practical implications of our work are substantial for the maritime industry. Ship captains and intelligent automated vessels can use our system to enhance situational awareness, enabling informed collision avoidance decisions. Our work aims to improve the safety and efficiency of ship navigation, reducing collision risks and ensuring the well-being of crew members and valuable cargo.

Fetal well-being during labor is currently assessed by medical professionals through visual interpretation of the cardiotocogram (CTG), a simultaneous recording of Fetal Heart Rate (FHR) and Uterine Contractions (UC). This method is disputed due to high inter- and intra-observer variability and a resulting increase in the number of unnecessary interventions. A method for computerized interpretation of the CTG, based on Contrastive Predictive Coding (CPC) is presented here. We hypothesize that the CPC framework, when trained on healthy fetuses only, can predict the FHR response of healthy fetuses to UC, but will provide significant prediction error in case of fetuses with compromised condition. To that end, we have extended the original CPC model by making stochastic, recurrent, and conditioned (upon Uterine Contractions) predictions. We, moreover, introduce a new training objective that was found more suitable for the task of anomaly detection. Based on the detection of out-of-distribution behaviour and deviations from subject-specific behaviour, the proposed model is capable of achieving promising results for identification of suspicious and anomalous FHR events in the CTG, with an average correlation coefficient of 0.80±0.13 with respect to expert annotations.

The aim of Probabilistic Programming (PP) is to automate inference in probabilistic models. One efficient realization of PP-based inference concerns variational message passing-based (VMP) inference in a factor graph. VMP is efficient but in principle only leads to closed-form update rules in case the model consists of conjugate and/or conditionally conjugate factor pairs. Recently, Extended Variational Message Passing (EVMP) has been proposed to broaden the applicability of VMP by importance sampling-based particle methods for non-linear and non-conjugate factor pairs. EVMP automates the importance sampling procedure by employing forward messages as proposal distributions, which unfortunately may lead to inaccurate estimation results and numerical instabilities in case the forward message is not a good representative of the unknown correct posterior. This paper addresses this issue by integrating an adaptive importance sampling procedure with message passing-based inference. The resulting method is a hyperparameter-free approximate inference engine that combines recent advances in stochastic adaptive importance sampling and optimization methods. We provide an implementation for the proposed method in the Julia package ForneyLab.jl.

Stochastic approximation methods for variational inference have recently gained popularity in the probabilistic programming community since these methods are amenable to automation and allow online, scalable, and universal approximate Bayesian inference. Unfortunately, common Probabilistic Programming Languages (PPLs) with stochastic approximation engines lack the efficiency of message passing-based inference algorithms with deterministic update rules such as Belief Propagation (BP) and Variational Message Passing (VMP). Still, Stochastic Variational Inference (SVI) and Conjugate-Computation Variational Inference (CVI) provide principled methods to integrate fast deterministic inference techniques with broadly applicable stochastic approximate inference. Unfortunately, implementation of SVI and CVI necessitates manually driven variational update rules, which does not yet exist in most PPLs. In this paper, we cast SVI and CVI explicitly in a message passing-based inference context. We provide an implementation for SVI and CVI in ForneyLab, which is an automated message passing-based probabilistic programming package in the open source Julia language. Through a number of experiments, we demonstrate how SVI and CVI extends the automated inference capabilities of message passing-based probabilistic programming.

In this paper we discuss variational message passing-based (VMP) inference in a switching Hierarchical Gaussian Filter (HGF). An HGF is a flexible hierarchical state space model that supports closed-form VMP-based approximate inference for tracking of both states and slowly time-varying parameters. Since natural signals often submit to regime-switching dynamics, there is a need for low-complexity closed-form inference in switching state space models. Here we extend the HGF model with parameter switching mechanics and derive closed-form VMP update rules for plug-in applications in factor graph-based models. These VMP rules support both tracking of latent variables and variational free energy as a model performance measure. We show that the switching HGF performs better than a non-switching HGF on modelling of a stock market data set.

Active Inference is an emerging framework for designing intelligent agents. In an Active Inference setting, any task is formulated as a variational free energy minimisation problem on a generative probabilistic model. Goal-directed behaviour relies on a clear specification of desired future observations. Learning desired observations would open up the Active Inference approach to problems where these are difficult to specify a priori. This paper introduces the BAyesian Target Modelling for Active iNference (BATMAN) approach, which augments an Active Inference agent with an additional, separate model that learns desired future observations from a separate data source. The main contribution of this paper is the design of a coupled generative model structure that facilitates learning desired future observations for Active Inference agents and supports integration of Active Inference and classical methods in a joint framework. We provide proof-of-concept validation for BATMAN through simulations.

Probabilistic programming provides a structured approach to signal processing algorithm design. The design task is formulated as a generative model, and the algorithm is derived through automatic inference. Efficient inference is a major challenge; e.g., the Shafer-Shenoy algorithm (SS) performs badly on models with large treewidth, which arise from various real-world problems. We focus on reducing the size of discrete models with large treewidth by storing intermediate factors in compressed form, thereby decoupling the variables through conditioning on introduced weights. This work proposes pruning of these weights using Kullback-Leibler divergence. We adapt a strategy from the Gaussian mixture reduction literature, leading to Kullback-Leibler Tensor Belief Propagation (KL-TBP), in which we use agglomerative hierarchical clustering to subsequently merge pairs of weights. Experiments using benchmark problems show KL-TBP consistently achieves lower approximation error than existing methods with competitive runtime.

Hierarchical autoregressive (AR) models can describe many complex physical processes. Unfortunately, online adaptation in these models under non-stationary conditions remains a challenge. In this paper, we track states and parameters in a hierarchical AR filter by means of variational message passing (VMP) in a factor graph. We derive VMP update rules for an "AR node" that can be re-used at various hierarchical levels and supports automated message passing-based inference for states and parameters. The proposed method is experimentally validated for a 2-level hierarchical AR model.

We address the problem of online state and parameter estimation in the Hierarchical Gaussian Filter (HGF), which is a multi-layer dynamic model with non-conjugate couplings between upper-layer hidden states and parameters of a lower layer. These non-conjugacies necessitate the approximation of marginalization and expectation integrals, while the online inference constraint renders batch learning and Monte Carlo sampling unsuitable. Here we formulate the problem as a message passing task on a factor graph and propose an online variational message passing-based state and parameter tracking algorithm, which uses Gaussian quadrature to deal with non-conjugacies. We present improved message update rules for all non-conjugate couplings, thus allowing a plug-in inference method for alternative models with equivalent non-conjugate layer couplings. The method is validated on a recorded time series of Bitcoin prices.

Development of hearing aid (HA) signal processing algorithms entails an iterative process between two design steps, namely algorithm development and the embedded implementation. Algorithm designers favor high-level programming languages for several reasons including higher productivity, code readability and, perhaps most importantly, availability of state-of-the-art signal processing frameworks that open new research directions. Embedded software, on the other hand, is preferably implemented using a low-level programming language to allow finer control of the hardware, an essential trait in real-time processing applications. In this paper we present a technique that allows deploying DSP algorithms written in Julia, a modern high-level programming language, on a real-time HA processing platform known as openMHA. We demonstrate this technique by using a model-based Bayesian inference framework to perform real-time audio processing.

Acoustic smart sensor networks can provide valuable actionable intelligence to authorities for managing safety in the urban environment. A spatial filter (beamformer) for localization and separation of acoustic sources is a key component of such a network. However, classical methods such as delay-and-sum beamforming fail, because sources are located at varying distances from the sensor array. This causes a regularization problem where either far-away sources are wrongly attenuated, or noise is wrongly amplified. We solve this by considering source strength and location as random variables. The posterior distributions are approximated using Gibbs sampling. Each marginal is computed by combining importance sampling and inverse transform sampling using Chebyshev polynomial approximation. This leads to an iterative algorithm with similarities to deconvolution beamforming. Our method is robust against deviations in manifold model, can deal with sources at different distances and power levels, and does not require an a priori known number of sources.

Hearing Aid (HA) algorithms need to be tuned (“fitted”) to match the impairment of each specific patient. The lack of a fundamental HA fitting theory is a strong contributing factor to an unsatisfying sound experience for about 20% of HA patients. This paper proposes a probabilistic modeling approach to the design of HA algorithms. The proposed method relies on a generative probabilistic model for the hearing loss problem and provides for automated inference of the corresponding (1) signal processing algorithm, (2) the fitting solution as well as (3) a principled performance evaluation metric. All three tasks are realized as message passing algorithms in a factor graph representation of the generative model, which in principle allows for fast implementation on HA or mobile device hardware. The methods are theoretically worked out and simulated with a custom-built factor graph toolbox for a specific hearing loss model.

Non-intrusive signal quality assessment in general, and its application to speech signal processing, in particular, builds extensively upon statistical regression models. Commonly, the raw preference scores used for fitting these models belong to a categorical scale. Averaging the scores over a number of test subjects results in smooth, close-to-continuous ratings, thus justifying the use of regression as opposed to classification models. A form of marginalization, averaging subjective ratings takes away useful information about the reliability of individual test points. Using a model tailored to the raw data achieves highly competitive performance in terms of conventional performance measures while providing the additional advantage of identifying the usability of individual test points. In this paper, we consider the application of discrete choice models to non-intrusive quality assessment of speech.

We propose a noise estimation algorithm for single-channel noise suppression in dynamic noisy environments. A stochastic-gain hidden Markov model (SG-HMM) is used to model the statistics of nonstationary noise with time-varying energy. The noise model is adaptive and the model parameters are estimated online from noisy observations using a recursive estimation algorithm. The parameter estimation is derived for the maximum-likelihood criterion and the algorithm is based on the recursive expectation maximization (EM) framework. The proposed method facilitates continuous adaptation to changes of both noise spectral shapes and noise energy levels, e.g., due to movement of the noise source. Using the estimated noise model, we also develop an estimator of the noise power spectral density (PSD) based on recursive averaging of estimated noise sample spectra. We demonstrate that the proposed scheme achieves more accurate estimates of the noise model and noise PSD, and as part of a speech enhancement system facilitates a lower level of residual noise.

Loss of understanding speech in noise, known as signal to noise ratio (SNR) loss, is the largest problem of most hearing aid users today. This paper discusses the most important noise reduction (NR) agents available in current hearing aids and their ability to improve SNR-loss. Then, it is investigated how the many different NR agents can best be integrated to maximize results. A global minimization shows that a (Generalized Sidelobe Canceller) beamformer followed by single microphone NR agents is the theoretically optimal configuration. Since the performance of the different NR agents depends on the environment, it is suggested to use an environmental classifier to select the optimal combination of NR agents. Either by a fixed set of rules or by using a self training network catered to the specific SNR-loss of the hearing impaired.

A method for adaptive (online) pruning and constructing a (layered) computational network is introduced. The dimensions of the network are updated for every new available sample, which makes this technique highly suitable for tracking nonstationary sources. This method extends work on predictive least squares by Rissanen (1986) and Wax (1988) to an adaptive updating scheme. The algorithm is demonstrated by an application to adaptive prediction of exchange rates.

Most contemporary speech recognizers are designed to operate with close-talking speech and they work best in a quiet laboratory condition. There is an apparent need to render environment robustness to these systems. The objective of the paper is to explore utility of existing speech recognition technology in adverse "real-world" environments for distant-talking applications. A synergistic system consisting of microphone array and neural network (MANN) is utilized to mitigate environmental interference introduced by reverberation, ambient noise, and channel mismatch between training and testing conditions. The MANN system is evaluated with experiments on continuous distant-talking speech recognition. The results show that the MANN system elevates the word recognition accuracy to a level which is competitive with a retrained speech recognizer and that the neural network compensation performs better than some previously researched techniques.

The authors introduce a new class of filters, the generalized feedforward structures, that combine attractive properties of the moving average (MA) filters for adaptation (i.e. fast algorithms, trivial stability) with some of the power of autoregressive moving average (ARMA) filters (i.e. decoupling of the length of the impulse response with filter order). Preliminary results show that this class of filters is much more efficient than conventional MA filters (i.e. for a given minimum mean square error (MSE) the filter order is much smaller). The authors have extended the Wiener-Hopf solution for this class of filters and have developed some design tools. The generalized feedforward structures accept Widrow's adaptive linear combiner as a special case. An identification example is presented.>