This paper explores the impact of relational state abstraction on sample efficiency and performance in collaborative Multi-Agent Reinforcement Learning. The proposed abstraction is based on spatial relationships in environments where direct communication between agents is not allowed, leveraging the ubiquity of spatial reasoning in real-world multi-agent scenarios. We introduce MARC (Multi-Agent Relational Critic), a simple yet effective critic architecture incorporating spatial relational inductive biases by transforming the state into a spatial graph and processing it through a relational graph neural network. The performance of MARC is evaluated across four collaborative tasks, including a novel environment with heterogeneous agents. We conduct a comprehensive empirical analysis, comparing MARC against state-of-the-art MARL baselines, demonstrating improvements in both sample efficiency and asymptotic performance, as well as its potential for generalization. Our findings suggest that a minimal integration of spatial relational inductive biases as abstraction can yield substantial benefits without requiring complex designs or task-specific engineering. This work provides insights into the potential of relational state abstraction to address sample efficiency, a key challenge in MARL, offering a promising direction for developing more efficient algorithms in spatially complex environments.

The special role of epistemic uncertainty in Machine Learning is now well recognised, and an increasing amount of research is focused on methods for dealing specifically with such a lack of knowledge. Yet, most often, a probabilistic representation is considered for both aleatoric and epistemic uncertainties, hence creating challenges in applications where decoupling these two types of uncertainty is necessary. In this work, we show that an alternative representation of epistemic uncertainty, based on possibility theory, maintains many of the convenient features of standard Bayesian inference while displaying specific behaviours and properties that closely match the ones of an intuitive notion of information. Our main contributions are: i) a general framework for jointly representing epistemic and aleatoric uncertainties, ii) a Bernstein-von Mises theorem for the analogue of Bayes’ rule in possibility theory, iii) a version of the law of large numbers and of the central limit theorem for the associated variables, and iv) an analysis of the properties of the possibilistic maximum a posteriori. These results highlight that a dedicated and principled representation of epistemic uncertainty, that is compatible with standard Bayesian inference and preserves many of its strengths, is attainable.

Quantifying negative information in an efficient way is a challenging task, especially when this information has to be communicated on a network. In this article we leverage the unique properties offered by possibility theory to quantify and approximate the negative information arising in the context of tracking a target with a sensor that has a limited field of view. We also verify experimentally that the corresponding target tracking methodology can be applied in a decentralised manner to a sensor network, while maintaining a performance close to the idealised case where the initial location of the target is better-known.

This paper is concerned with the Bayesian estimation with possibility theory of the number of targets in an area of interest when no prior information is available. The estimation relies solely on the number of detections that is reported every time the area is surveyed. It is assumed that detections can be true and false positives. Possibility theory is capable of quantifying the complete absence of prior information on the number of targets with the introduction of a prior possibility function that is equal to one everywhere on the set of natural numbers. This has no equivalent in probability theory. The prior possibility function is turned into a posterior possibility function from which a point estimate of the number of targets can be derived. Besides the point estimate, a meaningful notion of confidence interval can be derived from the posterior possibility and necessity associated with subsets of the natural numbers.

We consider a class of diffusion processes with finite-dimensional parameters and partially observed at discrete time instances. We propose a methodology to unbiasedly estimate the expectation of a given functional of the diffusion process conditional on parameters and data. When these unbiased estimators with appropriately chosen functionals are employed within an expectation-maximization algorithm or a stochastic gradient method, this enables statistical inference using the maximum likelihood or Bayesian framework. Compared to existing approaches, the use of our unbiased estimators allows one to remove any time-discretization bias and Markov chain Monte Carlo burn-in bias. Central to our methodology is a novel and natural combination of multilevel randomization schemes and unbiased Markov chain Monte Carlo methods, and the development of new couplings of multiple conditional particle filters. We establish under assumptions that our estimators are unbiased and have finite variance. We illustrate various aspects of our method on an Ornstein--Uhlenbeck model, a logistic diffusion model for population dynamics, and a neural network model for grid cells.

A key challenge in the design of a Resident Space Object (RSO) detection and tracking algorithm is the scarcity of available information on the various sources of uncertainty in the underlying estimation problem, such as the orbital mechanics or the data sources producing the observations. Some of these uncertain components lack statistical description, such as the Two-Line Elements (TLEs), so that their description as a random variable and their exploitation in a standard Bayesian filtering algorithm remains largely unexplored. This paper exploits uncertain variables and outer probability measures (o.p.m.s), a generalization of the concepts of random variables and probability distributions, in order to propose a representation of all the uncertain components of a typical RSO tracking problem that matches the information available to a space analyst, and fuse them into a coherent Bayesian estimation filter. This algorithm is then illustrated on a scenario where the kinematic state of a Low-Earth Orbit (LEO) satellite is estimated, using realistically simulated radar observations and real TLEs queried from the U.S. Strategic Command (USSTRATCOM)'S catalog.