This paper investigates the use of transformers to approximate the mean-field dynamics of interacting particle systems exhibiting collective behavior. Such systems are fundamental in modeling phenomena across physics, biology, and engineering, including opinion formation, biological networks, and swarm robotics. The key characteristic of these systems is that the particles are indistinguishable, leading to permutation-equivariant dynamics. First, we empirically demonstrate that transformers are well-suited for approximating a variety of mean field models, including the Cucker-Smale model for flocking and milling, and the mean-field system for training two-layer neural networks. We validate our numerical experiments via mathematical theory. Specifically, we prove that if a finite-dimensional transformer effectively approximates the finite-dimensional vector field governing the particle system, then the $L_\infty$ distance between the \textit{expected transformer} and the infinite-dimensional mean-field vector field can be bounded by a function of the number of particles observed during training. Leveraging this result, we establish theoretical bounds on the distance between the true mean-field dynamics and those obtained using the transformer.

Score matching based diffusion has shown to achieve the state of art results in generation modeling. In the original score matching based diffusion algorithm, the forward equation is a differential equation for which the probability density equation evolves according to a linear partial differential equation, the Fokker-Planck equation. A drawback of this approach is that one needs the data distribution to have a Lipschitz logarithmic gradient. This excludes a large class of data distributions that have a compact support. We present a deterministic diffusion process for which the vector fields are always Lipschitz and hence the score does not explode for probability measures with compact support. This deterministic diffusion process can be seen as a regularization of the porous media equation equation, which enables one to guarantee long term convergence of the forward process to the noise distribution. Though the porous media equation is itself not always guaranteed to have a Lipschitz vector field, it can be used to understand the closeness of the output of the algorithm to the data distribution as a function of the the time horizon and score matching error. This analysis enables us to show that the algorithm has better dependence on the score matching error than approaches based on stochastic diffusions. Using numerical experiments we verify our theoretical results on example one and two dimensional data distributions which are compactly supported. Additionally, we validate the approach on a modified MNIST data set for which the distribution is concentrated on a compact set. In each of the experiments, the approach using deterministic diffusion performs better that the diffusion algorithm with stochastic forward process, when considering the FID scores of the generated samples.

In this article, we introduce a model and a control approach for herding a swarm of “follower” agents to a target distribution among a set of states using a single “leader” agent. The follower agents evolve on a finite state space that is represented by a graph and transition between states according to a continuous-time Markov chain (CTMC), whose transition rates are determined by the location of the leader agent. The control problem is to define a sequence of states for the leader agent that steers the probability density of the forward equation of the Markov chain. For the case, when the followers are possibly interacting, we prove local approximate controllability of the system about equilibrium probability distributions. For the case, when the followers are noninteracting, we design two switching control laws for the leader that drive the swarm of follower agents asymptotically to a target probability distribution that is positive for all states. The first strategy is open-loop in nature, and the switching times of the leader are independent of the follower distribution. The second strategy is of feedback type, and the switching times of the leader are functions of the follower density in the leader's current state. We validate our control approach through numerical simulations with varied numbers of follower agents that evolve on graphs of different sizes, through a 3-D multirobot simulation in which a quadrotor is used to control the spatial distribution of eight ground robots over four regions, and through a physical experiment in which a swarm of ten robots is herded by a virtual leader over four regions.

This paper explores a stochastic approach for controlling swarms of independent robots toward a target distribution in a bounded domain. The robot swarm has no central controller, and individual robots lack both communication and localization capabilities. Robots can only measure a scalar field (e.g. concentration of a chemical) from the environment and from this deduce the desired local swarm density. Based on this value, each robot follows a simple control law that causes the swarm as a whole to diffuse toward the target distribution. Using a new holonomic drive robot, we present the first confirmation of this control law with physical experiment. Despite deviations from assumptions underpinning the theory, the swarm achieves the theorized convergence to the target distribution in both simulation and experiment. In fact, simulated and experimental performance agree with one another and with our hypothesis that the error from the target distribution is inversely proportional to the square root of the number of robots. This is evidence that the algorithm is both practical and easily scalable to large swarms.

This paper addresses a trajectory planning and task allocation problem for a swarm of resource-constrained robots that cannot localize or communicate with each other and that exhibit stochasticity in their motion and task-switching policies. We model the population dynamics of the robotic swarm as a set of advection-diffusion-reaction partial differential equations (PDEs), a linear parabolic PDE model that is bilinear in the robots' velocity and task-switching rates. These parameters constitute a set of time-dependent control variables that can be optimized and broadcast to the robots prior to their deployment. The planning and allocation problem can then be formulated as a PDE-constrained optimization problem, which we solve using techniques from optimal control. Simulations of a commercial pollination scenario validate the ability of our control approach to drive a robotic swarm to achieve predefined spatial distributions of activity over a closed domain, which may contain obstacles.