This paper presents a novel sequential variational inference algorithm for distributed multi-sensor tracking and fusion. The algorithm is based on a multi-sensor target representation where a target is represented jointly by its states at different sensors and a global state fusing all sensor data. A tree-structured graphical model is adopted to model the dependencies between these states at a time instant. In contrast to previous work, most of which is based on belief propagation, we propose an alternative variational inference algorithm that combines importance sampling techniques and variational methods for graphical models to infer the multi-sensor target states over time. In particular, the sequential variational inference algorithm distributes the global inference to each node in graphical models. Via a message-passing scheme similar to BP, the inference processes at different nodes collaborate so that each integrates information from all sensors. One contribution of this paper is the proper design of an importance function for generating samples to approximate the target distributions. Experiments on a synthetic example show that our method achieves results comparable to BP-based methods.

Multi-camera tracking poses a data fusion problem that integrates image measurements from different cameras. A novel approach to tracking using multiple cameras is proposed that combines particle filters and belief propagation in a unified framework. In each view, a target is tracked by a dedicated particle-filter-based local tracker. The trackers in different views collaborate via belief propagation so that a local tracker operating in one view is able to take advantage of additional information from other views. The message passing mechanism in belief propagation guarantees that wrong information is not propagated across views, thus avoiding a common problem in multi-camera tracking. Target states in each view and in 3D are inferred based on the multi-view image measurements by a set of particle filters, and a sequential belief propagation algorithm implements collaboration between the view-specific particle filters. We demonstrate the effectiveness of our approach on sequences of video surveillance and soccer games

A moving target produces a coherent cluster of feature points in the image plane. This motivates our novel method of tracking multiple targets via feature points. First, the Harris corner detector and the Lucas-Kanade tracker are applied in each frame to detect feature points and their associated velocities. Points that are both spatially co-located and exhibit similar motion are grouped into clusters. Due to the non-Gaussian distribution of the points in a cluster and the multi-modality resulting from multiple targets, a special particle filter, the mixture particle filter, is adopted to model the mixture point distribution over time. Each cluster is treated as a mixture component and is modeled by an individual particle filter. The filters in the mixture are instantiated and initialized by applying the EM algorithm, are reclustered by merging overlapping clusters and splitting spatially disjoint clusters, and are terminated when their component weights drop below a threshold. The advantage of using mixture particle filtering is that it is capable of tracking multiple targets simultaneously and also of handling appearing and disappearing targets. We demonstrate the effectiveness of our method on different PETS datasets.