Assignment games are a promising framework for autonomous management of a multimodal Tactical Sensing Grid (TSG). They provide theoretical guarantes and exhibit excellent empirical performance in maintaining custody of all observed targets in a scene. However, TSG sub-grid initiation of an assignment game requires multimodal data association between playing nodes (the sub-grid). Playing nodes must achieve consensus on target labels and identities before game play proceeds. Using the locally centralized communication network assumed by an assignment game, we propose the Triple-C distributed method for solving this association problem. The method we propose is suitable for fairly general scenarios, requiring that each node have an intrinsic notion of what constitutes an outlier and what constitutes similarity. At the core of the Triple-C method is a collection of parallelizable assignment games played between two nodes. Triple-C yields theoretical guarantees on output association consistency. We evaluate the performance of the Triple-C method on multiple simulations, showing that it computes associations quickly and accurately, thus enabling a TSG to maintain full situational awareness in a two-vehicle scenario.

Reinforcement learning is a promising candidate methodology for achieving situational awareness across an area of interest by controlling and processing data acquired from a multi-site, multi-modality sensing grid. We previously reported successful detection, tracking, classification, and identification of objects operating within the area of interest monitored by a sensing grid. We combined online kernel least squares policy iteration (an online reinforcement learning method combining dictionary learning with classical Q-learning) with a particle tracker to achieve these results. The work reported here extends these prior results to show that sensor fusion allows our online reinforcement learning methodology to successfully control a multi-modality sensor platform (consisting of a pan/tilt/zoom electro-optical camera, a radar, and passive radio frequency sensor) to maintain persistent surveillance of objects of interest. We evaluate our sensor fusion and online reinforcement learning methodology through Gazebo simulation of a realistic test and experimentation location. Our results demonstrate that policies based on our methodology trained on processed simulated sensor data perform as well as policies trained on known ground truth data. These results also show that the learned policies offer significant generalization ability, with the sensor platform being able to successfully track an observed object well past the observed training period. Moreover, we show that our reinforcement learning methodology shows promising results in dealing with the data association problem. These results further build towards our ultimate goal of achieving automated situational awareness across a heterogeneous sensor grid.