Assignment games are a promising framework for autonomous management of a multimodal Tactical Sensing Grid (TSG). They provide theoretical guarantes and exhibit excellent empirical performance in maintaining custody of all observed targets in a scene. However, TSG sub-grid initiation of an assignment game requires multimodal data association between playing nodes (the sub-grid). Playing nodes must achieve consensus on target labels and identities before game play proceeds. Using the locally centralized communication network assumed by an assignment game, we propose the Triple-C distributed method for solving this association problem. The method we propose is suitable for fairly general scenarios, requiring that each node have an intrinsic notion of what constitutes an outlier and what constitutes similarity. At the core of the Triple-C method is a collection of parallelizable assignment games played between two nodes. Triple-C yields theoretical guarantees on output association consistency. We evaluate the performance of the Triple-C method on multiple simulations, showing that it computes associations quickly and accurately, thus enabling a TSG to maintain full situational awareness in a two-vehicle scenario.

Reinforcement learning is a promising candidate methodology for achieving situational awareness across an area of interest by controlling and processing data acquired from a multi-site, multi-modality sensing grid. We previously reported successful detection, tracking, classification, and identification of objects operating within the area of interest monitored by a sensing grid. We combined online kernel least squares policy iteration (an online reinforcement learning method combining dictionary learning with classical Q-learning) with a particle tracker to achieve these results. The work reported here extends these prior results to show that sensor fusion allows our online reinforcement learning methodology to successfully control a multi-modality sensor platform (consisting of a pan/tilt/zoom electro-optical camera, a radar, and passive radio frequency sensor) to maintain persistent surveillance of objects of interest. We evaluate our sensor fusion and online reinforcement learning methodology through Gazebo simulation of a realistic test and experimentation location. Our results demonstrate that policies based on our methodology trained on processed simulated sensor data perform as well as policies trained on known ground truth data. These results also show that the learned policies offer significant generalization ability, with the sensor platform being able to successfully track an observed object well past the observed training period. Moreover, we show that our reinforcement learning methodology shows promising results in dealing with the data association problem. These results further build towards our ultimate goal of achieving automated situational awareness across a heterogeneous sensor grid.

Dimensionality reduction is a crucial first step for many unsupervised learning tasks including anomaly detection and clustering. Autoencoder is a popular mechanism to accomplish dimensionality reduction. In order to make dimensionality reduction effective for high-dimensional data embedding nonlinear low-dimensional manifold, it is understood that some sort of geodesic distance metric should be used to discriminate the data samples. Inspired by the success of geodesic distance approximators such as ISOMAP, we propose to use a minimum spanning tree (MST), a graph-based algorithm, to approximate the local neighborhood structure and generate structure-preserving distances among data points. We use this MST-based distance metric to replace the euclidean distance metric in the embedding function of autoencoders and develop a new graph regularized autoencoder, which outperforms a wide range of alternative methods over 20 benchmark anomaly detection datasets. We further incorporate the MST regularizer into two generative adversarial networks and find that using the MST regularizer improves the performance of anomaly detection substantially for both generative adversarial networks. We also test our MST regularized autoencoder on two datasets in a clustering application and witness its superior performance as well.