A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. A novel Bayesian theory is developed for detecting a change in the distribution of an i.p.i.d. process. It is shown that the Bayesian change point problem can be expressed as an optimal control problem of a Markov decision process (MDP) with periodic transition and cost structures. An optimal control theory is developed for periodic MDPs for discounted and undiscounted total cost criteria. A fixed-point equation is obtained that is satisfied by the optimal cost function. It is shown that a nonstationary but periodic policy is optimal. A value iteration algorithm is obtained to compute the optimal cost function. The results from the MDP theory are then applied to detect a change in the distribution of an i.p.i.d. process. It is shown that while a stopping rule based on a periodic sequence of thresholds is exactly optimal, a single-threshold policy is asymptotically optimal, as the probability of false alarm goes to zero. Numerical results are provided to demonstrate that the asymptotically optimal policy is not strictly optimal.

We develop a Distributed Event-Triggered Stochastic GRAdient Descent (DETSGRAD) algorithm for solving non-convex optimization problems typically encountered in distributed deep learning. We propose a novel communication triggering mechanism that would allow the networked agents to update their model parameters aperiodically and provide sufficient conditions on the algorithm step-sizes that guarantee the asymptotic mean-square convergence. The algorithm is applied to a distributed supervised-learning problem, in which a set of networked agents collaboratively train their individual neural networks to perform image classification, while aperiodically sharing the model parameters with their one-hop neighbors. Results indicate that all agents report similar performance that is also comparable to the performance of a centrally trained neural network, while the event-triggered communication provides significant reduction in inter-agent communication. Results also show that the proposed algorithm allows the individual agents to classify the images even though the training data corresponding to all the classes are not locally available to each agent.

Saliency maps are a popular approach to creating post-hoc explanations of image classifier outputs. These methods produce estimates of the relevance of each pixel to the classification output score, which can be displayed as a saliency map that highlights important pixels. Despite a proliferation of such methods, little effort has been made to quantify how good these saliency maps are at capturing the true relevance of the pixels to the classifier output (i.e. their “fidelity”). We therefore investigate existing metrics for evaluating the fidelity of saliency methods (i.e. saliency metrics). We find that there is little consistency in the literature in how such metrics are calculated, and show that such inconsistencies can have a significant effect on the measured fidelity. Further, we apply measures of reliability developed in the psychometric testing literature to assess the consistency of saliency metrics when applied to individual saliency maps. Our results show that saliency metrics can be statistically unreliable and inconsistent, indicating that comparative rankings between saliency methods generated using such metrics can be untrustworthy.

Building successful collaboration between humans and robots requires efficient, effective, and natural communication. Here we study a RGB-based deep learning approach for controlling robots through gestures (e.g., "follow me"). To address the challenge of collecting high-quality annotated data from human subjects, synthetic data is considered for this domain. We contribute a dataset of gestures that includes real videos with human subjects and synthetic videos from our custom simulator. A solution is presented for gesture recognition based on the state-of-the-art I3D model. Comprehensive testing was conducted to optimize the parameters for this model. Finally, to gather insight on the value of synthetic data, several experiments are described that systematically study the properties of synthetic data (e.g., gesture variations, character variety, generalization to new gestures). We discuss practical implications for the design of effective human-robot collaboration and the usefulness of synthetic data for deep learning.

Mixture-based algorithms are proposed for detecting a change in the distribution of a statistically periodic process in multislot and multistream settings. In the multislot change detection problem, the distribution of the observed process can change in any subset of time slots in each period. In the multistream change detection problem, there are parallel streams of observations, and the change can affect an arbitrary subset of streams. It is shown that the algorithms are asymptotically optimal in a Bayesian setting.

Motivated by decentralized approaches to machine learning, we propose a collaborative Bayesian learning algorithm taking the form of decentralized Langevin dynamics in a non-convex setting. Our analysis show that the initial KL-divergence between the Markov Chain and the target posterior distribution is exponentially decreasing while the error contributions to the overall KL-divergence from the additive noise is decreasing in polynomial time. We further show that the polynomial-term experiences speed-up with number of agents and provide sufficient conditions on the time-varying step-sizes to guarantee convergence to the desired distribution. The performance of the proposed algorithm is evaluated on a wide variety of machine learning tasks. The empirical results show that the performance of individual agents with locally available data is on par with the centralized setting with considerable improvement in the convergence rate.

A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. Algorithms are proposed to detect changes in such i.p.i.d. processes. It is shown that the algorithms can be computed recursively and are asymptotically optimal. This problem has applications in anomaly detection in traffic data, social network data, and neural data, where periodic statistical behavior has been observed.

Bayesian optimality theory is developed for quickest change detection in a class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes. This class of processes can be used to model periodically varying statistical behavior. An algorithm called the periodic-Shiryaev algorithm is proposed and is shown to asymptotically minimize the average detection delay subject to a constraint on the probability of false alarm. It is also shown that the statistic for this algorithm can be computed recursively and using a finite amount of memory. This problem has applications in anomaly detection problems in cyber-physical systems and biology, where periodic statistical behavior has been observed.

The problem of sequential detection of anomalies in multimodal data is considered. The objective is to observe physical sensor data from CCTV cameras, and social media data from Twitter and Instagram to detect anomalous behaviors or events. Data from each modality is transformed to discrete time count data by using an artificial neural network to obtain counts of objects in CCTV images and by counting the number of tweets or Instagram posts in a geographical area. The anomaly detection problem is then formulated as a problem of quickest detection of changes in count statistics. The quickest detection problem is then solved using the framework of partially observable Markov decision processes (POMDP), and structural results on the optimal policy are obtained. The resulting optimal policy is then applied to real multimodal data collected from New York City around a 5K race to detect the race. The count data both before and after the change is found to be nonstationary in nature. The proposed mathematical approach to this problem provides a framework for event detection in such nonstationary environments and across multiple data modalities.

Recent advances in Machine Learning (ML) have profoundly changed many detection, classification, recognition and inference tasks. Given the complexity of the battlespace, ML has the potential to revolutionise how Coalition Situation Understanding is synthesised and revised. However, many issues must be overcome before its widespread adoption. In this paper we consider two - interpretability and adversarial attacks. Interpretability is needed because military decision-makers must be able to justify their decisions. Adversarial attacks arise because many ML algorithms are very sensitive to certain kinds of input perturbations. In this paper, we argue that these two issues are conceptually linked, and insights in one can provide insights in the other. We illustrate these ideas with relevant examples from the literature and our own experiments.

In this paper, an algorithm to randomly select feature sub-spaces for hyperspectral image classification using the principle of coalition game theory is presented. The feature selection algorithms associated with non-linear kernel based Support Vector Machines (SVM) are either NP-hard or greedy and hence, not very optimal. To deal with this problem, a metric based on the principles of coalition game theory called Shapely value and a sampling approximation is used to determine the contributions of individual features towards the classification task. Feature subsets are randomly drawn from a probability distribution function generated using normalized Shapely values of the individual features. These feature subsets are then used to build kernels corresponding to individual weak classifiers in the Sparse Kernel-based Ensemble Learning (SKEL) framework. By weighting the kernels optimally and sparsely, a small number of useful subsets of features are selected which improve the generalization performance of the ensemble classifier. The algorithm is applied on real hyper-spectral datasets and the results are presented in the paper.

In this paper, an algorithm to select feature subsets for hyper-spectral image classification using the principle of coalition game theory is presented. The feature selection algorithms associated with non-linear kernel based Support Vector Machines (SVM) are either NP-hard or greedy and hence, not very optimal. To deal with this problem, a metric based on the principles of coalition game theory called Shapely value and a sampling approximation is used to determine the contribution of a subset of features towards the classification task. Starting with a few subsets of features, we successively partition each of them into smaller parts if the smaller parts contribute more than a pre-determined threshold compared to the parent subset. The algorithm is terminated when the subsets of features do not change from one iteration to the next. The final subsets of features are then used in multiple kernels and sparse weights of these kernels are optimally learned to build a maximum margin classifier. The algorithm is applied on real hyperspectral datasets and the results are presented in the paper.

In this letter, a kernel-based contextual classification approach built on the principle of a newly introduced mapping technique, called Hilbert space embedding, is proposed. The proposed technique, called contextual support vector machine (SVM), is aimed at jointly exploiting both local spectral and spatial information in a reproducing kernel Hilbert space (RKHS) by collectively embedding a set of spectral signatures within a confined local region into a single point in the RKHS that can uniquely represent the corresponding local hyperspectral pixels. Embedding is conducted by calculating the weighted empirical mean of the mapped points in the RKHS to exploit the similarities and variations in the local spectral and spatial information. The weights are adaptively estimated based on the distance between the mapped point in consideration and its neighbors in the RKHS. An SVM separating hyperplane is built to maximize the margin between classes formed by weighted empirical means. The proposed technique showed significant improvement over the composite kernel-based SVM on several hyperspectral images.

Recently, a kernel-based ensemble learning technique for hyperspectral detection/classification problems has been introduced by the authors, to provide robust classification over hyperspectral data with relatively high level of noise and background clutter. The kernel-based ensemble technique first randomly selects spectral feature subspaces from the input data. Each individual classifier, which is in fact a support vector machine (SVM), then independently conducts its own learning within its corresponding spectral feature subspace and hence constitutes a weak classifier. The decisions from these weak classifiers are equally or adaptively combined to generate the final ensemble decision. However, in such ensemble learning, little attempt has been previously made to jointly optimize the weak classifiers and the aggregating process for combining the subdecisions. The main goal of this paper is to achieve an optimal sparse combination of the subdecisions by jointly optimizing the separating hyperplane obtained by optimally combining the kernel matrices of the SVM classifiers and the corresponding weights of the subdecisions required for the aggregation process. Sparsity is induced by applying an$l1$norm constraint on the weighting coefficients. Consequently, the weights of most of the subclassifiers become zero after the optimization, and only a few of the subclassifiers with non-zero weights contribute to the final ensemble decision. Moreover, in this paper, an algorithm to determine the optimal full-diagonal bandwidth parameters of the Gaussian kernels of the individual SVMs is also presented by minimizing the radius-margin bound. The optimized full-diagonal bandwidth Gaussian kernels are used by the sparse SVM ensemble to perform binary classification. The performance of the proposed technique with optimized kernel parameters is compared to that of the one with single-bandwidth parameter obtained using cross-validation by testing them on various data sets. On an average, the proposed sparse kernel-based ensemble learning algorithm with optimized full-diagonal bandwidth parameters shows an improvement of 20$\%$over the existing ensemble learning techniques.

In this paper, a contextual Support Vector Machine (SVM) technique based on the principle of Hilbert Space Embedding (HSE) of a local hyperspectral data distribution into an Reproducing Kernel Hilbert Space (RKHS) is proposed to optimally exploit the spectral and local spatial information of the hyperspectral image. The idea of embedding is to map hyperspectral pixels in a local neighborhood into a single point in the RKHS that can uniquely represent those pixels collectively. Previously, the authors have employed an HSE called empirical mean map to build the contextual SVM. In this work, a weighted empirical mean map is utilized to exploit the similarities and variation in the local spatial information. For every pixel, a small set of the neighboring pixels in a hyperspectral image are mapped into an RKHS induced by a certain kernel (Eg. Gaussian RBF kernel) and then, the embedded point of these group of pixels is obtained by calculating the weighted empirical mean of these mapped points. The weights are determined based on the distance between the pixel in consideration and its neighbors. An SVM separating hyperplane is built to maximize the margin between classes formed by weighted empirical means. The proposed technique showed significant improvement over the existing contextual and composite kernels on two hyperspectral image data sets.

In this letter, a novel ensemble-learning approach for anomaly detection is presented. The proposed technique aims to optimize an ensemble of kernel-based one-class classifiers, such as support vector data description (SVDD) classifiers, by estimating optimal sparse weights of the subclassifiers. In this method, the features of a given multivariate data set representing normalcy are first randomly subsampled into a large number of feature subspaces. An enclosing hypersphere that defines the support of the normalcy data in the reproducing kernel Hilbert space (RKHS) of each respective feature subspace is estimated using standard SVDD. The joint hypersphere in the RKHS of the combined kernel is learned by optimally combining the weighted individual kernels while imposing the$l1$constraint on the combining weights. The joint hypersphere representing the optimal compact support of the multivariate data in the joint RKHS is then used to test a new data point to determine if it belongs to the normalcy data or not. A performance comparison between the proposed algorithm and regular SVDD is reported using hyperspectral image data as well as general multivariate data.

In this paper, a Generalized Kernel-based Ensemble Learning (GKEL) algorithm for hyperspectral classification problems is presented. The proposed algorithm generalizes the Sparse Kernel-based Ensemble Learning (SKEL) technique, developed previously by the authors. SKEL optimally and sparsely weights and aggregates an ensemble of individual SVM classifiers which independently conduct learning within their corresponding randomly selected spectral feature sub-space using a Gaussian kernel. This ensemble decision is fully optimal, if the dimensionality of the randomly selected feature subspaces and the initial number of the sub-classifiers are determined optimally and is sub-optimal, otherwise. This sub optimality issue is addressed by taking a bottom-up approach. Individual sub-classifiers are added one-by-one optimally to the ensemble until the ensemble converges. The feature sub space of each individual classifier is optimally selected. The ensemble is modeled as a Quadratically-Constrained Linear Programming (QCLP) problem and optimized by combining Multiple Kernel Learning (MKL) with a greedy, non-linear integer programming method for non-monotonic sparse feature sub-space selection. Hyperspectral image data as well as multivariate data are used to verify the performance improvement of the proposed GKEL algorithm over SKEL in detecting difficult targets.

In this letter, a method to optimally determine the kernel bandwidth of the Gaussian radial basis function (RBF) kernel for support vector (SV)-based hyperspectral anomaly detection is presented. In this method, the support of a local background distribution is first nonparametrically learned by a technique called SV data description (SVDD). The SVDD optimally models an enclosing hypersphere around the local background data in a high-dimensional feature space associated with the Gaussian RBF kernel. Any test pixel that lies outside this hypersphere surrounding the local background is considered an anomaly and, hence, a possible target pixel. Considerable improvement in detection performance due to kernel parameter optimization can be seen in the simulation results when the algorithm is applied to hyperspectral images.

Recently, a sparse kernel-based SVM ensemble learning technique has been introduced by the authors for hyperspectral plume detection/classification. This technique first randomly selects spectral feature subspaces from the input data. Each individual SVM classifier then independently conducts its own learning within its corresponding spectral feature space using a Gaussian kernel with a single bandwidth parameter. Each classifier constitutes a weak classifier. The sub-classifiers are sparsely weighted and aggregated to make an ensemble decision. In this paper, in order to further improve the generalization performance of the ensemble classifier, Gaussian kernel with full diagonal bandwidth parameter matrix is used for each sub-classifier where the parameters are optimally learned by minimizing a bound of the generalization error estimate using a gradient descent algorithm. A performance comparison between the aggregating techniques - sparse kernel-based technique and majority voting with single bandwidth and full diagonal optimized bandwidth parameters as applied to hyperspectral chemical plume detection is presented in the paper.

A kernel-based anomaly detection technique called Kernel RX algorithm has been developed earlier by one of the authors, to be used as a prescreening tool that non-linearly detects anomalous pixel spectra in hyperspectral images. Targets of interest are then identified among the prescreened anomalous spectra based on reference spectral information using supervised classification/detection techniques. Kernel RX algorithm uses kernels like the Gaussian radial basis function (RBF) kernel to transform the given data into higher-dimensional (possibly infinite) feature space before detecting the anomalies. The efficiency of the algorithm depends on this transformation which in turn depends on the respective kernel parameters. The Gaussian RBF kernel has a parameter called bandwidth parameter. In this paper, a new method to determine the optimal full diagonal bandwidth parameters of the Gaussian RBF kernel is presented. First, cross-validation technique is used to estimate an optimal single bandwidth parameter. Then, the full diagonal parameters are estimated from this single parameter using the variances of the spectral bands of the hyperspectral image. It will be shown that the optimal full diagonal bandwidth parameters provide a better probability of detection at a given false alarm rate compared to the optimal single bandwidth parameter and other suboptimal bandwidth parameters when tested on hyperspectral imagery for military target detection.

In this paper, we propose a segment-based triangulation method for building parallel-perspective stereo mosaics using ray interpolation. The currently available fast parallel ray interpolation for stereo mosaicing uses a uniform or regular triangulation technique that does not take advantage of the spatial information. This works well when the interframe displacement in the video is small. However, for large interframe displacements, hence large motion parallax, we observe visual artifacts in the mosaics when the source triangles in the video frames are warped into destination triangles in the mosaics using affine transformations. Our method uses the edge information present in the scene to ensure that triangles do not cross over from one planar facet to the other. The video frames are segmented, and the edges are obtained from the inner boundaries of different segments. A mesh is designed in each segment or region, over which motion between consecutive video frames is constant. Hence, this method avoids unwanted warping of objects with large motion parallax in the scene and reduces the visual artifacts. The performance of our proposed technique is demonstrated on a series of simple-to-complex video imagery collected by utilizing cameras and airborne sensors.