Abstract Achieving precise 6 degrees of freedom (6D) pose estimation of rigid objects from color images is a critical challenge with wide-ranging applications in robotics and close-contact aircraft operations. This study investigates key techniques in the application of YOLOv5 object detection convolutional neural network (CNN) for 6D pose localization of aircraft using only color imagery. Traditional object detection labeling methods suffer from inaccuracies due to perspective geometry and being limited to visible key points. This research demonstrates that with precise labeling, a CNN can predict object features with near-pixel accuracy, effectively learning the distinct appearance of the object due to perspective distortion with a pinhole camera. Additionally, we highlight the crucial role of knowledge about occluded features. Training the CNN with such knowledge slightly reduces pixel precision, but enables the prediction of 3 times more features, including those that are not initially visible, resulting in an overall better performing 6D system. Notably, we reveal that the data augmentation technique ofscalecan interfere with pixel precision when used during training. These findings are crucial for the entire system, which leverages the Solve Perspective-N-Point (Solve-PnP) algorithm, achieving 6D pose accuracy within 1 $$^\circ$$ ∘ and 7 cm at distances ranging from 7.5 to 35 m from the camera. Moreover, this solution operates in real-time, achieving sub-10ms processing times on a desktop PC.

Radio Frequency Fingerprinting (RFF) is the attribution of uniquely identifiable signal distortions to emitters via Machine Learning (ML) classifiers. RFF approaches relying on pre-determined expert features lack generalizability, and state-of-the-art approaches based on Convolutional Neural Networks (CNNs) can be too demanding for endpoint devices to train. This work presents Fingerprint Extraction through Distortion Reconstruction (FEDR), a best-of-both-worlds technique which employs a pre-trained CNN to identify and extract a small, salient set of unique features, amenable for use in lightweight machine learning models. Given a received distorted signal, the FEDR network encodes signal distortions into “fingerprints,” which can be used by lightweight ML classifiers to perform RFF with minimal resource consumption at the endpoint. FEDR learns by transforming generated signals into reconstructions of received signals, relying solely on the fingerprints as representations of the distortions – as the reconstructions improve, the fingerprints better encode the distortions. The FEDR technique was evaluated on synthetic IQ-imbalanced IEEE 802.11a/g data, where FEDR fingerprints were shown to encode actual IQ imbalance parameters, signifying successful isolation of distortion information and validating the FEDR technique. FEDR was further evaluated on a representative real-world WiFi dataset, where extracted fingerprints were coupled with a lightweight two-layer dense network. When compared against two common RFF techniques, the FEDR-based approach achieved state-of-the-art performance with Matthews Correlation Coefficient ranging from 0.984 (5 classes) to 0.851 (100 classes), using nearly 73% fewer training parameters than the next-best technique.

Using deeply recurrent neural networks to account for temporal dependence in electroencephalograph (EEG)-based workload estimation is shown to considerably improve day-to-day feature stationarity resulting in significantly higher accuracy (p < .0001) than classifiers which do not consider the temporal dependence encoded within the EEG time-series signal. This improvement is demonstrated by training several deep Recurrent Neural Network (RNN) models including Long Short-Term Memory (LSTM) architectures, a feedforward Artificial Neural Network (ANN), and Support Vector Machine (SVM) models on data from six participants who each perform several Multi-Attribute Task Battery (MATB) sessions on five separate days spread out over a month-long period. Each participant-specific classifier is trained on the first four days of data and tested using the fifth’s. Average classification accuracy of 93.0% is achieved using a deep LSTM architecture. These results represent a 59% decrease in error compared to the best previously published results for this dataset. This study additionally evaluates the significance of new features: all combinations of mean, variance, skewness, and kurtosis of EEG frequency-domain power distributions. Mean and variance are statistically significant features, while skewness and kurtosis are not. The overall performance of this approach is high enough to warrant evaluation for inclusion in operational systems.

Classification of operator functional state for workload estimation using electroencephalograph (EEG) has proven difficult in cross-day scenarios due to non-stationarity of the feature and target distributions. This study analyzes multi-day data collected from a Multi-Attribute Task Battery (MATB) workload study using a new feature generation methodology which examines not just the average power, but also the variability of the power distribution in the clinical frequency bands over a 10 second sliding temporal window. High versus low workload levels were predicted for day five of the study based on training three traditional classifiers-Linear Discriminant Analysis (LDA), random forest, and K-Nearest Neighbors (KNN)-on the first four days' results. Frequency-domain power distribution variance was statistically significant between conditions, suggesting it as a salient feature. Including variance as a feature enabled a crossday workload classification accuracy improvement of 5.8% above models only using mean power. Furthermore, the individual classifiers were combined into a time-smoothed composite classifier which capitalized on the differences in features selected in the models to improve overall classification accuracy to greater than 80%.

Real world operational environments often require the integration of complex multiple-objective tasks that necessitate split attention and individual prioritization in human operators. This study examines the effect of secondary task presence on operator electroencephalogram (EEG) activity in two different multiple-objective remotely piloted aircraft (RPA) simulations. Eight participants completed simulated aerial reconnaissance tasks of varying difficulties, while continuously monitoring and responding to radio traffic requesting distance, speed, and elevation calculations that required expedient mathematical reasoning. In these realistic dynamic task scenarios, balanced random forest and binary logistic regression classifiers are used to measure the effectiveness of 35 physiological markers in detecting operator workload changes. Results suggest that within-subject random forest models perform reasonably well even when trained using alternative primary tasks. Additionally, novel evidence supporting the importance of delta band (1-3Hz) brain activity for task detection is reported.

As surveillance becomes ubiquitous, the amount of data to be processed grows along with the demand for manpower to interpret the data. A key goal of surveillance is to detect behaviors that can be considered anomalous. As a result, an extensive body of research in automated surveillance has been developed, often with the goal of automatic detection of anomalies. Research into anomaly detection in automated surveillance covers a wide range of domains, employing a vast array of techniques. This review presents an overview of recent research approaches on the topic of anomaly detection in automated surveillance. The reviewed studies are analyzed across five aspects: surveillance target, anomaly definitions and assumptions, types of sensors used and the feature extraction processes, learning methods, and modeling algorithms.

We develop an upper bound for the potential performance improvement of an agent using a best response to a model of an opponent instead of an uninformed game-theoretic equilibrium strategy. We show that the bound is a function of only the domain structure of an adversarial environment and does not depend on the actual actors in the environment. This bounds-finding technique will enable system designers to determine if and what type of opponent models would be profitable in a given adversarial environment. It also gives them a baseline value with which to compare performance of instantiated opponent models. We study this method in two domains: selecting intelligence collection priorities for convoy defense and determining the value of predicting enemy decisions in a simplified war game.

Coalition formation algorithms are generally not applicable to real-world robotic collectives since they lack mechanisms to handle uncertainty. Those mechanisms that do address uncertainty either deflect it by soliciting information from others or apply reinforcement learning to select an agent type from within a set. This paper presents a coalition formation mechanism that directly addresses uncertainty while allowing the agent types to fall outside of a known set. The agent types are captured through a novel agent modeling technique that handles uncertainty through a belief-based evaluation mechanism. This technique allows for uncertainty in environmental data, agent type, coalition value, and agent cost. An investigation of both the effects of adding agents on processing time and of model quality on the convergence rate of initial agent models (and thereby coalition quality) is provided. This approach handles uncertainty on a larger scale than previous work and provides a mechanism readily applied to a dynamic collective of real-world robots.