Maintaining connectivity in directional mobile ad hoc networks, or Directional Network Maneuvering (DNM) in tactical environments places high cognitive demands on operators —whether human or autonomous— due to the need to fuse multiple sources, including deterministic models, measurements, and natural language. Unpredictable circumstances force operators to make trade-offs in connectivity, resources, and stealth. Given the large number of emergent factors involved, language models (LMs) show promise to serve as reasoning engines for decision support systems (DSS). This work explores the utility and boundaries of LMs as they apply to DNM decision support. We explore challenges, biases and opportunities for LMs in tactical environments. Our results suggest LMs show promise as decision support engines for network maneuvering at the physical layer, especially when delegating demanding tasks.

Evasion attacks in adversarial machine learning (AML) have been well-established for multi-class classification problems. However, for multi-label audio classification, several considerations and unique challenges exist in evasion AML that the attacker needs to consider. First, the attacker must cater the attack from the multi-class scenario assumed by the vast majority of evasion attacks to multi-label AML. Second, the attacker should consider creating imper-ceptible adversarial examples to reduce suspicion of the human element by factoring in properties inherent to the audio data set and model. Lastly, with the proliferation of AML techniques, the attacker should assume that the target model will implement a deep neural network (DNN) defense and should attempt a strategy to overcome those defenses. In this paper, we show how an attacker would cater and design an attack against multi-label audio classification using stochastic DNNs, highlighting some considerations for linking AML techniques to acoustic-based metrics.

Maintaining awareness of United States (US) coastline conditions is a challenging problem in the midst of rising climate volatility. One effective way to monitor the coast and measure the impact of extreme weather on a coastal community is with aerial imagery, which is now often obtained both pre-and post-extreme event by agencies like FEMA, NOAA, and the USGS [1]. However, confidence that these images are properly classified is crucial to draw meaningful and realistic insights. While Convolutional Neural Networks (CNNs) have been used in the past to classify coastal images [2], standard deterministic models lack uncertainty awareness about prediction decisions. Currently, there is not a sufficiently large coastal dataset to train probabilistic models. To overcome this obstacle, this study performs probabilistic transfer learning with a ResNet50 Monte Carlo (MC) Dropout model custom trained on ImageNet dataset to a custom-built dataset of aerial images of US East, Gulf, and West Coasts images obtained from US Geological Survey (USGS) and the California Coastal Records Project, as cited in [2]. To visualize predictive uncertainty of the model about which class a coastal image belongs to, we propose a Gradient-weighted Class Activation Mapping (Grad-CAM) ensembling method that portrays uncertainty about image feature importance for the model’s decision making.Overall, this study provides a proof-of-concept that transfer learning is a promising technique to preserve uncertainty awareness, along with high accuracy, from large-scale datasets such as ImageNet to small-scale, custom coastal image repositories while also presenting a useful way to visualize a model’s uncertainty about a given image.

The paper proposes using alternative loss functions beyond mean squared error (MSE) for geoscience image regression tasks with neural networks. The limitations of MSE for capturing spatial correlation and overall distribution are discussed. Loss functions incorporating earth mover’s distance, Pearson Correlation Coefficient, and selective input-driven weighting are presented. Experiments applying these to a passive microwave brightness temperature emulation task demonstrate improved performance over MSE baseline for preserving spatial patterns and distributions. The input-driven loss function utilizing the solar angle showed the greatest improvement in mean absolute error. Overall, the results demonstrate the importance of selecting appropriate loss functions in geoscience regression models to better capture physical properties.

Reconstructing spatially continuous 2-D fields out of their individually derived building blocks typically introduces artifacts that decrease the overall perceptual quality of the field. Machine learning (ML) applications encounter such a challenge when patching a U-net-like architecture output. Numerous techniques have been developed to mitigate this problem. Yet, few are informed scalable solutions. The present work manages the stitching of Unet-inferred images using Bayesian deep learning (BDL) probabilistic output. The ability to preserve a Bayesian prediction’s variance while effectively reducing the artifacts within a patched scene is presented through an example of predicting a field related to atmospheric radiance. In areas of high variance, adjacent patches of inferred atmospheric radiances may significantly vary in magnitude, leading to large undesirable spatial gradients in the combined (patched) product. Multiple weighted aggregation strategies and weighting schema are surveyed to investigate how to efficiently decrease artificial gradients in large images constructed by stitching several small predictions while maintaining naturally occurring gradients expected to appear in the mosaiced image. Structural similarity (SSIM) Index and visual information fidelity (VIF) are used to evaluate the perceptual quality of the resultant images and confirm the successful employment of Bayesian U-nets with well-calibrated uncertainty, yielding geospatial images with fewer artifacts than naive methods. Log-linear pooling (LLP) proved to be the optimal aggregation strategy tested for fusing patch uncertainties by retaining per-pixel Gaussian distributions and scaling uncertainties in a principled manner to maintain calibration across the spatial map.

Passive microwave (PMW) satellite data provides numerous benefits for forecasting tropical cyclone (TC) track and intensity when available, but due to the low-earth orbits of PMW sensors, several days may pass before a PMW swath fully captures a TC center, which is insufficient for operational needs. This study uses Bayesian deep learning to "fill in the gaps" of PMW data and explore the potential for deriving a high spatio-temporal "full-disk" synthetic PMW brightness temperature (TB) product and its uncertainty from geostationary infrared observations. Current results indicate that synthetic product uncertainty may be particularly beneficial for TC forecasters, and can be used to enhance the visibility of shallow low-level convection, environmental moisture, and nascent TC eyes that are thinly obscured.

This paper presents an empirical study that evaluates the impact of different types of augmentations on the performance of Deep Learning (DL) models for detecting imaging artifacts in Synthetic Aperture Sonar (SAS) imagery. Despite the popularity of using DL in the SAS community, the impact of augmentations that violate the geometry and physics of SAS has not been fully explored. To address this gap, we developed a unique dataset for detecting imaging artifacts in SAS imagery with DL and trained a Bayesian neural network with a ResNet architecture using widely used augmentations in DL for computer vision, as well as common augmentations used in the SAS literature. The study shows that augmentations that violate the geometry and imaging physics of SAS can negatively impact supervised classification, but can sometimes improve performance. Overall, the study provides important insights into the impact of different types of augmentations on the performance of DL models in SAS applications.

The data abundance that characterizes many remote sensing problems can be as difficult to deal with as data sparsity. Bayesian Deep Learning (BDL) is a tool that can help remote sensing researchers with this challenge. A classification problem and a regression problem are presented as examples of how the quantified uncertainty from BDL can be coupled with uncertainty decomposition to make more informed decisions during remote sensing research involving deep learning.

Precipitation is one of the primary drivers of earth’s water and energy cycles. While its role in the climate system is generally well-understood, our ability to accurately capture the spatio-temporal distribution of precipitation is limited. Satellite-borne passive microwave (PMW) radiometers and the associated research on their precipitation retrievals provide a highly accurate understanding of global total precipitation. Still, these precipitation retrievals are prone to large systematic errors at all spatial and temporal scales given their inability to accurately relate changes in precipitation intensity to the changes in radiometric signatures caused by variability in cloud system structure. Central to generating estimates of precipitation is proper classification of precipitation type, that is, convective or stratiform. This work presents a study on using a Bayesian deep learning (BDL) to help mitigate this problem by accurately classifying precipitation type and providing uncertainty in the classification. Specifically, it adopts a Bayesian form of Residual Networks (ResNet) architectures to extract the information from PMW observations vectors and identify these structural differences of cloud systems while providing, per pixel, classification uncertainty estimates. Benchmarked ResNet architectures in a deterministic configuration achieve accuracies above 86% on the classification task for precipitation type, while a Bayesian configuration of the same architectures reaches accuracy above 90%. Most importantly, uncertainty estimates are predicted by the Bayesian configuration to accompany each individual output value. This allows end-users to use them to calibrate the output by filtering the least certain predictions. The performance of the classifier, therefore, can be further improved depending on the application needs.

Collection of increasingly voluminous multispectral data from multiple instruments with high spatial resolution has posed both an opportunity and a challenge for maximizing their utilization, analysis, and impact. Obtaining accurate estimates of precipitation globally with high temporal resolution is crucial for assessing multiscale hydrologic impacts and providing a constraint for development of numerical models of the atmosphere that provide weather and climate predictions. Precipitation type classification plays an important role in constraining both the inverse problem in satellite precipitation retrievals and latent heat transfer within weather prediction simulations. Precipitation type, however, is often reported deterministically, without uncertainty attached to an estimate. Machine learning techniques are capable of extracting content of interest from large datasets and accurately retrieving discrete and continuous properties of physical systems, but with limited insights to the retrieval components—such as errors and the physical relationship between the observed and retrieved properties. To address this shortcoming, we perform precipitation type classification to introduce a novel tool for decomposing errors of satellite-retrieved products. We use Bayesian neural networks to map global precipitation measurement mission microwave imager observations to dual-frequency precipitation radar-derived precipitation type, which perform comparably to deterministic models, but with the added benefit of providing well-calibrated uncertainties. Through uncertainty decomposition, we demonstrate well-calibrated uncertainties as useful for making decisions concerning high uncertainty predictions, model selection, targeted data analysis, and data collection and processing. Additionally, our Bayesian models enable mathematical confirmation of a data distribution change as the cause for an unacceptable decline in model accuracy.

Audio adversarial attacks on deep learning models are of great interest given the commercial success and proliferation of these technologies. These types of attacks have been successfully demonstrated, however, artifacts introduced in the adversarial audio are easily detectable by a human observer. In this work, an expansion of the fast audio adversarial perturbation framework is proposed that can produce an adversarial attack that is imperceptible to a human observer in near-real time using black-box attacks. This is achieved by proposing a perceptually motivated penalty function. We propose a perceptual fast audio adversarial perturbation generator (PFAPG) that employs a loudness constrained loss function, in lieu of a conventional L-2 norm, between the adversarial example and original audio signal. We compare the performance of PFAPG against the conventional constraint based on the MSE on three audio recognition datasets: speaker recognition, speech command, and the Ryerson audiovisual database of emotional speech and song. Our results indicate that, on average, PFAPG equipped with the loudness-constrained loss function yields a 11% higher success rate, while reducing the undesirable distortion artifacts in adversarial audio by 10% dB compared to the prevalent MSE constraints.

In this work, we introduce and evaluate federated fine-tuning (FFT) toward developing decentralized systems for IoT applications using edge computing. We demonstrate a deployed, off-grid, FFT network composed of embedded hardware and assess the system and its performance. The federated averaging algorithm has become the popular approach in decentralized systems with multiple nodes due its low computational costs and simplicity. However, it is commonly implemented using a workstation or a cloud as its server node, typically demonstrated with unrealistic (small) neural network models and may have high communication cost for embedded applications. To address these challenges, we present two main contributions by: (1) Improving the federated averaging algorithm’s weight initialization step and by limiting percentage of weights being averaged to enhance system security and model performance, and (2) Demonstrating the proposed system’s effectiveness via deployment of realistic model using an edge device as the server node (first time for a federated system). For our evaluation, we use a centrally pre-trained MobileNetV2 model on the CelebA dataset. We record the transmitted model parameters across the network with the modified federated averaging algorithm and FFT, and capture metrics related to memory, power consumption, CPU load, and communication on the device. Overall, results demonstrate that FFT can improve federated system performance and model accuracy while providing stronger privacy, protection of intellectual property, and security against adversarial attacks on federated learning.

Generative Adversarial Networks (GANs) have recently shown promising results for audio synthesis, however, synthesized audio has artifacts and can sound unnatural/mechanical. In the case of speech synthesis, the artifacts can contribute to reduced intelligibility of the generated speech. A new architecture is proposed, TangGAN, that generates audio with less artifacts compared to the current state-of-the-art. This is achieved via the introduction of decimation layers in the discriminator and interpolation layers in the generator of a common GAN architecture to formally address aliasing due to improper sampling. Aliasing occurs in convolutional layers with strides larger than one. To quantify the reduction of artifacts in generated audio three new metrics for evaluating GANs for audio/speech synthesis are introduced: total harmonic distortion (THD), signal-to-noise ratio (SNR), and Speech-To-Reverberation Modulation Energy Ratio (SRMR). New metrics are necessary, given that results in this study indicate that the commonly used Inception score does not capture human listener perception of quality of generated speech. Results presented in this study demonstrate an improved Inception score over the benchmarked WaveGAN architecture as well as reduction of artifacts in generated speech as quantified by THD (up to 2dB), SNR (up to 5 dB), and SRMR. These improvements are achieved in half the training iterations of TangGAN compared to the WaveGAN architecture.