Reinforcement learning (RL) using world models has found significant recent successes. However, when a sudden change to world mechanics or properties occurs then agent performance and reliability can dramatically decline. We refer to the sudden change in visual properties or state transitions as novelties. Implementing novelty detection within generated world model frameworks is a crucial task for protecting the agent when deployed. In this paper, we propose straightforward bounding approaches to incorporate novelty detection into world model RL agents by utilizing the misalignment of the world model’s hallucinated states and the true observed states as a novelty score. We provide effective approaches to detecting novelties in a distribution of transitions learned by an agent in a world model. Finally, we show the advantage of our work in a novel environment compared to traditional machine learning novelty detection methods as well as currently accepted RL-focused novelty detection algorithms.

Numerous real-world control problems involve dynamics and objectives affected by unobservable hidden parameters, ranging from autonomous driving to robotic manipulation, which cause performance degradation during sim-to-real transfer. To represent these kinds of domains, we adopt hiddenparameter Markov decision processes (HIP-MDPs), which model sequential decision problems where hidden variables parameterize transition and reward functions. Existing approaches, such as domain randomization, domain adaptation, and meta-learning, simply treat the effect of hidden parameters as additional variance and often struggle to effectively handle HIP-MDP problems, especially when the rewards are parameterized by hidden variables. We introduce PrivilegedDreamer, a model-based reinforcement learning framework that extends the existing model-based approach by incorporating an explicit parameter estimation module. PrivilegedDreamer features its novel dual recurrent architecture that explicitly estimates hidden parameters from limited historical data and enables us to condition the model, actor, and critic networks on these estimated parameters. Our empirical analysis on five diverse HIP-MDP tasks demonstrates that PrivilegedDreamer outperforms state-of-the-art model-based, model-free, and domain adaptation learning algorithms. Additionally, we conduct ablation studies to justify the inclusion of each component in the proposed architecture.

Reinforcement Learning (RL) has made significant strides in enabling artificial agents to learn diverse behaviors. However, learning an effective policy often requires a large number of environment interactions. To mitigate sample complexity issues, recent approaches have used high-level task specifications, such as Linear Temporal Logic (LTLf) formulas or Reward Machines (RM), to guide the learning progress of the agent. In this work, we propose a novel approach, called Logical Specifications-guided Dynamic Task Sampling (LSTS), that learns a set of RL policies to guide an agent from an initial state to a goal state based on a high-level task specification, while minimizing the number of environmental interactions. Unlike previous work, LSTS does not assume information about the environment dynamics or the Reward Machine, and dynamically samples promising tasks that lead to successful goal policies. We evaluate LSTS on a gridworld and show that it achieves improved time-to-threshold performance on complex sequential decision-making problems compared to state-of-the-art RM and Automaton-guided RL baselines, such as Q-Learning for Reward Machines and Compositional RL from logical Specifications (DIRL). Moreover, we demonstrate that our method outperforms RM and Automaton-guided RL baselines in terms of sample-efficiency, both in a partially observable robotic task and in a continuous control robotic manipulation task.

The Active Cooling for Multispectral Earth Sensors (ACMES) is an advanced CubeSat mission selected for flight under the In-space Validation of Earth Science Technologies (InVEST) program in support of the NASA Science Mission Directorate and the Earth Science Technology Office (ESTO) [1]. The ACMES program is a joint development effort led by the Center for Space Engineering at Utah State University (CSE USU) in collaboration with Orion Space Solutions (OSS), and the Hawaii Space Flight Laboratory (HSFL) at the University of Hawaii Manoa. ACMES is currently scheduled to launch in 2025 to a ~550 km sun-synchronous Orbit (SSO) with a local node (ascending time) between 11:30 and 12:30. The ACMES satellite, a +16U CubeSat bus from the OSS Triton line, serves as a technology host platform for four unique remote and in situ earth-observing payloads. Including the second-generation Hyperspectral Thermal Imager (HyTI 2.0), The Filter Incidence Narrow-band Infrared Spectrometer (FINIS), the Planar Langmuir Impedance Diagnostic (PLAID) probe, and a novel active thermal control system. The Active Thermal Architecture technology (ATA). ACMES will consist of a 1-year primary mission to demonstrate and validate each of the hosted payloads in situ, raising the operational TRL to ~7, followed by up to a 3-year extended mission to gather valuable scientific data for the earth science community. The ACMES satellite and its payloads represent an important progression in satellite remote sensing and CubeSat technology/capabilities. Ultimately, the ACMES satellite represents a paradigm shift towards low-cost, fast-to-space, small satellite constellations that are scientifically valuable space platforms. Producing IR observations on par, if not exceeding traditional satellite missions. It is the author's hope, that in the future, small satellites will provide continuous observations of the near-earth environment and that ACMES will demonstrate that capability.

Abstract The purpose of this study was to examine how the framework of administrative tethering (AT) provides important guidance and insight at the organizational level to support a strategic management mechanism to ameliorate pressing political, social, and economic issues for the public good. This investigation was an empirical analysis of data collected from the 2020 Federal Employee Viewpoint (FEV) Survey and records attributed to the Department of the Navy (N = 32,416). The Department of the Navy consists of naval forces and marine forces for maritime operations, which conduct internal collaboration and represent cases of both external institution to institution (i2i) collaboration and internal unit to unit (u2u) collaboration. Structural equation modeling (SEM) is a useful analytical approach to examine the relationships among all the measured and latent variables. The resulting SEM analysis evoked the importance of individual behaviors on collaborative efforts.

The Active Cooling for Multispectral Earth Sensors (ACMES) is a 16U CubeSat mission funded under the NASA Earth Science Technology Office. ACMES will simultaneously advance two new technologies. The first technology is the Active Thermal Architecture, a complete end-to-end solution for active thermal control of cryogenic instruments on nanosatellites. The second technology is the hyperspectral imaging in the IR using both spatially modulated interferometric and spatially modulated spectral imaging techniques. The ACMES mission is being led by Utah State University with the University of Hawaii and implemented by Orion Space Solutions, with a delivery for launch in late 2024.

Despite recent progress in Reinforcement Learning for robotics applications, many tasks remain prohibitively difficult to solve because of the expensive interaction cost. Transfer learning helps reduce the training time in the target domain by transferring knowledge learned in a source domain. Sim2Real transfer helps transfer knowledge from a simulated robotic domain to a physical target domain. Knowledge transfer reduces the time required to train a task in the physical world, where the cost of interactions is high. However, most existing approaches assume exact correspondence in the task structure and the physical properties of the two domains. This work proposes a framework for Few-Shot Policy Transfer between two domains through Observation Mapping and Behavior Cloning. We use Generative Adversarial Networks (GANs) along with a cycle-consistency loss to map the observations between the source and target domains and later use this learned mapping to clone the successful source task behavior policy to the target domain. We observe successful behavior policy transfer with limited target task interactions and in cases where the source and target task are semantically dissimilar.

Automatic segmentation of the placenta in fetal ultrasound (US) is challenging due to the (i) high diversity of placenta appearance, (ii) the restricted quality in US resulting in highly variable reference annotations, and (iii) the limited field-of-view of US prohibiting whole placenta assessment at late gestation. In this work, we address these three challenges with a multi-task learning approach that combines the classification of placental location (e.g., anterior, posterior) and semantic placenta segmentation in a single convolutional neural network. Through the classification task the model can learn from larger and more diverse datasets while improving the accuracy of the segmentation task in particular in limited training set conditions. With this approach we investigate the variability in annotations from multiple raters and show that our automatic segmentations (Dice of 0.86 for anterior and 0.83 for posterior placentas) achieve human-level performance as compared to intra- and inter-observer variability. Lastly, our approach can deliver whole placenta segmentation using a multi-view US acquisition pipeline consisting of three stages: multi-probe image acquisition, image fusion and image segmentation. This results in high quality segmentation of larger structures such as the placenta in US with reduced image artifacts which are beyond the field-of-view of single probes.

Keystroke dynamics has been shown to be a promising method for user authentication based on a user's typing rhythms. Over the years, it has seen increasing applications such as in preventing transaction fraud, account takeovers, and identity theft. However, due to the variable nature of keystroke dynamics, a user's typing patterns may vary on a different keyboard or in a different keyboard language setting, which may affect the system accuracy. In other words, an algorithm modeled with data collected using a mechanical keyboard may perform significantly differently when tested with an ergonomic keyboard. Similarly, an algorithm modeled with data collected in one language may perform significantly differently when tested with another language. Hence, there is a need to study the impact of multiple keyboards and multiple languages on keystroke dynamics performance. This motivated us to develop two free-text keystroke dynamics datasets. The first is a multi-keyboard keystroke dataset comprising of four (4) physical keyboards - mechanical, ergonomic, membrane, and laptop keyboards - and the second is a bilingual keystroke dataset in both English and Chinese languages. Data were collected from a total of 86 participants using a non-intrusive web-based keylogger in a semi-controlled setting. To the best of our knowledge, these are the first multi-keyboard and bilingual keystroke datasets, as well as the data collection software, to be made publicly available for research purposes. The usefulness of our datasets was demonstrated by evaluating the performance of two state-of-the-art free-text algorithms.

Hyperspectral Thermal Imager (HyTI) is designed with the focus on understanding, modeling, mapping, and monitoring the world's agricultural lands, and water resources, as well as for studies pertaining to forest fires, volcanoes, and a number of other applications (e.g., stubble burning in agricultural lands). Climate variability and expanding populations are putting unprecedented pressure on agricultural croplands and their water use, which are vital for ensuring global food and water security in the twenty-first century. Currently, on average worldwide, croplands use 80-90% of all human water consumption. HyTI data and derived information products will be invaluable in advancing our knowledge on multiple fronts in crop type mapping, crop productivity modeling, crop water use assessments, and crop water productivity (“crop per drop”) mapping.

The HyTI (Hyperspectral Thermal Imager) mission, funded by NASA's Earth Science Technology Office InVEST (In-Space Validation of Earth Science Technologies) program, will demonstrate how high spectral and spatial long-wave infrared image data can be acquired from a 6U CubeSat platform. The mission will use a spatially modulated interferometric imaging technique to produce spectro-radiometrically calibrated image cubes, with 25 channels between$8-10.7\ \mu\mathrm{m}$, at 13 cm−1resolution), at a ground sample distance of ∼60 m. The HyTI performance model indicates narrow band$\text{NE}\Delta\text{Ts}$of <0.3 K. The small form factor of HyTI is made possible via the use of a no-moving-parts Fabry-Perot interferometer, and JPL's cryogenically-cooled HOT-BIRD FPA technology. Launch is scheduled for no earlier than Fall 2021. The value of HyTI to Earth scientists will be demonstrated via on-board processing of the raw instrument data to generate L1 and L2 products, with a focus on rapid delivery of data regarding volcanic degassing, and land surface temperature.

Traditionally, monitoring Earth's active volcanoes with a cadence that is useful for informing decision makers has been the domain of low spatial, but high temporal, resolution systems. The recent proliferation and maturation of CubeSat and small satellite technologies, combined with increased opportunity (and affordability) of getting to space, will mean that volcanologists will increasingly have access to data acquired specifically for the purposes of studying active volcanism from orbit. In this presentation, the historical

Deep learning based automatic feature extraction methods have radically transformed speaker identification and facial recognition. Current approaches are typically specialized for individual domains, such as Deep Vectors (D-Vectors) for speaker identification. We provide two distinct contributions: a generalized framework for biometric verification inspired by D-Vectors and novel models that outperform current stateof-the-art approaches. Our approach supports substitution of various feature extraction models and improves the robustness of verification tests across domains. We demonstrate the framework and models for two different behavioral biometric verification problems: keystroke and mobile gait. We present a comprehensive empirical analysis comparing our framework to the state-of-the-art in both domains. Our models perform verification with higher accuracy using orders of magnitude less data than state-of-the-art approaches in both domains. We believe that the combination of high accuracy and practical data requirements will enable application of behavioral biometric models outside of the laboratory in support of much-needed improvements to cyber security.

The HyTI (Hyperspectral Thermal Imager) mission, funded by NASA's Earth Science Technology Office InVEST (In-Space Validation of Earth Science Technologies) program, will demonstrate how high spectral and spatial long-wave infrared image data can be acquired from a 6U CubeSat platform. The mission will use a spatially modulated interferometric imaging technique to produce spectro-radiometrically calibrated image cubes, with 25 channels between 8-10.7 μm, at a ground sample distance of ~70 m. The HyTI performance model indicates narrow band NEΔTs of <; 0.3 K. The small form factor of HyTI is made possible via the use of a no-moving-parts Fabry-Perot interferometer, and JPL's cryogenically-cooled BIRD FPA technology. Launch is scheduled for no earlier than October 2020. The value of HyTI to Earth scientists will be demonstrated via on-board processing of the raw instrument data to generate L1 and L2 products, with a focus on rapid delivery of precision agriculture metrics.

We study an important yet under-addressed problem of quickly and safely improving policies in online reinforcement learning domains. As its solution, we propose a novel exploration strategy - diverse exploration (DE), which learns and deploys a diverse set of safe policies to explore the environment. We provide DE theory explaining why diversity in behavior policies enables effective exploration without sacrificing exploitation. Our empirical study shows that an online policy improvement algorithm framework implementing the DE strategy can achieve both fast policy improvement and safe online performance.

This technical note describes the use of an airborne hyperspectral imaging sensor (HICO - Hyperspectral Imager of the Coastal Ocean) to record the spectral reflectance characteristics of centerline ship wakes in the 400 to 900 nm wavelength region. Data were collected for a target of known provenance (the United States Coast Guard Cutter Kittiwake) off the Wai'anae coast of O'ahu, Hawai'i, on 8 April 2010. HICO acquired data in 60 spectral bands by flying along the long axis of the wake while the vessel travelled at three speeds (~3.6 m s-1or 7 knots; ~7.2 m s-1or 14 knots; and ~11.2 m s-1or 21 knots). A flying altitude of ~1500 m yielded a spatial resolution of ~1.5 m. Spectral profiles along and across the wake axes are presented which show how the spectral reflectance of the centerline wake varies spatially and temporally as a function of vessel speed. Length (and to a lesser extent, width) vary in proportion to speed. In common with previous studies and model predictions, the wakes show a pronounced greening of the wake (i.e. enhanced reflectance at ~550 nm), with evidence for elevated reflectance at 750-800 nm. Resampling the data from its raw 1.5 m spatial resolution yields insights into how the turbulent wake becomes spectrally inseparable from the background water as spatial resolution decreases (i.e. becomes increasingly coarse). Using a simple statistical test, the wake becomes spectrally similar to the background ocean as the resolution approaches 60 m.

This study aims at developing an evaluation framework of strategic information systems (SIS) and evaluating the SIS planning and implementation by using a cognitive approach called the repertory grid technique. The findings are based on in-depth interviews with chief information officers (CIOs) involved with SIS developments in their organisations. This exploratory study builds on a cognitive methodology and enables us to develop the evaluation framework of SIS within the CIO's mind. In a practical viewpoint, we evaluated the effectiveness of the essential activities in the SIS planning and implementation. Results showed that activities on analysing industry and environment, analysing information system weakness and strength, formulating SIS strategy, identifying SIS initiatives, prioritising and allocating resources for SIS, documenting SIS, and liaising with top management team are well performed.

We present a game theoretic model for multi-agent resource distribution and allocation where agents in the environment must help each other to survive. Each agent maintains a set of two-tuples T=(A, P) called friendship values representing actual friendship and perceived friendship. The model directly addresses problems in reputation management schemes in multi-agent systems and peer-to-peer distributed systems. We present algorithms for maintaining the friendship values as well as a utility equation used in each agent's decision making. For an application problem, we adapted our formal model to the military coalition support problem in peace-keeping missions. Simulation results show that efficient resource allocation and sharing with minimum communication cost is achieved without centralized control.

Much of what we might call 'high-art music' occupies the difficult end of listening for contemporary audiences. Concepts such as pitch, meter and even musical instruments often have little to do with such music, where all sound is typically considered as possessing musical potential. As a result, such music can be challenging to educationalists, for students have few familiar pointers in discovering and understanding the gestures, relationships and structures in these works. We describe ongoing projects at the University of Hertfordshire that adopt an approach of mapping interactions within visual spaces onto musical sound. These provide a causal explanation for the patterns and sequences heard, whilst incorporating Web interoperability thus enabling potential for distance learning applications. While so far these have mainly driven pitch-based events using MIDI or audio files, it is hoped to extend the ideas using appropriate technology into fully developed composition tools, aiding the teaching of both appreciation/analysis and composition of contemporary music.