Building generalist agents that can rapidly adapt to new environments is a key challenge for deploying AI in the digital and real worlds. Is scaling current agent architectures the most effective way to build generalist agents? We propose a novel approach to pre-train relatively small policies on relatively small datasets and adapt them to unseen environments via in-context learning, without any finetuning. Our key idea is that retrieval offers a powerful bias for fast adaptation. Indeed, we demonstrate that even a simple retrieval-based 1-nearest neighbor agent offers a surprisingly strong baseline for today's state-of-the-art generalist agents. From this starting point, we construct a semi-parametric agent, REGENT, that trains a transformer-based policy on sequences of queries and retrieved neighbors. REGENT can generalize to unseen robotics and game-playing environments via retrieval augmentation and in-context learning, achieving this with up to 3x fewer parameters and up to an order-of-magnitude fewer pre-training datapoints, significantly outperforming today's state-of-the-art generalist agents.

Imitation learning considerably simplifies policy synthesis compared to alternative approaches by exploiting access to expert demonstrations. For such imitation policies, errors away from the training samples are particularly critical. Even rare slip-ups in the policy action outputs can compound quickly over time, since they lead to unfamiliar future states where the policy is still more likely to err, eventually causing task failures. We revisit simple supervised "behavior cloning" for conveniently training the policy from nothing more than pre-recorded demonstrations, but carefully design the model class to counter the compounding error phenomenon. Our "memory-consistent neural network" (MCNN) outputs are hard-constrained to stay within clearly specified permissible regions anchored to prototypical "memory" training samples. We provide a guaranteed upper bound for the sub-optimality gap induced by MCNN policies. Using MCNNs on 10 imitation learning tasks, with MLP, Transformer, and Diffusion backbones, spanning dexterous robotic manipulation and driving, proprioceptive inputs and visual inputs, and varying sizes and types of demonstration data, we find large and consistent gains in performance, validating that MCNNs are better-suited than vanilla deep neural networks for imitation learning applications. Website: https://sites.google.com/view/mcnn-imitation

Cyber-physical systems (CPSs) leverage computations to operate physical objects in real-world environments, and increasingly more CPS-based applications have been designed for life-critical applications. Therefore, any vulnerability in such a system can lead to severe consequences if exploited by adversaries. In this paper, we present a data predictive recovery system to safeguard the CPS from sensor attacks, assuming that we can identify compromised sensors from data. Our recovery system guarantees that the CPS will never encounter unsafe states and will smoothly recover to a target set within a conservative deadline. It also guarantees that the CPS will remain within the target set for a specified period. Major highlights of our paper include (i) the recovery procedure works on nonlinear systems, (ii) the method leverages uncorrupted sensors to relieve uncertainty accumulation, and (iii) an extensive set of experiments on various nonlinear benchmarks that demonstrate our framework’s performance and efficiency.

In Cyber-Physical Systems (CPS), integrating new technologies that interact with and control physical systems raises new security risks beyond the classical cyber security domain. These risks motivated many attack detectors that focus on the binary outcome. However, one pressing risk in CPS is hidden sensor attacks that are well-designed by powerful attackers who gained full knowledge of our systems and detector. The hidden attacks inject such a small malicious signal into sensor measurement that they can stay undetected but eventually lead to a significant deviation. Thus, to secure the CPS, we propose a detection framework to identify these sensor attacks that can drive the system's physical states to an unsafe state within a given period, even if they are not detected. First, we solve optimization problems to find the optimal hidden sensor attack that leads to the minimal distance to a pre-defined unsafe state region within an observation window for a given system and detector. Then, based on this algorithm, we perform offline profiling to search for a conditionally safe region, where the system states are guaranteed to be safe within the observation window as long as the detector does not raise any alerts. Finally, the framework can online discover potential hidden sensor attacks that endanger the system by checking if the current system state moves out of the region and raising a yellow alert. The evaluation shows that the optimal hidden sensor attack results in the minimum distance to unsafe, within a given observation window among existing hidden sensor attacks. We implemented our method on four linear simulators to show the effectiveness of our method. Additionally, we provided a discussion on the challenges of applying the proposed method to non-linear systems.