Diffusion models have revolutionized imitation learning, allowing robots to replicate complex behaviours. However, diffusion often relies on cameras and other exteroceptive sensors to observe the environment and lacks long-term memory. In space, military, and underwater applications, robots must be highly robust to failures in exteroceptive sensors, operating using only proprioceptive information. In this paper, we propose ProDapt, a method of incorporating long-term memory of previous contacts between the robot and the environment in the diffusion process, allowing it to complete tasks using only proprioceptive data. This is achieved by identifying “keypoints”, essential past observations maintained as inputs to the policy. We test our approach using a UR10e robotic arm in both simulation and real experiments and demonstrate the necessity of this long-term memory for task completion.

Modeling dynamics is often the first step to making a vehicle autonomous. While on-road autonomous vehicles have been extensively studied, off-road vehicles pose many challenging modeling problems. An off-road vehicle encounters highly complex and difficult-to-model terrain/vehicle interactions, as well as having complex vehicle dynamics of its own. These complexities can create challenges for effective high-speed control and planning. In this paper, we introduce a framework for multistep dynamics prediction that explicitly handles the accumulation of modeling error and remains scalable for sampling-based controllers. Our method uses a specially-initialized Long Short-Term Memory (LSTM) over a limited time horizon as the learned component in a hybrid model to predict the dynamics of a 4-person seating all-terrain vehicle (Polaris S4 1000 RZR) in two distinct environments. By only having the LSTM predict over a fixed time horizon, we negate the need for long term stability that is often a challenge when training recurrent neural networks. Our framework is flexible as it only requires odometry information for labels. Through extensive experimentation, we show that our method is able to predict millions of possible trajectories in real-time, with a time horizon of five seconds in challenging off road driving scenarios.

The discovery of ocean worlds such as Enceladus, Titan, and Europa motivates the development of versatile autonomous mobility systems to enable the next era of space exploration where there is large uncertainty in terrain specifications due to a lack of prior surface reconnaissance missions. To explore these environments, we propose Exobiology Extant Life Surveyor (EELS): the first large-scale (4 lm long with 400 Nm peak torque) snake robot. The large scale is achieved by using a screw-based active skin mechanism to decouple motion and shape control. Autonomous mobility for such a system remains an open problem due to its many Degrees of Freedom (DoFs), complex terrain interactions, and intermittent localization failures in GPS-denied perceptually degraded environments due to the presence of fog, dust, featureless terrains, etc. We propose NEO, an autonomy architecture that scales to large DoFs to generate a versatile set of gaits to achieve mobility in unknown extreme environments. We also discuss the resilience capabilities of NEO that achieves closed-loop tracking performance by leveraging exteroception when available but can also operate with proprioception only, leading to resiliency against localization failures via graceful degradation in performance rather than unsafe behaviors. A quantitative hardware evaluation of exteroceptive leader-follower gait is performed indoors on synthetic ice along with qualitative results of field deployment of the proprioceptive leader-follower and sidewinding gaits in extreme environments of icy and sandy terrains with mobility-stressing elements such as trenches, undulations, and steep slopes (up to 35 degrees). We present a set of lessons learned from field deployments with a summary of challenges and open research problems. Video: www.rohanthakker.in/eels-neo-autonomy.html

Semantic object mapping in uncertain, perceptually degraded environments during long-range multi-robot autonomous exploration tasks such as search-and-rescue is important and challenging. During such missions, high recall is desirable to avoid missing true target objects and high precision is also critical to avoid wasting valuable operational time on false positives. Given recent advancements in visual perception algorithms, the former is largely solvable autonomously, but the latter is difficult to address without the supervision of a human operator. However, operational constraints such as mission time, computational requirements and mesh network bandwidth can make the operator's task infeasible unless properly managed. We propose the Early Recall, Late Precision (EaRLaP) semantic object mapping pipeline to solve this problem. EaRLaP was used by Team CoSTAR in DARPA Subterranean Challenge, where it successfully detected all the artifacts encountered by the team of robots. We will discuss these results and the performance of the EaRLaP on various datasets.

Aircraft that can launch ballistically and convert to autonomous, free-flying drones have applications in many areas such as emergency response, defense, and space exploration, where they can gather critical situational data using onboard sensors. This paper presents a ballistically-launched, autonomously-stabilizing multirotor prototype (SQUID - Streamlined Quick Unfolding Investigation Drone) with an onboard sensor suite, autonomy pipeline, and passive aerodynamic stability. We demonstrate autonomous transition from passive to vision-based, active stabilization, confirming the multirotor's ability to autonomously stabilize after a ballistic launch in a GPS-denied environment.

This paper presents a novel episodic method to learn a robot's nonlinear dynamics model and an increasingly optimal control sequence for a set of tasks. The method is based on the Koopman operator approach to nonlinear dynamical systems analysis, which models the flow of observables in a function space, rather than a flow in a state space. Practically, this method estimates a nonlinear diffeomorphism that lifts the dynamics to a higher dimensional space where they are linear. Efficient Model Predictive Control methods can then be applied to the lifted model. This approach allows for real time implementation in on-board hardware, with rigorous incorporation of both input and state constraints during learning. We demonstrate the method in a real-time implementation of fast multirotor landing, where the nonlinear ground effect is learned and used to improve landing speed and quality.

The operation of multirotors in crowded environments requires a highly reliable takeoff method, as failures during takeoff can damage more valuable assets nearby. The addition of a ballistic launch system imposes a deterministic path for the multirotor to prevent collisions with its environment, as well as increases the multirotor's range of operation and allows deployment from an unsteady platform. In addition, outfitting planetary rovers or entry vehicles with such deployable multirotors has the potential to greatly extend the data collection capabilities of a mission. A proof-of-concept multirotor aircraft has been developed, capable of transitioning from a ballistic launch configuration to a fully controllable flight configuration in midair after launch. The transition is accomplished via passive unfolding of the multirotor arms, triggered by a nichrome burn wire release mechanism. The design is 3D printable, launches from a three-inch diameter barrel, and has sufficient thrust to carry a significant payload. The system has been fabricated and field tested from a moving vehicle up to 50mph to successfully demonstrate the feasibility of the concept and experimentally validate the design's aerodynamic stability and deployment reliability. SUPPLEMENTARY MATERIAL Videos of the experiments: https://youtu.be/sQuKJfllyRM.