We demonstrate the capabilities of an attentionbased end-to-end approach for high-speed vision-based quadrotor obstacle avoidance in dense, cluttered environments, with comparison to various state-of-the-art learning architectures. Quadrotor unmanned aerial vehicles (UAVs) have tremendous maneuverability when flown fast; however, as flight speed increases, traditional model-based approaches to navigation via independent perception, mapping, planning, and control modules breaks down due to increased sensor noise, compounding errors, and increased processing latency. Thus, learning-based, end-to-end vision-to-control networks have shown to have great potential for online control of these fast robots through cluttered environments. We train and compare convolutional, U-Net, and recurrent architectures against vision transformer (ViT) models for depth image-to-control in high-fidelity simulation, observing that ViT models are more effective than others as quadrotor speeds increase and in generalization to unseen environments, while the addition of recurrence further improves performance while reducing quadrotor energy cost across all tested flight speeds. We assess performance at speeds of up to 7m/s in simulation and hardware. To the best of our knowledge, this is the first work to utilize vision transformers for end-to-end vision-based quadrotor control.

Tracking controllers enable robotic systems to accurately follow planned reference trajectories. In particular, reinforcement learning (RL) has shown promise in the synthesis of controllers for systems with complex dynamics and modest online compute budgets. However, the poor sample efficiency of RL and the challenges of reward design make training slow and sometimes unstable, especially for high-dimensional systems. In this work, we leverage the inherent Lie group symmetries of robotic systems with a floating base to mitigate these challenges when learning tracking controllers. We model a general tracking problem as a Markov decision process (MDP) that captures the evolution of both the physical and reference states. Next, we prove that symmetry in the underlying dynamics and running costs leads to an MDP homomorphism, a mapping that allows a policy trained on a lower-dimensional “quotient” MDP to be lifted to an optimal tracking controller for the original system. We compare this symmetry-informed approach to an unstructured baseline, using Proximal Policy Optimization (PPO) to learn tracking controllers for three systems: the Particle (a forced point mass), the Astrobee (a fully-actuated space robot), and the Quadrotor (an underactuated system). Results show that a symmetry-aware approach both accelerates training and reduces tracking error at convergence.

The safety and efficiency of an Autonomous Vehicle (AV) manoeuvre planning heavily depend on the future trajectories of surrounding vehicles. If an AV can predict its surrounding vehicles’ future trajectories, it can make safe and efficient manoeuvre decisions. In this paper, we present a Deep Spatio-Temporal Context-Aware decision Network (DST-CAN) for predictive manoeuvre decisions for AVs on highways. DST-CAN has two main components, namely spatio-temporal context-aware map generator and predictive manoeuvre decisions engine. DST-CAN employ a memory neuron network to predict the future trajectories of its surrounding vehicles. Using look-ahead prediction and past actual trajectories, a spatio-temporal context-aware probability occupancy map is generated. These context-aware maps as input to a decision engine generate a safe and efficient manoeuvre decision. Here, CNN helps extract feature space, and two fully connected network generates longitudinal and lateral manoeuvre decisions. Performance evaluation of DST-CAN has been carried out using two publicly available NGSIM US-101 and I-80 highway datasets. A traffic rule is defined to generate ground truths for these datasets in addition to human decisions. Two DST-CAN models are trained using imitation learning with human driving decisions from actual traffic data and rule-based ground truth decisions. The performances of the DST-CAN models are compared with the state-of-the-art Convolutional Social-LSTM (CS-LSTM) models for manoeuvre prediction. The results clearly indicate that the context-aware maps help DST-CAN to predict the decision accurately over CS-LSTM. Further, an ablation study has been carried out to understand the effect of prediction horizons of performance and a robustness study to understand the near collision scenarios over actual traffic observations. The context-aware map with a 3 second prediction horizon is robust against near collision.

This paper addresses the problem of position estimation in UAVs operating in a cluttered environment where GPS information is unavailable. A learning-based approach is proposed that takes in the rotor RPMs and past state as input and predicts the one-step-ahead position of the UAV using a novel spectral-normalized memory neural network (SN-MNN). The spectral normalization guarantees stable and reliable prediction performance. The predicted position is transformed to the global coordinate frame (GPS), which is then fused along with the odometry of other peripheral sensors like IMU, barometer, compass, etc., using the onboard extended Kalman filter (EKF) to estimate the states of the UAV. The experimental flight data collected from an RTK-GPS facility using a micro-UAV is used to train the SN-MNN. The PX4-ECL library is used to fuse the predicted data using the SN-MNN, and the estimated position is compared with actual ground truth data. The proposed algorithm doesn't require any additional onboard sensors and is computationally light. The performance of the proposed approach is compared with the current state-of-art GPS-denied algorithms, and it can be seen that the proposed algorithm has the least RMSE for position estimates.

Prognostication of vehicle trajectories in unknown environments is intrinsically a challenging and difficult problem to solve. The behavior of such vehicles is highly influenced by surrounding traffic, road conditions, and rogue participants present in the environment. Moreover, the presence of pedestrians, traffic lights, stop signs, etc., makes it much harder to infer the behavior of various traffic agents. This paper attempts to solve the problem of spatio-temporal look-ahead trajectory prediction using a novel recurrent neural network called the Memory Neuron Network. The Memory Neuron Network (MNN) attempts to capture the input-output relationship between the past positions and the future positions of the traffic agents. The proposed prediction model is computationally less intensive and has a simple architecture as compared to other deep learning models that utilize LSTMs and GRUs. It is then evaluated on the publicly available NGSIM dataset and its performance is compared with several state-of-art algorithms. Additionally, the performance is also evaluated on a custom synthetic dataset generated from the CARLA simulator. It is seen that the proposed model outperforms the existing state-of-art algorithms. Finally, the model is integrated with the CARLA simulator to test its robustness in real-time traffic scenarios.