The scale and diversity of demonstration data required for imitation learning is a significant challenge. We present EgoMimic, a full-stack framework which scales manipulation via human embodiment data, specifically egocentric human videos paired with 3D hand tracking. EgoMimic achieves this through: (1) a system to capture human embodiment data using the ergonomic Project Aria glasses, (2) a low-cost bimanual manipulator that minimizes the kinematic gap to human data, (3) cross-domain data alignment techniques, and (4) an imitation learning architecture that co-trains on human and robot data. Compared to prior works that only extract high-level intent from human videos, our approach treats human and robot data equally as embodied demonstration data and learns a unified policy from both data sources. EgoMimic achieves significant improvement on a diverse set of long-horizon, single-arm and bimanual manipulation tasks over state-of-the-art imitation learning methods and enables generalization to entirely new scenes. Finally, we show a favorable scaling trend for EgoMimic, where adding 1 hour of additional hand data is significantly more valuable than 1 hour of additional robot data. Videos and additional information can be found at https://egomimic.github.io/

This paper presents Neural Visibility Field (NVF), a novel uncertainty quantification method for Neural Radi-ance Fields (NeRF) applied to active mapping. Our key insight is that regions not visible in the training views lead to inherently unreliable color predictions by NeRF at this region, resulting in increased uncertainty in the synthesized views. To address this, we propose to use Bayesian Networks to composite position-based field uncertainty into ray-based uncertainty in camera observations. Consequently, NVF nat-urally assigns higher uncertainty to unobserved regions, aiding robots to select the most informative next viewpoints. Extensive evaluations show that NVF excels not only in un-certainty quantification but also in scene reconstruction for active mapping, outperforming existing methods. More de-tails can be found at https://sites.google.com/view/nvf-cvpr24/.

Drones have now found their way into diverse fields of application. Getting accustomed to the presence of these drones requires seamless integration into our lives and depends a lot on Human-Drone Interaction (HDI). This paper presents a novel approach to develop a non-invasive Brain-Computer Interface (BCI) for control of the quadcopters as an assistive device (AD) for those people suffering from neurodegenerative diseases or impaired mobility and have lost the ability to explore the world around them freely. Electroencephalography (EEG) signals of individuals are captured, and actions corresponding to the wearer's thoughts were classified and used to control a quadcopter. Our main contribution lies in modeling this problem as a Markovian process thus enabling us to maximize the accuracy of data post-classification especially since the data may still contain outliers due to the inability of the user to maintain a constant thought stream, sensor noise, or classification errors. We propose an algorithm that aims at achieving robust control by breaking the problem into sub-parts, which are- classification, outlier removal, maximum likelihood estimation, and autonomy, where each step is optimized individually. We also present a shared control algorithm incorporating visual feedback by three-dimensional reconstruction of the environment to augment user's decisions with autonomous obstacle avoidance. The entire algorithm of this BCI is built using the Robot Operating System (ROS) framework. Our results suggest that post-processing classified data improves accuracy and system reaction time with minimal detriment to computational efficiency.