Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited.Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various real-world factors. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark, encompassing low-level robotics problems and high-level tactics. Different from other machine learning-centric benchmarks, participants need to tackle practical challenges in robotics, such as the sim-to-real gap, low-level control issues, safety problems, real-time requirements, and the limited availability of real-world data. Furthermore, we focus on a dynamic environment, removing the typical assumption of quasi-static motions of other real-world benchmarks.The competition's results show that solutions combining learning-based approaches with prior knowledge outperform those relying solely on data when real-world deployment is challenging.Our ablation study reveals which real-world factors may be overlooked when building a learning-based solution.The successful real-world air hockey deployment of best-performing agents sets the foundation for future competitions and follow-up research directions.

Achieving reactive robot behavior in complex dynamic environments is still challenging as it relies on being able to solve trajectory optimization problems quickly enough, such that we can replan the future motion at frequencies which are sufficiently high for the task at hand. We argue that current limitations in Model Predictive Control (MPC) for robot manipulators arise from inefficient, high-dimensional trajectory representations and the negligence of time-optimality in the trajectory optimization process. Therefore, we propose a motion optimization framework that optimizes jointly over space and time, generating smooth and timing-optimal robot trajectories in joint-space. While being task-agnostic, our formulation can incorporate additional task-specific requirements, such as collision avoidance, and yet maintain real-time control rates, demonstrated in simulation and real-world robot experiments on closed-loop manipulation. For additional material, please visit https://sites.google.com/oxfordrobotics.institute/vp-sto.

Pushing is an essential motor skill involved in several manipulation tasks, and has been an important research topic in robotics. Recent works have shown that Deep Q-Networks (DQNs) can learn pushing policies (when, where to push, and how) to solve manipulation tasks, potentially in synergy with other skills (e.g. grasping). Nevertheless, DQNs often assume a fixed setting and task, which may limit their deployment in practice. Furthermore, they suffer from sparse-gradient backpropagation when the action space is very large, a problem exacerbated by the fact that they are trained to predict state-action values based on a single reward function aggregating several facets of the task, rendering the model training challenging. To address these issues, we propose a multi-head target-parameterized DQN to learn robotic manipulation tasks, in particular pushing policies, and make the following contributions: i) we show that learning to predict different reward and task aspects can be beneficial compared to predicting a single value function where reward factors are not disentangled; ii) we study several alternatives to generalize a policy by encoding the target parameters either into the network layers or visually in the input; iii) we propose a kernelized version of the loss function, allowing to obtain better, faster and more stable training performance. Extensive experiments on simulations validate our design choices, and we show that our architecture learned on simulated data can achieve high performance in a real-robot setup involving a Franka Emika robot arm and unseen objects.

Collaborating robots face rising challenges with respect to autonomy and safety as they are deployed in flexible automation applications. The ability to perform the required tasks in the presence of humans and obstacles is key for the integration of these machines in industry. In this work we introduce a framework for motion planning of manipulators that builds upon the most promising existing approaches by combining them in an advantageous way. It includes a new Obstacle-related Sampling Rejection Probabilistic Roadmap planner (ORSR-PRM) that represents the free workspace in an efficient way. Using this representation, dynamic obstacles can be avoided in real-time using an attractor-based online trajectory generation. The resulting motions satisfy kinematic and dynamic joint limits, ensuring a safe human-robot interaction. We validate the functionality and performance of the presented framework in simulations and experiments.

Interactions between robots and their environment give rise to external wrenches acting on the robot structure. The estimation of the resulting torques in the joints is fundamental in human-robot interaction to detect/identify collisions and perform suitable reaction strategies. Other applications may require to use the estimation for compensating the effects of the external torques within the control loop. The well-established momentum observer, which relies on proprioceptive sensors only, is usually used for these purposes. In this work, the momentum dynamics is used to derive new observers. While the classic momentum observer provides a first-order filtered version of the external torques, here a (theoretically) finite-time convergence is achieved. Simulations and experiments are used to validate the performance of the proposed methods.