Today’s robot policies exhibit subpar performance when faced with the challenge of generalizing to novel environments. Human corrective feedback is a crucial form of guidance to enable such generalization. However, adapting to and learning from online human corrections is a non-trivial endeavor: not only do robots need to remember human feedback over time to retrieve the right information in new settings and reduce the intervention rate, but also they would need to be able to respond to feedback that can be arbitrary corrections about high-level human preferences to low-level adjustments to skill parameters. In this work, we present Distillation and Retrieval of Online Corrections (DROC), a large language model (LLM)-based system that can respond to arbitrary forms of language feedback, distill generalizable knowledge from corrections, and retrieve relevant past experiences based on textual and visual similarity for improving performance in novel settings. DROC is able to respond to a sequence of online language corrections that address failures in both high-level task plans and low-level skill primitives. We demonstrate that DROC effectively distills the relevant information from the sequence of online corrections in a knowledge base and retrieves that knowledge in settings with new task or object instances. DROC outperforms other techniques that directly generate robot code via LLMs [1] by using only half of the total number of corrections needed in the first round and requires little to no corrections after two iterations. We show further results and videos on our project website: https://sites.google.com/stanford.edu/droc.

Intersection management systems, with the assistance of vehicular networks and autonomous vehicles, have the potential to perform traffic control more precisely than contemporary signalized intersections. However, as infrastructural intersection management controllers do not directly activate motions of vehicles, it is possible that the vehicles fail to follow the instructions from controllers, undermining system properties such as deadlock-freeness and traffic performance. In this article, we consider a class of robustness issues, the time violations, which stem from possible discrepancies between scheduled orders and real executions. We refine a graph-based intersection model to build our theoretical foundations and analyze potential deadlocks and their resolvability. We develop solutions that mitigate negative effects of time violations. In particular, we propose a Robustness-Aware Greedy Scheduling algorithm for robust scheduling and evaluate the deadlock-free robustness of different intersection models and scheduling algorithms. Experimental results show that the Robustness-Aware Greedy Scheduling algorithm is able to significantly improve robustness and keep a good balance with traffic performance.

The goal of intersection management is to organize vehicles to pass the intersection safely and efficiently. Due to the technical advance of connected and autonomous vehicles, intersection management becomes more intelligent and potentially unsignalized. In this paper, we propose a reinforcement-learning-based methodology to train a centralized intersection manager. We define the intersection scheduling problem with a graph-based model and transform it to the job-shop scheduling problem (JSSP) with additional constraints. To utilize reinforcement learning, we model the scheduling procedure as a Markov decision process (MDP) and train the agent with the proximal policy optimization (PPO). A grouping strategy is also developed to apply the trained model to streams of vehicles. Experimental results show that the learning-based intersection manager is especially effective with high traffic densities. This paper is the first work in the literature to apply reinforcement learning on the graph-based intersection model. The proposed methodology can flexibly deal with any conflicting scenario and indicate the applicability of reinforcement learning to Intelligent intersection management.

Intelligent intersection management aims to schedule vehicles so that vehicles can pass through an intersection efficiently and safely. However, inaccurate control, imperfect communication, and malicious information or behavior lead to robustness issues of intelligent intersection management. In this work, we focus on improving robustness against deadlocks by changing the trajectories of vehicles. To guarantee the resolvability of deadlocks, we limit the number of vehicles in an intersection to be smaller than or equal to an intersection-specific value called the maximal deadlock-free load. We develop an algorithm to compute the maximal deadlock-free load. We further reduce the computation time by computing the loads which are pessimistic (smaller) but still deadlock-free. Since the maximal deadlock-free load only depends on the given intersection, it can be integrated with different scheduling algorithms. Experimental results demonstrate that, by changing the trajectories of vehicles and limiting the number of vehicles under maximal deadlock-free loads, our approach can guarantee deadlock-freeness and maintain good traffic efficiency.

An easily retrofittable stereo vision based system for quick and temporary measurement of a mining shovel's swing angle is presented. The stereo camera is mounted externally to the upper swingable shovel house, with a clear view of the shovel's lower carbody. As the shovel swings from its 0° swing angle position, the camera revolves with the shovel house, seeing differing views of the carbody. In real-time, the camera position is tracked, which in turn is used to calculate the swing angle. The problem was solved using the Simultaneous Localization and Mapping (SLAM) approach in which the system learns a map of 3D features on the carbody while using the map to determine the camera pose. The contribution includes a locally maximal Harris corner selection technique and a novel use of 3D feature clusters as landmarks, for improving the robustness of visual landmark matching in an outdoor environment. Results show that the vision-based sensor has a maximum error of +/- 1° upon map convergence.

We propose a framework for skill synthesis for robots that exploits the human capacity to learn novel control tasks. The conceptual idea is to incorporate the target robotic platform into the experimenter's body schema so that it can be controlled effortlessly as if the robot were a part of the body. Once this stage is achieved, the dexterity on a task exhibited with the new external limb -the robot- can be used for designing controllers for the task under consideration. This article exemplifies the proposed framework by showing the derivation of an effective open-loop controller that can manipulate two balls with the fingers of a 16-DOF robotic hand.