Intelligent and reliable task planning is a core capability for generalized robotics, which requires a descriptive domain representation that sufficiently models all object and state information for the scene. We present CLIMB, a continual learning framework for robot task planning that leverages foundation models and feedback from execution to guide the construction of domain models. CLIMB can build a model from a natural language description, learn non-obvious predicates while solving tasks, and store that information for future problems. We demonstrate the ability of CLIMB to improve performance in common planning environments compared to baseline methods. We also developed the BlocksWorld++ domain, a simulated environment with an easily usable real counterpart, together with a curriculum of tasks with progressing difficulty to evaluate continual learning. Code and additional details for this system can be found at https://plan-with-climb.github.io/.

In order to accomplish various missions, autonomous ground vehicles must operate on a wide range of terrain. While many systems such as wheels and whegs can navigate some types of terrain, none are optimal across all. This creates a need for physical adaptation. This paper presents a broad new approach to physical adaptation that relies on manipulation. Specifically, we explore how multipurpose manipulators can enable ground vehicles to dramatically modify their propulsion system in order to optimize performance across various terrain. While this approach is general and widely applicable, this work focuses on physically switching between wheels and legs. We outline the design of "swappable propulsors" that combine the powerful adhesion forces of permanent magnets with geometric features for easy detachment. We provide analysis on how the swappable propulsors can be manipulated, and use these results to create a functional prototype robot. This robot can use its manipulator to change between wheeled and legged locomotion. Our experimental results illustrate how this approach can enhance energy efficiency and versatility.

This work introduces an affordance characterization employing mechanical wrenches as a metric for predicting and planning with workspace affordances. Although affordances are a commonly used high-level paradigm for robotic task-level planning and learning, the literature has been sparse regarding how to characterize the agent in this object-agent-environment framework. In this work, we propose decomposing a behavior into a vocabulary of characteristic requirements and capabilities that are suitable to predict the affordances of various parts of the workspace. Specifically, we investigate mechanical wrenches as a viable representation of these affordance requirements and capabilities. We then use this vocabulary in a planning system to compose complex motions from simple behavior types in continuous space. The utility of the framework for complex planning is demonstrated on example scenarios both in simulation and with real-world industrial manipulators.

In this paper, we describe a novel approach for representing state information for the purpose of intention recognition in cooperative human-robot environments. States are represented by a combination of spatial relationships in a Cartesian frame along with cardinal direction information. This approach is applied to a manufacturing kitting operation, where humans and robots are working together to develop kits. Based upon a set of predefined high-level states relationships that must be true for future actions to occur, a robot can use the detailed state information presented in this paper to infer the probability of subsequent actions occurring. This would enable the robot to better help the human with the operation or, at a minimum, better stay out of his or her way.

The IEEE RAS Ontologies for Robotics and Automation Working Group is dedicated to developing a methodology for knowledge representation and reasoning in robotics and automation. As part of this working group, the Industrial Robots sub-group is tasked with studying industrial applications of the ontology. One of the first areas of interest for this subgroup is the area of kit building or kitting. This is a process that brings parts that will be used in assembly operations together in a kit and then moves the kit to the assembly area where the parts are used in the final assembly. This paper examines the knowledge representations that have been developed and implemented for the kitting problem.

This article discusses a newly formed IEEE-RAS working group entitled Ontologies for Robotics and Automation (ORA). The goal of this working group is to develop a standard ontology and associated methodology for knowledge representation and reasoning in robotics and automation, together with the representation of concepts in an initial set of application domains. The standard provides a unified way of representing knowledge and provides a common set of terms and definitions, allowing for unambiguous knowledge transfer among any group of humans, robots, and other artificial systems. In addition to describing the goal and structure of the group, this article gives some examples of how the ontology, once developed, can be used by applications such as industrial kitting.

The Single-query, Bi-directional, Lazy roadmap (SBL) algorithm successfully builds upon the traditional Probabilistic Roadmaps (PRM) approach by introducing a number of related optimizations. While these optimizations are applicable in the domain of car-like robots, the non-holonomic constraints of these systems and non-independence of several of their degrees of freedom pose challenges that will be examined in this paper. We present several enhancements that improve the quality of the generated path in comparison with the simple adaptation of the SBL algorithm. Results demonstrate that this work provides a planner that quickly and reliably discovers efficient paths for car-like robots.

Simulation of robots in a virtual domain has multiple benefits, including serving as a training tool for end users and serving as a tool for robot testing and development. USARSim (Unified System for Automation and Robot Simulation) is a simulation tool that is being used to accomplish these goals. It is based on the Unreal Tournament 2004 gaming engine, which approximates the physics of how a robot interacts with its environment. A family of robots that can benefit from simulation in USARSim are Whegstrade* robots. Developed in the Biorobotics Laboratory at Case Western Reserve University (CWRU), Whegstrade robots are highly mobile ground vehicles that use abstracted biological principles to achieve a robust level of locomotion, including passive gait adaptation and enhanced climbing abilities. This paper describes a Whegstrade robot model that was constructed and benchmarked in USARSim. The model was given the same kinds of physical characteristics found in real Whegstrade vehicles. With these traits in place, a validation study was performed for the simulation by comparing the rotational body motions of the virtual and real robot. Specifically, we examine the robotpsilas pitch and roll ranges and frequencies (found to be about 10 degrees to 12 degrees and 5.5 Hz respectively). By altering the simulation parameters in a physically relevant way, the virtual robotpsilas performance was moved to be more in line with these values.

This paper presents USARSim, an open source high fidelity robot simulator that can be used both for research and education. USARSim offers many characteristics that differentiate it from most existing simulators. Most notably, it constitutes the simulation engine used to run the virtual robots competition within the Robocup initiative. We describe its general architecture, describe examples of utilization, and provide a comprehensive overview for those interested in robot simulations for education, research and competitions.

Rough terrain, such as the rubble that one would expect to find in urban disaster areas, poses a severe impediment to robot mobility. The goal of this paper' is to develop methods for quantifying the difficulty a robot would encounter traversing such a region of rough terrain. Towards this, three traversability metrics describing rough terrain robot mobility are developed. In order to simplify the problem, we assume that the rough terrain in question can be discretized in a certain manner and then develop the metrics for this discretized version of the terrain. Two of the developed metrics reflect the difficulty a robot would have trying to move over the entire region of terrain (referred to as coverability). The third metric, referred to as crossability, describes the difficulty a robot would encounter attempting to move from some fixed point on the terrain to some other fixed point. We verify the applicability of the developed metrics using NIST-developed repeatable test methods (step fields) and discuss the validity of the obtained results.

In this paper, we describe our recent efforts on symbolic representations enabling autonomous driving in dynamic environments such as on-road driving. In contrast to traditional bottom-up approaches where temporal accumulation of locally insignificant inaccuracies causes eventual failures in high-level scene interpretation, our proposed knowledge-driven top-down approach combined with vehicle's intentions can provide valuable information to guide low-level bottom-up tasks and vice-versa. We contend that such rich symbolic representations can reduce the burden on sensory processing thereby dynamically directing it to look for particular features in expected locations and subsequently facilitating the vehicle to better react to potentially dangerous situations, such as the appearance of pedestrians in the road. We demonstrate the proposed approaches in various scenarios pertaining to vehicle perception and control using field data obtained from a military unmanned ground vehicle (UGV) traversing urban environments

For an autonomous vehicle to drive in a dynamic environment, it must be able to detect moving objects, predict their future location, represent this information in its internal knowledge bases, and make appropriate plans based upon all of this information. The estimation of what the object is and where it is expected to be in the future must be continuously refined as time progresses. This paper discusses a multi-representational approach to capturing important characteristics about moving objects in a dynamic environment. Specifically we look at applying a combination of symbolic, equation-based, and grid-based representations to fully represent the information that is needed for different types of planners to develop appropriate plans in the presence of moving objects.

We define an architecture for course vehicle motion planning within the real-time control system hierarchy. Interaction between the planning system and world model is discussed, and results of this system being implemented on a real world system are presented.

This paper considers the characterization of three different image stabilization algorithms when used as a preprocessor for a computer vision application. These algorithms vary in computational complexity, accuracy, and performance. The first algorithm (developed by the Army Research Laboratory (ARL)) is capable of image alignment to an accuracy of one pixel. Algorithms 2 and 3 (developed by the University of Maryland) are capable of full subpixel stabilization with respect to translation, rotation, and scale. The evaluation tools incorporated include mean square error of the output data set and the overall performance of an automatic target acquisition (ATA) system (developed at ARL) that uses the algorithms as a front-end preprocessor. We show that for the ARL ATA application, extremely accurate subpixel stabilization is a requirement for proper operation. Based on experiments, we conclude that algorithm 3 performs significantly better than the other two algorithms.