Robots that process navigation instructions in large outdoor environments will need to operate at different levels of abstraction. For example, a land-surveying aerial robot receiving the instruction “go to Boston and go through the state forest on the way” must reason about a long-range goal like “go to Boston” while also processing a finer-grained constraint like “go through the state forest.” Existing approaches struggle to plan such commands because of the immense number of locations and constraints that can be expressed in language. We introduce a hierarchical representation of outdoor environments and a planning approach that dynamically compacts the robot's state space to enable tractable planning in city and state-scale environments. Our approach leverages natural abstractions in real-world map data, coupled with abstractions generated from users' instructions, to generate filtered environment views that accelerate planning while supporting a robot's ability to obey complex temporal goals and constraints at different levels of abstraction. We evaluate our approach on seven templates of LTLJ formulas and in an 80 kilometer-radius environment containing over 250,000 locations downloaded from OpenStreetMap. The results show our approach enables planning in seconds or minutes in a large outdoor environment while still satisfying the task specification.

Robots operating in outdoor, urban environments need the ability to follow complex natural language commands which refer to never-before-seen landmarks. Existing approaches to this problem are limited because they require training a language model for the landmarks of a particular environment before a robot can understand commands referring to those landmarks. To generalize to new environments outside of the training set, we present a framework that parses references to landmarks, then assesses semantic similarities between the referring expression and landmarks in a predefined semantic map of the world, and ultimately translates natural language commands to motion plans for a drone. This framework allows the robot to ground natural language phrases to landmarks in a map when both the referring expressions to landmarks and the landmarks themselves have not been seen during training. We test our framework with a 14-person user evaluation demonstrating an end-to-end accuracy of 76.19% in an unseen environment. Subjective measures show that users find our system to have high performance and low workload. These results demonstrate our approach enables untrained users to control a robot in large unseen outdoor environments with unconstrained natural language.

Human behavior is powerfully guided by social and moral norms. Robots that enter human societies must therefore behave in norm-conforming ways as well. However, there is currently no cognitive, let alone computational model available of how humans represent, activate, and learn norms. We offer first steps toward such a model and apply it to the design of a norm-competent social robot. We propose a general methodology for such a design, from empirical identification of relevant norms to computational implementations of norm learning to thorough and iterative evaluation of the robot's norm compliance by means of community feedback.