In this paper, a novel approach for commanding mobile robots using a probabilistic multistroke sketch interface is presented. Drawing from prior work in handwriting recognition, sketches are modeled as a variable duration hidden Markov model, where the distributions on the states and transitions are learned from training data. A forward search algorithm is used to find the most likely sketch given the observations on the strokes, interstrokes, and gestures. A heuristic is implemented to discourage breadth-first search behavior, and is shown to greatly reduce computation time while sacrificing little accuracy. To avoid recognition errors, the recognized sketch is displayed to the user for confirmation; a rejection prompts the algorithm to search for and display the next most likely sketch. Upon confirmation of the recognized sketch, the robot executes the appropriate behaviors. A set of experiments was conducted in which operators controlled a single mobile robot in an indoor search-and-identify mission. Operators performed two missions using the proposed sketch interface and two missions using a more conventional point-and-click interface. On average, missions conducted using sketch control were performed as well as those using the point-and-click interface, and results from user surveys indicate that more operators preferred using sketch control.

A novel method for controlling a mobile robot using qualitative inputs in the context of an approximate map, such as one sketched by a human, is presented. By defining a desired trajectory with respect to observable landmarks, human operators can send semi-autonomous robots into areas for which a truth map is not available. Waypoint planning is formulated as a quadratic optimization problem, resulting in robot trajectories in the true environment that are qualitatively similar to those provided by the human. The algorithm is implemented both in simulation and on a mobile robot platform in several different environments. A sensitivity analysis is per formed, illustrating how the method is robust to uncertainties, even large sketch distortions, and allows the robot to adapt and re-plan according to its most current perception of the world.

A fully probabilistic command interface for controlling robots using multi-stroke sketch commands is presented. Drawing from prior work in handwriting recognition, sketches are modeled as a variable duration hidden Markov model, where the distributions on the states and transitions are learned from training data. A forward search algorithm on the gesture, stroke, and stroke transition observations is used to find the most likely sketch, which is displayed to the user for confirmation. In cases where the most likely sketch is incorrect, the user can reject it, prompting the next most likely sketch to be displayed. Upon confirmation from the user, the robot executes the desired behaviors. A prototype sketch interface was implemented using a pen tablet; two sets of search-and-identify experiments were conducted using a single robot in an indoor environment to test the usability of the proposed framework. Even novice users were able to successfully complete the missions, including those on whom the algorithm was not trained. User surveys indicate that operators generally found the interface to be natural and easy to use.

In this paper, scalable collaborative human-robot systems for information gathering applications are approached as a decentralized Bayesian sensor network problem. Human-computer augmented nodes and autonomous mobile sensor platforms are collaborating on a peer-to-peer basis by sharing information via wireless communication network. For each node, a computer (onboard the platform or carried by the human) implements both a decentralized Bayesian data fusion algorithm and a decentralized Bayesian control negotiation algorithm. The individual node controllers iteratively negotiate anonymously with each other in the information space to find cooperative search plans based on both observed and predicted information that explicitly consider the platforms (humans and robots) motion models, their sensors detection functions, as well as the target arbitrary motion model. The results of a collaborative multi-target search experiment conducted with a team of four autonomous mobile sensor platforms and five humans carrying small portable computers with wireless communication are presented to demonstrate the efficiency of the approach.

A state-dependent model reduction approach to evaluating human-robotic system performance for the purpose of implementing Bayesian Networks is introduced. A procedure for reducing state dependencies on a specific class of operator failure events for probabilistic graph models is developed and verified with user data. In contrast to evaluating performance with respect to objective "workload" measurements, discrete failure "tag" events are classified and modeled as Bayesian network blocks with conditional dependencies on a subset of the total system states. Initial extraction of performance results are shown using data from the RoboFlag experiments.