Assistive robots offer agency to humans with severe motor impairments. Often, these users control high-DoF robots through low-dimensional interfaces—such as using a 1-D sip/puff interface to operate a 6-DoF robotic arm. This mismatch results in having access to only a subset of control dimensions at a given time, imposing unintended and artificial constraints on robot motion. As a result, interface-limited demonstrations embed suboptimal motions that reflect interface restrictions rather than user intent. To address this, we present a trajectory reconstruction algorithm that reasons about task, environment, and interface constraints to lift demonstrations into the robot’s full control space. We evaluate our approach using real-world demonstrations of ADL-inspired tasks performed via a 2-D joystick and 1-D sip/puff control interface, teleoperating two distinct 7-DoF robotic arms. Analyses of the reconstructed demonstrations and derived control policies show that lifted trajectories are faster and more efficient than their interface-constrained counterparts while respecting user preferences.

When individuals are paralyzed from injury or damage to the brain, upper body movement and function can be compromised. While the use of body motions to interface with machines has shown to be an effective noninvasive strategy to provide movement assistance and to promote physical rehabilitation, learning to use such interfaces to control complex machines is not well understood. In a five session study, we demonstrate that a subset of an uninjured population is able to learn and improve their ability to use a high-dimensional Body-Machine Interface (BoMI), to control a robotic arm. We use a sensor net of four inertial measurement units, placed bilaterally on the upper body, and a BoMI with the capacity to directly control a robot in six dimensions. We consider whether the way in which the robot control space is mapped from human inputs has any impact on learning. Our results suggest that the space of robot control does play a role in the evolution of human learning: specifically, though robot control in joint space appears to be more intuitive initially, control in task space is found to have a greater capacity for longer-term improvement and learning. Our results further suggest that there is an inverse relationship between control dimension couplings and task performance.

We present an assistance system that reasons about a human’s intended actions during robot teleoperation in order to provide appropriate modifications on unintended behavior. Existing methods typically treat the human and control interface as a black box and assume the measured user input is noise-free, and use this signal to infer task-level human intent. We recognize that the signal measured through the interface is masked by the physical limitations of the user and the interface they are required to use. With this key insight, we model the human’s physical interaction with a control interface during robot teleoperation, and distinguish between interface-level intended and measured physical actions explicitly. By reasoning over the unobserved intentions using model-based inference techniques, our assistive system provides customized modifications on a user’s issued commands. We validate our algorithm both in simulation and with a 10-person human subject study in which we evaluate the performance of the proposed assistance paradigms. Our results show that the assistance paradigms helped to significantly reduce task completion time, number of mode switches, cognitive workload, and user frustration, and improve overall user satisfaction.

Shared control for human-robot teams - where both the human and the robot's autonomy provide commands to the hardware - offers advantages over fully teleoperated or fully autonomous systems by utilizing the unique skill sets of both the human and robot's autonomy simultaneously. However, the mechanism by which control is shared is often static and many teams could benefit from adjusting this mechanism, such that the human or autonomy alternatively receive more control authority in different scenarios. The question then is: how do we know when these scenarios occur? In this paper, we present a method to estimate the performance of human-robot teams using a novel metric called Discrete N-Dimensional Entropy of Behavior (DNDEB). DNDEB utilizes knowledge of a high-performing human-robot team to build a model of how the team should operate. The model is used to predict the human's command. The error between the prediction and actual command is tracked and after a certain number of samples, entropy is estimated. A higher level of entropy corresponds to deviations from the high-performance model, which can be interpreted as poor performance by the human-robot team (e.g., long task time or a collision). Our formulation offers several advantages: it (1) accepts discrete inputs of any size, (2) does not require additional sensors, and (3) is tunable to the specific application. To validate this, we conduct a 15person study where subjects operated a powered wheelchair under three different shared-control paradigms. We find that entropy is higher for cases with longer task durations and cases where there is a collision. Moreover, we use DNDEB thresholds as a mechanism to predict the performance of the human-robot team online and find an average accuracy of 91% with a prescience rate of 72%.

This work presents an algorithm for automated real-time ramp detection using 3D point cloud data in the context of shared-control powered wheelchairs. Limitations in the interfaces available to those with severe motor impairments can make basic maneuvering tasks with powered wheelchairs difficult. Although a significant amount of work has been done on obstacle detection and avoidance, much less attention has been given to algorithms for the safe and reliable detection of ramps and inclines; even though navigating these structures is an important part of urban life. We provide an algorithmic solution for accurately detecting traversable inclines for applications with powered wheelchairs using the Point Cloud Library (PCL) within the Robotics Operating System (ROS) framework. All algorithms are implemented first in simulation and later evaluated on data obtained from indoor and outdoor urban environments. We measure the performance of our algorithm with systematic testing on several different ramp structures, observed from varied viewpoints. Results show that our algorithm is successful in detecting the orientation, slope, and width of traversable ramps with up to 100% accuracy and an average detection accuracy of 88%.