Even as autonomous capabilities improve, many robot manipulation tasks require human(s)-in-the-loop to resolve high-level problems in uncertain environments or ambiguous situations. Prior work in highly autonomous applications tends to use interfaces with few human interface modalities, poten-tially missing out on the benefits that multimodal interfaces have demonstrated in lower-level operation. In this work, we demonstrate a system with a multimodal interface for a controlling a robot at a high level of autonomy. This example highlights how multiple modalities could enable redundant and robust interactions, increased situational awareness, and compact representations of complex commands, such as how to grasp an object.

Imbuing furniture with robot properties reduces the physical labor and time needed for arranging spaces, such as homes, classrooms, and offices. Outsourcing labor tasks to robot furniture requires users’ involvement with functional user interfaces. We performed a user study on multi-robot furniture and added additional features based on the study results. The study involved 12 participants rearranging multiple non-robotic and robotic chairs (ChairBots). Results from the video and interview analysis revealed five high-level features missing in the original ChairBot: dual screen-based user interface, the ability to save and to set arrangements, the ability to move in multi-robot formations, the ability to snap to angles/gridlines, and higher movement precision. The improved system allows users to control multiple furniture robots, both locally and remotely. Such improvement sets the baseline functionalities of robot furniture arrangement systems while extending the potential utilization of established robotic chairs.