Mobile devices such as smartphones and tablets can provide additional sensing and interacting capabilities to a mobile robot, even extending its senses to remote locations. To do so, we developed ROS4iOS, a native port of ROS allowing to seamlessly use data from mobile iOS devices to be processed on a robot. To demonstrate this capability, this video presentation illustrates how ROS4iOS has been used to implement an assistance scenario: a person in a remote location asks our IRL-1 robot for assistance, and IRL-1 must recognize the person's voice, identify the remote location using images taken from the mobile device, navigate to the identified location and interact vocally with the person through the mobile device. When communication with the robot is established by launching a specific iOS application, audio from the mobile device is published on a single topic and directed toward two ROS nodes on IRL-1. PocketSphinx, a speech recognition toolkit, is used to obtain the person vocal commands. WISS, a speaker identification system, is used for speaker identification. From a vocal request made through the mobile device, the robot can identify the person and request to get images of the remote location using the rear-facing camera of the mobile device. RTABMap, a loop-closure detection system for visual location recognition, is used to locate the person. A map was previously built with the robot's laser range finder using the gmapping SLAM algorithm, thus permitting reuse of the ROS navigation stack to plan a path and follow it safely from its current location to the requested one. Depending on its current state, the robot can also display the most suited view (i.e., phone, camera, navigation) on the iOS application. ROS4iOS can also be used to teleoperate IRL-1 through the mobile device or to discuss remotely with the robot, using the same ROS modules. ROS4iOS opens up a rich set of possibilities for HRI, making ROS-compatible code accessible on mobile devices. In future work, we plan to integrate a dialog management system for other assistance scenarios, such as image-based object fetching and delivery.

Smartphones and tablets are now part of our everyday lives, facilitating access to information through intuitive graphical interfaces. They can make great devices to work with robots because they integrate a lot of sensors and reasonable computing power. Using ROS on those devices would facilitate code reuse and integration with existing robotics applications and libraries.