This paper investigates a hybrid solution which combines deep reinforcement learning (RL) and classical trajectory planning for the "following in front" application. Here, an autonomous robot aims to stay ahead of a person as the person freely walks around. Following in front is a challenging problem as the user’s intended trajectory is unknown and needs to be estimated, explicitly or implicitly, by the robot. In addition, the robot needs to find a feasible way to safely navigate ahead of human trajectory. Our deep RL module makes decisions at a high level by implicitly estimates the human trajectory and produces short-term navigational goals to guide the robot. These goals are used by a trajectory planner, which is responsible for low-level execution, to smoothly navigate the robot to the short-term goals, and eventually in front of the user. We employ curriculum learning in the deep RL module to efficiently achieve a high return. Our system outperforms the state-of-the-art in following ahead and is more reliable compared to end-to-end alternatives in both the simulation and real world experiments. In contrast to a pure deep RL approach, we demonstrate zero-shot transfer of the trained policy from simulation to the real world.

We propose a novel scheme that jointly addresses the problems of powering and coordinating a population of mini-robots for collective construction. In our setting, a population of simple mobile robots must push blocks into desired polygonal shapes. Each robot performs only simple phototaxis. Coordination is purely open-loop: a global light field guides and powers the robots. We demonstrate this concept in simulation and explore a series of dynamic light field design strategies that robustly result in assembled shapes including nonconvex polygons.

Fig. 1.Generating predictions of a future for a pedestrian attempting to cross the street. We pick out two key frames from the (a) input sequence and the (b) ground truth sequence, 16 frames apart. Image (c) shows our prediction at the same time instant as the ground truth.We explore prediction of urban pedestrian actions by generating a video future of the traffic scene, and show promising results in classifying pedestrian behaviour before it is observed. We compare several encoder-decoder network models that predict 16 frames (400-600 milliseconds of video) from the preceding 16 frames. Our main contribution is a method for learning a sequence of representations to iteratively transform features learnt from the input to the future. Then we use a binary action classifier network for determining a pedestrian's crossing intent from predicted video. Our results show an average precision of 81%, significantly higher than previous methods. The model with the best classification performance runs for 117 ms on commodity GPU, giving an effective look-ahead of 416 ms.

We demonstrate an autonomous ground robot capable of exploring unknown indoor environments for reconstructing their 2D maps. This problem has been traditionally tackled by geometric heuristics and information theory. More recently, deep learning and reinforcement learning based approaches have been proposed to learn exploration behavior in an end-to-end manner. We present a method that combines the strengths of these different approaches. Specifically, we employ a state-of-the-art generative neural network to predict unknown regions of a partially explored map, and use the prediction to enhance the exploration in an information-theoretic manner. We evaluate our system in simulation using floor plans of real buildings. We also present comparisons with traditional methods which demonstrate the advantage of our method in terms of exploration efficiency. We retain an advantage over end-to-end learned exploration methods in that the robot's behavior is easily explicable in terms of the predicted map.

We expand on previous work for negotiating human-robot navigation contention around doorways to produce a more socially-compliant autonomous robot behaviour. Our goal is to improve the integration of robots navigating in human environments by eliciting human recognition of the robot's right of way. This is achieved by incorporating feedback from a user study of our previous system to create a more communicative, reciprocal, and assertive behaviour. Our contribution includes both the updated behaviour and a new user study that evaluates and compares the system to its predecessor. Results show that participants are more likely to respect the robot's right of way given the new robot behaviour, but their responses also highlight the challenges of socially integrating robots into human spaces.

Image-based modeling techniques [1]-[3] can now generate photo-realistic 3D models from images. But it is up to users to provide high quality images with good coverage and view overlap, which makes the data capturing process tedious and time consuming. We seek to automate data capturing for image-based modeling. The core of our system is an iterative linear method to solve the multi-view stereo (MVS) problem quickly and plan the Next-Best-View (NBV) effectively. Our fast MVS algorithm enables online model reconstruction and quality assessment to determine the NBVs on the fly. We test our system with a toy unmanned aerial vehicle (UAV) in simulated, indoor and outdoor experiments. Results show that our system improves the efficiency of data acquisition and ensures the completeness of the final model.

This paper demonstrates an autonomous mobile robot that follows a walking user while staying ahead of them. Despite several useful applications for autonomous push-carts, this problem has received much less attention than the easier problem of following from behind. In contrast to previous work, we use multi-modal person detection and a human-motion model that considers obstacles to predict the future path of the user. We implement the system with a modular architecture of obstacle mapper, human tracker, human motion model, robot motion planner and robot motion controller. We report on the performance of the robot in real-world experiments. We believe that approaches to this largely overlooked problem could be useful in real industrial, domestic and entertainment applications in the near future.

We present a robust real-time system for simultaneous detection of hands and faces in RGB and gray-scale images, and a novel dataset used for training. Our goal is to provide a robust sensor front-end suitable for real-time human-robot interaction using face-engagement and gestures. Using hand-labelled videos obtained from real human-UAV interaction experiments, we re-trained the YOLOv2 Deep Convolutional Neural Network to detect only hands and faces. This model was then used to automatically label several much larger third-party datasets. After manual correction of these results, we modified and re-trained the model on all this labelled data. We obtain qualitatively good detection results at 60Hz on a commodity GPU: our simultaneous hand-and-face detector gives state of the art accuracy and speed in a hand detection benchmark and competitive results in a face detection benchmark. To demonstrate its effectiveness for human-robot interaction we describe its use as the input to a simple but practical gestural human-UAV interface for entertainment or industrial applications. All software, training and test data are freely available.

We propose and test an autonomous robot behavior for socially-compliant navigation of doorways with both human and robot interlocutors. Building on previous work for “aggressive” interaction between robots to resolve navigation deadlocks in corridors, we demonstrate an “assertive” robot that negotiates right-of-way when faced with a human or other robot. The negotiation is implemented using only motion and common navigation sensors, without explicit message-passing. Our goal is for the correct agent to take priority, as decided both by time-efficiency and as judged subjectively by naive human participants. Our contribution is a practical method for doorway negotiation, and a study of human users' responses to a robot that appears to participate in existing social customs surrounding doors. Our method is evaluated with robot-robot experiments and a human-robot interaction study with nonexpert users.

We present a simple probabilistic framework for multimodal sensor fusion that allows a mobile robot to reliably locate and approach the most promising interaction partner among a group of people, in an uncontrolled environment. Our demonstration integrates three complementary sensor modalities, each of which detects features of nearby people. The output is an occupancy grid approximation of a probability density function over the locations of people that are actively seeking interaction with the robot. We show empirically that simply driving towards the peak of this distribution is sufficient to allow the robot to correctly engage an interested user in a crowd of bystanders.

We report the actions of untrained users when instructed to “ make the robot come to you”. The robot is a generic wheeled mobile robot located 8m away and is driven by the experimenter without the knowledge of the participant. The results show a variety of calls and gestures made to the robot, that changed over time. We observed two distinct behaviour phases: (i) getting the robot's attention; (ii) maintaining the robot's behaviour as it approaches. This data could aid the design of future mobile-robot HRI.

This paper presents a computer vision based system for interaction between multiple humans and multiple robots. Each human can “select” (obtain the undivided attention of) a robot by simply looking directly at it. This extends previous work whereby a single human can select one or more robots from a population. Each robot optimally assigns human identities to tracked faces in its camera view using a local Hungarian algorithm. Then the system finds the global optimal allocation of robot-to-human selections using a second, centralized, Hungarian algorithm. This is the first demonstration of optimal many-to-many robot-selection HRI.

We present the first demonstration of end-to-end far-to-near situated interaction between an uninstrumented human user and an initially distant outdoor autonomous Unmanned Aerial Vehicle (UAV). The user uses an arm-waving gesture as a signal to attract the UAV's attention from a distance. Once this signal is detected, the UAV approaches the user using appearance-based tracking until it is close enough to detect the human's face. Once in this close-range interaction setting, the user is able to use hand gestures to communicate its commands to the UAV. Throughout the interaction, the UAV uses colored-light-based feedback to communicate its intent to the user. We developed this system to work reliably with a low-cost consumer UAV, with only computation off-board. We describe each component of this interaction system, giving details of the depth estimation strategy and the cascade predictive flight controller for approaching the user. We also present experimental results on the performance of the complete system and its individual components.

This paper presents a computer vision based system for interaction between multiple humans and multiple robots. Each human can “select” (obtain the undivided attention of) a robot by simply looking directly at it. This extends previous work whereby a single human can select one or more robots from a population. Each robot optimally assigns human identities to tracked faces in its camera view using a local Hungarian algorithm. The gaze-direction and location of the faces are estimated via vision, and a score for each robot-face pair is assigned. Then the system finds the global optimal allocation of robot-to-human selections using a centralized Hungarian algorithm. A useful feature of this method is that robots can be selected by people they cannot see. This is the first demonstration of optimal many-to-many robot-selection HRI.

We propose a novel method for non-uniform terrain coverage using Unmanned aerial vehicles (UAVs). The existing methods for coverage path planning consider a uniformly interesting target area and hence all the regions are covered with high resolution. However in many real world applications items of interest are not uniformly distributed but form patches of locally high interest. Therefore, with sparse sampling of uninteresting sections of the environment and high resolution sampling of interesting patches, the trajectory of the robot can become shorter. In this paper, we present a novel coverage strategy based on Space-Filling Curves that can accomplish non-uniform coverage of regions in the target area. Simulations and real robot experiments indicate that with the new strategy, travel time / cost of the task can be (almost) always less than a regular ‘lawnmower’ coverage pattern.

We present the first demonstration of establishing mutual attention between an outdoor UAV in autonomous normal flight and an uninstrumented human user. We use the familiar periodic waving gesture as a signal to attract the UAV's attention. The UAV can discriminate this gesture from human walking and running that appears similarly periodic. Once a signaling person is observed and tracked, the UAV acknowledges that the user has its attention by hovering and performing a “wobble” behavior. Both parties are now ready for further interaction. The system works on-board the UAV using a single camera for input and is demonstrated working reliably in real-robot trials.

We present an integrated human-robot interaction system that enables a user to select and command a team of two Unmanned Aerial Vehicles (UAV) using voice, touch, face engagement and hand gestures. This system integrates multiple human [multi]-robot interaction interfaces as well as a navigation and mapping algorithm in a coherent semi-realistic scenario. The task of the UAVs is to explore and map a simulated Mars environment.

We present a multi-modal multi-robot interaction whereby a user can identify an individual or a group of robots using haptic stimuli, and name them using a voice command (e.g."You two are green"). Subsequent commands can be addressed to the same robot(s) by name (e.g. "Green! Take off!"). We demonstrate this as part of a real-world integrated system in which a user commands teams of autonomous robots in a coordinated exploration task.

We present a path planning method for MAVs with vision-only MonoSLAM that generates safe paths to a goal according to the information richness of the environment. The planner runs on top of monocular SLAM and uses the available information about structure of the environment and features visibility to find trajectories that maintain visual contact with feature-rich areas. The MAV continuously re-plans as it explores and updates the feature-points in the map. In real-world experiments we show that our system is able to avoid paths that lead into visually-poor sections of the environment by considering the distribution of visual features. If the same system ignores the availability of visually-informative regions in the planning, it is unable to estimate its state accurately and fails to reach its goal.

Area coverage is one of the compelling applications of UAVs. The existing methods for coverage path planning assume a uniformly interesting target area. However in many real world applications items of interest are not uniformly distributed but form clusters. Here it can be advantageous to only sample regions of interest while skipping uninteresting sections of the environment. In this paper, we present a coverage tree structure that can accommodate non-uniform coverage of regions in the target area. Three strategies are proposed to traverse the coverage tree. Experiments indicate that in some situations our method can cover the interesting regions with about half the travel time / cost of a naive regular `lawnmower' coverage pattern.

We describe a system whereby multiple humans and mobile robots interact robustly using a combination of sensing and signalling modalities. Extending our previous work on selecting an individual robot from a population by face-engagement, we show that reaching toward a robot - a specialization of pointing - can be used to designate a particular robot for subsequent one-on-one interaction. To achieve robust operation despite frequent sensing problems, the robots use three phases of human detection and tracking, and emit audio cues to solicit interaction and guide the behaviour of the human. A series of real-world trials demonstrates the practicality of our approach.

Extending our previous work in real-time vision-based Human Robot Interaction (HRI) with multi-robot systems, we present the first example of creating, modifying and commanding teams of UAVs by an uninstrumented human. To create a team the user focuses attention on an individual robot by simply looking at it, then adds or removes it from the current team with a motion-based hand gesture. Another gesture commands the entire team to begin task execution. Robots communicate among themselves by wireless network to ensure that no more than one robot is focused, and so that the whole team agrees that it has been commanded. Since robots can be added and removed from the team, the system is robust to incorrect additions. A series of trials with two and three very low-cost UAVs and off-board processing demonstrates the practicality of our approach.

We present a multimodal system for creating, modifying and commanding groups of robots from a population. Extending our previous work on selecting an individual robot from a population by face engagement, we show that we can dynamically create groups of a desired number of robots by speaking the number we desire, e.g. “You three”, and looking at the robots we intend to form the group. We evaluate two different methods of detecting which robots are intended by the user, and show that an iterated election performs well in our setting. We also show that teams can be modified by adding and removing individual robots: “And you. Not you”. The success of the system is examined for different spatial configurations of robots with respect to each other and the user to find the proper workspace of selection methods.

We introduce the concept of Maximum Sustainable Yield (MSY) to the context of autonomous robot foraging. MSY is an optimal approach to the problem of maximizing sustainable foraging where the resources harvested are replenished by logistic growth, e.g. living things. Over-harvesting reduces both the instantaneous resource availability and growth rate, and above some threshold will permanently deplete resources. Under-harvesting is sustainable, but fails to maximally exploit the resources. We describe a system model and use it to determine the optimal allocation of robot work to resource-producing `patches'. We give a practical illustration of a troublesome feature of MSY: it is too sensitive for a fixed allocation to be sustainable in practice. We show how to centrally allocate a number of robots to each patch, and then locally adapt the work rate of each robot to achieve sustainable and near-optimal foraging. This is the first study of robot foraging where the robots' activity modifies the productivity and sustainability of the environment.

We present a novel method for a human user to select groups of robots without using any external instruments. We use computer vision techniques to read hand gestures from a user and use the hand gesture information to select single or multiple robots from a population and assign them to a task. To select robots the user simply draws a circle in the air around the robots that the user wants to command. Once the user selects the group of robots, he or she can send them to a location by pointing to a target location.

This video presents a computer vision based system for interaction between a single human and multiple robots. Face contact and motion-based gestures are used as two different non-verbal communication channels; a user first selects a particular robot by simply looking at it, then assigns it a task by waving his or her hand.

We consider the classical task of transporting resources from source to home by a group of autonomous robots. The robots use ant-like trail following to navigate between home and source. This paper studies the effect on global performance of changing the field of view of each robot's trail-following sensor. It is shown that, under certain conditions, a narrow field of view can improve system performance. We argue that the benefit is obtained by selectively degrading the individuals' trail-following ability so that more work space is exploited in parallel, thus decreasing mutual spatial interference. This “worse-is-better” idea may be applicable to other large-scale multi-robot systems.

In this paper we present two task-allocation strategies for a multi-robot transportation system. The first strategy is based on a centralized planner that uses domain knowledge to solve the assignment problem in linear time. In contrast in the second strategy, individual robots make rule-based allocation decisions using only locally obtainable information and single value communication. Both methods are tested and analysed in simulation experiments. We show that the rule-based method performs well but the lack of information has to be paid for with increased energy consumption.

We consider systems of mobile robots that execute a transportation task and periodically recharge from a docking station. The location of the docking station has a considerable effect on task performance. In nonstationary tasks the optimal dock location may vary over the length of the task. In multiple-robot systems, spatial interference between charging and working robots can make it difficult to find an optimal dock location, even in static tasks. We propose a new approach whereby the dock is itself an autonomous robot that attempts to incrementally improve its location. We show simulation results from a simple local controller that adapts to nonstationary tasks and spatial interference, and thus improves overall task performance compared to a static dock.

We describe a simple controller for swarms of foraging robots that reduces mutual spatial interference and adapts to non-uniform resource distributions. We discuss several sources of such non-uniformity, and show that some non-uniform distributions are not well-handled by previously described foraging schemes. Our new method adapts each robot's work site size and location during run time, to reflect the distribution encountered. The controller is very scalable, using only local information and no explicit communication. Simulation studies demonstrate the method's effectiveness.

We investigate the problem of a robot maximizing its long-term average rate of return on work. We present a means to obtain an estimate of the instantaneous rate of return when work is rewarded in discrete atoms, and a method that uses this to recursively maximize the long-term average return when work is available in localized patches, each with locally diminishing returns. We examine a puck-foraging scenario, and test our method in simulation under a variety of conditions. However, the analysis and approach applies to the general case.

We consider the problem of finding a set of trajectories for a heterogeneous group of mobile robots, such that a single service robot can rendezvous with every other member of the team in a prescribed order, and the total travel cost is minimized. We present a simple heuristic controller that runs independently on each robot, yet is guaranteed to achieve an approximation to global solution in bounded time linear in population size. Simulation experiments empirically show that the method finds approximated solutions of near-optimal quality on average.

As part of a program to find methods of reducing spatial interference in multi-robot systems, we propose the interaction grid (IG), a generalization of the occupancy grid that models the spatial distribution of interactions between robots. We describe alternative methods for building interaction grids, first by recording instances of actual robot-robot interaction, and then by a much faster approximation method. The resulting maps of interaction likelihood can subsequently be used to modify the robot's behaviour to avoid interference. For example, we show how to automatically generate an aggression map for input to the aggressive display behaviour we have previously shown to be effective in interference reduction.

We consider the problem of finding an energy-efficient route for a service robot to rendezvous with every member of a heterogeneous team of mobile worker robots. We analyze the general and special cases of the problem, finding it to be at least as hard as the travelling salesman problem. Decomposing the problem into two components: (i) an ordering of robot meetings; and (ii) finding an optimal set of meeting places given an ordering, we present useful solutions to part (ii) only. We propose and compare a discrete algorithm for the restricted meeting location case and two numerical algorithms for the continuous case with weighted Euclidean distance energy cost functions. Anticipating future work, we speculate briefly on suitable ordering heuristics and the need for an integrated method

A fundamental challenge for robotic target-tracking systems is to cope with cases in which the target is not seen for long periods of time. An additional challenge in multiple-robot systems is to coordinate robot activity to best track targets with limited visibility. We describe a novel technique that combines a particle filter target model with a potential field robot controller. Robots are attracted to points sampled from the particle cloud that models the probability distribution over the target's position, subject to environmental constraints. We show how this method can be used as a coordination strategy whereby a team of robots cooperatively minimize the uncertainty in the pose of a tracked target. Simulation results are presented

We examine a team of robots with no centralized control performing a transportation task in which robots frequently interfere with each other, thus impairing overall team's performance. It has previously been shown that stereo-typed robot-robot competitions, inspired by aggressive displays in animals, can be used to effectively reduce interference and improve system performance for this task. We describe an extension to the previous best-performing 'aggression function' to dynamic teams of robots. Experimental results show that the new method provides the best performance yet seen. Further, we examine the effects of interference-reduction methods over a range of population sizes, and we compare the results to a previously suggested theoretical model

In order to guide the design of a new multi-robot system, we seek to evaluate two different designs of audio direction sensor. We have implemented a simple but useful audio propagation simulator as an extension to the stage multi-robot simulator. We use the simulator to evaluate the use of audio signals to improve the performance of a team of robots at a prototypical search task. The results indicate that, for this task, (i) audio can significantly improve team performance, and (ii) binary discrimination of the direction of a sound source (left/right) performs no worse than high-resolution direction information. This result suggests that a simple two-microphone audio system would be useful for our real robots, without advanced signal processing to find sound direction

This paper extends and improves upon our previous work on the use of stereotypical aggressive display behavior to reduce interference in robot teams, and thus improve their overall efficiency. We examine a team of robots with no centralized control performing a transportation task in which robots frequently interfere with each other. The robots must work in the same space, so territorial methods are not appropriate. In our method, when robots come into competition for floor space, each selects an aggression level and the competition is resolved in favor of the more aggressive robot. Our recent work showed that choosing aggression proportional to task investment can produce better overall system performance compared to aggression chosen at random. This paper describes a new technique, local investment, for computing an aggression level that performs better than any previous method and relies only on local sensor data. The method is evaluated in a simulation study, and is then shown to be effective in a real-world robot implementation.

We seek to make robot programming more efficient by developing a standard abstract interface for robot hardware, based on familiar techniques from operating systems and network engineering. This paper describes the application of three well known abstractions, the character device model, the interface/driver model, and the client/server model to this purpose. These abstractions underlie Player/Stage, our open source project for rapid development of robot control systems. One product of this project is the Player Abstract Device Interface (PADI) specification, which defines a set of interfaces that capture the functionality of logically similar sensors and actuators. This specification is the central abstraction that enables Player-based controllers to run unchanged on a variety of real and simulated devices. We propose that PADI could be a starting point for development of a standard platform for robot interfacing, independent of Player, to enable code portability and re-use, while still providing access to the unique capabilities of individual devices.

We describe a team of robots that uses a trail of landmarks to navigate between places of interest. The landmarks are not physical; they are waypoint coordinates generated online by each robot and shared with teammates over the network. Waypoints are specified with reference to features in the world that are relevant to the team's task and common to all robots. Using such task-level features as landmarks avoids the need to sense and name physical landmarks. Using these common landmarks, each robot can transform waypoint coordinates into its local reference frame, avoiding the cost of maintaining a fixed global coordinate system. The algorithm is tested in an experiment in which a team of 4 autonomous mobile robots run in our office building for more than 3 hours, travelling a total of 8.2 km (5.1 miles). Despite significant divergence of their local coordinate systems, they are able to share waypoints, forming and following a common trail between two fixed locations.

We describe localization-space trails (LOST), a method that enables a team of robots to navigate between places of interest in an initially unknown environment using a trail of landmarks. The landmarks are not physical; they are waypoint coordinates generated online by each robot and shared with teammates. Waypoints are specified in each robot's local coordinate system, and contain references to features in the world that are relevant to the team's task and common to all robots. Using these task-level references, robots can share waypoints without maintaining a global coordinate system. The method is tested in a series of real-world multirobot experiments. The results demonstrate that the method: 1) copes with accumulating odometry error; 2) is robust to the failure of individual robots; 3) converges to the best route discovered by any robot in the team. In one experiment, a team of four autonomous mobile robots performs a resource transportation task in our uninstrumented office building. Despite significant divergence of their local coordinate systems, the robots are able to share waypoints, forming and following a common trail between two predetermined locations for more than three hours, traveling a total of 8.2 km (5.1 miles) before running out of power. Designed to scale to large populations, LOST is fully distributed, with low costs in processing, memory, and bandwidth. It combines metric data about the position of features in the world with instructions on how to get from one place to another; producing something between a map and a plan.

Wireless communication is an enabling factor in multiple mobile robot systems. There is significant interaction between robot controllers and communications subsystems. We present a method for evaluating combined robot control/communication strategies for a team of wireless-networked robots performing a resource transportation task. Two alternative controller designs are compared under established communication and radio propagation models. For each we measure the overall performance of the robot team including the cost of communication. The study illustrates how our evaluation tools can be used for designing controllers for robots operating in wireless communication environments.

Successful distributed sensing and control require data to flow effectively between sensors, processors and actuators on single robots, in groups and across the Internet. We propose a mechanism for achieving this flow that we have found to be powerful and easy to use; we call it Player. Player combines an efficient message protocol with a simple device model. It is implemented as a multithreaded TCP socket server that provides transparent network access to a collection of sensors and actuators, often comprising a robot. The socket abstraction enables platform- and language-independent control of these devices, allowing the system designer to use the best tool for the task at hand Player is freely available from http://robotics.usc.edu/player.