Developing software for large scale off-road robot applications is challenging and tedious due to cost, logistics, and rigor of field testing. High-fidelity sensor-realistic simulation can speed up the development process for perception and state estimation algorithms. We focus on Lidar simulation for robots operating in off-road environments. Lidars are integral sensors for robots, and Lidar simulation for off-road environments is particularly challenging due to the way Lidar rays interact with natural terrain such as vegetation. A hybrid geometric terrain representation has been shown to model Lidar observations well [1]. However, previous work has only been able to simulate a single, fixed scene, and the entire scene had to be precisely surveyed. In this work, we add semantic information to the hybrid geometric model. This allows us to extract terrain primitives, such as trees and shrubs, from data logs. Our approach uses these primitives to compose arbitrary scenes for Lidar simulation. We evaluate our simulator on a real-world environment of interest, and show that primitives derived using our approach generalize to new scenes.

Traditional control and planning algorithms for wheeled mobile robots (WMR) either totally ignore or make simplifying assumptions about the effects of wheel slip on the motion. While this approach works reasonably well in practice on benign terrain, it fails very quickly when the WMR is deployed in terrain that induces significant wheel slip. We contribute a novel control framework that predictively corrects for the wheel slip to effectively minimize path following errors. Our framework, the Receding Horizon Model Predictive Path Follower (RHMPPF), specifically addresses the problem of path following in challenging environments where the wheel slip substantially affects the vehicle mobility. We formulate the solution to the problem as an optimal controller that utilizes a slip-aware model predictive component to effectively correct the controls generated by a strictly geometric pure-pursuit path follower. We present extensive experimental validation of our approach using a simulated 6-wheel skid-steered robot in a high-fidelity data-driven simulator, and on a real 4-wheel skid-steered robot. Our results show substantial improvement in the path following performance in both simulation and real world experiments.

This workshop is the fourth in the series and continued to build upon the work carried out at the previous iterations of the International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering, which were held at ICSE in 2012, 2013 and 2014. RAISE 2015 brought together researchers and practitioners from the artificial intelligence (AI) and software engineering (SE) disciplines to build on the interdis- ciplinary synergies that exist and to stimulate further interaction across these disciplines. Mutually beneficial characteristics have appeared in the past few decades and are still evolving due to new challenges and technological advances. Hence, the question that motivates and drives the RAISE Workshop series is: "Are SE and AI researchers ignoring important insights from AI and SE?". To pursue this question, RAISE'15 explored not only the application of AI techniques to SE problems but also the application of SE techniques to AI problems. RAISE not only strengthens the AI- and-SE community but also continues to develop a roadmap of strategic research directions for AI and SE.

A human detection algorithm running on an indoor mobile robot has to address challenges including occlusions due to cluttered environments, changing backgrounds due to the robot's motion, and limited on-board computational resources. We introduce a fast human detection algorithm for mobile robots equipped with depth cameras. First, we segment the raw depth image using a graph-based segmentation algorithm. Next, we apply a set of parameterized heuristics to filter and merge the segmented regions to obtain a set of candidates. Finally, we compute a Histogram of Oriented Depth (HOD) descriptor for each candidate, and test for human presence with a linear SVM. We experimentally evaluate our approach on a publicly available dataset of humans in an open area as well as our own dataset of humans in a cluttered cafe environment. Our algorithm performs comparably well on a single CPU core against another HOD-based algorithm that runs on a GPU even when the number of training examples is decreased by half. We discuss the impact of the number of training examples on performance, and demonstrate that our approach is able to detect humans in different postures (e.g. standing, walking, sitting) and with occlusions.

Joint attention, which is the ability of coordination of a common point of reference with the communicating party, emerges as a key factor in various interaction scenarios. This paper presents an image-based method for establishing joint attention between an experimenter and a robot. The precise analysis of the experimenter's eye region requires stability and high-resolution image acquisition, which is not always available. We investigate regression-based interpolation of the gaze direction from the head pose of the experimenter, which is easier to track. Gaussian process regression and neural networks are contrasted to interpolate the gaze direction. Then, we combine gaze interpolation with image-based saliency to improve the target point estimates and test three different saliency schemes. We demonstrate the proposed method on a human-robot interaction scenario. Cross-subject evaluations, as well as experiments under adverse conditions (such as dimmed or artificial illumination or motion blur), show that our method generalizes well and achieves rapid gaze estimation for establishing joint attention.

Computationally efficient task execution is very important for autonomous mobile robots endowed with limited on-board computational capabilities. Most robot control approaches assume fixed state and action representations, and use a single algorithm to map states to actions. However, not all instances of a given task require equally complex algorithms and equally detailed representations. The main motivation for this work is a desire to reduce the computational footprint of performing a task by allowing the robot to run simpler algorithms whenever possible, and resort to more complex algorithms only when needed. We contribute the Multi-Resolution Task Execution (MRTE) algorithm that utilizes human feedback to learn a mapping from a given state to an appropriate detail resolution consisting of a state and action representation, and an algorithm. We then present Model Plus Correction (M+C), an algorithm that complements an existing robot controller with corrective human feedback to further improve the task execution performance. Finally, we introduce Multi-Resolution Model Plus Correction (MRM+C) as a combination of MRTE and M+C. We provide formal definitions of MRTE, M+C, and MRM+C, showing how they relate to general robot control problem and Learning from Demonstration (LfD) methods. We present detailed experimental results demonstrating the effectiveness of proposed methods on a simulated goal-directed humanoid obstacle avoidance task.

In this video we briefly illustrate the progress and contributions made with our mobile, indoor, service robots CoBots (Collaborative Robots), since their creation in 2009. Many researchers, present authors included, aim for autonomous mobile robots that robustly perform service tasks for humans in our indoor environments. The efforts towards this goal have been numerous and successful, and we build upon them. However, there are clearly many research challenges remaining until we can experience intelligent mobile robots that are fully functional and capable in our human environments.

Developing a robust, flexible, closed-loop walking algorithm for a humanoid robot is a challenging task due to the complex dynamics of the general biped walk. Common analytical approaches to biped walk use simplified models of the physical reality. Such approaches are partially successful as they lead to failures of the robot walk in terms of unavoidable falls. Instead of further refining the analytical models, in this work we investigate the use of human corrective demonstrations, as we realize that a human can visually detect when the robot may be falling. We contribute a two-phase biped walk learning approach, which we experiment on the Aldebaran NAO humanoid robot. In the first phase, the robot walks following an analytical simplified walk algorithm, which is used as a black box, and we identify and save a walk cycle as joint motion commands. We then show how the robot can repeatedly and successfully play back the recorded motion cycle, even if in open-loop. In the second phase, we create a closed-loop walk by modifying the recorded walk cycle to respond to sensory data. The algorithm learns joint movement corrections to the open-loop walk based on the corrective feedback provided by a human, and on the sensory data, while walking autonomously. In our experimental results, we show that the learned closed-loop walking policy outperforms a hand-tuned closed-loop policy and the open-loop playback walk, in terms of the distance traveled by the robot without falling.