In this paper, we address the problem of trajectory planning while simultaneously reacting to the presence of pedestrians for an autonomous car on urban roads. Past work limits jerk, velocities and acceleration for smooth trajectories, without considering reactive behaviors such as responding to pedestrians. Other systems based on collision avoidance, plan paths around obstacles and pedestrians in unstructured environments. In this paper, we present an integrated trajectory generation and tracking system. Our system simultaneously considers both parameter and reactive constraints for smooth trajectory and updates it in real-time. We present a novel online method for planning trajectories to follow a given urban path while honoring traffic regulations such as stop signs at intersections. We update the trajectory to safely avoid pedestrians on the road by slowing down or stopping. Our method has closed form solutions, runs at 20 Hz, and is efficient and reliable for use in online planning. We have confirmed this with a test vehicle and pedestrians with over 100 hours of testing under driverless operation.

Navigating a car at intersections is one of the most challenging parts of urban driving. Successful navigation needs predicting of intention of other traffic participants at the intersection. Such prediction is an important component for both Advanced Driver Assistance Systems (ADAS) and Autonomous Driving (AD) Systems. In this paper, we present a driver intention prediction model for general intersections. Our model incorporates lane-level maps of an intersection and makes a prediction based on past position and movement of the vehicle. We create a real-world dataset of 375 turning tracks at a variety of intersections. We present turn prediction results based on Hidden Markov Model (HMM), Support Vector Machine (SVM), and Dynamic Bayesian Network (DBN). SVM and DBN models give higher accuracy compared to HMM models. We get over 90% turn prediction accuracy 1.6 seconds before the intersection. Our work advances the state of art in ADAS/AD systems with a turn prediction model for general intersections.

In this paper, we address issues in situ-ated language understanding in a rapidly changing environment – a moving car. Specifically, we propose methods for un-derstanding user queries about specific tar-get buildings in their surroundings. Unlike previous studies on physically situated in-teractions such as interaction with mobile robots, the task is very sensitive to tim-ing because the spatial relation between the car and the target is changing while the user is speaking. We collected situated utterances from drivers using our research system, Townsurfer, which is embedded in a real vehicle. Based on this data, we analyze the timing of user queries, spa-tial relationships between the car and tar-gets, head pose of the user, and linguis-tic cues. Optimized on the data, our al-gorithms improved the target identification rate by 24.1 % absolute. 1

Deciding what mix of engine and battery power to use is critical to hybrid vehicles' fuel efficiency. Current solutions consider several factors such as the charge of the battery and how efficient the engine operates at a given speed. Previous research has shown that by taking into account the future power requirements of the vehicle, a more efficient balance of engine vs. battery power can be attained. In this paper, we utilize a probabilistic driving route prediction system, trained using Inverse Reinforcement Learning, to optimize the hybrid control policy. Our approach considers routes that the driver is likely to be taking, computing an optimal mix of engine and battery power. This approach has the potential to increase vehicle power efficiency while not requiring any hardware modification or change in driver behavior. Our method outperforms a standard hybrid control policy, yielding an average of 1.22% fuel savings.

Our goal in this work is to make high level decisions for mobile robots. In particular, given a queue of prioritized object delivery tasks, we wish to find a sequence of actions in real time to accomplish these tasks efficiently. We introduce a novel reinforcement learning algorithm called Smoothed Sarsa that learns a good policy for these delivery tasks by delaying the backup reinforcement step until the uncertainty in the state estimate improves. The state space is modeled by a Dynamic Bayesian Network and updated using a Region-based Particle Filter. We take advantage of the fact that only discrete (topological) representations of entity locations are needed for decision-making, to make the tracking and decision making more efficient. Our experiments show that policy search leads to faster task completion times as well as higher total reward compared to a manually crafted policy. Smoothed Sarsa learns a policy orders of magnitude faster than previous policy search algorithms. We demonstrate our results on the Player/Stage simulator and on the Pioneer robot.

We present an algorithm for creating globally consistent three-dimensional maps from depth fields produced by camera-based range measurement systems. Our approach is specifically suited to dealing with the high noise levels and the large number of outliers often produced by such systems. Range data is filtered to reject outliers within each scan. The point-to-plane variant of ICP is used for local alignment, including weightings that favor nearby points and a novel outlier rejection strategy that increases the robustness for this class of data while eliminating the burden of user-specified thresholds. Global consistency is imposed on cycles by optimally distributing the cyclic discrepancy according to the local fit correlation matrices. The algorithm is demonstrated on a dataset collected by an active unstructured-light space-time stereo vision system.

We propose a vision-based SLAM algorithm incorporating feature descriptors derived from multiple views of a scene, incorporating illumination and viewpoint variations. These descriptors are extracted from video and then applied to the challenging task of wide baseline matching across significant viewpoint changes. The system incorporates a single camera on a mobile robot in an extended Kalman filter framework to develop a 3D map of the environment and determine egomotion. At the same time, the feature descriptors are generated from the video sequence, which can be used to localize the robot when it returns to a mapped location. The kidnapped robot problem is addressed by matching descriptors without any estimate of position, then determining the epipolar geometry with respect to a known position in the map.

In this paper, we present a novel technique for estimating large camera displacement using stereo images. The relative transformation between two stereo image pairs is estimated using a hybrid registration algorithm which combines the robustness of multi-scale feature tracking for large movements and the accuracy of 3D normal flow constraints. Our hybrid technique takes advantage of depth information available from the stereo camera which makes it less sensitive to lighting variations. We tested the accuracy of our hybrid algorithm on real stereo sequences and showed that our technique handles displacements up to 150 cm and rotations up to 20 degrees between images. Our algorithm runs at 6 Hz on a Pentium 4 1.7GHz.