Predicting future events accurately is a task of great importance for autonomous vehicles. In this work we focus on lane change events. For this, we propose a novel attention mechanism on top of recurrent neural networks for the prediction task, which improves performance and yields more interpretable models. As critical corner cases are often not considered and reflected in traditional prediction metrics, we additionally introduce a new scenario-based evaluation scheme, which we posit be considered for further maneuver prediction works. Prediction and planning tasks often are correlated, usually sharing input representations and differing in expected outputs and their subsequent consideration. Here, we detail a supporting layer for planning tasks, which analyzes situations w.r.t. their suitability for lane changes and can serve as decision-making support for any planning algorithm. Exploitation of similarities between this task and the aforementioned prediction problem further improves performance of the prediction task, as well as labelling quality of the assessment task. Additionally, we extend our evaluation to urban scenarios, showcasing the generalizability of our proposed prediction models.

We introduce an extension of the Dubins Traveling Salesman Problem with Neighborhoods into the 3D space in which a fixed-wing aerial vehicle is requested to visit a set of target regions while the vehicle motion constraints are satisfied, i.e., the minimum turning radius and maximum climb and dive angles. The primary challenge is to address both the combinatorial optimization part of finding the sequence of target visits and the continuous optimization part of the final trajectory determination. Due to its high complexity, we propose to address both parts of the problem separately by a decoupled approach in which the sequence is determined by a new distance function designed explicitly for the utilized 3D Dubins Airplane model. The final trajectory is then found by a local optimization which improves the solution quality. The proposed approach provides significantly better solutions than using Euclidean distance in the sequencing part of the problem. Moreover, the found solutions are of the competitive quality to the sampling-based algorithm while its computational requirements are about two orders of magnitude lower.

One of the greatest challenges towards fully autonomous cars is the understanding of complex and dynamic scenes. Such understanding is needed for planning of maneuvers, especially those that are particularly frequent such as lane changes. While in recent years advanced driver-assistance systems have made driving safer and more comfortable, these have mostly focused on car following scenarios, and less on maneuvers involving lane changes. In this work we propose a situation assessment algorithm for classifying driving situations with respect to their suitability for lane changing. For this, we propose a deep learning architecture based on a Bidirectional Recurrent Neural Network, which uses Long Short-Term Memory units, and integrates a prediction component in the form of the Intelligent Driver Model. We prove the feasibility of our algorithm on the publicly available NGSIM datasets, where we outperform existing methods.

In this paper, we address the problem of human body pose estimation from depth data. Previous works Based on random forests relied either on a classification strategy to infer the different body parts or on a regression approach to predict directly the joint positions. To permit the inference of very generic poses, those approaches did not consider additional information during the learning phase, e.g. the performed activity. In the present work, we introduce a novel approach to integrate additional information at training time that actually improves the pose prediction during the testing. Our main contribution is a multi-task forest that aims at solving a joint regression-classification task: each foreground pixel from a depth image is associated to its relative displacements to the 3D joint positions as well as the activity class. Integrating activity information in the objective function during forest training permits a better partitioning of the 3D pose space that leads to a better modelling of the posterior. Thereby, our approach provides an improved pose prediction, and as a by-product, can give an estimate of the performed activity. We performed experiments on a dataset performed by 10 people associated with the ground truth body poses from a motion capture system. To demonstrate the benefits of our approach, poses are divided into 10 different activities for the training phase. Results on this dataset show that our multi-task forest provides improved human pose estimation compared to a pure regression forest approach.

In this paper, we introduce a flexible framework that can facilitate the definition of 3D gesture-based interfaces. Highlighting the need for context awareness in complex domains, such as the operating room, we argue how the proposed architecture can overcome integration challenges. Through a real-life scenario, an intra-operative medical image viewer, we demonstrate how the proposed framework can be used in practice to define user interfaces in collaborative environments, where the behavior and the system response can be adapted based on the current workflow stage and individual user requirements. Finally, we demonstrate how the defined interface can be manipulated using a high-level visual programming interface. The extensibility of the proposed architecture makes it applicable to a wide range of scenarios.

Computerized medical systems play a vital role in the operating room, however, sterility requirements and interventional workflow often make interaction with these devices challenging for surgeons. Typical solutions, such as delegating physical control of keyboard and mouse to assistants, add an undesirable level of indirection. We present a touchless, gesture-based interaction framework for the operating room that lets surgeons define a personalized set of gestures for controlling arbitrary medical computerized systems. Instead of using cameras for capturing gestures, we rely on a few wireless inertial sensors, placed on the arms of the surgeon, eliminating the dependence on illumination and line-of-sight. A discriminative gesture recognition approach based on kernel regression allows us to simultaneously classify performed gestures and to track the relative spatial pose within each gesture, giving surgeons fine-grained control of continuous parameters. An extensible software architecture enables a dynamic association of learned gestures to arbitrary intraoperative computerized systems. Our experiments illustrate the performance of our approach and encourage its practical applicability.

In this paper, we present a method for human full-body pose estimation from Time-of-Flight (ToF) camera images. Our approach consists of robustly detecting anatomical landmarks in the 3D data and fitting a skeleton body model using constrained inverse kinematics. Instead of relying on appearance-based features for interest point detection that can vary strongly with illumination and pose changes, we build upon a graph-based representation of the ToF depth data that allows us to measure geodesic distances between body parts. As these distances do not change with body movement, we are able to localize anatomical landmarks independent of pose. For differentiation of body parts that occlude each other, we employ motion information, obtained from the optical flow between subsequent ToF intensity images. We provide a qualitative and quantitative evaluation of our pose tracking method on ToF sequences containing movements of varying complexity.

In this paper, we propose a method for detection and tracking of multiple planes in sequences of Time of Flight (ToF) depth images. Our approach extends the recent J-linkage algorithm for estimation of multiple model instances in noisy data to tracking. Instead of randomly selecting plane hypotheses in every image, we propagate plane hypotheses through the sequence of images, resulting in a significant reduction of computational load in every frame. We also introduce a multi-pass scheme that allows detecting and tracking planes of varying spatial extent along with their boundaries. Our qualitative and quantitative evaluation shows that the proposed method can robustly detect planes and consistently track the hypotheses through sequences of ToF images.

International audience

We propose a novel numerical approach for solving the free-form deformable registration problem. The central idea is to utilize the well understood techniques from variational deformable registration problems. We demonstrate that it is possible to formulate the free-form deformable registration problem as the optimization of an energy functional as in the dense deformation case. This energy functional possesses image distance and regularization terms, which are both functions of the free-form deformation control points. We then setup a semi-backward (implicit) partial differential equation that optimizes the established energy functional. In addition to being mathematically justified, this approach provides both accuracy and speed. Our evaluation on synthetic, real, two dimensional, and three dimensional data demonstrates accuracy and computational effectiveness.