Vision for autonomous driving is a uniquely challenging problem: the number of tasks required for full scene understanding is large and diverse; the quality requirements on each task are stringent due to the safety-critical nature of the application; and the latency budget is limited, requiring real-time solutions. In this work we address these challenges with QuadroNet, a one-shot network that jointly produces four outputs: 2D detections, instance segmentation, semantic segmentation, and monocular depth estimates in real-time (>60fps) on consumer-grade GPU hardware. On a challenging real-world autonomous driving dataset, we demonstrate an increase of+2.4% mAP for detection, +3.15% mIoU for semantic segmentation, +5.05% [email protected] for instance segmentation and +1.36% in δ <; 1.25 for depth prediction over a baseline approach. We also compare our work against other multi-task learning approaches on Cityscapes and demonstrate state-of-the-art results.

We describe an interactive system featuring fluid-driven animation that responds to moving objects. Our system includes a GPU-accelerated Eulerian fluid solver that is suited for real-time use because it is unconditionally stable, takes constant calculation time per frame, and provides good visual fidelity. We dynamically translate the fluid simulation domain to track a user-controlled object. The fluid motion is visualized via its effects on particles which respond to the calculated fluid velocity field, but which are not constrained to stay within the bounds of the simulation domain. As particles leave the simulation domain, they seamlessly transition to purely particle-based motion, obscuring the point at which the fluid simulation ends. We additionally describe a hardware-accelerated volume rendering system that treats the particles as participating media and can render effects such as smoke, dust, or mist. Taken together, these components can be used to add fluid-driven effects to an interactive system without enforcing constraints on user motion, and without visual artifacts resulting from the finite extents of Eulerian fluid simulation methods.

It is usually difficult to resolve the fine details of turbulent flows, especially when targeting real-time applications. We present a novel, scalable turbulence method that uses a realistic energy model and an efficient particle representation that allows for the accurate and robust simulation of small-scale detail. We compute transport of turbulent energy using a complete two-equation k -ε model with accurate production terms that allows us to capture anisotropic turbulence effects, which integrate smoothly into the base flow. We only require a very low grid resolution to resolve the underlying base flow. As we offload complexity from the fluid solver to the particle system, we can control the detail of the simulation easily by adjusting the number of particles, without changing the large scale behavior. In addition, no computations are wasted on areas that are not visible. We demonstrate that due to the design of our algorithm it is highly suitable for massively parallel architectures, and is able to generate detailed turbulent simulations with millions of particles at high framerates.

Localization from sensor measurements is a fundamental task for navigation. Particle filters are among the most promising candidates to provide a robust and real-time solution to the localization problem. They instantiate the localization problem as a Bayesian altering problem and approximate the posterior density over location by a weighted sample set. In this paper, we introduce map-based priors for localization, using the semantic information available in maps to bias the motion model toward areas of higher probability. We, show that such priors, under a particular assumption, can easily be incorporated in the particle filter by means of a pseudo likelihood. The resulting filter is more reliable and more accurate. We show experimental results on a GPS based outdoor people tracker that illustrate the approach and highlight its potential.

Most of the work in head-pose tracking has concentrated on single-camera systems with a relatively small field of view which have limited accuracy because features are only observed in a single viewing direction. We present a multicamera pose tracker that handles an arbitrary configuration of cameras rigidly fixed to the observer's head. By using multiple cameras, we increase the robustness and accuracy by which a 6-DOF pose is tracked. However, in a multicamera rig setting, earlier methods for determining the unknown pose from three world-to-camera correspondences are no longer applicable. We present a RANSAC (M. Fischler and R. Bolles, 1981) based method that handles multicamera rigs by using a fast nonlinear minimization step in each RANSAC round.