Optical flow estimation is very challenging in situations with transparent or occluded objects. In this work, we address these challenges at the task level by introducing Amodal Optical Flow, which integrates optical flow with amodal perception. Instead of only representing the visible regions, we define amodal optical flow as a multi-layered pixel-level motion field that encompasses both visible and occluded regions of the scene. To facilitate research on this new task, we extend the AmodalSynthDrive dataset to include pixel-level labels for amodal optical flow estimation. We present several strong baselines, along with the Amodal Flow Quality metric to quantify the performance in an interpretable manner. Furthermore, we propose the novel AmodalFlowNet as an initial step toward addressing this task. AmodalFlowNet consists of a transformer-based cost-volume encoder paired with a recurrent transformer decoder which facilitates recurrent hierarchical feature propagation and amodal semantic grounding. We demonstrate the tractability of amodal optical flow in extensive experiments and show its utility for downstream tasks such as panoptic tracking. We make the dataset, code, and trained models publicly available at http://amodal-flow.cs.uni-freiburg.de.

Cooperative systems have finally entered the automotive mass market with the introduction of the 2020 VW Golf. These "Day-1" functions are aimed at driver information and warning only, but the integration of cooperative systems and automated driver assistance is already planned on several levels such as cooperative perception and cooperative driving maneuvers. The introduction of wireless open networks into highly critical systems sets highest demands to safety and security. In this paper we examine several cybersecurity attack vectors on Day-1 and future cooperative systems by applying the methodology used in functional safety. We evaluate attack difficulty (exposure), severity and controllability for a selection of current and next-gen functions. From this analysis, we derive associated risks and thus give recommendations to researchers and engineers. Finally, we simulate a selection of attacks on a platoon and evaluate function behavior and the possibility of critical system malfunction.

Recent years have seen steady advance in automated vehicle research and development. Similarly, communication between vehicles is not only seen as a telematics function anymore, but rather to enable better perception, higher coordination capabilities in groups of automated vehicles or even for directly plan cooperative driving maneuvers. Negotiation and planning for safety critical functions demands a high level of safety, security and robustness to external disturbances. The current state of the art in autonomous vehicles is impressive, but even after the huge amount of work that was invested in previous years, they still struggle with basic driving maneuvers such as in intersections and in lane merge scenarios. Cooperation between vehicles using wireless communication can solve these issues partially, as it improves perception, prediction and most of all planning. In this paper we present a cooperative automated lane merge function, specifically looking at challenging on-ramp merge scenarios. We detail the planning, cooperation and execution levels and show an implementation and testing approach using a 3D simulation environment. We present an evaluation of function performance and network load and discuss tradeoffs in function and network protocol design.

A highly accurate and reliable localization system is one of the keystones for automated driving in underground parking garages. Indoor parking areas pose specific challenges due to the narrow spaces, repetitive architectural layout, low lighting conditions and the unavailability of Global navigation satellite system (GNSS). To solve this challenge, we propose a novel Map Switching Monte Carlo Localization (MSMCL) approach based on a single close-to-production front-mounted LiDAR with a Field of View (FOV) of 110 degrees. To localize the vehicle, we use the correlation of LiDAR measurements with a 2Dreference map, with different switchable representations. The measurement distribution is computed in advance in the Fourier space, which decouples the calculation time from the number of particles. We compare our approach to a custom variant of a state-of-the-art Iterative Closest Point (ICP) algorithm. In an experimental evaluation, the ICP and MSMCL approaches achieve a median angular accuracy of 0.31° and 0.36° resp. as well as a median total position accuracy of 12.0cm and 16.5cmresp. The 95 percentile total positioning accuracy is 35.7cmfor ICP and 38.5cmfor MSMCL. While ICP achieves a higher accuracy, the proposed MSMCL approach can solve the lost-robot problem, i.e. estimate the vehicle position even if no initial guess is available. Both ICP and MSMCL can be operated at real-time with a frequency of at least 5Hz. We validated the localization system by successfully completing multiple fully automated test drives in the Fraunhofer FOKUS parking garage with a prototype Mercedes E-Class W213.

Indoor-Localization using mobile end-user devices such as smartphones remains a highly discussed research topic. Applications range from guidance in large structures, assistance of visually impaired or even localization and guidance in rescue operations. Current approaches are often based on Radio-Frequency strength localization, which show insufficient precision and robustness. In this paper we investigate a magneto-based approach using rotation-invariant features compared with a pre-recorded grid-based map. Furthermore, we show, how this approach can be combined with relative localization using step recognition from inertial measurements. We test and evaluate both systems in simulation as well as real-world tests. We show, that a sub-meter precision localization accuracy can be reached using our magneto-intertial approach.

Automated vehicles rely on the accurate and robust detection of the drivable area, often classified into free space, road area and lane information. Most current approaches use monocular or stereo cameras to detect these. However, LiDAR sensors are becoming more common and offer unique properties for road area detection such as precision and robustness to weather conditions. We therefore propose two approaches for a pixel-wise semantic binary segmentation of the road area based on a modified U-Net Fully Convolutional Network (FCN) architecture. The first approach UView-Cam employs a single camera image, whereas the second approach UGrid-Fused incorporates a early fusion of LiDAR and camera data into a multi-dimensional occupation grid representation as FCN input. The fusion of camera and LiDAR allows for efficient and robust leverage of individual sensor properties in a single FCN. For the training of UView-Cam, multiple publicly available datasets of street environments are used, while the UGrid-Fused is trained with the KITTI dataset. In the KITTI Road/Lane Detection benchmark, the proposed networks reach a MaxF score of 94.23% and 93.81% respectively. Both approaches achieve realtime performance with a detection rate of about 10 Hz.

Modern driver assistance systems enable a variety of use cases which rely on accurate localization information of all traffic participants. Due to the unavailability of satellite-based localization, the use of infrastructure cameras is a promising alternative in indoor spaces such as parking garages. This paper presents a parking management system which extends the previous work of the eValet system with a low-complexity tracking functionality on compressed video bitstreams (compressed-domain tracking). The advantages of this approach include the improved robustness to partial occlusions as well as a resource-efficient processing of compressed video bit-streams. We have separated the tasks into different modules which are integrated into a comprehensive architecture. The demonstrator setup includes a 2D visualizer illustrating the operation of the algorithms on a single camera stream and a 3D visualizer displaying the abstract object detections in a global reference frame.

Cooperative driver assistance systems (CoDAS) allow automated vehicles to perform cooperative driving maneuvers using vehicle to vehicle communication (V2V). For planning, negotiation and execution, these functions require precise localization of the vehicle itself and of other vehicles in the surroundings. Most vehicles today are equipped with Global Navigation Satellite System receivers (GNSS). However GNSS are prone to errors up to several meters, far exceeding CoDAS requirements. Yet, most of these errors are similar for vehicles in close proximity as they are caused by the satellite clock or atmospheric effects. In this paper we show how these characteristics can be used to improve absolute position of connected cars fusing GNSS and map data. Map matching is used to determine the GNSS offset of vehicles that share their position through V2V. The lateral error correction of a map matched vehicle that drives in perpendicular direction to the ego vehicle at an intersection is converted to a longitudinal error correction for the ego vehicle. We present an evaluation of this approach in simulation environment and real-world test drives using a LiDAR-based ground truth. We show that the cooperative positioning can reduce the mean GPS error below one meter.

Some of the most advanced functions in automated vehicles are in the parking domain, since it provides structured environments and slow speeds. As most parking garages deny GNSS reception, vehicles rely on dead reckoning or simple path-finding algorithms. To this regard, we have developed an infrastructure-based positioning system using cameras. In this paper, we present a novel approach on how to utilize dedicated short-range communication (often referred to as "Car-2-X communication" (C2X)) to transmit external positioning to vehicles with low latency. A session-based distributed state machine protocol for automated driving is used to ensure synchronicity between vehicle and infrastructure.

Modern vehicles are equipped with numerous driver assistance and telematics functions, such as Turn-by-Turn navigation. Most of these systems rely on precise positioning of the vehicle. While Global Navigation Satellite Systems (GNSS) are available outdoors, these systems fail in indoor environments such as a car-park or a tunnel. Alternatively, the vehicle can localize itself with landmark-based positioning and internal car sensors, yet this is not only costly but also requires precise knowledge of the enclosed area. Instead, our approach is to use infrastructure-based positioning. Here, we utilize off-the shelf cameras mounted in the car-park and Vehicle-to-Infrastructure Communication to allow all vehicles to obtain an indoor position given from an infrastructure-based localization service. Our approach uses a Convolutional Neural Network (CNN) with Deep Learning to identify and localize vehicles in a car-park. We thus enable position-based Driver Assistance Systems (DAS) and telematics in an underground facility. We compare the novel Deep Learning classifier to a conventional classifier using Haar-like features.

The introduction of dedicated short-range Vehicle- to-Vehicle communication (DSRC) enables the next step in advanced driver assistance systems (ADAS) - the cooperative automated driver assistance systems (CoDAS). Combined with automated functions and even autonomous driving, a host of novel functions become feasible. Some of these - such as platooning- have been in research for decades, while others are not tackled yet. In this paper we give an overview on research on automated cooperative functions, survey conceivable functions and present a way to classify them.

Vehicle-to-Vehicle communication has recently passed from a research topic to the subject of Field Operational Testing (FOT) and pilot deployment. Current state-of-the art Car-to-Car and Car-to-Infrastructure (C2X) functions will however only inform the driver, not interfere in actual vehicle operation. A logical next step after initial deployment will be sensor fusion to enhance actively intervening Advanced Driver Assistance Systems (ADAS) with information received over C2X. As the penetration rate of equipped vehicles increases over time, higher-level CoDAS functions become feasible. In this paper we present a concept for a cooperative active blind spot assistant (CABSA) as an exemplary function of these novel Cooperative Driver Assistance Systems. As the CABSA function improves an existing ADAS function, no negative effects are observed in low-penetration scenarios. The function was implemented with messages adhering European Telecommunications Standards Institute (ETSI) standards. Simulations and real-life tests show that the increase in operation range significantly expands the vehicle speed envelope upon which the system can prevent accidents compared to conventional blind-spot assistance.

In this paper we present an indoor micronavigation system for enclosed parking garages. It builds on car-to-infrastructure communication to provide layout information of the car park, the coordinates of the destination parking lot, as well as external positioning information to vehicles. In our approach we use customary network video cameras to detect and locate vehicles within the car park. Once a vehicle is detected, the system correlates the position of the vehicle to the car park layout and transmits this information to the appropriate vehicle to substitute the internal positioning system. With this information the vehicle is guided from the car park entrance to a destination parking lot.

The paper presents an association rule mining based learning approach for multimodal user interface adaptation in mobile environments. High-level knowledge about user preferences in multimodal interaction is inferred, using data mining techniques based on context parameters of the environment. The current approach facilitates automatic selection of multimodality capable interaction devices and their according rendering facilities for media output streams. An overview of the learning subsystem being part of the distributed communication sphere (DCS) management architecture, proposed within the EU IST-027617 project SPICE, will be introduced. Further the design of the learning approach will be discussed, including the definition and adaptation of snapshot data based on environment parameters. Frequent snapshots form the basis for learning, and therefore for the described association rule mining algorithm. Theoretical simulation results are presented and an outlook towards next research steps is given.