For autonomous driving, knowledge about the current environment and especially the driveable lanes is of utmost importance. Currently this information is often extracted from meticulously (hand-)crafted offline high-definition maps, restricting the operation of autonomous vehicles to few well-mapped areas and making it vulnerable to temporary or permanent environment changes. This paper addresses the issues of map-based road models by building the road model solely from online sensor measurements. Based on Dempster-Shafer theory and a novel frame of discernment, sensor measurements, such as lane markings, semantic segmentation of drivable and non-drivable areas and the trajectories of other observed traffic participants are fused into semantic grids. Geometrical lane information is extracted from these grids via an iterative path-planning method. The proposed approach is evaluated on real measurement data from German highways and urban areas.

In this paper, a method based on vector fields for the navigation of autonomous cars is developed. Vector fields-used to generate the desired heading angle of a vehicle toward a specified road lane—attract the car to the desired path and prevent the car from colliding with obstacles. Also, a control law is developed to define the velocity direction and the desired steering angle based on the angle between the car and the vector field. The efficacy of the proposed approach is investigated through several simulations and lab experimental tests.

Today's car traffic is dominated by human drivers. Autonomous cars must comprehend the behavior of human drivers in order to fit in current daily traffic scenarios. To achieve this goal, analysis of the behavior of other traffic participants is necessary. In this paper we present a system to record, store, and analyze the movements of other traffic participants with an autonomous car and evaluate traffic maps, which are obtained from real world experiments. The evaluation shows that the maps cover more than 80% of the driveable area with a precision of 80 to 90%. Additionally, we present the results of a traffic behavior change detection heuristic, which can detect anomalous traffic conditions.

We propose a trajectory planning approach for autonomous vehicles in highly dynamic traffic scenarios, using elastic bands to follow the observed trajectories of other vehicles. The focus of this paper is on the initialization of the elastic band. The proposed method does not rely on a map. We tested our method using recorded urban traffic data. The results show that the presented approach is valid and the proposed initialization process is clearly superior to naive initialization.

In the course of the development and integration of the autonomous driving the knowledge about the current environment and especially the road is one of the basic requirements to fulfill the automated driving task. This information is often extracted from a precise map provided in the vehicle. Therefore the road course and the individual lanes are known in advance by using the map and a suitable ego position estimation. However, in some situations such a map may be invalid and therefore unusable. This can be a driving area which was never mapped or regions where the map is outdated because the environment has changed. The following paper addresses the problem of building a road model without using a map by only fusing measurements from different sensors mounted on the ego vehicle. As sensor measurements various information like lane markings painted on the ground, the position of other cars or occupancy grids can be used. They are transformed into a grid-based model and a geometrical description is extracted out of this model by the use of a novel path-planning based method. The proposed approach was tested with a vehicle equipped with sensors and real measurement data from German highways.

In this paper we propose a trajectory planning approach for autonomous vehicles in highly dynamic traffic scenarios, capable of exploiting the observed trajectories of other vehicles. For this purpose, we introduce a novel variant of the timed elastic bands (TEB) approach by using fixed time intervals and a flexible goal position. We tested our method with a simulated merge-into-traffic scenario and compare it to a reference TEB implementation with focus on the impact of important parameters and the stability of planned trajectories. The results show that our method is an improvement over TEB in terms trajectory smoothness and stability.

Even for a human driver, urban intersections represent probably the most difficult scenarios, in which the driver could be overloaded by understanding the traffic rules, predicting the intention of other objects, etc. The complexity of these scenarios makes the task of automated driving at intersections a very difficult challenge. Thus, we propose an approach that aims to reduce the complexity of the scenario interpretation by breaking down the problem into a set of primary situations linked over time. Based on the combination of four primary situations, the scenario interpretation should enable the corresponding planning that guides the ego vehicle along a driving corridor.

This paper presents a system which constantly monitors the level of attention of a driver in traffic. The vehicle is instrumented and can identify the state of traffic-lights, as well as obstacles on the road. If the driver is inattentive and fails to recognize a threat, the assistance system produces a warning. Therefore, the system helps the driver to focus on crucial traffic situations. Our system consists of three components: computer vision detection of traffic-lights and other traffic participants, an eye tracking device used also for head localization, and finally, a human machine interface consisting of a head-up display and an acoustic module used to provide warnings to the driver. The orientation of the driver's head is detected using fiducial markers visible in video frames. We describe how the system was integrated using an autonomous car as experimental ADAS platform.

Localization is a key capability for autonomous vehicles especially in urban scenarios. We propose the use of pole-like landmarks as primary features in these environments, as they are distinct, long-term stable and can be detected reliably with a stereo camera system. Furthermore, the resulting map representation is memory efficient, allowing for easy storage and on-line updates. The localization is performed in real-time by a stereo camera system as a main sensor, using vehicle odometry and an off-the-shelf GPS as secondary information sources. Localization is performed by a particle filter approach, coupled with an Kalman filter for robustness and sensor fusion. This leads to a lateral accuracy below 20 cm in various urban test areas. The system has been included in our autonomous test vehicle and successfully demonstrated the full loop from mapping to autonomous driving.

We propose a novel approach for automated vehicle motion planning systems that introduces the likelihood of an information gain at future positions to trajectory optimization. In the same way as human drivers, computer controlled vehicles have to be fully aware of their surroundings and the current driving situation. Even though automated cars have a full 360 degrees field of view through sensor data fusion, objects can be hidden behind other obstacles. We optimize the vehicle's future pose (position and orientation) on the road and within the traffic stream, so that it can perceive as much as possible while fulfilling other constraints related to the overall safety or driving comfort. Our results show that perception benefits from maximizing the entropy in areas of interest (EAI) over field of view (FOV). The computation of an EAI is expensive and achieved by using an optimized algorithm for modern GPGPUs.

In this paper we present an approach for building a graph of drivable paths from the reconstructed trajectories of vehicles detected by lidar and radar sensors mounted in an autonomous car. The perceived objects are tracked, and their trajectories are merged, clustered and labeled with meta information. A graph of the underlying road infrastructure can be generated with this information. We report on the results of testing the validity and accuracy of the method. The generated path graph can be used either to update high precision maps or for generating local temporary maps, both of them useful for autonomous driving.

We have built a display wall system with four liquid crystal displays to provide a very large visual area for university lectures. A program was developed to transform the display wall into an electronic blackboard. It has been used in a long-term classroom practice. Based on the research results, we demonstrate the benefits of the digital multi-display system for classroom learning.

This paper presents a sampling-based planning method considering motion uncertainty to generate more human-like driving paths for automated vehicles. Given information in the form of a small set of rules and driving heuristics the planning system optimizes trajectories in a seven dimensional state space. In a post-processing step a set of candidates is evaluated considering the uncertainty of the vehicles motion executing the given trajectory using a Linear-Quadratic Gaussian (LQG). This addresses the problem of indecisive planning behavior in case the optimal solution is unlikely to be followed precisely. The results of our experiments show that the mobile graphics processing unit (GPU) technology can be used as an enabler for real-time applications of computationally expensive planning approaches.

The information about the road course and individual lanes is an important requirement in driver assistance systems and for automated driving applications. It is often stored in a highly accurate offline map so that the road and the lanes are known in advance. However, there exist situations where an offline map can become unusable or invalid. This paper presents a novel approach for a road model estimation solely based on online measurements from sensors mounted on the ego vehicle. It combines perception data like detected lane markings, the movement history of dynamic objects in the vehicle's environment and detected road boundaries into a grid-based road model. This approach allows for an estimation of the road model even when one source of information is not available and offers a redundant source of information about the road, which is necessary in critical applications such as automated driving. The presented approach was tested and evaluated with a prototype vehicle and real sensor data from German highway scenarios.

For a vehicle driving safe inside a parking garage autonomously, it is necessary to build a map with its surroundings and also to localize itself within this map. This is known as Simultaneous Localization And Mapping (SLAM). To enable the vehicle to drive autonomously to an assigned parking slot, a parking area, or the exit, the vehicle also needs knowledge about the whole map of the parking garage. This map only contains static elements of the parking garage. Variable elements are not known to the parking garage and therefore are not contained in this static map. In order to reach a target, the vehicle needs to localize itself with respect to this static map. In this contribution the use of such a static map is proposed to support SLAM. This enables SLAM to determine poses related to a static map. Also the performance of SLAM is improved.

Semi-Global Matching (SGM) is widely used for real-time stereo vision in the automotive context. Despite its popularity, only implementations using reconfigurable hardware (FPGA) or graphics hardware (GPU) achieve high enough frame rates for intelligent vehicles. Existing real-time implementations for general purpose PCs use image and disparity sub-sampling at the expense of matching quality. We study methods to improve the efficiency of SGM on general purpose PCs, through fine grained parallelization and usage of multiple cores. The different approaches are evaluated on the KITTI benchmark, which provides real imagery with LIDAR ground truth. The system is able to compute disparity maps of VGA image pairs with a disparity range of 128 values at more than 16 Hz. The approach is scalable to the number of available cores and portable to embedded processors.

In this article, we propose an objective method to evaluate stroke-width measures. With this aim, we discuss the relevance of features based on the stroke width for document analysis. Then, we point out that most of the consulted references have a vague definition of stroke width. Because of this, we propose a formal definition of the stroke-width and remark the linearity of the stroke-width as an important property. Inspired by these ideas, we propose a measure together with a dataset to evaluate the linearity of the measurements of the stroke width and conduct an evaluation for seven well-known stroke-width methods. Our experiments have interesting results, like the fact that the most popular method is the one with the worst performance and that the best method is the easiest to implement. We hope that our objective evaluation assists further authors to choose suitable stroke-width methods for their applications.

This paper describes the extension of an existing commercial lane detection system for marked roads by the detection and classification of soft shoulders, curbs and guardrails. Available sensors are a front directed gray scale camera and the CAN-bus data. To accomplish our goals we use two complementing methods. The first method is meant for localizing the border using a texture based area classification. We use Local Binary Patterns for their quality as well as their ability for being realized on an embedded system and a neural network as classifier. The second method is a kind of structure from motion for identification of raised structures like guardrails. The algorithm tracks Harris features using the Lucas and Kanade tracker and extracts 3D information out of it. The whole system works in real time and achieves an availability of about 80% with a detection rate of 94% on a test database, which includes roads under almost all weather conditions throughout a number of countries.

We attempt to evaluate the efficacy of six unsupervised evaluation method to tune Sauvola's threshold in optical character recognition (OCR) applications. We propose local implementations of well-known measures based on gray-intensity variances. Additionally, we derive four new measures from them using the unbiased variance estimator and gray-intensity logarithms. In our experiment, we selected the well binarized images, according each measure, and computed the accuracy of the recognized text of each. The results show that the weighted and uniform variance (using logarithms) are suitable measures for OCR applications.

This paper extends the transition method for binarization based on transition pixels, a generalization of edge pixels. This method originally computes transition thresholds using the quantile thresholding algorithm, that has a critical parameter. We achieved an automatic version of the transition method by computing the transition thresholds with the Rosin's algorithm. We experimentally tested four variants of the transition method combining the density and cumulative distribution functions of transition values, with gray-intensity thresholds based on the normal and lognormal density functions. The results of our experiments show that these unsupervised methods yields superior binarization compared with top-ranked algorithms.

This paper presents a method for creating a low cost stereo vision system which fits perfectly to the needs of cheap mobile robots. As mass-produced low cost CMOS cameras have no direct means for taking synchronized images, we present a method for taking synchronous images with these devices by manipulating their system clock. We also show a method of merging the data of two or more cameras for interfacing all of them to a single camera controller.

A new biomimetic honeybee robot capable of dancing and mimicking all known signals in the honeybee dance communication system has been built. This paper describes the hard- and software design of the first honeybee robot with computer vision. The robot can robustly recognize obstacles and react on imminent collisions.

This paper presents a new algorithm for fusioning images of text-documents taken with different exposures.It is compared to several standard block oriented exposure- and focus-blending-algorithms.The recognition rate of a publicly available OCR-engine is used as a benchmark to quantify the results.Experiments show in average an improvement in the recognition rate from 0.46 to 0.64 by employing exposure blending as preprocessing step to an OCR. The presented algorithm of blending high-pass filtered images instead of original images further increases the recognition rate to 0.95.

We present a method for the recognition of on-line hand-written commutative diagrams. Diagrams are formed with arrows that join relatively simple mathematical expressions. Diagram recognition consists in grouping isolated symbols into simple expressions, recognizing the arrows that join such expressions, and finding the layout that best describes the diagram. We model the layout of the diagram with a grid that optimally fits a tabular arrangement of expressions. Our method maximizes a linear function that mea-sures the quality of our layout model. The recognition results are translated into the LaTeX library xy-pic.

In this paper, a new binarization algorithm for degraded document images is proposed. The method is based on positive and negative pixel energies using the Laplacian of an image. After a filtering step and morphological operations our local contrast segmentation method is able to detecting connected components. The given approach is applied to cases of sophisticated, challenging documents and other application scenarios like whiteboard and chalk images.

We describe a threshold-based local algorithm for im- age binarization. The main idea is to compute a transition energy using pixel value differences taken from a neighbor- hood around the pixel of interest. By filtering the pixels with low positive and negative energy, we keep two subsets in the neighborhood, corresponding to higher positive and nega- tive energy values. The binarization threshold is calculated using a statistical model of the high energy pixels. Experi- ments show that this new approach is faster and better than current state-of-the-art algorithms.

This paper describes a new versatile algorithm for correcting nonlinear distortions, such as curvature of book pages, in camera based document processing. We introduce the idea of using local orientation features to interpolate a vector field from which a warping mesh is derived. Ultimately, the image is corrected by approximating the nonlinear distortion with multiple linear projections. Since the algorithm does not derive the mesh directly from text baselines it is robust over arbitrarily complex text layouts. We describe a baseline detector for extracting the required local orientation features. We also sketch a method for correcting nonlinear distortions of a document's vertical axis with our algorithm.

This paper presents recent improvements on a previous published practical approach to object segmentation from still images called SIOX: simple interactive object extraction. The basic method has been improved further and applied to a variety of applications. In the case of videos, this technique allows scale and rotation invariant real-time tracking of foreground objects including the classification of newly introduced objects. In the case of still-images, the method has been extended to cope with highly detailed textures with sub-pixel accuracy. The approach is robust in the presence of noise and can be applied to a variety of problems where objects have to be extracted, tracked and/or identified. The approach has been released as an open source framework. We discuss various applications of the algorithm and incorporated experiences and user feedback from several projects that have begun to integrate the algorithm

In our system for recording and transmitting lectures over the Internet, the board content is transmitted as vector graphics, producing thus a high quality image, while the video of the lecturer is sent as a separate stream. It is easy for the viewer to read the board but the lecturer appears in a separate window. As a result, two areas of the screen are competing for the viewer's attention, causing the widely known split attention effect. To eliminate this problem, the lecturer is extracted from the video stream and his or her image is pasted onto the board image at video stream rates. The lecturer can be dimmed from opaque to semitransparent, or even transparent. The article presents a detailed analysis of the underlying psychological problems and explains the multimedia techniques that are applied to achieve the solution

This article introduces the new version of our system for the recognition of on-line handwritten mathematical expressions. The previous version was capable to recognize the most common mathematical expressions, such as expressions of calculus, following the usual mathematical conventions. This version improves and extends our technique, allowing the recognition of structures described by tabular and multiline arrangements, such as matrices and systems of equations. The recognition results are stored in a composite of symbols, which are used by computer algebra systems for algebraic computations, and Junction plotting. The system is embedded as an intelligent assistant in the electronic chalkboard (E-Chalk), a system for the live transmission and storage of lectures.

The following article presents an approach for interactive foreground extraction in still images that is currently being integrated into the GIMP. The presented approach has been derived from color signatures, a technique originating from image retrieval. The article explains the algorithm and presents some benchmark results to show the improvements in speed and accuracy compared to state of the art solutions. The article also describes how the algorithm can easily be adapted for video segmentation tasks.

Abstract. We present a structural analysis method for the recognition of on-line handwritten mathematical expressions based on a minimum spanning tree construction and symbol dominance. The method handles some layout irregularities frequently found in on-line handwritten formula recognition systems, like symbol overlapping and association of arguments of sum-like operators. It also handles arguments of operators with non-standard layouts, as well as tabular arrangements, like matrices. 1

It is now well known that neural classifiers can learn to compute a posteriori probabilities of classes in input space. This note offers a shorter proof than the traditional ones. Only one class has to be considered and straightforward minimization of the error function provides the main result. The method can be extended to any kind of differentiable error function. We also present a simple visual proof of the same theorem, which stresses the fact that the network must be perfectly trained and have enough plasticity.