In this paper we propose a novel boosting-based sliding window solution for object detection which can keep up with the precision of the state-of-the art deep learning approaches, while being 10 to 100 times faster. The solution takes advantage of multisensorial perception and exploits information from color, motion and depth. We introduce multimodal multiresolution filtering of signal intensity, gradient magnitude and orientation channels, in order to capture structure at multiple scales and orientations. To achieve scale invariant classification features, we analyze the effect of scale change on features for different filter types and propose a correction scheme. To improve recognition we incorporate 2D and 3D context by generating spatial, geometric and symmetrical channels. Finally, we evaluate the proposed solution on multiple benchmarks for the detection of pedestrians, cars and bicyclists. We achieve competitive results at over 25 frames per second.

In this paper we introduce a novel multimodal boosting based solution for semantic segmentation of traffic scenarios. Local structure and context are captured from both monocular color and depth modalities in the form of image channels. We define multiple channel types at three different levels: low, intermediate and high order channels. The low order channels are computed using a multimodal multiresolution filtering scheme and capture structure and color information from lower receptive fields. For the intermediate order channels, we employ deep convolutional channels that are able to capture more complex structures, having a larger receptive field. The high order channels are scale invariant channels that consist of spatial, geometric and semantic channels. These channels are enhanced by additional pyramidal context channels, capturing context at multiple levels. The semantic segmentation is achieved by a boosting based classification scheme over superpixels using multi-range channel features and pyramidal context features. A pre-segmentation is used to generate semantic channels as input for more powerful final segmentation. The final segmentation is refined using a superpixel-level dense conditional random field. The proposed solution is evaluated on the Cityscapes segmentation benchmark and achieves competitive results at low computational costs. It is the first boosting based solution that is able to keep up with the performance of deep learning based approaches.

Scene labeling enables very sophisticated and powerful applications for autonomous driving. Training classifiers for this task would not be possible without the existence of large datasets of pixelwise labeled images. Manually annotating a large number of images is an expensive and time consuming process. In this paper, we propose a new semi-automatic annotation tool for scene labeling tailored for autonomous driving. This tool significantly reduces the effort of the annotator and also the time spent to annotate the data, while at the same time it offers the necessary features to produce precise pixel-level semantic labeling. The main contribution of our work represents the development of a complex annotation framework able to generate automatic annotations for 20 classes, which the user can control and modify accordingly. Automatic annotations are obtained in two separate ways. First, we employ a pixelwise fully-connected Conditional Random Field (CRF). Second, we perform grouping of similar neighboring superpixels based on 2D appearance and 3D information using a boosted classifier. Polygons represent the manual correction mechanism for the automatic annotations.

Pedestrian detection and semantic segmentation are high potential tasks for many real-time applications. However most of the top performing approaches provide state of art results at high computational costs. In this work we propose a fast solution for achieving state of art results for both pedestrian detection and semantic segmentation. As baseline for pedestrian detection we use sliding windows over cost efficient multiresolution filtered LUV+HOG channels. We use the same channels for classifying pixels into eight semantic classes. Using short range and long range multiresolution channel features we achieve more robust segmentation results compared to traditional codebook based approaches at much lower computational costs. The resulting segmentations are used as additional semantic channels in order to achieve a more powerful pedestrian detector. To also achieve fast pedestrian detection we employ a multiscale detection scheme based on a single flexible pedestrian model and a single image scale. The proposed solution provides competitive results on both pedestrian detection and semantic segmentation benchmarks at 8 FPS on CPU and at 15 FPS on GPU, being the fastest top performing approach.

In this paper we describe a fast solution for the semantic segmentation of traffic scenarios. We propose a multiresolution filtering scheme over LUV + HOG image channels using high pass and low pass filtered channels at multiple scales. To add spatial context, we extend the filtered channels with horizontal and vertical position channels. We introduce multi-range classification features that capture local structure and context for achieving fast semantic segmentation of traffic scenarios. Binary boosting based pixel classifiers are trained for each semantic class. Finally, we use these classifiers to provide the unary potential term in a dense Conditional Random Field. We evaluate the proposed solution on the CamVid traffic scene segmentation benchmark and achieve state of art results at 25 FPS, being the fastest top performing approach.

In this work we present an obstacle detection and tracking solution applied to Automated Guided Vehicles (AGVs) in industrial environments. The proposed method relies on information provided by an omnidirectional stereo vision system enabling 360 degree perception around the AGV. The stereo data is transformed into a classified digital elevation map (DEM). Based on this intermediate representation we are able to generate a set of obstacle hypotheses, each represented by a 3D cuboid and a free-form polygonal model. The cuboidal model is used for the classification of each hypothesis as “Pedestrian”, “AGV”, “Large Obstacle” or “Small Obstacle”, while the free-form polylines are used for object motion estimation relying on an Iterative Closest Point (ICP) method. The obtained measurements are subjected to a Kalman filter based tracking approach, in which the data association takes into account also the classification results.

In this paper we propose an approach for obstacle localization and recognition using omnidirectional stereovision applied to autonomous fork-lifts in industrial environments. We use omnidirectional stereovision with two fisheye cameras for the 3D perception of the surrounding environment. Using the reconstructed 3D points, a Digital Elevation Map (DEM) is constructed consisting of a 2.5D grid of elevation cells. Each cell is then classified as ground or obstacle. Further, we use the classified DEM to generate obstacle hypotheses. To ensure a higher detection rate we also propose a fast sliding window based approach relying on the monocular fisheye intensity image. The detections from both approaches are merged and are subjected to a tracking mechanism. Finally each obstacle is classified using boosting over Visual Codebook type features. The classification is refined using the classification history available from tracking. The presented approaches are integrated into a 3D visual perception system for AGVs and are of real time performance.

Most pedestrian detection approaches that achieve high accuracy and precision rate and that can be used for realtime applications are based on histograms of gradient orientations. Usually multiscale detection is attained by resizing the image several times and by recomputing the image features or using multiple classifiers for different scales. In this paper we present a pedestrian detection approach that uses the same classifier for all pedestrian scales based on image features computed for a single scale. We go beyond the low level pixel-wise gradient orientation bins and use higher level visual words organized into Word Channels. Boosting is used to learn classification features from the integral Word Channels. The proposed approach is evaluated on multiple datasets and achieves outstanding results on the INRIA and Caltech-USA benchmarks. By using a GPU implementation we achieve a classification rate of over 10 million bounding boxes per second and a 16 FPS rate for multiscale detection in a 640×480 image.

Multi-class segmentation assigns a class label to each pixel in an image. It represents a significant task for the semantic understanding of images and has received plentiful attention over the last years. The current state of art is dominated by conditional random field based approaches, defined over pixels or image segments. However, high accuracy segmentation comes at a high computational cost. The best performing methods can barely run at few frames per second and are far from real-time applications. Our goal is to bridge the gap between current state of the art segmentation approaches and real-time applications. In this paper we propose an efficient approach for individual pixel classification. Multiple local descriptors are computed densely and then quantized using visual codebooks. Joint boosting is used to classify each pixel based on the quantized local descriptors. We show that using careful design choices and GPU optimization we can achieve sate of the art segmentation results at over 50 FPS. We also propose a Conditional Random Field (CRF) model defined over superpixels that uses the proposed pixel classifier for the estimation of unary potentials. The CRF based multi-class segmentation can run at over 30 FPS. The proposed approach is validated on the MSRC21 and CamVid multi-class segmentation benchmarks, the former one consisting of urban traffic sequences.