Local image features matching is a useful technique for 3D reconstruction and localization in ITS field. In recent years, machine learning-based local feature matching methods have been proposed. However, it is impractical to manually generate the ground truth of corresponding points used as training data on these methods. Therefore, for in-vehicle camera images we propose an automatic training data collection method using geometric constraints and continuity with the movement of the vehicle. The experimental results show that the proposed method can provide effective corresponding points for training a machine learning-based matcher.

Generation of high density depth values of the driving environment is indispensable for autonomous driving. Stereo vision is one of the practical and effective methods to generate these depth values. However, the accuracy of the stereo vision is limited by texture-less regions, such as sky and road areas, and repeated patterns in the image. To overcome these problems, we propose to enhance the stereo generated depth by incorporating prior information of the driving environment. Prior information, generated by deep learning-based U-Net model, is utilized in a novel post-processing mathematical framework to refine the stereo generated depth. The proposed mathematical framework is formulated as an optimization problem, which refines the errors due to texture-less regions and repeated patterns. Owing to its mathematical formulation, the post-processing framework is not a black-box and is explainable, and can be readily utilized for depth maps generated by any stereo vision algorithm. The proposed framework is qualitatively validated on the acquired dataset and KITTI dataset. The results obtained show that the proposed framework improves the stereo depth generation accuracy.

We propose a robust and reliable method to generate 3D dense map of surrounding environment using thermal stereo sensor. To realize level 4 and 5 of fully automated driving, we should develop reliable and robust perception modules that works 24 hours every 365 day and night at any weather condition. Today, sensors such as cameras, LIDAR and RADAR are extensively used by OEMs to develop different Advanced Driving Assistance Systems (ADAS) applications. Though any of these sensors have specific challenges to overcome and we are still far from a reliable and robust perception module for automated driving. Sensors fusion may solve some problems of individual sensor but there are still some challenges that cannot be solved by advanced fusion solutions. Last decade, the thermal sensors had great advances to improve the resolution and accuracy and the sensor price is getting cheaper. The thermal stereo is robust and reliable sensors that may solve current challenges of perception and would be a necessary item of any sensory solutions for future vehicles. In this paper, we propose a thermal stereo solution to generate reliable 3D maps from driving environment. The experiment results shows that we are able to generate accurate and dense 3D map by using the thermal images in any environment.

In this paper, a vision-based multilabel deep learning framework is combined with an extra trees regression framework to estimate the free space, visible ego-lane markers and missing ego-lane markers. The multilabel deep learning framework, the PsiNet, with two semantic segmentation layers and one multiclass classifer layer, estimates the free space and visible ego-lane markers, while the deep learning-based extra trees regression framework estimates the missing ego-lane markers. The missing ego-lane markers are predicted using image-based deep features extracted from the multilabel framework. To account for spatial variation in the missing ego lane markers, multiple extra trees regression models are trained. During testing, the multiclass label estimated by the multilabel framework is used to retrieve the corresponding extra trees regression model. The proposed framework combining the deep learning-based semantic segmentation and regression frameworks is termed the PsiNet-ET framework. We validate our proposed framework using the multiple acquired datasets. A comparative analysis with baseline algorithms, along with a parametric analysis is performed. The experimental results show that the proposed framework robustly estimates the free space, visible and missing lane markers even for challenging road scenes.

This paper proposes a novel concept of a multi-level disparity map (distance image) characterized by input images from the Wide Angle Fovea (WAF) lens. The authors focuses on a unique property of the WAF lens, i.e., the lens magnification is the highest locally in the central field of view (FOV) and decreases rapidly towards the peripheral FOV. Thus, the input image by the WAF lens achieves more accurate and wider-angle observation simultaneously without increasing the number of image pixels. Our proposed algorithm generates the multi-level disparity map based on the parallel stereo vision method. By using the disparity map, generated from the central region having the high-spatial resolution, we can measure the distance of a target object being quite far away ahead from the WAF stereo vision system very accurately. On the same time, by using the disparity map generated from the peripheral region, we can obtain 3D information of a wider space comparatively close to the vision system by adequate accuracy to some degree. We have implemented the proposed algorithm to the WAF stereo vision system and have experimented in order to verify its performance. The authors think our proposed multi-level disparity map is quite applicable for improving safety of the automatic driving assistance system.

This paper presents a high-quality wide-angle fovea lens, i.e., the WAF lens, for the autonomous robot's and vehicle's super-sensing vision system. The WAF lens is well-known in the field of robotic vision with respect to its unique design concept, biologically-inspired from a visual system of the primates. The WAF lens achieves the following two conflicting properties in imaging simultaneously: (1) wide field of view (FOV) and (2) high magnification factor (although only the central FOV achieves it partially). In this paper, the authors designs the WAF lens for the high-resolution photosensitive imaging chip more than 3M pixels. For this design, we decide the following targets on the assumption of applying this WAF lens for the stereo vision system: (1) The WAF lens can measure a very far distance over 100m ahead from the imager accurately. (2) The WAF lens can observe approximately 100-degree wide FOV on the same time. We produce a prototype of this WAF lens with much higher optical performance than our previous developments. The compound system of the prototype includes four aspherical surfaces in its front part to project enough bright images so that the WAF lens is available not only at daytime but also in dark situations at night. The authors experiment and demonstrate the projection tests using the prototype, and discuss about the results as the inspection of this challenging development.

Vision-based environment perception is an important research topic for autonomous driving and advanced driver assistance systems. Vision sensors, such as the monocular camera and stereo camera, are widely used for environment perception. The monocular camera provides the appearance information like intensity, and the stereo camera provides the depth information. The appearance and depth information are complementary, and their effective fusion would result in robust environment perception. Consequently, in this paper, we propose a novel deep learning framework, termed as the ChiNet, for the effective sensor fusion of the appearance and depth information for free space and road object estimation. The ChiNet has two input branches and two output branches. The ChiNet input branches contains separate branches for the intensity and depth information. For the output branches, the ChiNet contains separate branches for the free space and road object semantic segmentation. A comparative of the proposed framework with state-of-the-art baseline algorithms is performed using an acquired dataset. Moreover, a detailed parameter analysis is performed to validate the ChiNet architecture as well as the advantages of sensor fusion. The experimental results show that the ChiNet is better than baseline algorithms. We also show that the proposed ChiNet architecture is better than other variations of the ChiNet architecture.

Path planning for automated driving vehicle in wide, unstructured toll gate is a difficult task while several surrounding vehicles are driving with almost random behavior. To deal with this problem, we propose a decision tree based method to control speed & acceleration of the host vehicle along a predefined path. Our proposed method is able to reach the destination without collision and with minimum changes in acceleration or deceleration. To validate the performance, we simulated the toll gate region in Nagoya Expressway, where driving area is wide with no lane markers. Simulation results show that the host vehicle is able to avoid potential collisions through acceleration or deceleration, which is determined by the proposed decision tree. Speed profiles show that host vehicle is driving in a comfortable range of acceleration/deceleration.

In this paper, a combined vision and dead reckoning-based parking system for end-to-end driving are proposed. Standard autonomous parking frameworks contain multiple modules with each module having its own limitation. On the other hand, the proposed parking framework consists of a single end-to-end module, which reduces these inherent limitations. In the proposed deep learning-based parking system, a novel iterative two-stage learning framework is utilized to predict the steering angles and gear status using a front and back mounted monocular camera. In the first stage of the proposed framework, the encoder-decoder architecture is used to predict an initial estimate of the steering angle trajectory from multiple frames of the front or the back monocular camera. The camera used for steering estimated is selected using the gear status estimate. The gear status is predefined during initialization and estimated subsequently in the second stage of the proposed framework. In the second stage of the proposed framework, the initial estimate of the steering angle trajectory along with the vehicles heading angle, and absolute position is given as an input to the long short-term memory network to estimate the optimal steering angle and gear status. The proposed framework is validated on an acquired dataset. A comparative analysis of baseline algorithms and detailed parametric analysis are performed. The experimental results show that the proposed framework is better than the baseline end-to-end algorithms.

The stereo vision system is one of the most important sensors for 3D reconstruction of the environment and scene understanding. It plays an essential role for the automated driving and ADAS applications. Although recently great progress has been made in this field both in the density and accuracy, the stereo vision in the challenging environment, such as rain, snow and low light conditions, are still open problems and need to be improved. To address this issue, we propose a novel real-time stereo vision algorithm, we transform the rough confidence score of the matching cost to the outlier probability rate and use it to guide the multi-path Viterbi method to estimate the disparity. Our proposed algorithm costs less than 40ms and improves the accuracy of stereo results in difficult scenarios. Real-world experiments show that the proposed method can significantly improve the disparity results in the various challenging environment.

In this paper, we propose a real-time vision-based filtering algorithm for steering angle estimation in autonomous driving. A novel scene-based particle filtering algorithm is used to estimate and track the steering angle using images obtained from a monocular camera. Highly accurate proposal distributions and likelihood are modeled for the second order particle filter, at the scene-level, using deep learning. For every road scene, an individual proposal distribution and likelihood model is learnt for the corresponding particle filter. The proposal distribution is modeled using a novel long short term memory network-mixture-of-expert-based regression framework. To facilitate the learning of highly accurate proposal distributions, each road scene is partitioned into straight driving, left turning and right turning sub-partitions. Subsequently, each expert in the regression framework accurately model the expert driver's behavior within a specific partition of the given road scene. Owing to the accuracy of the modelled proposal distributions, the steering angle is robustly tracked, even with a limited number of sampled particles. The sampled particles are assigned importance weights using a deep learning-based likelihood. The likelihood is modeled with a convolutional neural network and extra trees-based regression framework, which predicts the steering angle for a given image. We validate our proposed algorithm using multiple sequences. We perform a detailed parameter analysis and a comparative analysis of our proposed algorithm with different baseline algorithms. Experimental results show that the proposed algorithm can robustly track the steering angles with few particles in real-time even for challenging scenes.

In this study, a vision based in-car entertainment user interface is presented. The user interface is designed using a hand posture and gesture recognition algorithm in deep learning framework. The hand posture recognition algorithm is formulated using the convolutional neural network to perform the fundamental tasks in the user interface. The hand gesture recognition algorithm is formulated using the long-term recurrent convolutional neural network to intuitively interact with the touchless automotive user interface in a detailed manner. In the recurrent deep learning framework, typically, the gesture frames are taken from a uniformly sampled image sequence. In this work, the recurrent structure is enhanced using a reduced number of input frames captured from the image sequence. The reduced input frames or key frames represent the action present in the video sequence. Sparse dictionary learning provide reliable key frame extraction from video sequences. However, sparse dictionary learning is computationally expensive, and are individually optimized for every video sequence. In this paper, we propose to approximate sparse dictionary learning using a non-linear regression framework. The multilayer perceptron is utilized to model the non-linear regression framework. The optimal neural network architecture is identified after a detailed evaluation. We evaluate the proposed recognition methods on public datasets. The proposed methods yield a recognition accuracy of 92% and 90% for pose and gestures, respectively. The combined hand posture and gesture recognition takes 82ms which is a reasonable for real time implementation.

Nowadays, the driverless automobiles have become a near reality and are going to become widely available. For autonomous navigation, the vehicles need to precise localize itself within a pre-defined map. In this paper, we propose a novel algorithm for the problem of three-dimensional (3D) point cloud map (PCL) based localization using a stereo camera. This 3D point cloud map consists of dense 3D geometric information and intensity measures of surface reflectivity value generated by the 3D light detection and ranging (LIDAR) scanner based mapping system. Although some LIDAR based localization algorithms have been proposed, in this paper we present a comparable centimeter-level accuracy localization algorithm using much cheaper and commodity stereo camera. Specifically, at each candidate position we transform the 3D data points from the real-world coordinate system to the camera coordinate system and synthetic the virtual depth and intensity images from the 3D PCL map. We localize the ego vehicle by estimating the transformation between the real-world and vehicle coordinates in each frame by matching these virtual images with the stereo depth and intensity images. In the experiment part, we show that although the 3D map was generated 3 years ago, the proposed algorithm still can produce reliable localization results even in many difficult cases, such as shadow, dynamic objects, new lane marker and night.

Pixel-labeling approaches using semantic segmentation play an important role in road scene understanding. In recent years, deep learning approaches such as the deconvolutional neural network have been used for semantic segmentation, obtaining state-of-the-art results. However, the segmentation results have limited object delineation. In this paper, we adopt the de-convolutional neural network to perform the semantic segmentation of the road scene using colour and depth information. Moreover, we improve the network's limited object delineation within a computationally efficient framework using novel features that are learnt at the pixel-level and patch-level for different road scenes. The patch-level features represent the road scene geometry. On the other hand, the learnt pixel-level features represent the appearance and depth information. The features learnt for the different road scenes are indexed with the scene's pre-defined label. Following the indexing, the random forest classifier is trained to retrieve the relevant geometric and appearance-depth features for a given road scene. The retrieved features are then used to refine identified error regions in the initial semantic segmentation estimate. Our proposed algorithm is evaluated on an acquired dataset and compared with state-of-the-art baseline algorithms. We also perform a detailed parametric evaluation of our proposed framework. The experimental results show that our proposed algorithm reports better accuracy.

Reconstructing the depth information from the 3D scene using stereo vision is a key element in the development of advanced driver assistance systems. We previously proposed a novel real-time stereo matching method based on the Multi-paths Viterbi that outperforms the well-known SGBM (Semi-Global Block-Matching Algorithm) algorithm in both disparity accuracy and density. In this paper, we extend the previous framework to estimate the depth information for challenging environments such as nighttime. Estimating the depth at nighttime is generally challenging as the night images are dark and noisy and the estimated depth information is not accurate. In our proposed work, we modify the non-local means filter and propose a new non-local cost function to combine the noise reduction and stereo vision within a single framework. We evaluate our proposed algorithm on both natural and synthetic datasets and show that the proposed algorithm can significantly improve the quality of the stereo results in the low light condition. Moreover, our proposed method can be implemented in real-time for autonomous driver applications.

Visual-based approaches have been extensively studied for on-road vehicle detection; however, it faces great challenges as the visual appearance of a vehicle may greatly change across different viewpoints and as a partial observation sometimes happens due to occlusions from infrastructure or scene dynamics and/or a limited camera vision field. This paper presents a visual-based on-road vehicle detection algorithm for a multilane traffic scene. A probabilistic inference framework based on part models is proposed to overcome the challenges from a multiview and partial observation. Geometric models are learned for each dominant viewpoint to describe the configuration of vehicle parts and their spatial relations in probabilistic representations. Viewpoint maps are generated based on the knowledge of the road structure and driving patterns, which provide a prediction of the viewpoints of a vehicle whenever it happens at a certain location. Extensive experiments are conducted using an onboard camera on multilane motor ways in Beijing. A large-scale data set that contains more than 30 000 labeled ground truths for both fully and partially observed vehicles in different viewpoints across various traffic density scenes is developed. The data set will be opened to the society together with this publication.

Object detection is a fundamental challenge facing intelligent applications. Image processing is a promising approach to this end, but its computational cost is often a significant problem. This paper presents schemes for accelerating the deformable part models (DPM) on graphics processing units (GPUs). DPM is a well-known algorithm for image-based object detection, and it achieves high detection rates at the expense of computational cost. GPUs are massively parallel compute devices designed to accelerate data-parallel compute-intensive workload. According to an analysis of execution times, approximately 98 percent of DPM code exhibits loop processing, which means that DPM could be highly parallelized by GPUs. In this paper, we implement DPM on the GPU by exploiting multiple parallelization schemes. Results of an experimental evaluation of this GPU-accelerated DPM implementation demonstrate that the best scheme of GPU implementations using an NVIDIA GPU achieves a speed up of 8.6x over a naive CPU-based implementation.

Lane change maneuver is a cause for many severe highway accidents and automatic lane change has great potentials to reduce the impact of human error and number of accidents. Previous researches mostly tried to find an optimal trajectory and ignore the behavior model. Presented methods can be applied for simple lane change scenario and generally fail for complicated cases or in the presence of time/distance constraints. Through analysis and inspiring of human driver lane change data, we propose a multi segments lane change model to mimic the human driver for challenging scenarios. We also propose a method to convert behavior/motion selection to a time-based pattern recognition problem. We developed a simulation platform in PreScan and evaluated proposed automatic lane change method for challenging scenarios.

As a key technique in ADAS (Advanced Driving Assistant System) or autonomous driving systems, visual-based on-road vehicle detection has been studied widely, while it faces still great challenges, among which are the complexity, diversity and unpredictable changes of the real-world environments. In the authors' previous work, an algorithm was developed in a probabilistic inference framework with its focus on solving the multi-view and occlusion problems at multi-lane motor way scenes. In this research, we seek to answer the questions: how efficient is the system during a long-term operation across a large area of changed conditions? To this end, a large scale experiment is conducted, where three testing data sets are developed containing the samples of more than 30,000 on Beijing's ring roads, 800 on Nagoya's fast road, and 3,000 on Nagoya's downtown streets, and the performance of visual-based vehicle detection concerning the multi-view and occlusion problems across extensive regions and at transnational environments are studied. We present our preliminary findings in this paper, which leads to a more extensive study in future work.

In recent years, automated vehicle researches move on to the next stage, that is, auto-driving experiments on public roads. This study focuses on how to realize accurate localization based on the use of Lidar data and precise map. On different roads such as urban roads and expressways, the observed information of surrounding is significantly different. For example, on the urban roads, many buildings can be observed around the upper part of the vehicle. Such observation realizes accurate map matching. On the other hand, the upper part has no specific observation on the expressway. Therefore, it is necessary to observe the lower part for the map matching. To adapt the situation changes, we propose a localization method based on self-adaptive multi-layered scan matching and road line segment matching. The main idea is to effectively match the features observed from different heights and to improve the results by applying the line segment matching in certain scenes. Localization experiments show the ability to estimate accurate vehicle pose in urban driving.

Visual based approach has been studied extensively for on-road vehicle detection, while it faces great challenges, as visual appearance of a vehicle may change greatly across different viewpoints, and partial observation happens sometime due to occlusions from infrastructure or scene dynamics, and/or limited camera vision field. Inspired by the works on part-based detection, this research proposes a probabilistic framework for on-road vehicle detection, where focus is cast on vehicle pose inference on the set of part instances by addressing the issues of partial observation and varying viewpoints. To this end, geometric models describing the configuration of vehicle parts as well as their spatial relations in probabilistic representations are learned for each dominant viewpoint, and viewpoint maps are generated on each typical road structure, which provide probabilistic prediction to the viewpoints of a vehicle at each location at ego frame. Experiments have been conducted using a data set that was developed in the authors' previous work on the ring roads in Beijing. Viewpoint-discriminative part appearance models (VDPAM) and viewpoint-discriminative part-based geometric models (VDPGM) are learned on the image samples of the data set, and the road structure-based probabilistic viewpoint maps (RSPVM) are generated by taking the statistics of the Lidar-based vehicle detection results. On-road vehicle detection is examined using an on-road video stream that has been labelled with ground truth. Experimental results are presented and efficiency on detecting the partially observed vehicles on varying viewpoints is demonstrated.

This paper introduces a novel path planning method under non-holonomic constraint for car-like vehicles, which associates map discovery and heuristic search to attain an optimal resultant path. The map discovery applies fast marching method to investigate the map geometric information. After that, the support vector machine is performed to find obstacle clearance information. This information is then used as a heuristic function which helps greatly reduce the search space. The fast marching is performed again, guided by this function to generate vehicle motions under kinematic constraints. Experimental results have shown that this method is able to generate motions for non-holonomic vehicles. In comparison with related methods, the path generated by proposed method is smoother and stay farther away from the obstacles.

Deformable part models (DPM) already proved great performance for objects detection and many extensions have already published in literature. DPMs generally have high performance, though they dramatically fail to detect objects in challenging environments such as night time. This paper proposes a method based on the idea of latent parts to optimize the structure of objects in deformable part models. Even in challenging environment some parts of the objects are still visible and latent parts can optimize the structure of DPM's object model to catch significant features. We have evaluated proposed method to detect cars at night time in urban area using IR camera. It is a challenging problem due to low visibility, light distortion and illumination/glare in urban area. Experimental results prove the effectiveness of the model to detect close and medium range cars in urban scenes at night time.

Google already showed the self-driving car can do stopping and accelerating more smoothly than the human drivers by doing extensive experiments. In this paper, we compare a human driver with computer generated motion for expressway lane changing. We did experiments at Nagoya ring route expressway and recorded all the detail motion information regarding lateral and longitudinal movement of a taxi driver. We developed an algorithm to generate and evaluate the alternative lane change motion sets considering both lateral and longitudinal trajectories. We defined a smoothness measure that integrates lateral and longitudinal motions to evaluate generated lane change motions. Evaluation results for 19 sets of lane change labeled data show the computer motion generator is able to generate close or better motion compared with a taxi driver using proposed evaluation function.

In recent years, automated vehicle researches move on to the next stage, that is auto-driving experiments on public roads. Major challenge is how to robustly drive at complicated situations such as narrow or non-featured road. In order to realize practical performance, some static information should be kept on memory such as road topology, building shape, white line, curb, traffic light and so on. Currently, some measurement companies have already begun to prepare map database for automated vehicles. They are able to provide highly-precise 3-D map for robust automated driving. This study focuses on what kind of data should be observed during automated driving with such precise database. In particular, we focus on the accurate localization based on the use of lidar data and precise 3-D map, and propose a feature quantity for scan data based on distribution of clusters. Localization experiment shows that our method can measure surrounding uncertainty and guarantee accurate localization.

This paper proposes an image fusion algorithm based on a sparse linear equation system, which uses local extreme of high resolution image and intensity of multi-spectral image to construct the system. Based on this sparse system, the multi-scale image decomposition algorithm can be implemented. This algorithm extracts the details of the high-resolution image and integrates with the multi-spectral information to derive the fused image.

We propose a path planning method for autonomous vehicle in cluttered environment with narrow passages. Different from traditional methods, we use a learning approach based on RBF kernel SVM to maximize the safety margin for driving. We use the Lagrange multipliers of SVM dual model to find most critical points in map and generate optimized hyperplane for path. The method is implemented on autonomous vehicle for outdoor parking and compared to well-known method in autonomous vehicle literatures. The experiments prove that the method is able to generate smooth and safe path in shorter time compared to other methods.

Occlusion handling is crucial for developing ADAS in urban environment. Deformable Part Models (DPM) have already demonstrated state of art results in object detection. However, they fail to handle the occlusion due to inaccurate scores for part detectors when some object parts are occluded and not visible. To handle the imperfectness of part detectors in occlusion, we propose a two layers classifier using DPM and conditional random field (CRF). We use parts contextual information and their spatial configuration from DPM to train and optimize CRF parameters. Occlusion states are defined based on visibility of parts and considered as latent variables in CRF. To learn CRF parameters, stochastic gradient descent with belief propagation is used to optimize CRF objective function for latent variables. Experimental results on recorded data in real urban environment and PASCAL VOC dataset prove the effectiveness of the proposed approach to handle difficult occlusion situations.

This paper presents a stereovision-based detection and tracking approach of the drivable road boundary, designed for navigating an intelligent vehicle through challenging traffic scenarios, and increment road safety in such scenarios with advanced driver assistance systems (ADAS). It is based on a formulation of stereo with homography associated with a semantic graph constructed from the traffic scene. Under this formulation, we employ the Viterbi algorithm and propose a sophisticated measure of the probability of the state sequence in the semantic graph to find the most likely boundary between the road and non-road regions. The results are then refined by a post-processing step with the RANdom Sample Consensus (RANSAC) algorithm to obtain the locations and curvatures of the lateral road boundaries. Experimental results on a wide variety of typical but challenging real road scenes have substantiated the effectiveness as well as robustness of the proposed approach.

Introduction of new local and semi-local features has played an important role in advancing the performance of object recognitions. Deformable part models prepare elegant framework for representing object categories and both efficient and accurate, achieving state-of the-art results. In this paper, We consider the problem of training a part-based model with variable size from images labeled only with bounding boxes around the objects. We consider part size as a latent variable and try to optimize simultaneously size and place of part templates to cover high-energy regions of the object. Extensive experiments in urban scenarios for vehicle detection show that the average precision of deformable part model significantly is improved from 72.10% to 82.41% without losing the average recall.

This paper presents a robust stereo-vision-based drivable road detection and tracking system that was designed to navigate an intelligent vehicle through challenging traffic scenarios and increment road safety in such scenarios with advanced driver-assistance systems (ADAS). This system is based on a formulation of stereo with homography as a maximum a posteriori (MAP) problem in a Markov random held (MRF). Under this formulation, we develop an alternating optimization algorithm that alternates between computing the binary labeling for road/nonroad classification and learning the optimal parameters from the current input stereo pair itself. Furthermore, online extrinsic camera parameter reestimation and automatic MRF parameter tuning are performed to enhance the robustness and accuracy of the proposed system. In the experiments, the system was tested on our experimental intelligent vehicles under various real challenging scenarios. The results have substantiated the effectiveness and the robustness of the proposed system with respect to various challenging road scenarios such as heterogeneous road materials/textures, heavy shadows, changing illumination and weather conditions, and dynamic vehicle movements.

This paper proposes a novel method for multivehicle detection and tracking using a vehicle-mounted monocular camera. In the proposed method, the features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOGs). The detection algorithm combines both global and local features of the vehicle as a deformable object model. Detected vehicles are tracked through a particle filter, which estimates the particles' likelihood by using a detection scores map and template compatibility for both root and parts of the vehicle while considering the deformation cost caused by the movement of vehicle parts. Tracking likelihoods are iteratively used as a priori probability to generate vehicle hypothesis regions and update the detection threshold to reduce false negatives of the algorithm presented before. Extensive experiments in urban scenarios showed that the proposed method can achieve an average vehicle detection rate of 97% and an average vehicle-tracking rate of 86% with a false positive rate of less than 0.26%.

Many roads made for land vehicles are not totally planar and present uphill and downhill slopes that follow the environment topography. Moreover, the road appearance is often affected by a number of factors in challenging conditions. In this paper, we present an adaptive non-planar road detection and tracking approach which overcomes these difficulties by a piecewise planar road model as well as a Markov Random Field (MRF)-based alternating optimization using belief propagation (BP) on segmented images and a hard conditional Expectation Maximization (EM) algorithm to achieve adaptability and optimality. The proposed framework incorporates image evidence, geometry information, and temporal support such that the graph we build and the well-defined energy minimization formulation can exploit the essence of the roads that is invariant in challenging environments. Experimental results in various real challenging traffic scenes show the effectiveness of the proposed approach.

Maneuvering autonomous vehicles in constrained environments, such as autonomous vehicle parking, is not a trivial task and has received increasing attention from both the academy and industry. However, the traditional methods divide the problem into parallel parking, perpendicular parking, and echelon parking, then different methods are applied for the parking motion planning. In this paper a Rapidly-exploring Random Tree (RRT) based path planner is implemented for autonomous vehicle parking problem, which treats all the situations in a unified manner. As the RRT method sometimes generates some complicated paths, a smoother is also implemented for smoothing generated paths. The proposed algorithm is verified in simulation and generates applicable solutions for the proposed application scenarios.

This paper proposes a method for on road detecting and tracking of multi vehicles at nighttime in urban environment. The features of vehicles including root and part filters are learned as a weighted deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOG). Detected vehicles are tracked through a particle filter which estimates near optimum likelihoods by calculating the maximum HOG features compatibility for both root and parts of the tracked vehicles. Tracking likelihoods are iteratively used as a priori probability to generate vehicle hypothesis regions. Extensive experiments with close range IR camera in urban scenarios showed that the efficiency of the proposed method for detecting and tracking of multi vehicles at night time.

This paper proposed a practical path-planning algorithm for an autonomous vehicle or a car-like robot in an unknown semi-structured (or unstructured) environment, where obstacles are detected online by the vehicle's sensors. The algorithm is based on particle filter, Bézier curves and support vector machine to provide a safe path among various static and moving obstacles and to satisfy the vehicle's curvature constraints. The algorithm has been implemented and verified on the simulation software. Experimental results demonstrate the effectiveness of the proposed method in complicated conditions with existing of multi objects.

Comprehensive situational awareness is paramount to the effectiveness of proprietary navigational and higher-level functions of intelligent vehicles. In this paper, we address a graph-based approach for 2D road representation of 3D point clouds with respect to the road topography. We employ the gradient cues of the road geometry to construct a Markov Random Filed (MRF) and implement an efficient belief propagation (BP) algorithm to classify the road environment into four categories, i.e. the reachable region, the drivable region, the obstacle region and the unknown region. The proposed approach can overcome a wide variety of practical challenges, such as sloped terrains, rough road surfaces, rolling/pitching of the host vehicle, etc., and represent the road environment accurately as well as robustly. Experimental results in typical but challenging environments have substantiated that the proposed approach is more sensitive and reliable than the conventional vertical displacements analysis and show superior performance against other local classifiers.

This paper proposes an embedded real time method for detecting and tracking of multiobjects including vehicles, pedestrians, motorbikes and bicycles in urban environment. The features of different objects are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOG). Laser depth data have been used as a priori to generate objects hypothesis regions and estimate HOG feature pyramid level to reduce the detection time of previously presented algorithm. Detected objects are tracked through a particle filter which fuses the observations from laser map and sequential images. We use the accurate laser data for state predication and use image HOG information for likelihood calculation. The likelihood finds the maximum HOG feature compatibility for both root and parts of the tracked objects to increase tracking accuracy for deformable objects such as pedestrians in crowded scenes. Extensive experiments with urban scenarios showed that the proposed method can improve the detection and tracking in urban environment.

In this paper, we study no 4-cycle, high-rate LDPC codes based on finite geometries for use in data storage devices and prove that these codes cannot be classified as quasi-cyclic (QC) codes but should be considered as broader generalized quasi-cyclic (GQC) codes. Because of the GQC structure of such codes, they can be systematically encoded using Groebner bases and their encoder can be implemented using simple feedback-shift registers. In order to demonstrate the efficiency of the encoder, we show that the hardware complexity of the serial-in serial-out encoder architecture of these codes is of linear order O(n). To encode a binary codeword of length n, less than 2n adders and 3n memory elements are required. Furthermore, we evaluated the error performances of these codes with sum product algorithm (SPA) decoding over additive white Gaussian noise (AWGN) channels. At a bit error rate (BER) of 10^-5, they perform 1-dB away from the Shannon limit after 10 decoding iterations.

Real-time lane detection and localization is one of the key issues for many intelligent transportation systems. In this paper, we present a lane detection and tracking approach designed to work in challenging environments where lane boundaries may be low-contrast and changeful with noise due to a number of factors such as wear, type, lighting and weather conditions, etc. In the method, a sophisticated cascade lane feature detector is applied to cope with challenging environments at the very beginning of the detection and a weighted graph is subsequently constructed by integrating intensity as well as geometry cues, reflecting the confidence of each pixel as a lane feature. In order to deal with complex road geometry, we employ Catmull-Rom splines to represent lane boundaries and the left and right lane boundaries are estimated separately in a tracking process using particle filter based on the weighted graph. In the proposed framework, unlike most of previous methods we lay a strong emphasis on accurate and effective lane feature detection since the challenges happen in the very first step of lane detection, and accurately detected lane features can be expected to reduce the complexity and difficulty, as well as improve the accuracy of lane detection in the following steps.

This paper presents a vision-based approach with unsupervised learning for robust, accurate and stable detection of the drivable road to deal with autonomous driving in changing environments. This approach is based on a formulation of stereo with homography as a Maximum A Posteriori (MAP) problem in a Markov Random Field (MRF). Under this formulation, we develop an alternating optimization algorithm that alternates between computing the binary labeling and learning the optimal parameters from the stereo pair itself. The labeling is optimized by minimizing a well-defined energy function that consists of matching energy, smoothness energy and tracking energy. The parameters, including nine homography parameters and four MRF parameters, are learned online by applying a hard Expectation Maximization (EM) algorithm to maximize conditional likelihood. The proposed automatic parameter tuning procedure not only improves the accuracy of road detection but also makes the approach adaptive to changing environments without any a priori knowledge of the road. Experimental results show the optimality as well as adaptability of the proposed approach on a wide variety of challenging roads with changing environments.

This paper presents a Bézier curve based path planner which enables the anti-collision behavior of an electronic car. The anti-collision system is a fundamental module in the architecture. The path tracking implementation uses pure pursuit algorithm. The anti-collision system based on laser scanner data consists of estimating the trajectories and behavior of surrounding objects, and a Bézier curve based path planner. Experimental results are presented showing the effectiveness of the overall navigation control system.

In urban environment, we need accurate and precise estimation of vehicle state for real time navigation and control. This paper presents an architecture to fuse different data from onboard sensors to estimate the vehicle state when observations are noisy. We are trying to compensate the GPS errors by data fusion from different sensors in a probabilistic way. A particle filter with joint observation model has been proposed to real timely estimate the vehicle state. An adaptive joint observation model has been developed to fuse different observations according to accuracy and reliability of the corresponding sensor. Finally a navigation architecture has been proposed for fully autonomous driving with dynamic obstacles. Experiments with real vehicle show the proposed method is able to estimate the vehicle state precisely when the individual observations fail to be enough accurate.

This paper proposes a novel method for vehicle detection and tracking using a vehicle-mounted monocular camera. In this method, features of vehicles are learned as a deformable object model through the combination of a latent support vector machine (LSVM) and histograms of oriented gradients (HOG). The vehicle detector uses both global and local features as the deformable object model. Detected vehicles are tracked by using a particle filter with integrated likelihoods, such as the probability of vehicles estimated from the deformable object model and the intensity correlation between different picture frames. Tracking likelihoods are iteratively used as the a priori probability for the next frame. The experimental results showed that the proposed method can achieve an average vehicle detection rate of 98% and an average vehicle tracking rate of 87% with a false positive rate of less than 0.3%.

We define generalized quasi-cyclic (GQC) codes as linear codes with nontrivial automorphism groups. Therefore, GQC codes, unlike quasi-cyclic codes, can include many important codes such as Hermitian and projective geometry (PG) codes; this capability is important in practical applications. Further, we propose the echelon canonical form algorithm for computing Grobner bases from their parity check matrices. Consequently, by applying Grobner base theory, GQC codes can be systematically encoded and implemented with simple feedback shift registers. Our algorithm is based on Gaussian elimination and requires a sufficiently small number of finite-field operations, which is related to the third power of code-length. In order to demonstrate our encoder's efficiency, we prove that the number of circuit elements in the encoder architecture is proportional to the code-length for finite geometry (FG) LDPC codes (a class of GQC codes). We show that the hardware complexity of a serial-in-serial-out encoder architecture for FG-LDPC codes is related to the linear order of the code-length; less than 2n adder and 2n memory elements are required to encode a binary codeword of length n.

Road detection is a crucial problem for intelligent vehicles and mobile robots. Most of the methods proposed nowadays only achieve reliable results in relatively well-arranged environments. In this paper, we proposed a stereovision-based road boundary detection method by combining homography estimation and MRF-based belief propagation to cope with challenging scenarios such as unstructured roads with unhomogeneous surfaces. In the method, each pixel in the reference image is firstly labeled as ¿road¿ or ¿non-road¿ by minimizing a well defined energy function that accounts for the planar road region. Subsequently, both of the road boundaries are generated using Catmull-Rom splines based on RANdom SAmple Consensus (RANSAC) algorithm with varying road structure models to help the intelligent vehicle understand the structure as well as safe range of current road. In the suggested framework, both intensity and geometry information of road scenarios are used to contain all the regions belonging to the planar road plane, and the left and right road boundaries are generated separately using a robust fitting algorithm to handle different road structures. Therefore, more accurate as well as robust detection of the road can be expected. Experimental results on a wide variety of typical but challenging scenarios have demonstrated the effectiveness of the proposed method.

In this paper, a road detection method based on an image segmentation algorithm is presented. Road detection process is a key issue for an autonomous driving system in urban environment. Image-based road detection algorithm is applied on sources of visual information recorded by cameras when our car is running on road. Our method combines a posteriori probability and visual information for image segmentation. The method is composed of two steps. Firstly, a road identifier is trained with supervised learning algorithm. Secondly, road regions are detected by combining a posteriori probability and visual information using image segmentation algorithm. Experimental results are presented for road images in actual driving video acquired in urban areas.

We propose a novel encoding scheme for algebraic codes such as codes on algebraic curves, multidimensional cyclic codes, and hyperbolic cascaded Reed-Solomon codes and present numerical examples. We employ the recurrence from the Grobner basis of the locator ideal for a set of rational points and the two- dimensional inverse discrete Fourier transform. We generalize the functioning of the generator polynomial for Reed-Solomon codes and develop systematic encoding for various algebraic codes.

The authors have already proposed small-scale decoder for codes on algebraic curves, which updates decoding data serially and has the same number of calculators for the finite-field computations as the decoders for Reed-Solomon codes except for one calculator for inverse. In this research, we eliminate divisions of the finite field from error-location algorithm, and propose an inverse-free architecture for the error-location of codes on algebraic curves which reduces Kotter's architecture

We construct a two-dimensional systolic array implementing the Berlekamp-Massey-Sakata (BMS) algorithm to provide error-locator polynomials for codes on selected algebraic curves. This array is constructed by introducing some new polynomials in order to increase the parallelism of the algorithm. The introduced polynomials are used in the majority logic scheme by Sakata et al. to correct errors up to the designed minimum distance without affecting its high speed. The arrangement of the nearest local connection of processing units in the systolic array is obtained for the general case. Furthermore, shortened systolic arrays that reduce the circuit scale and have the same function are constructed with only a slight modification of the connections and controls; this enables the adjustment of the circuit scale for different types of systems.

A soft-output data-detection scheme optimized for data-dependent media-noise recording channels is proposed that uses signal-dependent correlation-sensitive (SDCS) metric estimation for post-processing decoding. This media-noise soft-output (MNS) decoding scheme achieves sub-optimal maximum-likelihood (ML) sequence detection in a non-stationary media-noise channel, while still using traditional Viterbi detection. Because it drastically reduces SDCS metric computation by focusing on only specified dominant error-events in the ML detector, it is less complex than other sub-optimal detection schemes. Moreover, its one-shot post-processing scheme enables the use of a simple lookup-table architecture suitable for high-speed circuit implementation. Simulation shows that the MNS decoding scheme in conjunction with a conventional ME/sup 2/PRML system provides an excellent tradeoff between data-detection performance and computation complexity for a media-noise-dominant high-density recording channel.

In this paper, we propose a turbo decoding method for a low-density generator matrix code. It is based on the Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm and the belief propagation (BP) algorithm. Userbits with high error probability are selected for the BP algorithm by using the soft-output of the BCJR algorithm. Since the amount of calculation in the BP algorithm increases exponentially with the number of the userbits, the amount of calculation is reduced drastically by the selection of the userbits with higher error probability. We showed that the use of the proposed algorithm and 16/17 MTR code on an MEEPR4 channel results in a performance better than that obtained without turbo decoding.

Erasure error correction based on List-SOVA has been applied to partial-response channel code for high-density magnetic recording. Soft-output information extracted from modified EEPR4ML is used. To achieve as high a total coding rate as possible, a 16/17MTR(3;11) code and a rate 544/561 cyclic redundancy check code (CRCC) are combined. CRCC-false-correction deteriorates erasure error correction performance. To solve this problem, a method for detecting CRCC-false-correction has been proposed. Simulation results show that our method can reduce block error rates to 1/10 the case without erasure error correction at the user density of 2.8.

A soft-output decoding scheme was developed for the cyclic-redundancy-check error-correction-code (CRCC) concatenated error correction by applying the list and soft-symbol-output Viterbi algorithm (list-SOVA). By focusing on just the specified dominant error-events in the maximum-likelihood (ML) sequence detector, a soft-output data-reliability generator and a list generator were efficiently simplified for soft-output decoding. The soft-output decoding enables multiple dominant error events within a data block to be corrected without increasing the CRCC redundancy and modifying the CRCC structure. It can also help prevent the CRCC miscorrection when correcting the multiple error-events. It was confirmed that the soft-output decoding with a reasonably sized data-block list can perfectly correct the double error-events within a data-block on a 16/17 rate quasi-MTR-coded modified E/sup 2/PRML (ME/sup 2/PRML) channel at a practical error-rate.

In digital storage systems, receivers for rate 1/2 modulation codes are usually oversampled by a factor of two with respect to the data stream that they attempt to reconstruct. It is shown that oversampling may be avoided by using partial-response techniques to detect, instead of the encoded binary signal, a decimated ternary one, from which the original data can be recovered by means of a simple decoder. A method is described to find all such decoders for a given rate 1/2 code. Examples treated are FM, MFM, Miller-squared,