This paper proposes an end-to-end trainable one-stage parking slot detection method for around view monitor (AVM) images. The proposed method simultaneously acquires global information (entrance, type, and occupancy of parking slot) and local information (location and orientation of junction) by using a convolutional neural network (CNN), and integrates them to detect parking slots with their properties. This method divides an AVM image into a grid and performs a CNN-based feature extraction. For each cell of the grid, the global and local information of the parking slot is obtained by applying convolution filters to the extracted feature map. Final detection results are produced by integrating the global and local information of the parking slot through non-maximum suppression (NMS). Since the proposed method obtains most of the information of the parking slot using a fully convolutional network without a region proposal stage, it is an end-to-end trainable one-stage detector. In experiments, this method was quantitatively evaluated using the public dataset (PS2.0) and outperforms previous methods by showing both recall and precision of 99.77%, type classification accuracy of 100%, and occupancy classification accuracy of 99.31% while processing 60 frames per second.

Backing crashes have become a serious problem, resulting in the United States announcing a rule requiring all vehicles to come equipped with rearview cameras. As a result of this situation, this paper proposes a method that detects obstacles behind a vehicle using a rearview camera with a fisheye lens. The proposed method represents obstacles by means of the Stixel World, and it consists of two stages: free space boundary detection and obstacle height estimation. In the first stage, obstacles are three-dimensionally reconstructed, and the 3D points are spatially interpolated and sequentially accumulated. An occupancy grid map is generated with depth and incidence angle axes to effectively handle a wide field-of-view situation. In the second stage, this method combines some complementary information to reliably estimate the obstacle height using spare 3D points. Three cost maps based on 3D points, color similarity, and edge strength are calculated and fused in height and incidence angle axes. The free space boundary and the obstacle height are estimated by finding the optimal paths in the occupancy grid map and the fused cost map, respectively. Finally, Stixels are generated by integrating the free space boundary and obstacle height. In the experiment, the proposed method was quantitatively evaluated and compared with the previous methods in various road, obstacle, and background conditions.

This paper proposes a practical backover warning system using a wide-angle rearview camera. The proposed system utilizes simple but cost-effective techniques for pedestrian detection, verification, and tracking to achieve real-time operation. This system first transforms fisheye images via Mercator projection to reduce pedestrian shape variations. Second, it detects pedestrians based on lower and upper body detectors. This is for handling upright and non-upright poses as well as severe occlusions of upper bodies. Then, it confirms pedestrians by checking the existence of feet or a head inside the lower or upper body region. This step ensures that the system achieves a low false alarm rate. Finally, the confirmed pedestrians are visually tracked by fusing generative and discriminative models in a score level. In the experiment, the proposed system was quantitatively evaluated and compared with the previous method.

This paper proposes a sensor fusion-based low-cost vehicle localization system. The proposed system fuses a global positioning system (GPS), an inertial measurement unit (IMU), a wheel speed sensor, a single front camera, and a digital map via the particle filter. This system is advantageous over previous methods from the perspective of mass production. First, it only utilizes low-cost sensors. Second, it requires a low-volume digital map where road markings are expressed by a minimum number of points. Third, it consumes a small computational cost and has been implemented in a low-cost real-time embedded system. Fourth, it requests the perception sensor module to transmit a small amount of information to the vehicle localization module. Last, it was quantitatively evaluated in a large-scale database.

As free space boundary defined as the boundary between the closest obstacles and a road surface provides information about drivable space and obstacles, it has been one of the most important research topics in the automotive field. This paper proposes a method enhancing the accuracy of u-disparity-based free space boundary estimation to the subpixel level. The conventional method detects peak series by exploiting dynamic programming in u-disparity made by accumulating disparity maps depending on the u coordinate. It assumes that the bigger accumulation value of the u-disparity of a position and the smaller vertical coordinate change between the position and its previous column, the higher is the possibility that the position belongs to peak series. However, as the conventional method considers only vertical coordinate change with respect to the previous column, it has a problem that even a straight line will be penalized if it is not seen to be horizontal. To address this problem, this paper proposes preview dynamic programming, which is modified to consider the slope change with the previous and next column. Additionally, to solve the problem that a slope smaller than 45 cannot be represented by integer coordinates, this paper proposes modified u-disparity, which uses average coordinates weighted by u-disparity values within a fixed size window. Through qualitative evaluations with an open dataset, it is confirmed that the proposed method significantly mitigates the quantization noise of free space boundary estimated by the conventional method.

This paper proposes a fast method for detecting symbolic road markings (SRMs) and stop-lines. The proposed method efficiently restricts the search area based on the lane detection results and finds SRMs and stop-lines in a cost-effective manner. The SRM detector generates multiple SRM candidates using a top-hat filter and projection histogram and classifies their types using a histogram of oriented gradient (HOG) feature and total error rate (TER)-based classifier. The stop-line detector creates stop-line candidates via random sample consensus (RANSAC)-based parallel line pair estimation and verifies them using the HOG feature and TER-based classifier. The proposed method achieves reasonable detection rates and extremely low false positive rates along with a fast computing time.

This paper presents a Monte Carlo localization algorithm for an autonomous car based on an integration of multiple sensors data. The sensor system is composed of onboard motion sensors, a low-cost GPS receiver, a precise digital map, and multiple cameras. Data from the onboard motion sensors, such as yaw rate and wheel speeds, are used to predict the vehicle motion, and the GPS receiver is applied to establish the validation boundary of the ego-vehicle position. The digital map contains location information at the centimeter level about road surface markers (RSMs), such as lane markers, stop lines, and traffic sign markers. The multiple images from the front and rear mono-cameras and the around-view monitoring system are used to detect the RSM features. The localization algorithm updates the measurements by matching the RSM features from the cameras to the digital map based on a particle filter. Because the particle filter updates the measurements based on a probabilistic sensor model, the exact probabilistic modeling of sensor noise is a key factor to enhance the localization performance. To design the probabilistic noise model of the RSM features more explicitly, we analyze the results of the RSM feature detection for various real driving conditions. The proposed localization algorithm is verified and evaluated through experiments under various test scenarios and configurations. From the experimental results, we conclude that the presented localization algorithm based on the probabilistic noise model of RSM features provides sufficient accuracy and reliability for autonomous driving system applications.

This paper proposes a dense stereo-based robust vertical road profile estimation method. The vertical road profile is modeled by a cubic B-spline curve, which is known to be accurate and flexible but difficult to estimate under a large proportion of outliers. To robustly estimate a cubic B-spline curve, the proposed method utilizes a two-step strategy that initially estimates a piecewise linear function and then obtains a cubic B-spline curve based on the initial estimation result. A Hough transform and dynamic programming are utilized for estimating a piecewise linear function to achieve robustness against outliers and guarantee optimal parameters. In the experiment, a performance evaluation and comparison were conducted using three publicly available databases. The result shows that the proposed method outperforms three previous methods in all databases. In particular, its performance is superior to the others in the cases of a large proportion of outliers and road surfaces distant from the ego-vehicle.

This paper proposes a vacant parking slot detection and tracking system that fuses the sensors of an Around View Monitor (AVM) system and an ultrasonic sensor-based automatic parking system. The proposed system consists of three stages: parking slot marking detection, parking slot occupancy classification, and parking slot marking tracking. The parking slot marking detection stage recognizes various types of parking slot markings using AVM image sequences. It detects parking slots in individual AVM images by exploiting a hierarchical tree structure of parking slot markings and combines sequential detection results. The parking slot occupancy classification stage identifies vacancies of detected parking slots using ultrasonic sensor data. Parking slot occupancy is probabilistically calculated by treating each parking slot region as a single cell of the occupancy grid. The parking slot marking tracking stage continuously estimates the position of the selected parking slot while the ego-vehicle is moving into it. During tracking, AVM images and motion sensor-based odometry are fused together in the chamfer score level to achieve robustness against inevitable occlusions caused by the ego-vehicle. In the experiments, it is shown that the proposed method can recognize the positions and occupancies of various types of parking slot markings and stably track them under practical situations in a real-time manner. The proposed system is expected to help drivers conveniently select one of the available parking slots and support the parking control system by continuously updating the designated target positions.

This paper proposes a noise-resilient road surface and free space estimation method using dense stereo. The proposed road surface estimation method selects 3D points expected to compose a road surface using YZ-plane accumulation, and then finds the road surface by sequentially estimating a piece-wise linear function based on a RANSAC framework. This makes our method insensitive to 3D points on obstacles and stereo matching errors on textureless road regions. The proposed free space estimation method is based on the fact that disparities from roads and obstacles should be equal at the free space boundary. This method calculates disparity consistency between road and obstacle surfaces, and finds free space that gives the best disparity consistency and depth smoothness using dynamic programming. This approach achieves robustness against stereo matching errors on obstacle surfaces and objects located in the air since its estimation process is independent of disparity accumulation unlike previous occupancy grid-based method. The experimental results show that the proposed method is able to estimate road surfaces and free spaces in various severe situations.

This letter proposes a background subtraction method for Bayer-pattern image sequences. The proposed method models the background in a Bayer-pattern domain using a mixture of Gaussians (MoG) and classifies the foreground in an interpolated red, green, and blue (RGB) domain. This method can achieve almost the same accuracy as MoG using RGB color images while maintaining computational resources (time and memory) similar to MoG using grayscale images. Experimental results show that the proposed method is a good solution to obtain high accuracy and low resource requirements simultaneously. This improvement is important for a low-level task like background subtraction since its accuracy affects the performance of high-level tasks, and is preferable for implementation in real-time embedded systems such as smart cameras.

This letter proposes an efficient and robust background compensation method for pan-tilt-zoom cameras. The proposed method approximates the relation between consecutive images to a three-parameter similarity transformation, which is separable in horizontal and vertical axes, and extracts and matches 1-D features that are local minima and maxima of intensity projection profiles in each axis. These correspondences are used to estimate transformation parameters via an outlier rejection approach. Experimental results show that the proposed method is more robust with respect to blurring effects and moving object proportion while dramatically decreasing computational costs compared to previous methods.

This paper proposes a face occlusion verification method for an automated teller machine (ATM) application. The proposed method mainly consists of three steps. Firstly, a head and shoulder shape is detected by applying B-spline active contour to motion edges. This motion edge is generated by a kurtosis-based frame selection and distance transformation-based motion edge detection. Secondly, a face area is estimated by fitting an ellipse to the detected head and shoulder shape. Finally, occlusion of the face area is determined by measuring skin color area ratio (SCAR) of whole face area and facial component areas. Experimental results show that the proposed head and shoulder detection method has 94.8% detection rate even though there are various types of severe occlusions in faces, and the proposed occlusion verifier has 86.7% verification rate.

This paper proposes non-intrusive iris image capturing system, which consists of pan-tilt-zoom camera and light stripe projection. Light stripe projection provides the position of user. After panning according to user's position, AdaBoost-based face detection finds tilt angle. With user's position and tilt angle, zoom and focus position are initialized. User's position replaces 2D face search with ID face search. Exact zoom and focus position enable fast control and narrow search range. Consequently, experimental results show that proposed system can capture iris image within acceptable time.