Efficient and precise localization is a prerequisite for the intelligent navigation of mobile robots. Traditional visual localization systems, such as visual odometry (VO) and simultaneous localization and mapping (SLAM), suffer from two shortcomings: a drift problem caused by accumulated localization error, and erroneous motion estimation due to illumination variation and moving objects. In this paper, we propose an enhanced VO by introducing a panoramic camera into the traditional stereo-only VO system. Benefiting from the 360° field of view, the panoramic camera is responsible for three tasks: (1) detecting road junctions and building a landmark library online; (2) correcting the robot’s position when the landmarks are revisited with any orientation; (3) working as a panoramic compass when the stereo VO cannot provide reliable positioning results. To use the large-sized panoramic images efficiently, the concept of compressed sensing is introduced into the solution and an adaptive compressive feature is presented. Combined with our previous two-stage local binocular bundle adjustment (TLBBA) stereo VO, the new system can obtain reliable positioning results in quasi-real time. Experimental results of challenging long-range tests show that our enhanced VO is much more accurate and robust than the traditional VO, thanks to the compressive panoramic landmarks built online.

Stereo cameras are widely used in autonomous vehicles as environmental perception sensors because of their availability and low cost. However, efficiently utilizing the obtained disparity images to generate a desirable local path for the vehicle still remains a challenging problem. In this paper, we present a novel navigation framework for autonomous vehicles equipped with stereo cameras, featuring finishing all of the perception and path planning tasks directly within the disparity space. Comparing with the popular 3-D counterpart, disparity is a projected geometric space that contains more primitive information directly computed from the stereo images. Furthermore, disparity image is a more compact representation for large field of 3-D Cartesian space, which makes perception and path finding in longer distance possible. The proposed framework is composed of three modules, namely, local disparity map building, slope analysis and obstacle detection, and path planning. Two important properties concerning the motion and slope in disparity space are presented for the first time, i.e., the motion model and the slope model in disparity space. With the motion model, the framework first fuses consecutive disparity maps to construct a more reliable and complete local map. Then, a novel slope analysis method called V-Intercept is developed based on the slope model. It can efficiently analyze slopes and obstacles, generating a reasonable cost map directly from the disparity image. Finally, the obstacles in the cost map are expanded properly, and a customized A* search algorithm is performed to find a reasonable path in disparity space. The experimental results show that our framework works well under various kinds of environments. The resulting system can efficiently perceive and plan on a much larger range and react to obstacles further beyond the traditional Cartesian-based method.

Robust vehicle detection is one of the key task for autonomous vehicle under the complex urban environment. Using 3D Lidar, the difficulty of the task lies in that the appearance of a vehicle in the sparse range data changes greatly with the distance, the angle of view, as well as occlusions. In this paper we present a new algorithm to detect vehicles using the finely segmented 3D object points. In segmentation, RGLOS (Ring Gradient based Local Optimal Segmentation) algorithm is proposed. Instead of processing in the grid map, the point-wise segmentation method keeps the connection between points and is able to extract object points in far distance. Using the local optimal ground height, it produces much more correct object points with less wrong ground points. In feature extraction stage, three novel features: position-related shape, object height along length and reflective intensity histogram are proposed. Finally the kernel based SVM is used to finish the classification task. Experiments are carried out using the real data acquired from urban environment. The results demonstrate the superior performance comparing with previous methods, thanks to the improved segmentation and new features.

This paper presents a new method for detecting drivable road surfaces in a single image. The method takes advantage of range and visual information so that reliable results are achieved. Specifically, given LIDAR data and an aligned image, it first makes use of 3D points to estimate the ground plane and determine the horizon. Then, subsets of road and obstacle points are extracted from the 3D points based on the plane and LIDAR properties. The pixels registered to the extracted points are used to build apriori road and non-road appearance models. The road detection problem is further formulated using Markov random field whose energy function is defined based on the learned models. Constraints are also added on the energy function to place high confidence on the pixels that are registered to extracted 3D points. Extensive experiments on urban roads and highways show that our method is robust even in complicated environments.

Visual odometry becomes an important method to deal the localization work in intelligent vehicle and robotics. A high performance visual odometry needs to achieve two requirements: high accuracy and high frequency. So we propose a two-stage local binocular bundle adjustment algorithm doing the optimization and construct a parallel pipeline using GPU acceleration. Finally, our system can run at about 35~40 frames per second with the maximum RMS 3D localization error less than 1%.

This paper presents a rock detection method for planetary terrain scenes. Our approach first segments an image into a set of superpixels. Then we formulate the rock detection task as an energy minimization problem and solve it efficiently via a novel graph cut which is constructed on the superpixels. In order to deal with complex rock scenarios, we integrate a discriminative observation model into the graph cut framework to enhance the discrimination power. Meanwhile, a couple of features, for instance, gradient based texture and contextual shading features, are employed to characterize superpixels. With the representative features, as well as the powerful optimization model, the rock detection problem is addressed well. We test our algorithm on a real Lunar terrain image set drawn from NASA which contains diverse scenarios. The attained qualitative and quantitative results show that our algorithm is effective.

Stereo matching and 3D reconstruction has been studied for decades as a fundamental problem in the field of computer vision. Recent years, stereo matching and 3D reconstruction with a large field of view, especially using omnidirectional vision and panoramic images, have received increasing attention. As a pre-step for dense stereo matching, methods are proposed to rectify different kinds of omnidirectional stereo image pairs. However, no one has described a rectification method applied to multi-camera omnidirectional systems yet. In this work, we proposed a rectification algorithm based on spherical camera model for rectifying omnidirectional stereo pairs, especially well suitable for the multi-camera omnidirectional systems as long as a spherical camera model is able to be applied. We describe the geometrical framework of the algorithm and implement it. Also we present the experimental results of the real stereo image pairs captured by Ladybug3. As the experimental results show, the effect of rectification is promising.

The compass operator is a useful tool for the detection of edges, junctions and corners. However, it is time-consuming, especially when the circular window is large. In this paper, we propose a new compass operator based on the sampling scheme utilized in the DAISY descriptor. We first generalize the original DAISY for representing a variety of types of features. Then, the metric of measuring the difference between two wedges is derived with respect to the descriptor. Our DAISY-like compass operator in essence captures cues in multi-scales but is efficient. It achieves results on boundary detection which are comparable to the conventional compass operator.

Epipolar rectification greatly simplifies the stereo matching which is important for vision based robotics navigation. In this paper we propose a novel rectification algorithm for coaxial omni-directional stereo system, i.e. systems aligned on the same axis of symmetry. The essential matrix is derived and computed for an arbitrary omni-directional stereo configuration based on Taylor model firstly. Using the one-to-one corresponding relationship between epipolar curve and its tangential space, we rectify the system to the direction of the line connecting two cameras' centers. This step removes the translation misalignment. After that, the corresponding points are constraint to the radial line. Image-resample based on the essential matrix compensates the rotation misalignment and makes corresponding points lie on the same scan column. The proposed algorithm is only performed on image plane so that the computation time and error caused by multi-projections in 3D space are decreased. Finally, experimental results on simulation and real data illustrate the effectiveness and accuracy of our algorithm.

A motor-less fast climbing robot driven by fluid is introduced in this study. Designed with a simple five-linkage and one-piston mechanism, the robot has kinematics similar to a gecko's motion. Due to bionic design of gecko, the robot has an advantage of climbing fast on flat surfaces with any slope angles, even overhanging surfaces. Applied with fluid driven and fluidic vacuuming, the robot can carry twice extra weight as its bodyweight on vertical climbing. As a result of motor-less design, the robot owns a unique benefit to get almost all power from the pressure of fluid working medium like water, compressed air, oil and etc., regardless of little energy supplied by internal batteries for low-power control circuit and solenoid valves. Especially for home application, the robot can get power from the pressure of tap water to finish a high-rise glass window cleaning by water-washing. This robot has potential applications to cleaning, fire lighting, painting, and repairing.

Central catadioptric cameras have been extensively adopted in robotics and surveillance due to their extensive field of view. To attain precise 3D information in these applications, it is important to calibrate the catadioptric cameras accurately. The existing calibration techniques either require prior knowledge of the mirror types, or highly depend on a conic estimation procedure, which might be ruined if there are only small portions of the conic visible on calibration images. In this paper, we design a novel planar pattern with concurrent lines as a calibration rig, which is more robust in conic estimation since the relationship among lines is taken into account. Based on the line properties, we propose a rough-to-fine approach suitable for the new planar pattern to calibrate central catadioptric cameras. This method divides the nonlinear optimization calibration problem into several linear sub-problems that are much more robust against noise. Our calibration method can estimate intrinsic parameters and the mirror parameter simultaneously and accurately, without a priori knowledge of the mirror type. The performance is demonstrated by both simulation and a real hyperbolic catadioptric imaging system.

Motion estimation is an important issue in H.264 video coding systems because it occupies a large amount of encoding time. In this paper, a novel search algorithm which utilizes an adaptive hexagon and small diamond search (AHSDS) is proposed to enhance search speed. The search pattern is chosen according to the motion strength of the current block. When the block is in active motion, the hexagon search provides an efficient search means; when the block is inactive, the small diamond search is adopted. Simulation results showed that our approach can speed up the search process with little effect on distortion performance compared with other adaptive approaches.

Central catadioptric camera is a class of omnidirectional sensor having single effective viewpoint with reflective surfaces and lenses. This work studies the geometry of central catadioptric projection of a set of lines and adopts it into calibration. In this paper, we propose a practical calibration method for central catadioptric cameras by manually labeling the corners on planar grids. This method analytically parameterizes the intersection of a set of lines, regardless whether they are parallel or not. Algebraic distance is introduced in order to robustify this method since the conic which is the projection of line, is difficult to be obtained precisely. Moreover, it is proved that central catadioptric is able to calibrate by at least one image with two lines. The performance of this method is validated by both simulation data and real image data.

A polarization-based method for water hazards detection is presented. The concept of polarization is introduced to computer vision to detect water hazards for autonomous off-road navigation. This method is based on the physical principle that the light reflected from water surface is partial linearly polarized and the polarization phases of them are more similar than those from the scenes around. Water hazards can be detected by comparison of polarization degree and similarity of the polarization phases. Experiments show that the method has good performance in water detection in complex natural backgrounds, especially when there is vegetation reflected in the water. This method has significant complementary advantages with respect to existing techniques, is computationally efficient, and can be easily implemented with existing imaging technology.

In this paper, a low-complexity spatial-domain error concealment algorithm is proposed for recovering consecutive blocks error in still images or intra-coded (I) frames of video sequences. The proposed algorithm works with the following steps. Firstly the Sobel operator is performed on the top and bottom adjacent pixels to detect the most likely edge direction of current block area. After that one-dimensional (1D) matching is used on the available block boundaries. Displacement between edge direction candidate and most likely edge direction is taken into consideration as an important factor to improve stability of ID boundary matching. Then the corrupted pixels are recovered by linear weighting interpolation along the estimated edge direction. Finally the interpolated values are merged to get last recovered picture. Simulation results demonstrate that the proposed algorithms obtain good subjective quality and higher PSNR than the methods in literatures for most images.

An automatic extraction method of container identity codes based on template matching is proposed. With different kinds of noises on the image, the container code can hardly be extracted. Initially, the container image is filtered with both adaptive linear and nonlinear filters in order to reduce noise so that the candidate text lines can be properly located. Then, a series of standard templates has been brought forward according to the standard align modes of the container identification (ID) codes. Finally, the align mode of each candidate text line is obtained and then matched with those standard templates and the container ID codes can be extracted automatically. Results show that this method can segment the container ID codes with high performance.

This paper presents a new method for the extraction of face image edges and a scheme for facial expression analysis based on the binary edge images (BEIs). Making use of the multi-resolution property of wavelet transform, the edge detection method includes two binarization steps and a noise removing step. Our method provides a robust solution to face image edge extraction, in particular, if changes of lighting conditions are encountered. Experimental results show that different components in a BEI are well separated and pixels of the same face component are connected well. Preliminary experimental results on facial expression analysis on the subsets of AR and Yale face databases show that a satisfactory recognition rate can be achieved when our facial analysis scheme is used for recognizing four action units.

In this paper, a new algorithm for automatic generating pen-and-ink drawings from photos is presented. The basic processing step of our system is the edge extraction based on the multiresolution property of wavelet transform. In order to enhance the variations in texture and tone, the proposed algorithm includes two binarization steps and a noise removing step. Experimental results show that the generated pen-and-ink drawings are of high quality, which accurately illustrate the scenic contents of the original photos.

A new method for eyes localization is presented. By incorporating some eye geometrical constraints, our method includes such procedures as eyes image segmentation, eye candidates selection and eyeball detection. In order to locate the eyes more precisely, the valley map transformed from the grayscale face image using an arithmetical morphology method is successively binarized with a series of threshold values determined adaptively, and many possible eyeball candidates are extracted from them. The final position of the two eyes is obtained using a statistical method. Experimental results on AR and Yale face databases show that a locating rate of over 94% and an average locating disparity of below 2 pixels can be achieved

In this paper an integrated vision system for autonomous land vehicle is described. The vision system includes 2D and 3D vision modules and information fusion module. The task of 2D vision is to provide the physical and geometry information of road, and the task of 3D vision system is to detect the obstacles in the surrounding. Fusion module combines 2D and 3D information to generate a feasible region provided for vehicle navigation.