The semantic SLAM (simultaneous localization and mapping) system is an indispensable module for autonomous indoor parking. Monocular and binocular visual cameras constitute the basic configuration to build such a system. Features used in existing SLAM systems are often dynamically movable, blurred and repetitively textured. By contrast, semantic features on the ground are more stable and consistent in the indoor parking environment. Due to their inabilities to perceive salient features on the ground, existing SLAM systems are prone to tracking loss during navigation. Therefore, a surround-view camera system capturing images from a top-down viewpoint is necessarily called for. To this end, this paper proposes a novel tightly-coupled semantic SLAM system by integrating Visual, Inertial, and Surround-view sensors, VIS SLAM for short, for autonomous indoor parking. In VIS SLAM, apart from low-level visual features and IMU (inertial measurement unit) motion data, parking-slots in surround-view images are also detected and geometrically associated, forming semantic constraints. Specifically, each parking-slot can impose a surround-view constraint that can be split into an adjacency term and a registration term. The former pre-defines the position of each individual parking-slot subject to whether it has an adjacent neighbor. The latter further constrains by registering between each observed parking-slot and its position in the world coordinate system. To validate the effectiveness and efficiency of VIS SLAM, a large-scale dataset composed of synchronous multi-sensor data collected from typical indoor parking sites is established, which is the first of its kind. The collected dataset has been made publicly available at https://cslinzhang.github.io/VISSLAM/.

Since the low quality of document images will greatly undermine the chances of success in automatic text recognition and analysis, it is necessary to assess the quality of document images uploaded in online business process, so as to reject those images of low quality. In this paper, we attempt to achieve document image quality assessment and our contributions are twofold. Firstly, since document image quality assessment is more interested in text, we propose a text line based framework to estimate document image quality, which is composed of three stages: text line detection, text line quality prediction, and overall quality assessment. Text line detection aims to find potential text lines with a detector. In the text line quality prediction stage, the quality score is computed for each text line with a CNN-based prediction model. The overall quality of document images is finally assessed with the ensemble of all text line quality. Secondly, to train the prediction model, a large-scale dataset, comprising 52,094 text line images, is synthesized with diverse attributes. For each text line image, a quality label is computed with a piecewise function. To demonstrate the effectiveness of the proposed framework, comprehensive experiments are evaluated on two popular document image quality assessment benchmarks. Our framework significantly outperforms the state-of-the-art methods by large margins on the large and complicated dataset.

Document image quality assessment (DIQA) is an important and challenging problem in real applications. In order to predict the quality scores of document images, this paper proposes a novel no-reference DIQA method based on character gradient, where the OCR accuracy is used as a ground-truth quality metric. Character gradient is computed on character patches detected with the maximally stable extremal regions (MSER) based method. Character patches are essentially significant to character recognition and therefore suitable for use in estimating document image quality. Experiments on a benchmark dataset show that the proposed method outperforms the state-of-the-art methods in estimating the quality score of document images.

Biometrics authentication has been corroborated to be an effective method for recognizing a person's identity with high confidence. In this field, the use of three-dimensional (3D) ear shape is a recent trend. As a biometric identifier, the ear has several inherent merits. However, although a great deal of efforts have been devoted, there is still large room for improvement in developing a highly effective and efficient 3D ear identification approach. In this paper, we attempt to fill this gap to some extent by proposing a novel 3D ear classification scheme that makes use of the label consistent K-SVD (LC-KSVD) framework. As an effective supervised dictionary learning algorithm, LC-KSVD learns a single compact discriminative dictionary for sparse coding and a multi-class linear classifier simultaneously. To use the LC-KSVD framework, one key issue is how to extract feature vectors from 3D ear scans. To this end, we propose a blockwise statistics-based feature extraction scheme. Specifically, we divide a 3D ear region of interest into uniform blocks and extract a histogram of surface types from each block; histograms from all blocks are then concatenated to form the desired feature vector. Feature vectors extracted in this way are highly discriminative and are robust to mere misalignment between samples. Experiments demonstrate that our approach can achieve better recognition accuracy than the other state-of-the-art methods. More importantly, its computational complexity is extremely low, making it quite suitable for the large-scale identification applications. MATLAB source codes are publicly online available at http://sse.tongji.edu.cn/linzhang/LCKSVDEar/LCKSVDEar. htm.

Conditional nonlinear optimal perturbation (CNOP) is an initial perturbation evolving into the largest nonlinear evolution at the prediction time. It has become a useful tool in meteorology and oceanography. The common method for solving the CNOP is the adjoint-based method which is always referred to as the benchmark. Unfortunately, many numerical models have no corresponding adjoint models, and developing a new one is usually a huge engineering, which consequently limits the application of the CNOP. In order to avoid adjoint models, we propose a principal components-based great deluge method to solve the CNOP. Through extracting principal components, the original problem is reduced into a low-dimensional space to hunt the coordinate of the optimal CNOP with the great deluge method. A regeneration strategy is also designed for the great deluge method to break away from local optimal positions. In addition, the proposed method can be parallelized to improve the computing efficiency. To demonstrate the validity, the proposed method has been studied in the Zebiak-Cane model to solve the CNOP. Experimental results show that the proposed method can efficiently obtain a satisfactory CNOP, approximate to the one computed with the adjoint-based method, and the parallelizing version can reach the speedup of 9.7 times with 12 cores.

In this paper, we propose a novel 3D ear classification scheme, making use of the label consistent K-SVD (LC-KSVD) framework. As an effective supervised dictionary learning algorithm, LC-KSVD learns a compact discriminative dictionary for sparse coding and a multi-class linear classifier simultaneously. To use LC-KSVD, one key issue is how to extract feature vectors from 3D ear scans. To this end, we propose a block-wise statistics based scheme. Specifically, we divide a 3D ear ROI into blocks and extract a histogram of surface types from each block; histograms from all blocks are concatenated to form the desired feature vector. Feature vectors extracted in this way are highly discriminative and are robust to mere misalignment. Experimental results demonstrate that the proposed approach can achieve much better recognition accuracy than the other state-of-the-art methods. More importantly, its computational complexity is extremely low at the classification stage.

Conditional nonlinear optimal perturbation (CNOP) is an extension of linear singular vector(LSV) to nonlinear optimization. Generally, CNOP is solved with such adjoint based algorithms as SPG2, SQP. Unfortunately, it is often difficult to obtain the corresponding adjoint models for some nonlinear models. In addition, for nonlinear models containing discontinuous “on-off” switches, the adjoint based methods can hardly find the correct gradient direction for solving the CNOP. These two factors restrict the application of CNOP. Intelligence algorithms are utilized to handle these problems and has made some improvements. Nevertheless, the intelligent method cannot be applied to solve CNOP of complex models due to its dimensional limitation. Therefore, the principal component analysis based genetic algorithm (PCAGA) is proposed to solve the CNOP of complex models. The PCAGA is composed of two key processes: dimension reduction and genetic optimization. To demonstrate the validity, PCAGA is applied to solve the CNOP of the Zebiak-Cane (ZC) model for studying ENSO predictability and compared with the adjoint based method. Experimental results show that PCAGA can achieve similar results to the adjoint based method in the high-dimensional space of a medium- complexity model without the adjoint models. The proposed method also can future be applied to solve variational data assimilation (VDA).

Developing 3D palmprint recognition systems has recently begun to draw attention of researchers. Compared with its 2D counterpart, 3D palmprint has several unique merits. However, most of the existing 3D palmprint matching methods are designed for one-to-one verification and they are not efficient to cope with the one-to-many identification case. In this paper, we fill this gap by proposing a collaborative representation (CR) based framework with l1-norm or l2-norm regularizations for 3D palmprint identification. The effects of different regularization terms have been evaluated in experiments. To use the CR-based classification framework, one key issue is how to extract feature vectors. To this end, we propose a block-wise statistics based feature extraction scheme. We divide a 3D palmprint ROI into uniform blocks and extract a histogram of surface types from each block; histograms from all blocks are then concatenated to form a feature vector. Such feature vectors are highly discriminative and are robust to mere misalignment. Experiments demonstrate that the proposed CR-based framework with an l2-norm regularization term can achieve much better recognition accuracy than the other methods. More importantly, its computational complexity is extremely low, making it quite suitable for the large-scale identification application. Source codes are available at http://sse.tongji.edu.cn/linzhang/cr3dpalm/cr3dpalm.htm.

With the rapid development of the usage of digital imaging and communication technologies, there appears to be a great demand for fast and practical approaches for image quality assessment (IQA) algorithms that can match human judgements. In this paper, we propose a novel general-purpose no-reference IQA (NR-IQA) framework by means of learning quality-aware filters (QAF). Using these filters for image encoding, we can obtain effective image representations for quality estimation. Additionally, random forest is used to learn the mapping from feature space to human subjective scores. Extensive experiments conducted on LIVE and CSIQ databases demonstrate that the proposed NR-IQA metric QAF can achieve better prediction performance than all the other state-of-the-art NR-IQA approaches in terms of both prediction accuracy and generalization capabilities.

Recent years have witnessed a growing interest in developing methods for 3D face recognition. However, 3D scans often suffer from the problems of missing parts, large facial expressions, and occlusions. In this paper, we propose a novel general approach to deal with the 3D face recognition problem by making use of multiple keypoint descriptors (MKD) and the sparse representation-based classifier (SRC). We call the proposed method 3DMKDSRC for short. Specifically, with 3DMKDSRC, each 3D face scan is represented as a set of descriptor vectors extracted from keypoints by meshSIFT. Descriptor vectors of gallery samples form the gallery dictionary. Given a probe 3D face scan, its descriptors are extracted at first and then its identity can be determined by using a multitask SRC. The effectiveness of 3DMKDSRC has been corroborated by extensive experiments.

Perceptual image quality assessment (IQA) aims to use computational models to measure the image quality in consistent with subjective evaluations. Visual saliency (VS) has been widely studied by psychologists, neurobiologists, and computer scientists during the last decade to investigate, which areas of an image will attract the most attention of the human visual system. Intuitively, VS is closely related to IQA in that suprathreshold distortions can largely affect VS maps of images. With this consideration, we propose a simple but very effective full reference IQA method using VS. In our proposed IQA model, the role of VS is twofold. First, VS is used as a feature when computing the local quality map of the distorted image. Second, when pooling the quality score, VS is employed as a weighting function to reflect the importance of a local region. The proposed IQA index is called visual saliency-based index (VSI). Several prominent computational VS models have been investigated in the context of IQA and the best one is chosen for VSI. Extensive experiments performed on four large-scale benchmark databases demonstrate that the proposed IQA index VSI works better in terms of the prediction accuracy than all state-of-the-art IQA indices we can find while maintaining a moderate computational complexity. The MATLAB source code of VSI and the evaluation results are publicly available online at http://sse.tongji.edu.cn/linzhang/IQA/VSI/VSI.htm.

Salient regions detection from images is an important and fundamental research problem in neuroscience and psychology and it serves as an indispensible step for numerous machine vision tasks. In this paper, we propose a novel conceptually simple salient region detection method, namely SDSP, by combining three simple priors. At first, the behavior that the human visual system detects salient objects in a visual scene can be well modeled by band-pass filtering. Secondly, people are more likely to pay their attention on the center of an image. Thirdly, warm colors are more attractive to people than cold colors are. Extensive experiments conducted on the benchmark dataset indicate that SDSP could outperform the other state-of-the-art algorithms by yielding higher saliency prediction accuracy. Moreover, SDSP has a quite low computational complexity, rendering it an outstanding candidate for time critical applications. The Matlab source code of SDSP and the evaluation results have been made online available at http://sse.tongji.edu.cn/linzhang/va/SDSP/SDSP.htm.

Recently, ear shape has attracted tremendous interests in biometric research due to its richness of feature and ease of acquisition. In this paper, we present a novel 3D ear identification approach based on the sparse representation framework. To this end, at first, we propose a template-based ear detection method. By utilizing such a method, the extracted ear regions are represented in a common standard coordinate system determined by the template, which facilitates the following feature extraction and classification. For each 3D ear, a feature vector can be generated as its representation. With respect to the ear identification, we resort to the l1-minimization based sparse representation. Experiments conducted on a benchmark dataset corroborate the effectiveness and efficacy of the proposed approach. The associated Matlab source code and the evaluation results have been made online available at http://sse.tongji.edu.cn/linzhang/ear/srcear/srcear.htm.

The goal of no-reference image quality assessment (NR-IQA) is to estimate the quality of an image consistent with the human perception of the image automatically without any prior of the reference image. In this paper, we present a simple yet efficient and effective approach to learn a blind Image Quality index based on Visual saliency guided sampling and Gabor filtering, namely IQVG. Given an image, we at first randomly sample a sufficient number of image patches guided by the image's visual saliency map and convolve each patch with Gabor filters to get a bag of features. Then, the image is represented by using a histogram to encode the bag of features. Support vector regression (SVR) is used to learn the mapping from feature space to image quality. Extensive experiments conducted on the LIVE IQA database demonstrate the overall superiority of our IQVG over the other state-of-the-art NR-IQA algorithms evaluated. The Matlab source code of IQVG and the evaluation results are available online at http://sse.tongji.edu.cn/linzhang/IQA/IQVG/IQVG.htm.

Compared with tristimulus, spectrum contains much more information of a color, which can be used in many fields, such as disease diagnosis and material recognition. In order to get an accurate and stable reconstruction of spectral data from a tristimulus input, a method based on locally linear approximation is proposed in this paper, namely SR-LLA. To test the performance of SR-LLA, we conduct experiments on three Munsell databases and present a comprehensive analysis of its accuracy and stability. We also compare the performance of SR-LLA with the other two spectral reconstruction methods based on BP neural network and PCA, respectively. Experimental results indicate that SR-LLA could outperform other competitors in terms of both accuracy and stability for spectral reconstruction.

Automatic image quality assessment (IQA) attempts to use computational models to measure the image quality in consistency with subjective ratings. In the past decades, dozens of IQA models have been proposed. Though some of them can predict subjective image quality accurately, their computational costs are usually very high. To meet real-time requirements, in this paper, we propose a novel fast and effective IQA index, namely spectral residual based similarity (SR-SIM), based on a specific visual saliency model, spectral residual visual saliency. SR-SIM is designed based on the hypothesis that an image's visual saliency map is closely related to its perceived quality. Extensive experiments conducted on three large-scale IQA datasets indicate that SR-SIM could achieve better prediction performance than the other state-of-the-art IQA indices evaluated. Moreover, SR-SIM can have a quite low computational complexity. The Matlab source code of SR-SIM and the evaluation results are available online at http://sse.tongji.edu.cn/linzhang/IQA/SR-SIM/SR-SIM.htm.

In this paper, we present a simple yet efficient and effective multi-resolution approach to gray-scale and rotation invariant texture classification. Given a texture image, we at first convolve it with J Gabor filters sharing the same parameters except the parameter of orientation. Then by binarizing the obtained responses, we can get J bits at each location. Then, each location can be assigned a unique integer, namely “rotation invariant binary Gabor pattern (BGPri)”, formed from J bits associated with it using some rule. The classification is based on the image's histogram of its BGPris at multiple scales. Using BGPri, there is no need for a pre-training step to learn a texton dictionary, as required in methods based on clustering such as MR8. Extensive experiments conducted on the CUReT database demonstrate the overall superiority of BGPriover the other state-of-the-art texture representation methods evaluated. The Matlab source codes are publicly available at http://sse.tongji.edu.cn/linzhang/IQA/BGP/BGP.htm.

Recent years have witnessed a growing interest in developing automatic palmprint recognition methods. Among them, coding-based ones, representing the texture of a palmprint using a binary code, are most prevalent and successful. We find that not all bits in a code map generated by a specific coding scheme are equally consistent. A bit is deemed fragile if its value changes across code maps created from different images of the same palmprint. In this paper, we first analyze the fragile bits phenomenon in a state-of-the-art palmprint coding scheme, namely, binary orientation co-occurrence vector (BOCV). Then, based on our analysis, we extend BOCV to E-BOCV by incorporating fragile bits information in appropriate ways. Experiments conducted on the benchmark dataset demonstrate that E-BOCV can achieve the highest verification accuracy among all the state-of-the-art palmprint verification methods evaluated. To our knowledge, this is the first work investigating the fragile bits of coding-based palmprint recognition approaches.

Eigen-decomposition is a key step in spectral clustering and some kernel methods. The Nyström method is often used to speed up kernel matrix decomposition. However, it cannot effectively update eigenvectors of matrices when datasets dynamically increase with time. In this paper, we propose an incremental Nyström method for dynamic learning. Experimental results demonstrate the feasibility and effectiveness of the proposed method.

In this paper, we present a novel 3D model alignment method by analyzing the voxels of 3D meshes and a visual similarity based 3D model matching and retrieving method using active tabu search. Firstly, each 3D model is voxelized and applied voxels based PCA transformation, then it is represented by six depth images which are projected by rendering in the PCA coordinate system. Hybrid descriptors are extracted from these depth images to represent the origin 3D model shape features. Matching and retrieving is performed when geometric manifold entropy based active tabu search is used to index all the models in the library by its associated sets of depth images, then the dissimilarity between 3D models are computed from this indexed depth images dataset. Finally, in order to accelerate our proposed approach, all the key operations were implemented on GPU platform using its high parallel architecture. Experimental results show that our proposed method achieve better shape matching effect and gain absolutely improvement in retrieval performances on the Princeton 3D Shape Benchmark database.

Manifold clustering aims to partition a set of input data into several clusters each of which contains data points from a separate, simple low-dimensional manifold. This paper presents a novel solution to this problem. The proposed algorithm begins by randomly selecting some neighboring orders of the input data and defining an energy function that is described by geometric features of underlying manifolds. By minimizing such energy using the tabu search method, an approximately optimal sequence could be found with ease, and further different manifolds are separated by detecting some crucial points, boundaries between manifolds, along the optimal sequence. We have applied the proposed method to both synthetic data and real image data and experimental results show that the method is feasible and promising in manifold clustering.

In this paper, we propose an approach for 2D discrete vector field segmentation based on the Green function and normalized cut. The method is inspired by discrete Hodge Decomposition such that a discrete vector field can be broken down into three simpler components, namely, curl-free, divergence-free, and harmonic components. We show that the Green Function Method (GFM) can be used to approximate the curl-free and the divergence-free components to achieve our goal of the vector field segmentation. The final segmentation curves that represent the boundaries of the influence region of singularities are obtained from the optimal vector field segmentations. These curves are composed of piecewise smooth contours or streamlines. Our method is applicable to both linear and nonlinear discrete vector fields. Experiments show that the segmentations obtained using our approach essentially agree with human perceptual judgement.

Eddy tracking is just finding all the vectors with similar motion in its influence region and recording its moving route. Virtually, a flow field is able to be replaced with a scalar dataset, if its feature information can be faithfully preserved by the scalar dataset. Therefore, the eddy tracking problem can be solved by tracking the moving objects in an image sequence after a certain transformation. Here, we propose a new method to track several moving eddies simultaneously and three main technologies are introduced: green function method, motion probability distribution and quadratic program.