The semantic-segmentation-based scene text detection algorithms always use the bounding-box regions or their shrinks to represent the text pixels. However, the non-text pixel information in these regions easily results in the poor performance of text detection, because these semantic segmentation methods need accurate pixel-level annotated training data to achieve approving performance and they are sensitive to noise and interference. In this work, we propose a fully convolutional network (FCN) based method termed TextEdge for multi-oriented scene text detection. Compared with previous methods simply using bounding-box regions as a segmentation mask, TextEdge introduces the text-region edge map as a new segmentation mask. Edge information is more representative for text areas and is proved to be effective in improving detection performance. TextEdge is optimized in an end-to-end way with multi-task outputs: text and non-text classification, text-edge prediction and the text boundaries regression. Experiments on standard datasets demonstrate that the proposed method achieves state-of-the-art performance in both accuracy and efficiency. Specifically, it achieves an F-score of 0.88 on ICDAR 2013 dataset and 0.86 on ICDAR 2015 dataset.

Recently, numerous deep learning based scene text detection methods have achieved promising performances in different text detecting tasks. Most of these methods are trained in a supervised way, which requires a large amount of annotated data. In this paper, we explore a weakly supervised method to locate text regions in scene images. We propose a fully convolutional network (FCN) architecture to implement binary classification. The training data we used do not need any text location annotation, we only need to divide the training data into two categories according to whether it contains text or not. We can obtain the text localization map (TLM) directly from the last convolutional layer. By setting a fixed threshold, the TLM is converted to a mask map. Then the connected component analysis and the text proposals method based on Maximally Stable Extremal Regions (MSERs) are used to get the text region bounding boxes. We conduct comprehensive experiments on standard text datasets. The results show that our text localization method achieves comparable recall performance with other methods and has more stable property.

In ground-based remote sensing cloud image observation, images with the highest possible resolution are captured to obtain sufficient information about clouds. However, when features are extracted and classification is performed on the basis of the original images, a high-resolution probably means a high (or even more, unacceptable) computation cost. In practical application, a simple and commonly adopted method is to appropriately resize the original image to a version with a decreased resolution. An inevitable problem is whether useful information is lost in this resizing operation. This paper demonstrates that information loss is inevitable and poor classification results may be obtained from the analysis of local binary pattern (LBP) histogram features. However, this problem has been always neglected in previous studies, and the original image is arbitrarily resized without any criterion. In particular, the histogram features based on LBPs actually reflect the distribution of features. Thus, a criterion based on the Kullback-Leibler divergence between LBP histograms from the original and resized images and a penalty term imposed on the resolution are proposed to select the resolution of the resized image. The optimal resolution of the resized image can be selected by minimizing this criterion. Furthermore, experiments based on three ground-based remote sensing cloud image data sets with different original resolutions validate this criterion by analyzing the LBP histogram features.

In this paper, we propose a simple but effective semantic-based aggregation (SBA) method. The proposed SBA utilizes the discriminative filters of deep convolutional layers as semantic detectors. Moreover, we propose the effective unsupervised strategy to select some semantic detectors to generate the "soft region proposals," which highlight certain discriminative pattern of objects and suppress the noise of background. The final global SBA representation could then be acquired by aggregating the regional representations weighted by the selected "soft region proposals" corresponding to various semantic content. Our unsupervised SBA is easy to generalize and achieves excellent performance on various tasks. We conduct comprehensive experiments and show that our unsupervised SBA outperforms the state-of-the-art unsupervised and supervised aggregation methods on image retrieval, place recognition, and cloud classification.

Existing manifold learning methods are not appropriate for image retrieval tasks, because most of them are unable to process query images and they have much greater computational cost especially for large-scale database. Therefore, we propose the iterative manifold embedding (IME) layer, of which the weights are learned offline by an unsupervised strategy, to explore the intrinsic manifolds by incomplete data. On the large-scale database that contains 27 000 images, the IME layer is more than 120 times faster than other manifold learning methods to embed the original representations at query time. We embed the original descriptors of database images that lie on manifold in a high-dimensional space into manifold-based representations iteratively to generate the IME representations in an offline learning stage. According to the original descriptors and the IME representations of database images, we estimate the weights of the IME layer by ridge regression. In the online retrieval stage, we employ the IME layer to map the original representation of a query image with an ignorable time cost (2 ms per image). We experiment on five public standard datasets for image retrieval. The proposed IME layer significantly outperforms the related dimension reduction methods and manifold learning methods. Without postprocessing, our IME layer achieves a boost in the performance of state-of-the-art image retrieval methods with postprocessing on most datasets, and needs less computational cost. The code is available at https://github.com/XJhaoren/IME_layer.

In this paper, we propose a simple but effective semantic part-based weighting aggregation (PWA) for image retrieval. The proposed PWA utilizes the discriminative filters of deep convolutional layers as part detectors. Moreover, we propose the effective unsupervised strategy to select some part detectors to generate the "probabilistic proposals," which highlight certain discriminative parts of objects and suppress the noise of background. The final global PWA representation could then be acquired by aggregating the regional representations weighted by the selected "probabilistic proposals" corresponding to various semantic content. We conduct comprehensive experiments on four standard datasets and show that our unsupervised PWA outperforms the state-of-the-art unsupervised and supervised aggregation methods.

Cloud type classification based on ground-based cloud image observations is an important task in atmospheric research. Currently, two kinds of cloud image observations with infrared and visible light images are widely used for cloud classification. However, they are only independently analyzed and simply compared in the current study. The useful information from these two kinds of images is not fully utilized and integrated. The classification performance could be improved if taking full advantage of the complementary information of these two observations. Thus, first, a database containing these two kinds of cloud images with same temporal resolution is released in this study. Then, a two-observation joint encoding strategy of LBP (local binary pattern) features is proposed to implement cloud classification by encoding the joint distribution of LBP patterns in different observations, which captures the correlation between two observations. Experimental results based on this database show the significant superiority of the proposed method compared to the results based on the single observation.

The faint text document images possess shallow characters inherently and the camera-captured form introduces more degradations such as low-resolution, non-uniform illumination and out-of-focus blur, which make the text binarization very difficult. In this paper, we propose a grayscale-projection based optimal character segmentation method for camera-captured faint text recognition. Instead of extracting the character candidates, we use the gradient projection to extract a series of segmentation candidates which contain inter-character gaps and intra-character gaps as well. In order to select the optimal segmentation path from all possible situations, we construct a segmentation tree and set a evaluation score for each path. The score integrates the information of single point projection, overall distribution and recognition probability. Finally the optimal segmentation path is obtained by selecting the path with the highest score. We collect a faint text recognition dataset and evaluate our method on it. Experimental results show that our method outperforms the binary-projection method and the convolutional recurrent neural network approach in terms of text segmentation and recognition accuracy.

Recognizing scene character is extremely challenging due to various interference factors such as character translation, blur and uneven illumination, etc. Considering that characters are composed of a series of parts and different parts attract diverse attentions when people observe a character, we should assign different importance to each part to recognize scene character. In this paper, we propose a discriminative character representation by aggregating the responses of the spatially embedded salient part detectors. Specifically, we first extract the convolution activations from the pre-trained convolutional neural network (CNN). These convolutional activations are considered as the local descriptors of the character parts. Then we learn a set of part detectors and pick the distinctive convolutional activations which respond to the salient parts. Moreover, to alleviate the effect of character translation, rotation and deformation, etc, we assign a response region for each part detector and search the maximal response in this region. Finally, we aggregate the maximal outputs of all the salient part detectors to represent character. The experiments on three datasets show the effectiveness of the proposed method for scene character recognition.

Several recent works interpret convolutional features produced by deep convolutional neural networks as local descriptors. Existing high-dimensional aggregation based methods, e.g., Fisher Vector (FV) obtain inferior performance to pooling based methods in most situations, and we observe that it is mainly caused by the ignorance of spatial weights. In this paper, we propose a novel method named spatial weighted Fisher Vector (SWFV) to enhance the representation of FV by injecting the spatial weight map to FV. In addition, we further analyze the distribution of spatial weights and propose truncated spatial weighted FV (TSWFV). Experimental results on two benchmark datasets demonstrate that the two proposed methods achieve competitive results compared with other global representation based methods.

Cloud detection plays an important role in climate models, climate predictions, and meteorological services. Although researchers have given increasing efforts on cloud detection, the performance is still unsatisfactory due to the diverse nature of clouds. Considering the fact that one source of information (color or texture) is not enough to segment cloud from clear sky, in this letter, we propose a novel ground-based cloud detection method using graph model (GM) built upon superpixels to integrate multiple sources of information. First, we use the superpixel segmentation to divide the image into a series of subregions according to the color similarity and spatial continuity. Next, adjacent superpixels are merged according to their similarity of extracted features. Finally, we build a GM on the merged superpixels by considering each superpixel as a node and adding edges between neighboring ones. The unary cost is set according to the classification score of Random Forests, while pairwise cost reflects the penalties for color and texture discontinuity between neighboring components. The final segmentation could be acquired by minimizing the cost function. Moreover, the algorithm is computationally efficient as we use the superpixels rather than raw pixels as computation units. Experimental results demonstrate the effectiveness and efficiency of the proposed method for cloud detection.

Ground-based cloud classification is crucial for meteorological research and has received great concern in recent years. However, it is very challenging due to the extreme appearance variations under different atmospheric conditions. Although the convolutional neural networks have achieved remarkable performance in image classification, no one has evaluated their suitability for cloud classification. In this letter, we propose to use the deep convolutional activations-based features (DCAFs) for ground-based cloud classification. Considering the unique characteristic of cloud, we believe the local rich texture information might be more important than the global layout information and, thus, give a comprehensive evaluation of using both shallow convolutional layers-based features and DCAFs. Experimental results on two challenging public data sets demonstrate that although the realization of DCAF is quite straightforward without any use-dependent tricks, it outperforms conventional hand-crafted features considerably.

Currently, ground-based cloud images taken by using a whole-sky imager are especially popular in the field of meteorology because of their high resolution and accurate cloud information. Cloud images are natural texture images, and thus texture features based on local binary patterns (LBPs) are widely used to analyze texture images. However, the high-computation cost of extracting LBP features from high-resolution cloud texture images may make this technique unacceptable in practical image processing. A commonly adopted method is to resize the original image to an appropriate version with a decreased resolution. But this process will inevitably result in information loss. Accordingly, a measure based on the Kullback-Leibler (KL) divergence of the difference between LBP histogram features extracted from the original and resized images with varying resolutions is reported in this letter. Furthermore, a confidence interval technique is introduced to validate the significance of such difference. Experiments based on real ground-based cloud images show the measurement results of KL divergence in LBP features extracted from original and resized images. The experimental results indicate that images should be resized with caution when performing image processing.

This letter presents a novel approach for logo retrieval. Considering the fact that logo only occupies a small portion of an image, we apply Faster R-CNN to detect logo proposals first, and then use a two-step pooling strategy with adaptive weight to obtain an accurate global signature. The adaptive weighted pooling method can effectively balance the recall and precision of proposals by incorporating the probability of each proposal being a logo. Experimental results show that the proposed method interprets the similarity between query and database image more accurately and achieves state of the art performance.

In this paper, a novel local threshold binarization method using structural symmetry of strokes is proposed. Different from most existing local threshold methods which use the whole region to compute the threshold, we estimate the local threshold by only using the structural symmetric pixels (SSP) of the region so as to suppress the non-text pixels and maintain the text ones as well. The SSP is defined as those pixels around strokes whose gradient magnitudes are big enough and directions are opposite. As the gradient map is our basis for computing the SSP, we further propose to estimate background surface first and extract potential SSP in the compensated image so as to deal with degradations of document images such as uneven illumination, low contrast and stain. To prove the effectiveness of our method, tests on two public document image datasets are preformed and the experimental results show that our method outperforms other local threshold binarization approaches on both F-measure and PSNR.

One difficulty in handwritten Chinese character recognition (HCCR) is due to the large number of similar characters. In this study, we propose a connection reduced network (CRN) to discriminate similar pairs. Each hidden neuron in CRN is restricted to has one input signal and the strength of this input is set as a variable which is selected from the input of the network. Experimental results based on 100 similar pairs demonstrate that the proposed method yields highly competitive test recognition results compared to the state-of-the-art methods, while consuming less memory and time resources.

Gamma correction, a nonlinear operation, has long been used to code and decode luminance or tristimulus values in video or still image systems [1]. In this paper, we make the following observations: for CAPTCHA images which could not be well binarized using the threshold of OTSU, there exists a gamma corrected image which could be well segmented by the OTSU threshold and the value of the best gamma could be revealed by observing the maximal inter-class variance (MICV) values of different images transformed by different values of gamma. Concretely, we convert the R, G, B channels of the original CAPTCHA image with different gamma values and transform the color images to gray-level images. Each gray-level image could be then segmented by the threshold acquired by OTSU. By linking each gamma value with the corresponding maximal inter-class variance value, we could draw a changing curve of variance values versus gamma. The best gamma could be acquired by finding the point whose related MICV starts to change slowly. Moreover, the polarity of the image could also be revealed by the changing trend of the curve. Experimental results on different categories of CAPTCHA images demonstrate the effectiveness of the observations for binarizing the CAPTCHA images and telling the polarity as well.

In this paper, we propose a novel method for cross-view action recognition via multiple continuous virtual paths which connect the source view and the target view. Each point on one virtual path is a virtual view which is obtained by a linear transformation of an action descriptor. All the virtual views are concatenated into an infinite-dimensional feature to characterize continuous changes from the source to the target view. To utilize these infinite-dimensional features directly, we propose a virtual view kernel (VVK) to compute the similarity between two infinite-dimensional features, which can be readily used to construct any kernelized classifiers. In addition, a constraint term is introduced to fully utilize the information contained in the unlabeled samples which are easier to obtain from the target view. The rationality behind the constraint is that any action video belongs to only one class. To further explore complementary visual information, we utilize multiple continuous virtual paths. The original source and target views are projected to different auxiliary source and target views using the random projection technique. Then we fuse all the VVKs generated from all pairs of auxiliary views. Our method is verified on the IXMAS and MuHAVi datasets, and the experimental results demonstrate that our method achieves better performance than the state-of-the-art methods.

This paper presents a novel binarization technique for text images based on Markov Random Field (MRF) framework. We regard stroke as an obvious feature of text to produce clustering result, which will be optimized by MRF model combining color, texture, context features to get the final binarization. The main innovations of our method are: (1) the integrated image is split into sub-images on which we can automatically acquire seed pixels of foreground and background using stroke feature; and (2) diverse weights are attached to seed pixels according to their location information, then highly confident cluster centers of sub-image can be acquired by gathering weighted seeds. The experimental results show that our method is robust and accurate on both video and scene images.

Text information contained in scene images is very helpful for high‐level image understanding. In this study, the authors propose to learn co‐occurrence of local strokes for scene text recognition by using a spatiality embedded dictionary (SED). Unlike spatial pyramid partitioning images into grids to incorporate spatial information, the authors SED associates every codeword with a particular response region and introduces more precise spatial information for robust character recognition. After localised soft coding and max pooling of the first layer, a sparse dictionary is learned to model co‐occurrence of several local strokes, which further improves classification performance. Experimental results on two scene character recognition datasets ICDAR2003 and CHARS74 K demonstrate that their character recognition method outperforms state‐of‐the‐art methods. Besides, competitive word recognition results are also reported for four benchmark word recognition datasets ICDAR2003, ICDAR2011, ICDAR2013 and street view text when combining their character recognition method with a conditional random field language model.

View variation is a major challenge in face recognition. In this study, the authors propose a novel cross‐view face recognition method by seeking potential intermediate domains between the source and target views to model the connection of varying‐views faces. Specifically, each intermediate domain is associated with a dictionary subspace. Learning proceeds in two phases. First, the authors discriminatively train a sub‐dictionary for each subclass of data, which then compose a structured dictionary of powerful reconstructive and discriminative capability on the source data. Secondly, the authors gradually adapt the source domain dictionary to the target domain by incrementally reducing the reconstruction error on the target data, which forms a smooth transition path connecting the source and target domains. Instead of updating the structured dictionary integrally, the authors develop a refined sub‐dictionary‐based updating algorithm, which makes the intermediate dictionaries fit on the target data better and faster. Finally, the authors apply invariant sparse codes across the source, intermediate and target domains to render domain‐shared representations, where the sample differences caused by view changes are reduced. Experiments on the CMU‐PIE and Multi‐PIE dataset demonstrate the effectiveness of the proposed method.

Cloud detection plays an essential role in meteorological research and has received considerable attention in recent years. However, this issue is particularly challenging due to the diverse characteristics of clouds. In this letter, a novel algorithm based on superpixel segmentation (SPS) is proposed for cloud detection. In our proposed strategy, a series of superpixels could be obtained adaptively by SPS algorithm according to the characteristics of clouds. We first calculate a local threshold for each superpixel and then determine a threshold matrix for the whole image. Finally, cloud can be detected by comparing with the obtained threshold matrix. Experimental results show that our proposed algorithm achieves better performance than the current cloud detection algorithms.

Characters, which are man-made symbols composed of strokes arranged in a certain structure, could provide semantic information and play an indispensable role in our daily life. In this paper, we try to make use of the intrinsic characteristics of characters and explore the stroke and structure-based methods for character recognition. First, we introduce two existing part-based models to recognize characters by detecting the elastic strokelike parts. In order to utilize strokes of various scales, we propose to learn the discriminative multi-scale stroke detector-based representation (DMSDR) for characters. However, the part-based models and DMSDR need to manually label the parts or key points for training. In order to learn the discriminative stroke detectors automatically, we further propose the discriminative spatiality embedded dictionary learning-based representation (DSEDR) for character recognition. We make a comparative study of the performance of the tree-structured model (TSM), mixtures-of-parts TSM, DMSDR, and DSEDR for character recognition on three challenging scene character recognition (SCR) data sets as well as two handwritten digits recognition data sets. A series of experiments is done on these data sets with various experimental setup. The experimental results demonstrate the suitability of stroke detector-based models for recognizing characters with deformations and distortions, especially in the case of limited training samples.

In Still-to-Video (S2V)face recognition, only a few high resolution images are enrolled for each subject, while the probe is videos of complex variations. As faces present distinct characteristics under different scenarios, recognition in the original space is obviously inefficient. In this paper, we propose a novel discriminant analysis method to learn separate mappings for different scenarios (still, video), and further pursue a common discriminant space based on these mappings. Concretely, by modeling each video as a set of local models, we form the scenario-oriented mapping learning as an Image-Model discriminant analysis framework. The learning objective is formulated by incorporating the intra-class compactness and inter-class separability for good discrimination. Moreover, a weighted learning scheme is introduced to concentrate on the discriminating information of the most confusing samples and then further enhance the performance. Experiments on the COX-S2V dataset demonstrate the effectiveness of the proposed method.

In the Still-to-Video (S2V) face recognition, each subject is enrolled with only few high resolution images, while the probe is video clips of complex variations. As faces present distinct characteristics under different scenarios, recognition in the original space is obviously inefficient. Therefore, in this paper, we propose a novel discriminant analysis method to learn separate mappings for different scenario patterns (still, video), and further pursue a common discriminant space for the cross-scenario samples. To maximize the intra-individual correlation of samples in the mapping space, we formulate the learning objective by incorporating the intra-class compactness and the inter-class dispersion. The gradient descend algorithm is used to get the optimal solution. Experimental results on the COX-S2V dataset demonstrate the effectiveness of the proposed method and remarkable superiority over state-of-art methods.

Pose variation is a major challenge in face recognition. In this paper, we propose a novel cross-pose face recognition method by learning associate appearance manifolds to model the connection of faces under different poses. The associate manifolds are built on an auxiliary set, in which each identity contains cross-pose face images. The basic assumption is that cross-pose face images from two similar identities can be projected onto similar appearance manifolds by pose-specific transforms. We first associate the input faces with alike identities from the auxiliary set. Then the manifolds of cross-pose faces in the training set are confined close to that of the associate identities in the auxiliary set. Thus, the connection of cross-pose faces is well modeled by the associate appearance manifolds on the auxiliary set. Formally, we formulate the assumption as a manifold-based distance minimization problem, so as to learn the optimal transforms. Experiments on the Multi-PIE dataset demonstrate the effectiveness of the proposed method.

Robust scene-text-extraction system can be used in lots of areas. In this work, we propose to learn co-occurrence of local strokes for robust character recognition by using a spatiality embedded dictionary (SED). Different from spatial pyramid partitioning images into grids to incorporate spatial information, our SED associates every codeword with a particular response region and introduces more precise spatial information for character recognition. After localized soft coding and max pooling of the first layer, a sparse dictionary is learned to model co-occurrence of several local strokes, which further improves classification performance. Experiment on benchmark datasets demonstrates the effectiveness of our method and the results outperform state-of-the-art algorithms.

Text information contained in scene images is very useful for image understanding. In this paper, we propose a high-level representation named stroke bank for scene character recognition. Inspired by the work of object bank, we train stroke detectors and use detectors' maximal output as features. Specifically, we collect training samples for stroke detectors based on labeled key points. We also propose to restrict classification areas of each stroke detector to particular local regions, which alleviates computation burden and retains discrimination power at the same time. Experiments on benchmark datasets demonstrate the effectiveness of our method and the results outperform state-of-the-art algorithms.

Pooling strategies, such as max pooling and sum pooling, have been widely used to obtain the global representations for action videos. However, these pooling strategies have several disadvantages. First, they are easily affected by unwanted background local features, the absence of discriminative local features and the times of actions periodically performed by actors. Second, most pooling strategies only use local features to build the global representation that captures little mid‐level features for action representation. In this study, the authors propose a novel weighted pooling strategy based on actionlets representation for action recognition. The actionlets are defined as the movements of large bodies such as legs, arms and head, which capture rich mid‐level features for action representation. Besides, the authors’ method also incorporates the distribution information of actionlets into pooling procedure. Specifically, a pooling weight, which determines the importance of actionlet on the final video representation, is assigned to each actionlet. To learn the weight, they propose a novel discriminative learning algorithm to capture the discriminative information for pooling operation. They evaluate their weighted pooling on three datasets: KTH actions dataset, UCF sports dataset and Youtube actions dataset. Experimental results show the effectiveness of the proposed method.

Image matching based on local features is a challenging task because it is difficult to build a robust local descriptor which is invariant to large variations in scale, viewpoints, illumination and rotation. To address these issues, Scale Invariant Feature Transform (SIFT) descriptor has been proposed to build a robust and distinctive local descriptor. However, it is not fully affine invariant. In this letter, we propose a novel robust descriptor: Sampling based Local Descriptor (SLD) to perform reliable image matching under large variations in scale, viewpoints, illumination and rotation. We build the descriptor based on elliptical sampling which samples image pixels according to the elliptic equations. The main advantage of elliptical sampling is that two controllable parameters of elliptical sampling can generate descriptors with different viewpoints and rotations. Besides, the descriptor has two notable properties: 1) it is fully invariant to affine changes; 2) it enables fast matching process because we only need to search two controllable parameters for elliptical sampling, which is more efficient than other affine invariant descriptors. We test the proposed descriptor on standard benchmark for evaluation. Experimental results show the robustness of the proposed method under large variations in illumination, viewpoints and scale.

Scene text recognition has inspired great interests from the computer vision community in recent years. In this paper, we propose a novel scene text-recognition method integrating structure-guided character detection and linguistic knowledge. We use part-based tree structure to model each category of characters so as to detect and recognize characters simultaneously. Since the character models make use of both the local appearance and global structure informations, the detection results are more reliable. For word recognition, we combine the detection scores and language model into the posterior probability of character sequence from the Bayesian decision view. The final word-recognition result is obtained by maximizing the character sequence posterior probability via Viterbi algorithm. Experimental results on a range of challenging public data sets (ICDAR 2003, ICDAR 2011, SVT) demonstrate that the proposed method achieves state-of-the-art performance both for character detection and word recognition.

Recently, maximum margin clustering (MMC) has been proposed for a cross-view action recognition. However, such a method neglects the temporal relationship between contiguous frames in the same action video. In this paper we propose a novel method called contextual maximum margin clustering (CMMC) to tackle cross-view action recognition. In CMMC, we add temporal regularization to give a high penalty when the contiguous frames are dissimilar. Thus, the CMMC not only achieves the goal of finding maximum margin hyperplanes, but also explicitly considers the temporal information among contiguous frames. Our method is verified on the IXMAS dataset and the experimental results demonstrate that our method can achieve better performance than the state-of-the-art methods.

Scene text recognition has inspired great interests from the computer vision community in recent years. In this paper, we propose a novel scene text recognition method using part-based tree-structured character detection. Different from conventional multi-scale sliding window character detection strategy, which does not make use of the character-specific structure information, we use part-based tree-structure to model each type of character so as to detect and recognize the characters at the same time. While for word recognition, we build a Conditional Random Field model on the potential character locations to incorporate the detection scores, spatial constraints and linguistic knowledge into one framework. The final word recognition result is obtained by minimizing the cost function defined on the random field. Experimental results on a range of challenging public datasets (ICDAR 2003, ICDAR 2011, SVT) demonstrate that the proposed method outperforms state-of-the-art methods significantly both for character detection and word recognition.

In this paper, we propose a novel method for cross-view action recognition via a continuous virtual path which connects the source view and the target view. Each point on this virtual path is a virtual view which is obtained by a linear transformation of the action descriptor. All the virtual views are concatenated into an infinite-dimensional feature to characterize continuous changes from the source to the target view. However, these infinite-dimensional features cannot be used directly. Thus, we propose a virtual view kernel to compute the value of similarity between two infinite-dimensional features, which can be readily used to construct any kernelized classifiers. In addition, there are a lot of unlabeled samples from the target view, which can be utilized to improve the performance of classifiers. Thus, we present a constraint strategy to explore the information contained in the unlabeled samples. The rationality behind the constraint is that any action video belongs to only one class. Our method is verified on the IXMAS dataset, and the experimental results demonstrate that our method achieves better performance than the state-of-the-art methods.

Detecting text in scene images is very challenging due to complex backgrounds, various fonts and different illumination conditions. Without prior knowledge, a detector previously trained using lots of samples still perform badly on a test image because of the disparities in distributions between the training samples and the testing ones. In this paper, we propose to adapt a pre-trained generic scene text detector towards new scenes by transfer learning. In particular, we choose cascade Adaboost as the detector style and try to re-weight pre-selected features according to their abilities to classify high confidence samples. The proposed adaptation mechanism has been evaluated on ICDAR 2011 scene text detection competition dataset and the encouraging experiments results can be compared with the latest published algorithms.

One of the most difficult challenges in automatic face recognition is computing facial similarity between two images captured in different modalities, called heterogeneous face recognition. In this paper, we propose a novel method, named as coupled latent least squares regression, to improve the heterogeneous face recognition performance. The basic assumption is that the images of one person captured in different modalities can be viewed as modality-specific transforms of a latent ideal object. We formulate this assumption in the least squares regression framework, so as to learn the coupled transforms for different modalities. In particular, the local consistency information in the each modality is considered as a constraint to improve the generalization. Extensive experiments on two cases of heterogeneous face recognition (visible light vs. near infrared, and photo vs. sketch) validate the efficiency of the proposed method.

Feature representations play a crucial role in modern face recognition systems. Most hand-crafted image descriptors usually provide low-level information. In this paper, we propose a novel feature learning method based on deep neural networks to obtain high-level, hierarchical representations for face verification. Learning proceeds in two phases. In the pre-training phase, we train Restricted Boltzmann Machine(RBM) networks for each modular region in the image separately. In the fine-tuning phase, in order to develop good discriminative ability, we stack the RBM networks of each region in deep architecture and combine deep learning with side information constraints in the whole image scale. Finally, we formulate the proposed method as an appropriate optimization problem and adopt gradient descent algorithm to get the optimal solution. We evaluate our method on the LFW dataset. Representations learned from the networks achieve comparable performance (93.11%) to the state-of-art method.

Since clouds are one of the most important meteorological phenomena related to the hydrological cycle and affect Earth radiation balance and climate changes, cloud analysis is a crucial issue in meteorological research. Most researchers only consider the classification task of cloud images while less attention has been paid to the synthesis one. In addition, all the existing research on cloud identification from sky images is based on single cloud images. However, the cloud-measuring devices on the ground actually take one image of the clouds every few minutes and collect a series of cloud images. Thus, the existing methods neglect the temporal information exhibited by contiguous cloud images. To overcome this drawback, in this letter we treat ground-based cloud sequences (GCSs) as dynamic texture. We then propose the Tensor Ensemble of Ground-based Cloud Sequences (eTGCS) model which represents the ensemble of GCSs in a tensor manner. In the eTGCS model, all GCSs form a single tensor, and each GCS is a subtensor of the single tensor. There are two main characteristics of the eTGCS model: 1) All GCSs share an identical mode subspace, which makes the classification convenient, and 2) a new GCS can be synthesized as long as the parameters of the eTGCS model are used. Therefore, less storage space is required. Comprehensive experiments are conducted to prove the superiority of our eTGCS model. The classification accuracy achieves 92.31%, and the synthesized GCSs are similar to the original ones in visual appearance.

Recently, attributes have been introduced as a kind of high-level semantic information to help improve the classification accuracy. Multitask learning is an effective methodology to achieve this goal, which shares low-level features between attributes and actions. Yet such methods neglect the constraints that attributes impose on classes, which may fail to constrain the semantic relationship between the attributes and actions. In this paper, we explicitly consider such attribute-action relationship for human action recognition, and correspondingly, we modify the multitask learning model by adding attribute regularization. In this way, the learned model not only shares the low-level features, but also gets regularized according to the semantic constrains. In addition, since attribute and class label contain different amounts of semantic information, we separately treat attribute classifiers and action classifiers in the framework of multitask learning for further performance improvement. Our method is verified on three challenging datasets (KTH, UIUC, and Olympic Sports), and the experimental results demonstrate that our method achieves better results than that of previous methods on human action recognition.

Background modeling from a stationary camera is a crucial component in video surveillance. Traditional methods usually adopt single feature type to solve the problem, while the performance is usually unsatisfactory when handling complex scenes. In this paper, we propose a multi-scale strategy, which combines both texture and color features, to achieve a robust and accurate solution. Our contributions are two folds: one is that we propose a novel textureoperator named Scale-invariant Center-symmetric Local Ternary Pattern, which is robust to noise and illumination variations, the other is that a multi-scale fusion strategy is proposed for the issue. Our method is verified on several complex real world videoswith illumination variation, soft shadows and dynamic backgrounds. We compare our method with four state-of-the-art methods, and the experimental results clearly demonstrate that our method achievesthe highest classification accuracy in complex real world videos.

Recently, attributes have been introduced to help object classification. Multi-task learning is an effective methodology to achieve this goal, which shares low-level features between attribute and object classifiers. Yet such a method neglects the constraints that attributes impose on classes which may fail to constrain the semantic relationship between the attribute and object classifiers. In this paper, we explicitly consider such attribute-object relationship, and correspondingly, we modify the multi-task learningmodel by adding attribute regularization. In this way, the learned model not only shares the low-level features, but also gets regularized according to the semantic constrains. Our method is verified on two challenging datasets (KTH and Olympic Sports), andthe experimental results demonstrate that our method achieves better results than previous methods in human action recognition.

In this paper, we propose a discriminative low-rank dictionary learning algorithm for sparse representation. Sparse representation seeks the sparsest coefficients to represent the test signal as linear combination of the bases in an over-complete dictionary. Motivated by low-rank matrix recovery and completion, assume that the data from the same pattern are linearly correlated, if we stack these data points as column vectors of a dictionary, then the dictionary should be approximately low-rank. An objective function with sparse coefficients, class discrimination and rank minimization is proposed and optimized during dictionary learning. We have applied the algorithm for face recognition. Numerous experiments with improved performances over previous dictionary learning methods validate the effectiveness of the proposed algorithm.

Interactive image segmentation which needs the user to give certain hard constraints has shown promising performance for object segmentation. In this paper, we consider characters in text image as a special kind of object, and propose an adaptive graph cut based text binarization method to segment text from background. The main contributions of the paper lie in: 1) in order to make the binarization local adaptive with uneven background, the text region image is firstly roughly split into several sub-images on which graph cut is applied, and 2) considering the unique characteristics of the text, we propose to automatically classify some pixels as text or background with high confidence, severed as hard constraints seeds for graph cut to extract text from background by spreading the seeds into the whole sub-image. The experimental results show that our approach could get better performance in both character extraction accuracy and recognition accuracy.

Detecting text in video or natural scene image is quite challenging due to the complex background, various fonts and illumination conditions. The preprocessing period, which suppresses the nontext areas so as to highlight the text areas, is the basis for further text detection. In this paper, a novel graph-based background suppression method for scene text detection is proposed. Considering each pixel as a node in the graph, our approach incorporates pixel-level and context-level features into a graph. Various factors contribute to the unary and pair wise cost function which is optimized via max-flow/min-cut algorithm [16] to get a binary image whose nontext pixels are suppressed so that text pixels are highlighted. Furthermore, the proposed background suppression method could be easily combined with other detection methods to improve the performance. Experimental results on ICDAR 2011 competition dataset show promising performance.

Text in image or video frames contains a lot of high-level semantics which can be useful for multimedia indexing, management. Coarse text detection results may contain many false alarms, which makes it necessary to eliminate the false alarms for further recognition. As text has distinct textural features, texture-based classifier such as SVM, MLP and Adaboost has been used to classify the detection regions as text or non-text region. In this paper, a random forests based method for text verification is proposed. The reason of choosing random forests lies in: 1) its ability of maintaining accuracy in small labeled dataset and 2) its good performance in unbalanced dataset as in the case of unbalanced text and non-text distribution. Furthermore, we propose to merge different random forests trained with different kinds of features to improve the accuracy of classification. The comprehensive experimental results show that our methods are effective.

Text detection and extraction in images with complex background can provide useful information for video annotation and indexing. More attention is paid to text detection for its importance, but text extraction is necessary for the text recognition, and it can test the validity of text detection. In this paper, we conclude text extraction is to segment the image and to remove noises, and then a robust text extraction method incorporating local information is proposed. First, we get the gray image from the original image and reprocess the gray image with edge enhancement. Then a binarization method incorporating local information is used to segment the gray image, by which the text-noises are removed and a binary image is obtained. Finally, the connected component analysis based on the character's density and geometric feature is performed on the binary image, by which background-noises are removed. The preliminary experiments show some promising results.

Making recognition more reliable under unconstrained environment is one of the most important challenges for realworld face recognition. In this paper, we propose a novel approach for unconstrained face verification. First, we use a spectral-clustering method based on Structural Similarity index to estimate the captured environments of facial images. Then for each pair of environments, we learn two coupled metrics, such that facial images captured in different environments can be transformed into a media subspace, and high recognition performance can be achieved. The coupled transformations are jointly determined by solving an optimization problem in the multi-task learning framework. Experimental results on the benchmark dataset (LFW) show the effectiveness of the proposed method in face verification across varying environments.

In this paper, we propose a sparse coding algorithm based on matrix rank minimization and k-means clustering and for recognition. We consider the problem of removing the noise in the training samples and generating more samples at the same time. To accomplish this, we extended the matrix rank minimization problem to cope with complex data. Samples from the same class are segmented into several groups by k-means clustering algorithm, and matrix rank minimization is applied on the clustered data to separate the noises and recover the low-rank structures in the grouped data. An over-complete dictionary is constructed by connecting the low-rank structures and the training samples together to keep the samples diversity. Sparse representation is operated based on this over-complete dictionary for recognition. Furthermore, a parameter is introduced to adjust the weighting of the coefficients that code the noises. We apply the proposed algorithm for character and face recognition. Experiments with improved performances validate the effectiveness of the proposed algorithm.

Face verification is the task of determining by analyzing face images, whether a person is who he/she claims to be. It is a very challenge problem, due to large variations in lighting, background, expression, hairstyle and occlusion. The crucial problem is to compute the similarity of two face vectors. Metric learning has provides a viable solution to this problem. Until now, many metric learning algorithms have been proposed, but they are usually limited to learning a linear transformation (i.e. finding a global Mahalanobis metric). In this brief, we propose a nonlinear metric learning method, which learns an explicit mapping from the original space to an optimal subspace, using deep Independent Subspace Analysis network. Compared to kernel methods, which can also learn nonlinear transformations, our method is a deep and local learning architecture, and therefore exhibits more powerful ability to learn the nature of highly variable dataset. We evaluate our method on the LFW benchmark, and results show very comparable performance to the state-of-art methods (achieving 92.28% accuracy), while maintaining simplicity and good generalization ability.

Although traditional bag-of-words model has shown promising results for action recognition, it takes no consideration of the relationship among spatio–temporal points; furthermore, it also suffers serious quantization error. In this letter, we propose a novel coding strategy called context-constrained linear coding (CLC) to overcome these limitations. We first calculate the contextual distance between local descriptors and each codeword by considering the spatio–temporal contextual information. Then, linear coding using contextual distance is adopted to alleviate the quantization error. Our method is verified on two challenging databases (KTH and UCF sports), and the experimental results demonstrate that our method achieves better results than previous methods in action recognition.

This paper proposes a modified two-class LDA based compound distance for similar handwritten Chinese characters discrimination. First the definition of the Intersecting Subspace (IS) between two classes and the modified between-class scatter matrix is given. Then we prove that the modified between-class scatter matrix can supply additional information. Our experiments demonstrate that the additional information can be used to discriminate points in the IS and the proposed method outperforms the previous LDA based method.

This paper proposes a Multiple Instance Learning based method for similar handwritten Chinese characters discrimination. The similar handwritten Chinese characters recognition problem is first defined as a Multiple-instance learning problem. Then the problem is solved by the AdaBoost framework. The proposed method selects some self-adapting critical regions as weak classifiers, and therefore it is more suitable for the wide variability of writing styles. Our experimental results demonstrate that the proposed method outperforms the other state-of-the-art methods.

The locally linear embedding (LLE) algorithm is considered as a powerful method for the problem of nonlinear dimensionality reduction. In this paper, a new method called globally-preserving based LLE (GPLLE) is proposed. It not only preserves the local neighborhood, but also keeps those distant samples still far away, which solves the problem that LLE may encounter, i.e. LLE only makes local neighborhood preserving, but can't prevent the distant samples from nearing. Moreover, GPLLE can estimate the intrinsic dimensionality d of the manifold structure. The experiment results show that GPLLE always achieves better classification performances than LLE based on the estimated d.

Scene classification is a hot topic in pattern recognition and computer vision area. In this paper, based on the past research on vision neuroscience, we proposed a new biologically inspired feature method for scene image classification. The new feature accounts for the visual processing from simple cell to complex cell in V1 area, and also the spatial layout for scene gist signature. It provides a different line and model revision to consider some nonlinearities inV1 area. We compare it with traditional HMAX model and recently proposed ScSPM model, and experiment on a popular 15 scenes dataset. We show that our proposed method has many important differences and merits. The experiment results also show that our method outperforms the state-of-the-art like ScSPM and KSPM model.

Detecting objects in images is very important for several application domains in computer vision. This paper presents an experimental study on data transformation of the feature vector in object detection. We use the modified Pyramid of Histograms of Orientation Gradients descriptor and the SVM classifier to form an object detection model. We apply a simple transformation to the histogram features before training and testing. This transformation equals a small change in the kernel function for Support Vector Machines. This change is much quicker than the χ2kernel, but obtains better results. Experimental evaluations on the UIUC Image Database and TU Darmstadt Database show that the transformed features perform better than the raw features, and this transformation improves the linear separability of the histogram feature.

The locally linear embedding (LLE) algorithm is considered as a powerful method for the problem of nonlinear dimensionality reduction. In this paper, first, a new method called clustering-based locally linear embedding (CLLE) is proposed, which is able to solve the problem of high time consuming of LLE and preserve the data topology at the same time. Then, how the proposed method achieves decreasing the time complexity of LLE is analyzed. Moreover, the further comparison shows that CLLE performs better in most cases than LLE on the time cost, topology preservation, and classification performance with several different data sets.

In this paper we propose a new clustering algorithm which combines the FCM clustering algorithm with the supervised learning normal mixture model; we call the algorithm as the FCM-SLNMM clustering algorithm. The FCM-SLNMM clustering algorithm consists of two steps. The FCM algorithm was applied in the first step. In the second step the supervised learning normal mixture model was applied and the clustering result of the first step was used as training data. The experiments on the real world data from the UCI repository show that the supervised learning normal mixture model can improve the performance of the FCM algorithm sharply, and which also show that the FCM-SLNMM perform much better than the unsupervised learning normal mixture model and other comparison clustering algorithms. This indicates that the FCM-SLNMM algorithm is an effective clustering algorithm.

In the area of multi-label image categorization, there are two important issues: label classification and label ranking. The former refers to whether a label is relevant or not, and the latter refers to what extent a label is relevant to an image. However, few existing papers have considered them in a holistic way. In this paper we will suggest a concrete improved method, named calibrated RankSVM, to bridge the gap between multi-label classification and label ranking. Through incorporating a virtual label as a calibrated scale [1], the threshold selection stage is embedded into ranking learning stage. This holistic way is essentially different from conventional rank methods, making our proposed method more suitable for multi-label classification task. The experiments on image have demonstrated that our algorithm has better multi-label classification performances than conventional ranksvm while preserving its good ranking characteristics.

In this paper, an adaptive edge-based text detection approach in images and video frames is proposed. The proposed approach can adopt different edge detection methods according to the image background complexity. It mainly consists of four stages: Firstly, images are classified into different background complexities. Secondly, different edge detectors are applied on the images according to their background complexities. Thirdly, connected component analysis is adopted on the edge image to obtain text candidates. Finally, the text candidates undergo the refinement algorithm to find the exact position. Experimental results demonstrate that the proposed approach is robust to text size and could effectively detect text lines in images and video frames in both simple background and complex background.

Web spam detection has become one of the top challenges for the Internet search industry. Instead of using some heuristic rules, we propose a feature re-extraction strategy to optimize the detection result. Based on the predicted spamicity obtained by the preliminary detection, through the host level web graph, three types of features are extracted. Experiments on WEBSPAM-UK2006 benchmark show that with this strategy, the performance of web spam detection can be improved evidently. Categories and Subject Descriptors

With the rapid growth of wireless technologies and mobile devices, there is a great demand for personalized services in m-commerce. Collaborative filtering (CF) is one of successful techniques to produce personalized recommendations for users. This paper proposes a novel approach to improve CF algorithms, where the contextual information of a user and the multicriteria ratings of an item are considered besides the typical information on users and items. The multilinear singular value decomposition (MSVD) technique is utilized to explore both explicit relations and implicit relations among user, item and criterion. We implement the approach in an existing m-commerce platform, and encouraging experimental results demonstrate its effectiveness.

The localization of texts in image/video is the first step in a text processing system. Its effect will do great impact on the following processing steps. Although many studies have been done on text localization algorithms, there is not a universally accepted performance evaluation method. In this paper we propose two sets of metrics to evaluate the performance of text localization algorithms in different usage conditions. The metrics also consider the text distribution characteristics, and the difficulties of the underlying task. Some experiments on the proposed metrics are also given.

This paper presents a general frame to integrate segmentation and recognition and gives a novel method to identify lingual attribute of mixed Chinese/English characters. The outstanding performance of this method is as follows. First, a text- line rather than a character segment is regarded as a process unit. Second, multi-feature is adopted based on multi-phase segmentation. Third, two types of feedbacks, including from character recognition and from character feature statistic within a text-line, are adopted throughout the whole segmentation and recognition. Fourth, it is adaptive to the quality and genre of documents.

With the rapid growth of wireless technologies and handheld devices, m-commerce is becoming a promising research area. Personalization is especially important to the success of m-commerce. This paper proposes a novel collaborative filtering-based framework for personalized services in m-commerce. The framework extends our previous work by using Online Analytical Processing (OLAP) to represent the relations among user, content and context information, and adopting a multi-dimensional collaborative filtering model to perform inference. It provides a powerful and well-founded mechanism to personalization for m-commerce. We implemented it in an existing m-commerce platform, and experimental results demonstrate its feasibility and correctness.

In this paper, an algorithm is proposed for detecting texts in images and video frames. It is performed by three steps: edge detection, text candidate detection and text refinement detection. Firstly, it applies edge detection to get four edge maps in horizontal, vertical, up-right, and up-left direction. Secondly, the feature is extracted from four edge maps to represent the texture property of text. Then k-means algorithm is applied to detect the initial text candidates. Finally, the text areas are identified by the empirical rules analysis and refined through project profile analysis. Experimental results demonstrate that the proposed approach could efficiently be used as an automatic text detection system, which is robust for font size, font color, background complexity and language.

In this paper, a new integration scheme with multilayer perceptron (MLP) networks is proposed to solve handwritten Chinese character recognition problem. The idea of meta-synthesis is emphasized in this scheme, human intelligence and computer capabilities are combined together through a procedure of two-step supervised learning. Compared with previous integration schemes, this scheme has much better performance and provides a promising way of applying MLP to large vocabulary classification.

Motivated by the idea of metasynthesis, a new adaptive classifier combination approach is proposed in this paper. Compared with previous integration methods, parameters of the proposed combination approach are dynamically acquired by a coefficient predictor based on neural network and vary, with the input pattern. It is also shown that many existing integration schemes can be considered as special cases of the proposed method. This approach is tested in application on handwritten Chinese character recognition. The experimental results demonstrate that this method can result in substantial improvement in overall performance.