Various investigations have shown that driver fatigue is the main cause of traffic accidents. Research on the use of computer vision techniques to detect signs of fatigue from facial actions, such as yawning, has demonstrated good potential. However, accurate and robust detection of yawning is difficult because of the complicated facial actions and expressions of drivers in the real driving environment. Several facial actions and expressions have the same mouth deformation as yawning. Thus, a novel approach to detecting yawning based on subtle facial action recognition is proposed in this study to alleviate the abovementioned problems. A 3D deep learning network with a low time sampling characteristic is proposed for subtle facial action recognition. This network uses 3D convolutional and bidirectional long short-term memory networks for spatiotemporal feature extraction and adopts SoftMax for classification. A keyframe selection algorithm is designed to select the most representative frame sequence from subtle facial actions. This algorithm rapidly eliminates redundant frames using image histograms with low computation cost and detects outliers by median absolute deviation. A series of experiments are also conducted on YawDD benchmark and self-collected datasets. Compared with several state-of-the-art methods, the proposed method has high yawning detection rates and can effectively distinguish yawning from similar facial actions.

Convolutional neural networks (CNNs) have achieved great success in visual recognition because of the availability of large-scale image datasets, such as the ImageNet. The transfer of convolutional features to challenging scene recognition remains an open problem. Multiple non-linear transforms endow the convolutional features with abundant information. On the other side, CNNs are adept at capturing the holistic appearances of scenes, whereas the lack of some critical local details may reduce the recognition accuracy. To address these problems, we propose a novel hierarchical coding algorithm to learn effective representations. To adapt the scale variations, many useful patches with various scales sampled from the whole image are considered to provide the sufficient details. Non-negative sparse decomposition model (NNSD) based on convolutional features is proposed to learn the sharable components for each scale and further produce global signatures. Based on the global signatures, inter-class linear coding (ICLC) is proposed to learn the discriminative components and ultimate image representations. Experimental results indicate that our approach significantly improves the recognition accuracy compared with general CNN models and achieves excellent performance on five standard benchmarks.

Text-independent Chinese writer identification does not depend on the text content of the query and reference handwritings. In order to deal with the uncertainty of the text content, text-independent approaches usually give special attention to the global writing style of handwriting, rather than the properties of each individual character or word. Thanks to the existence of high-frequency characters, some characters probably appear in both the query and reference handwritings in most cases. If character images in the query handwriting are similar to those in the reference handwriting, this query handwriting and the corresponding reference handwriting are very likely to be written by the identical writer. In this paper, we exploit the above characteristic to improve the performance of Chinese writer identification. We first present an identification scheme using edge co-occurrence feature (ECF). Then, we detect the character pairs in the query and reference handwritings using a two-step framework and propose the displacement field-based similarity (DFS) to determine whether a character pair is written by the identical writer. The character pairs help to re-rank the candidate list obtained by text-independent ECF-based similarity and finally decide the writer of the query handwriting. The proposed method is evaluated on the HIT-MW and CASIA-2.1 datasets. Experimental results demonstrate that our proposed method outperforms the existing ones, and its Top-1 accuracy on the two datasets reaches 97.1% and 98.3%, respectively.

Falls sustained by subjects can have severe consequences, especially for elderly persons living alone. A fall detection method for indoor environments based on the Kinect sensor and analysis of three‐dimensional skeleton joints information is proposed. Compared with state‐of‐the‐art methods, the authors’ method provides two major improvements. First, possible fall activity is quantified and represented by a one‐dimensional float array with only 32 items, followed by fall recognition using a support vector machine (SVM). Unlike typical deep learning methods, the input parameters of their method are dramatically reduced. Hence, videos are trained and recognised by an SVM with a low time cost. Second, the torso angle is imported to detect the start key frame of a possible fall, which is much more efficient than using a sliding window. Their approach is evaluated on the telecommunication systems team (TST) fall detection dataset v2. The results show that their approach achieves an accuracy of 92.05%, better than other typical methods. According to the characters of machine learning, when more samples are imported, their method is expected to achieve a higher accuracy and stronger capability of fall‐like discrimination. It can be used in real‐time video surveillance because of its time efficiency and robustness.

We propose a new approach to detect fake coins using their images in this paper. A coin image is represented in the dissimilarity space, which is a vector space constructed by comparing the image with a set of prototypes. Each dimension measures the dissimilarity between the image under consideration and a prototype. In order to obtain the dissimilarity between two coin images, the local keypoints on each image are detected and described. Based on the characteristics of the coin, the matched keypoints between the two images can be identified in an efficient manner. A post-processing procedure is further proposed to remove mismatched keypoints. Due to the limited number of fake coins in real life, one-class learning is conducted for fake coin detection, so only genuine coins are needed to train the classifier. Extensive experiments have been carried out to evaluate the proposed approach on different data sets. The impressive results have demonstrated its validity and effectiveness.

We propose a variable-length signature for near-duplicate image matching in this paper. An image is represented by a signature, the length of which varies with respect to the number of patches in the image. A new visual descriptor, viz., probabilistic center-symmetric local binary pattern, is proposed to characterize the appearance of each image patch. Beyond each individual patch, the spatial relationships among the patches are captured. In order to compute the similarity between two images, we utilize the earth mover's distance which is good at handling variable-length signatures. The proposed image signature is evaluated in two different applications, i.e., near-duplicate document image retrieval and near-duplicate natural image detection. The promising experimental results demonstrate the validity and effectiveness of the proposed variable-length signature.

A new near-duplicate document image matching approach is proposed. Globally, we model the spatial arrangements of objects in an image. Locally, the micro-patterns within each object are captured. To define a micro-pattern, the N-nary center-symmetric gray value differences in an image local neighborhood of a variable radius are exploited. A visual descriptor is proposed to characterize the appearance of the object based on micro-pattern distributions. By combining the global and local features, each document image is represented by a compact signature with a variable length. We employ Earth Mover's Distance for image dissimilarity computation, which stands out for its remarkable ability to tolerate the instability of object segmentation by allowing many-to-many correspondence among objects. Extensive experiments on two data sets demonstrate the effectiveness of the proposed approach.

The issue of near duplicate document image retrieval is addressed in this paper, which is characterized by not only encoding each individual word in the image but also modeling its local spatial configuration. On representing each word in the image as a string in terms of its shape characteristics, a lexicon is first learnt from a training set. Then a word in an arbitrary document image can be soft assigned to a weighted combination of several nearest neighbors in the lexicon. The rationale behind soft-assignment is to tolerate the distortions induced by character segmentations which are error-prone in degraded document images. Most importantly, we look beyond the single word and capture the local spatial configuration for each word which plays a very important role in human perception. It provides much useful information in discriminating between different document images compared with the single word. A graph, benefitting from its great representative power, is built for each word to model its relationships with the neighborhoods locally. The local word spatial configurations are integrated within the inverted file index structure to achieve scalable retrieval. Thus the retrieval of near duplicate document images is formulated as a voting problem. Experimental results on 45,000 document images demonstrate that the proposed approach brings significant improvements in successful retrieval of near duplicate images.

Most traditional postcode recognition systems implicitly assumed that the distribution of the 10 numerals (0–9) is balanced. However it is far from a reasonable setting because the distribution of 0–9 in postcodes of a country or a city is generally imbalanced. Some numerals appear in more postcodes, while some others do not. In this paper, we study cost-sensitive neural network classifiers to address the class imbalance problem in postcode recognition. Four methods, namely: cost-sampling, cost-convergence, rate-adapting and threshold-moving are considered in training neural networks. Cost-sampling adjusts the distribution of the training data such that the costs of classes are conveyed explicitly by the appearances of their instances. Cost-convergence and rate-adapting are carried out in training phase by modifying the architecture of training algorithms of the neural network. Threshold-moving tries to increase the probability estimations of expensive classes to avoid the samples with higher costs to be misclassified. 10,702 postcode images are experimented using five cost matrices based on the distribution of numerals in postcodes. The results suggest that cost-sensitive learning is indeed effective on class imbalanced postcode analysis and recognition. It also reveals that cost-sampling on a proper cost matrix outperforms others in this application.

A graph matching approach is proposed to retrieve envelope images from a large image database. First, the graph representation of an envelop image is generated based on the image segmentation results, in which each node corresponds to one segmented region. The attributes of nodes and edges in the graph are described by characteristics of the envelope image. Second, a minimum weighted bipartite graph matching method is employed to compute the distance between two graphs. Finally, the whole retrieval system including two principal stages is presented, namely, rough matching and fine matching. The experiments on a database of envelope images captured from real-life mail pieces demonstrate that the proposed method achieves promising results.