Sign language spotting is the task of detection and recognition of signs (words in the predefined vocabulary) and fingerspellings (a combination of continuous alphabets that are not found in signs) in a signed utterance. The internal structures of signs and fingerspellings differ significantly. Therefore, it is difficult to spot signs and fingerspellings simultaneously. In this paper, a novel method for spotting signs and fingerspellings is proposed, which can distinguish signs, fingerspellings, and nonsign patterns. This is achieved through a hierarchical framework consisting of three steps; (1) Candidate segments of signs and fingerspellings are discriminated with a two-layer conditional random field (CRF). (2) Hand shapes of detected signs and fingerspellings are verified by BoostMap embeddings. (3) The motions of fingerspellings are verified in order to distinguish those which have similar hand shapes and differ only in hand trajectories. Experiments demonstrate that the proposed method can spot signs and fingerspellings from utterance data at rates of 83% and 78%, respectively.

Sign language spotting is the task of detecting the start and end points of signs from continuous data and recognizing the detected signs in the predefined vocabulary. The difficulty with sign language spotting is that instances of signs vary in terms of both motion and shape. Moreover, signs have variable motion in terms of both trajectory and length. Especially, variable sign lengths result in problems with spotting signs in a video sequence, because short signs involve less information and fewer changes than long signs. In this paper, we propose a method for spotting variable lengths signs based on semi-CRF (semi-Markov Conditional Random Field). We performed experiments with ASL (American Sign Language) and KSL (Korean Sign Language) datasets of continuous sign sentences to demonstrate the efficiency of the proposed method. Experimental results showed that the proposed method outperforms both HMM and CRF.

Sign language spotting is the task of detecting and recognizing signs in a signed utterance, in a set vocabulary. The difficulty of sign language spotting is that instances of signs vary in both motion and appearance. Moreover, signs appear within a continuous gesture stream, interspersed with transitional movements between signs in a vocabulary and nonsign patterns (which include out-of-vocabulary signs, epentheses, and other movements that do not correspond to signs). In this paper, a novel method for designing threshold models in a conditional random field (CRF) model is proposed which performs an adaptive threshold for distinguishing between signs in a vocabulary and nonsign patterns. A short-sign detector, a hand appearance-based sign verification method, and a subsign reasoning method are included to further improve sign language spotting accuracy. Experiments demonstrate that our system can spot signs from continuous data with an 87.0 percent spotting rate and can recognize signs from isolated data with a 93.5 percent recognition rate versus 73.5 percent and 85.4 percent, respectively, for CRFs without a threshold model, short-sign detection, subsign reasoning, and hand appearance-based sign verification. Our system can also achieve a 15.0 percent sign error rate (SER) from continuous data and a 6.4 percent SER from isolated data versus 76.2 percent and 14.5 percent, respectively, for conventional CRFs.

Visual interpretation of gestures can be useful in accomplishing natural human-robot interaction (HRI). Previous HRI research focused on issues such as hand gestures, sign language, and command gesture recognition. Automatic recognition of whole-body gestures is required in order for HRI to operate naturally. This presents a challenging problem, because describing and modeling meaningful gesture patterns from whole-body gestures is a complex task. This paper presents a new method for recognition of whole-body key gestures in HRI. A human subject is first described by a set of features, encoding the angular relationship between a dozen body parts in 3-D. A feature vector is then mapped to a codeword of hidden Markov models. In order to spot key gestures accurately, a sophisticated method of designing a transition gesture model is proposed. To reduce the states of the transition gesture model, model reduction which merges similar states based on data-dependent statistics and relative entropy is used. The experimental results demonstrate that the proposed method can be efficient and effective in HRI, for automatic recognition of whole-body key gestures from motion sequences

In this paper, we present a method of tracking and identifying persons in video images taken by a fixed camera situated at an entrance. In video sequences a person may be totally or partially occluded in a scene for some period of time. The proposed approach uses the appearance model for the identification of persons and the weighted temporal texture features. The weight is related to the size, duration as well as the number of persons adjacent to the target person. Most systems have built an appearance model for each person to solve occlusion problems. The appearance model contains certain information on the target person. We have compared the proposed method with other related methods using color and shape features, and analyzed the features' stability. Experimental results with various real video data sequences revealed that real time person tracking and recognition is possible with increased stability in video surveillance applications even under situations of occasional occlusion.