To develop a nonverbal communication channel between an operator and a system, we built a tracking system called the Adaptive Visual Attentive Tracker (AVAT) to track and zoom in to the operator's behavioral sequence which represents his/her intention. In our system, hidden Markov models (HMMs) first roughly model the gesture pattern. Then, the state transition probabilities in HMMs are used to assign as the rewards in temporal difference (TD) learning. Later, the TD learning method is utilized to adjust the action model of the tracker for its situated behaviors in the tracking task. Identification of the hand sign gesture context through wavelet analysis autonomously provides a reward value for optimizing AVAT's action patterns. Experimental results of tracking the operator's hand sign action sequences during her natural walking motion with higher accuracy are shown which demonstrate the effectiveness of the proposed HMM-based TD learning algorithm of AVAT. During TD learning experiments, the exploring randomly chosen actions sometimes exceed the predefined state area, and thus involuntarily enlarge the domain of states. We describe a method utilizing HMMs with continuous observation distribution to detect whether the state would be split to make a new state. The generation of new states brings the ability of enlarging the predefined area of states.

Recent advances in vision technology lead to the establishment of non-verbal communication channels from human behaviors toward understanding their intention. An adaptive vision-based attentive tracker (AVAT) is proposed for isolating such human intended actions from the ordinary walking behavior. The algorithm which drives the attentive tracker is divided into two sub-processes: one is for modeling the movement of human body parts as the environment using hidden Markov models, and the other is for learning the model of the tracker's action using a temporal difference algorithm. In the paper, we describe in detail the integration of the two sub-processes and finally show an experimental result of isolating the human sign action during his natural walking motion for demonstrating the feasibility of our proposal. Identification of the sign context determines the action of the tracker with the simulated rewards supplied.