This paper proposes an approach to construct a system which allows humanoid robots to recognize human behaviors and predict his or her future behaviors. The system consists of two modules : "motion symbol tree" and "motion symbol graph", Human demonstrator motion patterns are stored as motion symbols, which abstract the motion data by using Hidden Markov Models. The stored motion patterns are organized into a hierarchical tree structure, which represents the similarity among the motion patterns and provides abstracted motion patterns. The formed hierarchical structure is the motion symbol tree. Concatenated sequences of motion patterns are stochastically represented as transitions between the motion patterns by using an Ngram Model, and the causality among the human behaviors are extracted. This structure is the motion symbol graph. The behavioral hierarchy and transition model make it possible to predict human behaviors during observation and to generate sequences of motion patterns automatically while maintaining a natural motion stream, as if the system is a "crystal ball" to reflect future behaviors. The experiments demonstrate the validity of the proposed framework on a large scale motion data.

This paper proposes an approach to hierarchy formation of human behaviors, extraction of the behavioral transitions, and their application to prediction and automatic generation of behaviors. Human demonstrator motion patterns are stored as motion symbols, which abstract the motion data by using Hidden Markov Models. The stored motion patterns are organized into a hierarchical tree structure, which represents the similarity among the motion patterns and provides abstracted motion patterns. Concatenated sequences of motion patterns are stochastically represented as transitions between the abstracted motion patterns by using an Ngram Model, and the transitional relationships of the human behaviors are extracted. The behavioral hierarchy and transition model make it possible to predict human behaviors during observation and to generate sequences of motion patterns automatically while maintaining a natural motion stream, as if the system is a “crystal ball” to reflect future behaviors. The experiments validates the proposed framework by using a developed visualization system, which shows the demonstrator or the operator the established hierarchical tree and the transition network of the motion patterns, predicted behaviors and generated sequences of the motion patterns.

This paper proposes an on-line, interactive approach for incremental learning and visualization of full body motion primitives from observation of human motion. The human demonstrator motion is captured in a motion capture studio. The continuous observation sequence is first partitioned into motion segments, using stochastic segmentation. Motion segments are next incrementally clustered and organized into a hierarchical tree structure representing the known motion primitives. At the same time, the sequential relationship between motion primitives is learned, to enable the generation of coherent sequences of motion primitives. An on-line visualization system is also developed to allow the demonstrator to visualize the motion database and the motion primitives learned by the system, thus giving the demonstrator insight into the learning process and the ability to interactively modify the demonstration based on the current state of the knowledge base. The developed system has many potential applications for motion analysis, prediction and imitation learning for humanoid robots.