We present a hands-free AR work support system that provides work instructions to workers without interrupting normal work procedures. This system estimates the work progress by monitoring the status of work objects only on the basis of 3D data captured from a depth sensor mounted on a helmet, and it selects appropriate information to be displayed on a head-mounted display (HMD) on the basis of the estimated work progress. We describe a prototype of the proposed system and the results of primary experiments carried out to evaluate the accuracy and performance of the system.

This paper presents our recent attempt to make a super-large scale spoken-term detection system, which can detect any keyword uttered in a 2,000-hour speech database within a few seconds. There are three problems to achieve such a system. The system must be able to detect out-of-vocabulary (OOV) terms (OOV problem). The system has to respond to the user quickly without sacrificing search accuracy (search speed and accuracy problem). The pre-stored index database should be sufficiently small (index size problem). We introduced a phoneme-based search method to detect the OOV terms, and combined it with the LVCSR-based method. To search for a keyword from large-scale speech databases accurately and quickly, we introduced a multistage rescoring strategy which uses several search methods to reduce the search space in a stepwise fashion. Furthermore, we constructed an out-of-vocabulary/in-vocabulary region classifier, which allows us to reduce the size of the index database for OOVs. We describe the prototype system and present some evaluation results.

Speech recognition errors are inevitable in a speech dialog system. It is important to provide a dialog flow that allows the user to correct system errors and quickly return to the original dialog. This paper describes explicit, final and implicit confirmation styles that are implemented in the CAMMIA speech dialog system, and compares them from the viewpoint of usability. Our results show that a final confirmation with fewer confirmation turns is preferred by the user when there is no error in the dialog. On the other hand, an explicit confirmation is preferred when an error occurs.

In recent years, the number of sign language learners is increasing in Japan. And there are many teaching materials of sign language such as textbooks, videotapes and software for PCs. However, these teaching materials have several problems that learners cannot study sign language sufficiently because the learners can mainly study manual gestures, cannot change the direction to see signed gestures, and cannot check their signed gestures by themselves. We developed a sign language teaching system applying sign language recognition and sign language generation technologies to solve these problems and realize effective sign language study. Further, we carried out an evaluation experiment by several users and confirmed that the developed system was effective.

To automatically interpret Japanese sign language (JSL), the recognition of signed words must be more accurate and the effects of extraneous gestures removed. We describe the parameters and the algorithms used to accomplish this. We experimented with 200 JSL sentences and demonstrated that recognition performance could be considerably improved.

Sign language gestures are inflected in accordance with the context. To recognize such sign language properly, the structure of sign kmgoage must be made clear. It is well known that the structure of sign Iangoage is represented as a combination of basic components of gestures. Sign language can be recognized by using such components. In this paper, a format to describe sign language gestures and a method to recognize the meaning of the gesture based on the components of gestures are discussed.

Sign language is one means of communication for hearing-impaired people. Words and sentences in sign language are mainly represented by hands' gestures. In this report, we show a sign language translation system which we are developing. The system translates Japanese sign language into Japanese and vice versa. In this system, hand shape and position data are inputted using DataGlove. Inputted hand motions are recognized and translated into Japanese sentences. Japanese text is translated into sign language represented as 3-D computer-graphic animation of sign language gestures.