Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by a Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.

In this paper, real-time multiple speaker tracking is addressed, because it is essential in robot perception and humanrobot social interaction. The difficulty lies in treating a mixture of sounds, occlusion (some talkers are hidden) and real-time processing. Our approach consists of three components; (1) the extraction of the direction of each speaker by using interaural phase difference and interaural intensity difference, (2) the resolution of each speaker's direction by multi-modal integration of audition, vision and motion with canceling inevitable motor noises in motion in case of an unseen or silent speaker, and (3) the distributed implementation to three PCs connected by TCP/IP network to attain real-time processing. As a result, we...

Nakadai et al. (2001) have developed a real-time auditory and visual multiple-talker tracking technique. In this paper, this technique is applied to human-robot interaction including a receptionist robot and a companion robot at a party. The system includes face identification, speech recognition, focus-of-attention control, and sensorimotor task in tracking multiple talkers. The system is implemented on a upper-torso humanoid and the talker tracking is attained by distributed processing on three nodes connected by 100Base-TX network. The delay of tracking is 200 msec. Focus-of-attention is controlled by associating auditory and visual streams by using the sound source direction and talker position as a clue. Once an association is established, the humanoid keeps its face to the direction of the associated talker.

Linear discriminant analysis (LDA) is applied to broad areas, e.g. image recognition. However, successive learning algorithms for LDA are not sufficiently studied while they have been well established for principal component analysis (PCA). A successive learning algorithm which does not need N/spl times/N matrices has been proposed for LDA (Hiraoka and Hamahira, 1999, and Hiraoka et al., 2000), where N is the dimension of data. In the present paper, an improvement of this algorithm is examined based on Sanger's (1989) idea. By the original algorithm, we can obtain only the subspace which is spanned by major eigenvectors. On the other hand, we can obtain major eigenvectors themselves by the improved algorithm.

Convergence of a matrix dynamics for online LDA is analyzed. Especially, stable spurious solutions are pointed out and two schemes to prevent the spurious solutions are proposed. The performance of the algorithm is confirmed by simulations of face identification.

Proposes a face detection method intended to be used for a practical intelligent environment and human-interactive robot. Face detection and recognition are very crucial for such applications. However in the real situation, it is not easy to realize the robust detecting function because position, size, and brightness of the face image are very changeable. The proposed method solves these problems by combining correlation-based pattern matching, histogram equalization, skin color extraction, and multiple scale images generation. The authors have implemented a prototype system based upon the proposed method and conducted some experiments using the system. Results support the effectiveness of the proposed idea.