A controller design to suppress vertical vibration of a position controlled biped walking robot is presented. The system model of structural vibration is estimated from a measurement of frequency response. By using the model, an optimal feedback controller with full state observer is designed. With a consideration of leg support phase, the vibration controller is combined into our walking controller. The effectiveness of our controller is experimentally demonstrated.

The performance of sound source localization is often reduced by the presence of colored noise in the environment, such as room reverberation. In this study, a method for estimating the noise spatial covariance using a hierarchical model is proposed and its performance is evaluated. By employing the hierarchical model in joint Bayesian estimation, robust estimation of the covariance is expected with a relatively small amount of data. Moreover, a method of jointly estimating the number of sources is introduced so that it can be used for cases in which the number of active sources dynamically changes, for example, speech signals. The results of the experiments performed using actual room reverberation show the effectiveness of the proposed method.

In this paper, source localization in spatially colored noise is addressed. The covariance of colored noise is estimated using a hierarchical model in the joint Bayesian estimation. The results of the experiment show that the spatial resolution was improved compared with the approach without hierarchical modeling.

Noise generated due to the motion of a robot deteriorates the quality of the desired sounds recorded by robot-embedded microphones. On top of that, a moving robot is also vulnerable to its loud fan noise that changes its orientation relative to the moving limbs where the microphones are mounted on. To tackle the non-stationary ego-motion noise and the direction changes of fan noise, we propose an estimation method based on instantaneous prediction of ego noise using parameterized templates. We verify the ego noise suppression capability of the proposed estimation method on a humanoid robot by evaluating it on two important applications in the framework of robot audition: (1) automatic speech recognition and (2) sound source localization. We demonstrate that our method improves recognition and localization performance during both head and arm motions considerably.

We have assessed robust tracking of humans based on intelligent Sound Source Localization (SSL) for a robot in a real environment. SSL is fundamental for robot audition, but has three issues in a real environment: robustness against noise with high power, lack of a general framework for selective listening to sound sources, and tracking of inactive and/or noisy sound sources. To address the first issue, we extended Multiple SIgnal Classification by incorporating Generalized EigenValue Decomposition (GEVD-MUSIC) so that it can deal with high power noise and can select target sound sources. To address the second issue, we proposed Sound Source Identification (SSI) based on hierarchical gaussian mixture models and integrated it with GEVD-MUSIC to realize a selective listening function. To address the third issue, we integrated audio-visual human tracking using particle filtering. Integration of these three techniques into an intelligent human tracking system showed: 1) GEVD-MUSIC improved the noise-robustness of SSL by a signal-to-noise ratio of 5-6 dB; 2) SSI performed more than 70% in F-measure even in a noisy environment; and 3) audio-visual integration improved the average tracking error by approximately 50%.

In this paper, we propose a system that can detect unusual sounds and directions by observing sound environment with microphone arrays. One of the attractive features of the system is to detect the unusual information through daily environmental sound measurement. Therefore the system does not require such troublesome processes that the detected sounds must be predefined, the predefined sounds must be collected, and using the collected sounds, their features must be modeled, where the conventional systems have such troublesomeness. Moreover, unlike conventional systems using video cameras, our system is not limited by video camera angles. A simple experimental result shows the validity of the proposed system.

In this paper, a method for extracting and classifying movement-related brain signals is proposed. A single-trial MEG observation is first processed with a pre-whitening filter so that strong stationary interference is eliminated. Next, a brain signal effective for classification is extracted using an adaptive spatial filter. The extracted signal is then classified with a support vector machine. From the experimental results, it is shown that the classification rate of 62.6 % is obtained for the brain signals related to the three types of hand movements (ldquoscissors-paper-rockrdquo).

As robotic technology plays an increasing role in human lives, ¿robot audition¿, human-robot communication, is of great interest, and robot audition needs to be robust and adaptable for dynamic environments. This paper addresses sound source localization working in dynamic environments for robots. Previously, noise robustness and dynamic localized sound selection have been enormous issues for practical use. To correct the issues, a new localization system ¿Selective Attention System¿ is proposed. The system has four new functions: localization with Generalized EigenValue Decomposition of correlation matrices for noise robustness(¿Localization with GEVD¿), sound source cancellation and focus (¿Target Source Selection¿), human-like dynamic Focus of Attention (¿Dynamic FoA¿), and correlation matrix estimation for robotic head rotation (¿Correlation Matrix Estimation¿). All are achieved by the dynamic design of correlation matrices. The system is implemented into a humanoid robot, and the experimental validation is successfully verified even when the robot microphones move dynamically.

In this paper, a generalized eigenvalue decomposition (GEVD)-based approach is proposed to remove continuous interferences included in MEG observations. The GEVD-based approach can be interpreted as a general framework of pre-whitening that can be combined with various types of signal separation techniques such as principal component analysis (PCA), independent component analysis (ICA) or adaptive beamforming (ABF). An example of extracting movement-related brain activities from single trial data is presented.

The extension of particle filtering techniques to the multiple speaker case is difficult as two distinct problems must now be addressed. Firstly, the active speakers must be identified and their locations estimated, requiring the use of multi-dimensional likelihoods, and then each speaker must be correctly associated with his corresponding location. In this paper we propose a multi-speaker tracking algorithm in which the number of active speakers is determined by estimating the profile of the noise-plus-reverberation covariance matrix eigenvalues. The multi-dimensional likelihoods are then decoupled using the Expectation Maximization (EM) algorithm. The tracking accuracy is improved by the inclusion of a pause detection step and estimation of the noise-plus-interference covariance matrix. The results show the benefits of the proposed methods under difficult tracking situations.

A particle filtering algorithm using the parameters in the EM (expectation-maximization) algorithm is proposed for tracking multiple sound sources. Differently from the conventional EM based algorithms, the proposed algorithm can track multiple sound sources without knowing their starting points. Moreover, an idea of the group tracking is applied to the particle filtering algorithm so that better tracking performances can be obtained. Experimental results show the validity of the proposed algorithm.

A method of detecting and tracking moving human speech events by integrating audio and video signals using a particle filter is proposed and evaluated. Using the particle filter, location, on/off status, and human/non-human determination can be estimated simultaneously for multiple sound sources. Experiments demonstrate that the proposed method performs well for the data obtained in an ordinary meeting room using a microphone array and a monocular camera.

In this paper, we describe an HMM-based featurecompensation method. The proposed method compensates for noise-corrupted features in the MFCC domain using the output probability density functions (pdf) of the Hidden Markov Models (HMM). In compensating the features, the output pdfs are adaptively weighted according to forward path probabilities. Because of this, the proposed method can minimize degradation of featurecompensation accuracy due to temporally changing noise environment. We evaluated the proposed method based on the AURORA2 database. All the experiments were conducted in a clean condition. The experiment results indicate that the proposed method, combined with cepstral mean subtraction, can achieve a word accuracy of 87.64%. We also show that the proposed method is useful in a transient pulse noise environment.

For cooperative work of robots and humans in the real world, a communicative function based on speech is indispensable for robots. To realize such a function in a noisy real environment, it is essential that robots be able to extract target speech spoken by humans from a mixture of sounds by their own resources. We have developed a method of detecting and extracting speech events based on the fusion of audio and video information. In this method, audio information (sound localization using a microphone array) and video information (human tracking using a camera) are fused by a Bayesian network to enable the detection of speech events. The information of detected speech events is then utilized in sound separation using adaptive beam forming. In this paper, some basic investigations for applying the above system to the humanoid robot HRP-2 are reported. Input devices, namely a microphone array and a camera, were mounted on the head of HRP-2, and acoustic characteristics for sound localization/separation performance were investigated. Also, the human tracking system was improved so that it can be used in a dynamic situation. Finally, overall performance of the system was tested via off-line experiments.

A method of estimating the number of sound sources in a reverberant sound field is proposed in this paper. It is known that the eigenvalue distribution of the spatial correlation matrix calculated from a multiple microphone input reflects information on the number of sources. However, in a reverberant sound field, the feature of the number of sources in the eigenvalue distribution is degraded by the room reverberation. In this paper, support vector machines is applied to classify the eigenvalue distributions which are not clearly separable. The proposed method is then applied to the source separation system and is evaluated via automatic speech recognition.

In a real environment, it is essential to adapt an acoustic model to variations in background noises in order to realize robust speech recognition. In this paper, we construct an extended acoustic model by combining a mismatch model with a clean acoustic model trained using only clean speech. We assume the mismatch model conforms to a Gaussian distribution with timevarying population parameters. The proposed method adapts the extended acoustic model to the noises by estimating the population parameters using a Gaussian Mixture Model (GMM) and Gain-Adapted Hidden Markov Model (GA-HMM) decomposition method. We performed recognition experiments under noisy conditions using the AURORA2 database in order to confirm the effectiveness of the proposed method.

Two array signal processing techniques are combined with independent component analysis (ICA) to enhance the performance of blind separation of acoustic signals in a reflective environment. The first technique is the subspace method which reduces the effect of room reflection when the system is used in a room. Room reflection is one of the biggest problems in blind source separation (BSS) in acoustic environments. The second technique is a method of solving permutation. For employing the subspace method, ICA must be used in the frequency domain, and precise permutation is necessary for all frequencies. In this method, a physical property of the mixing matrix, i.e., the coherency in adjacent frequencies, is utilized to solve the permutation. The experiments in a meeting room showed that the subspace method improved the rate of automatic speech recognition from 50% to 68% and that the method of solving permutation achieves performance that closely approaches that of the correct permutation, differing by only 4% in recognition rate.

Two major objectives of a binaural microphone array system are reducing background noise and retaining interaural relationships, for example, ILD and ITD. Hence, a new adaptive microphone array signal processing based on a weighted least squares algorithm for binaural hearing is proposed in this paper. Instead of Lagrange's method of indetenninant coefficients, which is often used in conventional binaural microphone array systems, a weighted least squares algorithm is employed. Results of computer simulations carried out for purposes of evaluation showed that the proposed method is characterized by flexible and robust adaptivity to various conditions.

The sound data for open evaluation is necessary for studies such as sound source localization, sound retrieval, sound recognition and hands-free speech recognition in real acoustic environments. This paper reports on our project for acoustic data collection. There are many kinds of sound scenes in real environments. The sound scene is specified by sound sources and room acoustics. The number of combinations of the sound sources, source positions and rooms is huge in real acoustic environments. We assumed that the sound in the environments can be simulated by convolution of the isolated sound sources and impulse responses. As an isolated sound source, hundred kinds of environment sounds and speech sounds are collected. The impulse responses are collected in various acoustic environments. Additionally we collected sounds from a moving source. In this paper, progress of our sound scene database collection project and application to environment sound recognition and hands-free speech recognition are described.

Two array signal processing techniques are combined with independent component analysis to enhance the performance of blind separation of acoustic signals in a reflective environment such as rooms. The first technique is the subspace method which reduces the effect of room reflection. The second technique is a method of solving the permutation, in which the coherency of the mixing matrix in adjacent frequencies is utilized.

A real-time sound localization/separation system for near-field sound sources was constructed and evaluated in a real office environment. As for the sound localization, the experimental results showed that the direction of the two sources was estimated with high accuracy while the range of the sources was estimated with moderate accuracy. As for the sound separation, a recognition rate of 70 % for an on-line recognizer on a network and of 90% for an off-line recognizer were achieved, respectively.

A method of speech enhancement using microphone-array signal processing based on the subspace method is proposed and evaluated. The method consists of the following two stages corresponding to the different types of noise. In the first stage, less-directional ambient noise is reduced by eliminating the noise-dominant subspace. It is realized by weighting the eigenvalues of the spatial correlation matrix. This is based on the fact that the energy of less-directional noise spreads over all eigenvalues while that of directional components is concentrated on a few dominant eigenvalues. In the second stage, the spectrum of the target source is extracted from the mixture of spectra of the multiple directional components remaining in the modified spatial correlation matrix by using a minimum variance beamformer. Finally, the proposed method is evaluated in both a simulated model environment and a real environment.

This paper describes a sound scene database necessary for studies such as sound source localization, sound retrieval, sound recognition and hands-free speech recognition in real acoustical environments.This paper reports on a project for collection of the sound scene data supported by Real World Computing Partnership(RWCP).There are many kinds of sound scenes in real environments.The sound scene is denoted by sound sources and room acoustics.The numb e r o f c o m bination of the sound sources, source positions and rooms is huge in real acoustical environments.Two approaches are taken to build the sound scene database in the early stage of the project.The rst approach is to collect isolated sound sources of many kinds of non-speech sounds and speech sounds.The second approach is to collect impulse responses in various acoustical environments.The sound in the environments can be simulated by convolution of the isolated sound sources and impulse responses.In a later stage, the sound scene data in real acoustical environments is planned to be collected using a three dimensional microphone array.In this paper, the plan and progress of our sound scene database project are described.1.

In active control systems, the configuration of the control sources greatly affects the performance. In this paper, a method for optimizing the number and location of control sources is proposed. As a criterion for the optimization, the linear independence of the transfer impedance vector is employed. A physical interpretation of the linear independence of the transfer impedance vector is the independent contribution of each control source that is not substitutable by the other sources. In the optimization process, control sources are determined one by one using Gram-Schmidt orthogonalization so that the corresponding transfer impedance vectors are the most linearly independent. The results of the simulation show the optimized configuration to have small control error. Moreover, for the optimized configuration, the transfer impedance matrix exhibits a small condition number, which results in robustness of the system in the face of changes in the environment.

A method for optimizing the number and the configuration of control sources in an active control system is proposed. In the optimization process, sources are selected one by one so that the corresponding transfer impedance vector is the most linearly independent. From the results of the simulation, it is shown that the optimized configuration yields not only a small average control error but also a small condition number in the transfer impedance matrix, which contributes to the robustness of the system against environmental change.

A method for recovering the LPC spectrum from a microphone array input signal corrupted by ambient noise is proposed. This method is based on the CSS (coherent subspace) method, which is designed for DOA (direction of arrival) estimation of broadband array input signals. The noise energy is reduced in the subspace domain by the maximum likelihood method. To enhance the performance of noise reduction, elimination of the noise-dominant subspace using projection is further employed, which is effective when the SNR is low and classification of noise and signals in the subspace domain is difficult. The results of the simulation show that some small formants, which cannot be estimated by the conventional delay-and-sum beamformer, were well estimated by the proposed method.

For the effective reduction of noise that is suddenly switched on or transient, attenuation is required from its onset. A weighted recursive least square approach, where an extra term providing an initial directivity is combined with the power minimization term, is proposed. A noise reduction system is also proposed.>

A method of designing a digital hearing aid suitable for sensorineural hearing loss with a narrow audible range is proposed. The input signal is divided into short-time blocks (8 ms), and its spectrum is calculated using short-time fast Fourier transforms. The frequency-gain characteristic of the block is determined using the calculated spectrum and loudness compensation functions which describe the relation between the loudness perceived by normal listeners and that perceived by an impaired listener. The input signal within the block is processed by a single digital filter using the calculated gain characteristics. This algorithm is realized using a digital signal processor. The results of monosyllabic speech tests were that the scores of the speech test for the proposed system were improved over a wide range of input levels compared with those for the linear system for nine of 13 sensorineural impaired subjects.>