Motivated by the emerging internet of things (IoT) applications, wireless systems encounter the challenges of providing accurate indoor localization to massive IoT devices. Although the received signal strength indicator (RSSI)-based fingerprinting can provide accurate localization with low system requirements, it still suffers from multipath and fading effects. To resolve this, we propose a beam domain-based fingerprinting localization that can leverage the spatial feature with multiple antenna systems to improve the localization. Specifically, we consider using the beam domain receive power map (BDRPM), which is an RSSI-based map that captures important features of spatial fingerprints of the environment, for localization. To learn the environmental fingerprints via using BDRPMs and to conduct the localization, we propose a deep-learning approach based on the 2D convolutional neural network and auto-encoder structure. We conduct practical simulations to evaluate our proposed localization approach. The results show that our approach can provide very accurate localization, be resistant to environmental changes, and outperform the reference schemes in the literature.

To realize the intelligent transportation, environmental awareness of roadside units (RSUs) is of paramount importance. One of the approaches to enable the environmental awareness of RSUs is to equip RSUs with radar systems. However, as more and more radar systems are installed, manually monitoring whether these radar systems work in their normal states becomes impossible. To resolve this issue, a radar system state self-diagnosis method is proposed in this paper by using the radar sensing information with deep learning techniques. Specifically, by using the proposed feature extraction approach, we first effectively convert the huge amount of radar sensing data into useful features. Then, by using the proposed deep neural network to interpret the extracted features, the radar systems can self-diagnose whether there exist faults on the systems. We verify our proposed method via real-world experiments. Results show that our proposed method can accurately diagnose the radar system and report the faults.

This paper presents a novel approach to implement the robust minimum variance distortionless response (MVDR) beamformer. The robust MVDR beamformer is based on the optimization of worst-case performance and provides an excellent robustness against an arbitrary but norm-bounded desired signal steering vector mismatch. For real-time consideration, the beamformer was formulated into state-space observer form and the second-order extended (SOE) Kalman filter was derived. However, the SOE Kalman filter assumes an accurate system dynamic and statistics of the noise signals. These assumptions limit the performance under uncertainties. This paper develops the SOEH∞filter for the implementation of the robust MVDR beamformer. The estimation criterion in the SOEH∞filter design is to minimize the worst possible effects of the disturbance signals on the signal estimation errors without a prior knowledge of the disturbance signals statistics. Experimental results demonstrate the performance of the proposed algorithm in a noisy and reverberant environment and show its superiority of the robustness against mismatches over the robust MVDR beamformer based on the SOE Kalman filter.

In many practical environments, the desired speech signal is usually contaminated not only by stationary noise but also nonstationay interferences, such as competing speech. This paper proposes a speech enhancement method which can extract desired speech in a multiple interferences and reverberant environment. The proposed method uses transfer function ratio beamformer and multi-channel adaptive filter algorithm. The virtual sound source concept is proposed to simplify the theoretical treatment for multiple competing speeches. In addition, a transfer function ratio estimation method in a more practical scenario is also proposed. The experiments are performed in a real room acoustic environment.

Sound source localization is an important feature in robot audition. This work proposes a sound source number and directions estimation method by using the delay information of microphone array. An eigenstructure-based generalized cross correlation method is proposed to estimate time delay between microphones. Upon obtaining the time delay information, the sound source direction and velocity can be estimated by least square method. In multiple sound source case, the time delay combination among microphones is arranged such that the estimated sound speed value falls within an acceptable range. By accumulating the estimation results of sound source direction and using adaptive K-means++ algorithm, the sound source number and directions can be estimated.

This work proposes an adaptive beamformer based on generalized sidelobe canceller (GSC) structure with novel blocking matrix design. The classical GSC presented by Griffiths and Jim suffers from desired signal cancellation problems due to the complicated acoustic environment. This work utilizes the pseudo-inverse property of matched filters and subarray structure to design a new blocking matrix of the GSC. For practical implementation, matched filter ratios between microphone pairs are estimated, instead of estimating matched filters from the sound source to each microphone. Simulation results are presented to show the effectiveness of the proposed method.

Human detection and tracking is important for user-friendly human-robot interaction. The robot should be able to find the user autonomously and keep its attention to the user in a human-like manner. In this paper, a design and experimental study of robust human detection and tracking is presented through fusion several modalities of sensory information. The multi-modal interaction design utilizes a combination of visual, audio, and laser scanner data for reliable detection and tracking of an interested user. During tracking motion, obstacle avoidance behavior will be activated any time required to ensure safety. Furthermore, user can further assign the robot to interact with other user by speech command. Experimental results show that the robot can robustly tracks person under complex scenarios.

This paper presents a real-time human-robot interface system (HRIS), which processes both speech and vision information to improve the quality of communication between human and an autonomous mobile robot. The HRIS contains a real-time speech attention system and a real-time face tracking system. In the speech attention system, a microphone-array voice acquisition system has been developed to estimate the direction of speaker and purify the speaker's speech signal in a noisy environment. The developed face tracking system aims to track the speaker's face under illumination variation and react to the face motion. The proposed HRIS can provide a robot with the abilities of finding a speaker's direction, tracking the speaker's face, moving its body to the speaker, focusing its attention to the speaker who is talking to it, and purifying the speaker's speech. The experimental results show that the HRIS not only purifies speech signal with a significant performance, but also tracks a face under illumination variation in real-time

In this paper, the feasibility of utilizing sound for robot's pose detection is investigated, and a novel and robust robot location and orientation detection method based on sound field features for noisy environment is proposed. Unlike traditional methods, the proposed method does not explicitly consider the characteristic of direct path from sound source to microphones, nor attempt to suppress the effect of reverberations and noise signals. Instead, it utilizes the sound field features of a robot at different location and orientation in a normal environment. The sound field feature is captured by using a probability distribution estimation method called Gaussian mixture model (GMM). The experimental results show that this method can detect robot's location and orientation under both line-of-sight and non-line-of-sight conditions using only two microphones and is robust to environmental noise. Moreover, it can also solve the microphones' mismatch problem and can be applied to both near-field and far-field conditions. Since this method can provide global location and orientation detection, it is suitable to fuse with other localization methods to provide initial conditions for reduction of the search effort, or provide the compensation for localizing certain locations that cannot be detected using other localization methods

This work proposes a novel and robust adaptive speech enhancement system, which contains both time-domain and frequency-domain beamformers using Hinfinfiltering approach in vehicle environments. A corresponding microphone array data acquisition hardware is also designed and implemented. Traditionally, mutually matched microphones are needed, but this requirement is not practical. To conquer this issue, the proposed system adapts the mismatch dynamics to allow unmatched microphones to be used in an array. Furthermore, to achieve a satisfactory speech recognition performance, the speech recognizer is usually required to be retrained for different vehicle environments due to different noise characteristics and channel effects. The channel effect usually causes the modeling error in a channel recovery process because of the long channel response. The proposed system using the Hinfinfiltering approach, which makes no assumptions about noise and disturbance, is robust to the modeling error. Consequently, the proposed frequency-domain beamformer provides a satisfactory performance without the need to retrain the speech.