Joint speech enhancement and bandwidth extension are challenging for through-barrier radar vibrometry. Deep learning approaches achieve reasonable success but lack temporal context and bandwidth extension capability for significantly band-limited speech. In this work, we propose a joint speech enhancement and bandwidth extension module consisting of a newly formulated features fusion transformer that leverages temporal context to facilitate the consistency of spectral features of the early extended subbands by large dilation deconvolution operations with the enhancement operation. The proposed EBEnet was evaluated under a through-barrier setup and demonstrated higher speech intelligibility than encoder-decoder-based CNNs.

Complex-valued independent component analysis (ICA) is a celebrated method in blind separation of complex-valued signals. In this paper, we propose to transform the constrained optimization problems of complex-valued ICA into unconstrained optimization problems which can be solved by limited-memory Broyden-Fletcher-Goldfarb-Shanno update (L-BFGS). As opposed to previous approaches, the proposed method does not apply any restriction on the Hessian matrix of ICA cost function. It can separate mixed sub-Gaussian, super-Gaussian, circular, and non-circular sources. Simulations show promising results.

We address the problem of direction-of-arrival (DOA) estimation for multiple speech sources in an enclosed environment using a single acoustic vector sensor. The challenges in such scenario include reverberation and overlapping of the source signals. In this work, we exploit low-reverberant-single-source (LRSS) points in the time-frequency (TF) domain, where a particular source is dominant with high signal-to-reverberation ratio. Unlike conventional algorithms having limitation that such potential points need to be detected at “TF-zone” level, the proposed algorithm performs LRSS detection at “TF-point” level. Therefore, for the proposed algorithm, the potential LRSS points need not be neighbors of each other within a TF zone to be detected, resulting an increased number of detected LRSS points. The detected LRSS points are further screened by an outlier removal step such that only reliable LRSS points will be used for DOA estimation. Simulations and experiments were conducted to demonstrate the effectiveness of the proposed algorithm in multisource reverberant environments.

We propose an algorithm to localize impacts on rigid surfaces using induced vibration signals. This allows for the conversion of daily objects, such as tabletops and glass panels, into human–computer touch interfaces using low-cost piezoelectric sensors. Impact localization is achieved by estimating the time-of-arrivals and subsequently time-difference-of-arrivals of the sensor-received signals. Time-of-arrival estimation is highly challenging with increasing source–sensor distance due to the occurrence of a gradual noise-to-signal transition at the sensor output. We address this problem by first converting the signal into Hermitian angle distributions. The time-varying probability contributions of the background noise and vibration signal in each of the distributions are subsequently monitored to identify the instant when the signal begins to dominate the noise, signifying the signals arrival. The proposed framework also allows simultaneous time-of-arrival estimation across all the sensors to minimize errors in the resultant time-difference-of-arrival estimates. Experimental results show that the proposed algorithm outperforms existing techniques for source localization on solid surfaces of different materials.

We propose a TDOA-based algorithm for source localization on rigid surfaces. This allows the conversion of readily available large surfaces into touch interfaces using surface-mounted vibration sensors. To achieve this, we characterize the arrival of each sensor-received signal by the arrival times of its frequency components. To estimate the arrival time of each frequency component, we first model each component as a harmonic random process. A kurtosis sequence, which exhibits a sharp rising edge when the signal begins to deviate from the background noise, can then be obtained for each component. By accurately estimating the starting point of the rising edge, our algorithm can avoid the uncertainty due to gradual noise-to-signal transition. Experiment results show that the proposed algorithm achieves better localization performance than existing techniques on large surfaces.

We propose a new sparse coding technique based on the power mean of phase-invariant cosine distances. Our approach is a generalization of sparse filtering and K-hyperlines clustering. It offers a better sparsity enforcer than the L1/L2norm ratio that is typically used in sparse filtering. At the same time, the proposed approach scales better than the clustering counterparts for high-dimensional input. Our algorithm fully exploits the prior information obtained by preprocessing the observed data with whitening via an efficient row-wise decoupling scheme. In our simulating experiments, the algorithm produces better estimates than previous approaches do. It yields better separation of live recorded speech mixtures as well.

We address the problem of localizing and tracking alternating (moving or stationary) talkers using microphone arrays in a room environment. One of the main challenges is the frequent (and possibly abrupt) change of talker positions, which requires the algorithm to capture the active talker rapidly. In addition, the presence of interference, background noise, and room reverberation degrades the tracking performance. We propose a new algorithm that jointly exploits the advantages of the particle filter (PF) and particle swarm intelligence. The PF is used as a general tracking framework, which incorporates a proposed alternating source-dynamic model for recursive estimation of talker position. Unlike the conventional PF, where particles operate independently in the particle sampling stage, the use of swarm intelligence allows particles to interact with each other, thereby improving convergence toward the active talker location. In addition, the memory mechanism in swarm intelligence allows particles to remain at their previous best-fit state estimate when signals are corrupted by interference, noise, and/or reverberation. Simulations and experiments were conducted to demonstrate the effectiveness of the proposed algorithm.

We propose a new algorithm for source localization on rigid surfaces, which allows one to convert daily objects into human-computer touch interfaces using surface-mounted vibration sensors. This is achieved via estimating the time-difference-of-arrivals (TDOA) of the signals across the sensors. In this work, we employ a smooth parametrized function to model the gradual noise-to-signal energy transition at each sensor. Specifically, the noise-to-signal transition is modeled by a four-parameter logistic function. The TDOA is then estimated as the difference in time shifts of the functions fitted to the sensor data. Experiment results show that the proposed algorithm significantly outperforms existing techniques which adopt the abrupt change model for time-of-arrival estimation.

We address the problem of estimating direction-of-arrivals (DOAs) for multiple sound sources using a single acoustic vector sensor (AVS) in an enclosed room environment. It is well-known that multi-source DOA estimation in an enclosed environment is challenging due to room reverberation, environmental noise and overlapping of the source spectra. In this work, we propose a multi-source DOA estimation algorithm which exploits co-location of the sensor elements in AVS. We identify time-frequency (TF) zones of the received signals in which only one source is dominant with a high signal-to-reverberation ratio. DOA estimation is then achieved via the use of clustering of the Hermitian angle feature. Simulation results show that the proposed DOA estimation algorithm is robust to both reverberation and environmental noise.

This paper focuses on the problem of single-channel noise reduction in a transient noise environment for speech enhancement application. A typical speech enhancement algorithm requires an estimate of the noise statistics. However, the problem of noise estimation is challenging when the statistics of the noise vary significantly with time. By exploiting the fact that for speech signal most of the energy is concentrated on the harmonic bands in voiced frames, we propose an algorithm for the estimation of speech presence probability in the time-frequency domain. The estimated speech presence probability is then used for noise estimation for speech enhancement application. Evaluations are conducted to compare the speech enhancement performance between the proposed algorithm and the existing algorithm for various types of transient noise.

We propose a new approach for source localization on solids with applications to human-computer interface. We analyze the wave propagation of flexural modes of vibration, generated by an impact on a solid surface, to characterize the dispersive linear time-varying system having non-linear phase response. We show that a difference in dispersion between two signals propagating through solids can be mapped directly to the relative propagation distance if the signals are appropriately time-warped. We then exploit this important property for source localization by computing the similarity of the warped signals in the time-frequency domain. As the proposed source localization algorithm jointly estimates warping-based polynomial parameters and source location, the method does not require pre-calibration.

Existing algorithms employ the Wiener filter to suppress residual crosstalk in the outputs of blind source separation algorithms. We show that, in the context of BSS, the Wiener filter is optimal in the maximum likelihood (ML) sense only for normally-distributed signals. We then propose to model the distribution of speech signals using the Gaussian mixture model (GMM) and then derive a post-filter in the ML sense using the expectation-maximization algorithm. We show that the GMM introduces a probabilistic sample weight that is able to emphasize speech segments that are free of crosstalk components in the BSS output and this results in a better estimate of the post-filter. Simulation results show that the proposed post-filter achieves better crosstalk suppression than the Wiener filter for BSS.

In this work, we exploit, in addition to sparseness, the temporal structure of the source signals to address the problem of underdetermined blind source separation. To achieve good separation performance and reduction of artifacts, a two-stage algorithm is proposed. In the first stage, the auto-regressive (AR) coefficients of the source signals are estimated using partially separated sources that have been derived from conventional sparseness-based algorithm. In the second stage, the AR model is combined with the mixing equation to form a state-space model. This model is subsequently solved using the Kalman filter in order to obtain the refined source estimate. Simulation results show the effectiveness of proposed sparseness-based AR-Kalman (SPARK) algorithm compared to the conventional sparseness-based algorithms.

Localization of impacts on solid surfaces is a challenging task due to dispersion where the velocity of wave propagation is frequency dependent. In this work, we develop a source localization algorithm on solids with applications to human-computer interface. We employ surface-mounted piezoelectric shock sensors that, in turn, allow us to convert existing flat surfaces to a low-cost touch interface. The algorithm estimates the time-differences-of-arrival between the signals via onset detection in the time-frequency domain. The proposed algorithm is suitable for vibration signals generated by a metal stylus and a finger. The validity of the algorithm is then verified on an aluminium and a glass plate surface.

In this paper, we consider the problem of separation of unknown number of sources from their underdetermined convolutive mixtures via time-frequency (TF) masking. We propose two algorithms, one for the estimation of the masks which are to be applied to the mixture in the TF domain for the separation of signals in the frequency domain, and the other for solving the permutation problem. The algorithm for mask estimation is based on the concept of angles in complex vector space. Unlike the previously reported methods, the algorithm does not require any estimation of the mixing matrix or the source positions for mask estimation. The algorithm clusters the mixture samples in the TF domain based on the Hermitian angle between the sample vector and a reference vector using the well knownk-means or fuzzyc-means clustering algorithms. The membership functions so obtained from the clustering algorithms are directly used as the masks. The algorithm for solving the permutation problem clusters the estimated masks by usingk-means clustering of small groups of nearby masks with overlap. The effectiveness of the algorithm in separating the sources, including collinear sources, from their underdetermined convolutive mixtures obtained in a real room environment, is demonstrated.

In this paper, we derive a relation for the circular convolution operation in the discrete sine and cosine transform domains. The transform coefficients are either symmetric or asymmetric and hence we need to calculate only half of the total coefficients. Since fast algorithms are available for the computation of discrete sine and cosine transforms, the proposed method is an alternative to the discrete Fourier transform method for filtering applications.