Deep neural networks (DNNs) have greatly benefited direction of arrival (DoA) estimation methods for speech source localization in noisy environments. However, their localization accuracy is still far from satisfactory due to the vulnerability to nonspeech interference. To improve the robustness against interference, we propose a DNN based normalized time-frequency (T-F) weighted criterion which minimizes the distance between the candidate steering vectors and the filtered snapshots in the T-F domain. Our method requires no eigendecomposition and uses a simple normalization to prevent the optimization objective from being misled by noisy filtered snapshots. We also study different designs of T-F weights guided by a DNN. We find that duplicating the Hadamard product of speech ratio masks is highly effective and better than other techniques such as direct masking and taking the mean in the proposed approach. However, the best-performing design of T-F weights is criterion-dependent in general. Experiments show that the proposed method outperforms popular DNN based DoA estimation methods including widely used subspace methods in noisy and reverberant environments.

The incompleteness of speech inputs severely degrades the performance of all the related speech signal processing applications. Although many researches have been proposed to address this issue, they controlled the data missing conditions by simulation with self-defined masking lengths or sizes. Besides, the masking definitions are different among all these experimental settings. This paper presents a novel intermittent speech recovery (ISR) system for real-world self-powered intermittent devices. Three contributive stages: interpolation, enhancement, and combination are applied to the ISR system for speech reconstruction. The experimental results show that our recovery system increases speech quality by up to 591.7%, while increasing speech intelligibility by up to 80.5%. Most importantly, the proposed ISR system improves the WER scores by up to 52.6%. The promising results not only confirm the effectiveness of the reconstruction but also encourage the utilization of these battery-free wearable/IoT devices.

A deep neural network using rectified linear units represents a continuous piecewise linear (CPWL) function and vice versa. Recent results in the literature estimated that the number of neurons needed to exactly represent any CPWL function grows exponentially with the number of pieces or exponentially in terms of the factorial of the number of distinct linear components. Moreover, such growth is amplified linearly with the input dimension. These existing results seem to indicate that the cost of representing a CPWL function is expensive. In this paper, we propose much tighter bounds and establish a polynomial time algorithm to find a network satisfying these bounds for any given CPWL function. We prove that the number of hidden neurons required to exactly represent any CPWL function is at most a quadratic function of the number of pieces. In contrast to all previous results, this upper bound is invariant to the input dimension. Besides the number of pieces, we also study the number of distinct linear components in CPWL functions. When such a number is also given, we prove that the quadratic complexity turns into bilinear, which implies a lower neural complexity because the number of distinct linear components is always not greater than the minimum number of pieces in a CPWL function. When the number of pieces is unknown, we prove that, in terms of the number of distinct linear components, the neural complexities of any CPWL function are at most polynomial growth for low-dimensional inputs and factorial growth for the worst-case scenario, which are significantly better than existing results in the literature.

Models recently used in the literature proving residual networks (ResNets) are better than linear predictors are actually different from standard ResNets that have been widely used in computer vision. In addition to the assumptions such as scalar-valued output or single residual block, the models fundamentally considered in the literature have no nonlinearities at the final residual representation that feeds into the final affine layer. To codify such a difference in nonlinearities and reveal a linear estimation property, we define ResNEsts, i.e., Residual Nonlinear Estimators, by simply dropping nonlinearities at the last residual representation from standard ResNets. We show that wide ResNEsts with bottleneck blocks can always guarantee a very desirable training property that standard ResNets aim to achieve, i.e., adding more blocks does not decrease performance given the same set of basis elements. To prove that, we first recognize ResNEsts are basis function models that are limited by a coupling problem in basis learning and linear prediction. Then, to decouple prediction weights from basis learning, we construct a special architecture termed augmented ResNEst (A-ResNEst) that always guarantees no worse performance with the addition of a block. As a result, such an A-ResNEst establishes empirical risk lower bounds for a ResNEst using corresponding bases. Our results demonstrate ResNEsts indeed have a problem of diminishing feature reuse; however, it can be avoided by sufficiently expanding or widening the input space, leading to the above-mentioned desirable property. Inspired by the densely connected networks (DenseNets) that have been shown to outperform ResNets, we also propose a corresponding new model called Densely connected Nonlinear Estimator (DenseNEst). We show that any DenseNEst can be represented as a wide ResNEst with bottleneck blocks. Unlike ResNEsts, DenseNEsts exhibit the desirable property without any special architectural re-design.

SSR methods that majorize the regularized objective function during the optimization process. We show that introducing the majorizers leads to the same algorithm as simply using the gradient update of the regularized objective function, as is done in existing approaches. Different from the past works, the reweighting formulation naturally leads to an affine scaling transformation (AST) strategy, which effectively introduces a diagonal weighting on the gradient, giving rise to new algorithms that demonstrate improved convergence properties. Interestingly, setting the regularization coefficient to zero in the proposed AST-based framework leads to the Sparsity-promoting LMS (SLMS) and Sparsity-promoting Normalized LMS (SNLMS) algorithms, which exploit but do not strictly enforce the sparsity of the system response if it already exists. The SLMS and SNLMS realize proportionate adaptation for convergence speedup should sparsity be present in the underlying system response. In this manner, we develop a new way for rigorously deriving a large class of proportionate algorithms, and also explain why they are useful in applications where the underlying systems admit certain sparsity, e.g., in acoustic echo and feedback cancellation.

While deep neural networks (DNNs) have achieved state-of-the-art results in many fields, they are typically over-parameterized. Parameter redundancy, in turn, leads to inefficiency. Sparse signal recovery (SSR) techniques, on the other hand, find compact solutions to overcomplete linear problems. Therefore, a logical step is to draw the connection between SSR and DNNs. In this paper, we explore the application of iterative reweighting methods popular in SSR to learning efficient DNNs. By efficient, we mean sparse networks that require less computation and storage than the original, dense network. We propose a reweighting framework to learn sparse connections within a given architecture without biasing the optimization process, by utilizing the affine scaling transformation strategy. The resulting algorithm, referred to as Sparsity-promoting Stochastic Gradient Descent (SSGD), has simple gradient-based updates which can be easily implemented in existing deep learning libraries. We demonstrate the sparsification ability of SSGD on image classification tasks and show that it outperforms existing methods on the MNIST and CIFAR-10 datasets.

In this letter, we propose a novel conjugate gradient (CG) adaptive filtering algorithm for online estimation of system responses that admit sparsity. Specifically, the Sparsity-promoting Conjugate Gradient (SCG) algorithm is developed based on iterative reweighting methods popular in the sparse signal recovery area. We propose an affine scaling transformation strategy within the reweighting framework, leading to an algorithm that allows the usage of a zero sparsity regularization coefficient. This enables SCG to leverage the sparsity of the system response if it already exists, while not compromising the optimization process. Simulation results show that SCG demonstrates improved convergence and steady-state properties over existing methods.

Acoustic feedback control continues to be a challenging problem due to the emerging form factors in advanced hearing aids (HAs) and hearables. In this paper, we present a novel use of well-known all-pass filters in a network to perform frequency warping that we call "freping." Freping helps in breaking the Nyquist stability criterion and improves adaptive feedback cancellation (AFC). Based on informal subjective assessments, distortions due to freping are fairly benign. While common objective metrics like the perceptual evaluation of speech quality (PESQ) and the hearing-aid speech quality index (HASQI) may not adequately capture distortions due to freping and acoustic feedback artifacts from a perceptual perspective, they are still instructive in assessing the proposed method. We demonstrate quality improvements with freping for a basic AFC (PESQ: 2.56 to 3.52 and HASQI: 0.65 to 0.78) at a gain setting of 20; and an advanced AFC (PESQ: 2.75 to 3.17 and HASQI: 0.66 to 0.73) for a gain of 30. From our investigations, freping provides larger improvement for basic AFC, but still improves overall system performance for many AFC approaches.

State-of-the-art noise power spectral density (PSD) estimation techniques for speech enhancement utilize the so-called speech presence probability (SPP). However, in highly non-stationary environments, SPP-based techniques could still suffer from inaccurate estimation, leading to significant amount of residual noise or speech distortion. In this paper, we propose to improve speech enhancement by deploying the bone-conduction (BC) sensor, which is known to be relatively insensitive to the environmental noise compared to the regular air-conduction (AC) microphone. A strategy is suggested to utilized the BC sensor characteristics for assisting the AC microphone in better SPP-based noise estimation. To our knowledge, no previous work has incorporated the BC sensor in this noise estimation aspect. Consequently, the proposed strategy can possibly be combined with other BC sensor assisted speech enhancement techniques. We show the feasibility and potential of the proposed method for improving the enhanced speech quality by both objective and subjective tests.

In this letter, we consider the problem of joint blind source separation of multiple datasets simultaneously using an Independent vector analysis (IVA) framework. In particular we propose a new paradigm of reweighted algorithms for IVA by employing a source prior from a multivariate generalized scale mixture distribution family. In addition, our proposed reweighted algorithms can also exploit second-order statistical information across datasets by learning intrasource correlation matrix of each source component vector (SCV) along with higher order statistics. Experimental results are provided to show the efficacy of our proposed algorithms in achieving reliable source separation for both the cases, i.e., when there is correlation present within an SCV, and also when the sources are uncorrelated, i.e., no second order dependencies across datasets.

Conventional EEG systems are extremely power hungry due to a reliance on high resolution analog-to-digital converters (ADCs) to cover their required dynamic range. This paper presents a novel architecture for acquiring multi-channel high-resolution EEG using low power techniques. Instead of digitizing the output of each analog front-end (AFE) amplifier with an individual ADC, a voltage-controlled oscillator is used to generate an up-converted frequency-modulated (FM) signal at a unique carrier frequency. All channels then share a single wire via frequency-domain multiplexing (FDM), enabling a rugged mechanical design. The composite FM signal is then digitized with a single ADC optimized for time-domain resolution (1 MHz, 12 bits) rather than employing multiple ADCs optimized for voltage-domain resolution (1 kHz, 16–24 bits), thereby enabling a low-power implementation. To validate this approach, a discrete prototype is developed and achieves 75 dB of dynamic range per channel.

Motion artifacts interfere with electrocardiogram (ECG) detection and information processing. In this paper, we present an independent component analysis based technique to mitigate these signal artifacts. We propose a new statistical measure to enable an automatic identification and removal of independent components, which correspond to the sources of noise. For the first time, we also present a signal-dependent closed-loop system for the quality assessment of the denoised ECG. In one experiment, noisy data is obtained by the addition of calibrated amounts of noise from the MIT-BIH NST database to the AHA ECG database. Arrhythmia classification based on a state-of-the-art algorithm with the direct use of noisy data thus obtained shows sensitivity and positive predictivity values of 87.7% and 90.0%, respectively, at an input signal SNR of -9 dB. Detection with the use of ECG data denoised by the proposed approach exhibits significant improvement in the performance of the classifier with the corresponding results being 96.5% and 99.1%, respectively. In a related lab trial, we demonstrate a reduction in RMS error of instantaneous heart rate estimates from 47.2% to 7.0% with the use of 56 minutes of denoised ECG from four physically active subjects. To validate our experiments, we develop a closed-loop, ambulatory ECG monitoring platform, which consumes 2.17 mW of power and delivers a data rate of 33 kbps over a dedicated UWB link.

Audio coding based on frequency domain linear prediction (FDLP) uses auto-regressive model to approximate Hilbert envelopes in frequency sub-bands for relatively long temporal segments. Although the basic technique achieves good quality of the reconstructed signal, there is a need for improving the coding efficiency. In this paper, we present a novel method for the application of temporal masking to reduce the bit-rate in a FDLP based codec. Temporal masking refers to the hearing phenomenon, where the exposure to a sound reduces response to following sounds for a certain period of time (up to 200 ms). In the proposed version of the codec, a first order forward masking model of the human ear is implemented and informal listening experiments using additive white noise are performed to obtain the exact noise masking thresholds. Subsequently, this masking model is employed in encoding the sub- band FDLP carrier signal. Application of the temporal masking in the FDLP codec results in a bit-rate reduction of about 10% without degrading the quality. Performance evaluation is done with perceptual evaluation of audio quality (PEAQ) scores and with subjective listening tests.

Audio coding based on Frequency Domain Linear Prediction (FDLP) uses auto-regressive models to approximate Hilbert envelopes in frequency sub-bands. Although the basic technique achieves good coding efficiency, there is a need to improve the reconstructed signal quality for tonal signals with impulsive spectral content. For such signals, the quantization noise in the FDLP codec appears as frequency components not present in the input signal. In this paper, we propose a technique of Spectral Noise Shaping (SNS) for improving the quality of tonal signals by applying a Time Domain Linear Prediction (TDLP) filter prior to the FDLP processing. The inverse TDLP filter at the decoder shapes the quantization noise to reduce the artifacts. Application of the SNS technique to the FDLP codec improves the quality of the tonal signals without affecting the bit-rate. Performance evaluation is done with Perceptual Evaluation of Audio Quality (PEAQ) scores and with subjective listening tests.

In this paper, we propose an efficient scheme to transport video over wireless networks, specifically cdma2000® 1x. Speech transmission over cdma2000® uses a variable rate voice coder (vocoder) over a channel with multiple fixed rates. We apply these ideas to compressed video transmission over wireless IP networks. Explicit Bit Rate (EBR) video compression is designed to match the video encoder output to a set of fixed channel rates. We show that in comparison with VBR video transmission over a fixed rate wireless channel, EBR video transmission provides improved error resilience, reduced latency and improved efficiency.

PureVoice™ VR is a speech recognition system characterized by noise robustness performance, small footprint, and limited computing requirement. The system was embedded into Qualcomm's Code Division Multiple Access (CDMA) Mobile Station Modem (MSM) chipsets. The PureVoice™ VR engine enabled many manufactures of CDMA cellular phones to add highly accurate voice-activated dialing features to their handsets and hands-free car kits without significant increase in hardware cost. Users of these cellular phones could make calls by simply speaking names or phone numbers. The current embedded system has the capacity to recognize 100 speaker-dependent (SD) nametags and user defined commands, or 30 speaker-independent (SI) digits and commands in realtime. It also works in a hybrid mode, in which an input utterance can be either SD namtags or SI commands. PureVoice™ VR has successfully supported the voice-activated dialing function for millions of CDMA handsets sold in the world market.

Our feature extraction module for the Aurora task is based on a combination of a conventional noise supression technique (Wiener filtering) with our temporal processing technigues (linear discriminant RASTA filtering and nonlinear TempoRAl Pattern (TRAP) classifier). We observe better than 58% relative error improvement on the prescribed Aurora Digit Task, a performance level that is somewhat better than the new ETSI Advanced Feature standard. Further- more, to test generalization of our approach to an independent test set not available during development, we evaluate performance on American English SpeechDatCar digits and show 10.54% relative improvement over the new ETSI stan- dard.

This paper describes an automatic speech recognition frontend that combines low-level robust ASR feature extraction techniques, and higher-level linear and non-linear feature transformations. The low-level algorithms use data-derived filters, mean and variance normalization of the feature vectors, and dropping of noise frames. The feature vectors are then linearly transformed using Principal Components Analysis (PCA). An Artificial Neural Network (ANN) is also used to compute features that are useful for classification of speech sounds. It is trained for phoneme probability estimation on a large corpus of noisy speech. These transformations lead to two feature streams whose vectors are concatenated and then used for speech recognition. This method was tested on the set of speech corpora used for the “Aurora” evaluation. Using the feature stream generated without the ANN yields an overall 41% reduction of the error rate over Mel-Frequency Cepstral Coefficients (MFCC) reference features. Adding the ANN stream further reduces the error rate yielding a 46% reduction over the reference features.

First pass acoustic models in a two pass, large vocabulary, speaker dependent, continuous speech recognizer are discussed. The models are based on single state VQ, compared to our earlier 5-state continuous HMMs. The loss in performance was recovered using context dependent models, multiple codebooks and improvements in the computation of the observation vector. This resulted in a first pass inclusion of >97% while reducing the computational burden by a factor of 30. Currently, the system runs real-time on an AT&T DSP32C module and an HP-735 workstation, with recognition accuracies of up to 90% for a 5000 word vocabulary task.>

Bilinear representations of Cohen's class are expressed as the Wigner distribution (WD) of the signal convolved with a kernel defined as a 2-D window in time and frequency. Gaussian, exponential and cone kernels are compared by examining their local behavior in the time-frequency plane. It is demonstrated that kernel spread and localization properties are important concerns for analyzing time-varying signals. The cone kernel has good cross-term suppression for speech-like signals because of its large time extent. An effect of this is to enhance stationary terms and attenuate time-varying terms.>