Historically, sparse methods and neural networks, particularly modern deep learning methods, have been relatively disparate areas. Sparse methods are typically used for signal enhancement, compression, and recovery, usually in an unsupervised framework, while neural networks commonly rely on a supervised training set. In this paper, we use the specific problem of sequential sparse recovery, which models a sequence of observations over time using a sequence of sparse coefficients, to show how algorithms for sparse modeling can be combined with supervised deep learning to improve sparse recovery. Specifically, we show that the iterative soft-thresholding algorithm (ISTA) for sequential sparse recovery corresponds to a stacked recurrent neural network (RNN) under specific architecture and parameter constraints. Then we demonstrate the benefit of training this RNN with backpropagation using supervised data for the task of column-wise compressive sensing of images. This training corresponds to adaptation of the original iterative thresholding algorithm and its parameters. Thus, we show by example that sparse modeling can provide a rich source of principled and structured deep network architectures that can be trained to improve performance on specific tasks.

This paper proposes a robust ground-moving-platform-based human tracking system, which effectively integrates visual simultaneous localization and mapping (V-SLAM), human detection, ground plane estimation, and kernel-based tracking techniques. The proposed system systematically detects humans from recorded video frames of a moving camera and tracks the humans in the V-SLAM-inferred 3-D space via a tracking-by-detection scheme. To efficiently associate the detected human frame by frame, we propose a novel human tracking framework, combining the constrained-multiple-kernel tracking and the estimated 3-D information (depth), to globally optimize the data association between consecutive frames. By taking advantage of the appearance model and 3-D information, the proposed system not only achieves high effectiveness but also well handles occlusion in the tracking. Experimental results show the favorable performance of the proposed system, which efficiently tracks humans in a camera equipped on a ground-moving platform such as a dash camera and an unmanned ground vehicle.

In this paper, we propose an innovative human tracking algorithm, which efficiently integrates the deformable part model (DPM) into the multiple-kernel based tracking using a moving camera. By representing each part model of a DPM detected human as a kernel, the proposed algorithm iteratively mean-shift the kernels (i.e., part models) based on color appearance and histogram of gradient (HOG) features. More specifically, the color appearance features, in terms of kernel histogram, are used for tracking each body part from one frame to the next, the deformation cost provided by DPM detector is further used to constrain the movement of each body kernel based on the HOG features. The proposed deformable multiple-kernel (DMK) tracking algorithm takes advantage of not only low computation owing to the kernelbased tracking, but also robustness of the DPM detector. Experimental results have shown the favorable performance of the proposed algorithm, which can successfully track human using a moving camera more accurately under different scenarios.

Many voice activity detection (VAD) systems use the magnitude of complex-valued spectral representations. However, using only the magnitude often does not fully characterize the statistical behavior of the complex values. We present two novel methods for performing VAD on single- and dual-channel audio that do completely account for the second-order statistical behavior of complex data. Our methods exploit the second-order noncircularity (also known as impropriety) of complex subbands of speech and noise. Since speech tends to be more improper than noise, higher impropriety suggests speech activity. Our single-channel method is blind in the sense that it is unsupervised and, unlike many VAD systems, does not rely on non-speech periods for noise parameter estimation. Our methods achieve improved performance over other state-of-the-art magnitude-based VADs on the QUT-NOISE-TIMIT corpus, which indicates that impropriety is a compelling new feature for voice activity detection.

The analysis of signals consisting of multiple components with non-linear frequency modulation is required in a large number of applications, including the study of marine mammals vocalizations. This analysis has multiple motivations, such as investigating the impact of anthropogenic noise on marine mammal behavior, and species identification to avoid collision between ships and marine mammals. Such applications are normally conducted in a Passive Acoustics Monitoring (PAM) context, where there is low SNR together with very little a priori knowledge on the signals being analyzed. Recently, time-frequency tracking based on local analysis of the instantaneous phase has been successfully applied to underwater signals. In this paper, we present a robust version of this method, based on the use of the multiple analysis windows. The results provided for simulated and real signals demonstrate improved tracking of the instantaneous frequency in noise.

This paper presents an approach for adapting a tracking algorithm to the acoustic propagation environment. This adaptation is performed by incorporating the expected target signal-to-noise ratio (SNR) into the data association step through the measured contact amplitude. In this work, expected SNR is provided via acoustic modeling; estimates of bottom loss and scattering strength, required by the acoustic model, are obtained via inversion of the acoustic model based on measured multi-static sonar reverberation data. This paper shows that the use of distributed sensors provides improved estimates of the environmental parameters, and hence better estimates of the expected SNR

The modulation spectrum is a promising method to incorporate dynamic information in pattern classification. It contains important cues about the nonstationary content of a signal and yields complementary improvements when it is combined with conventional features derived from short-term analysis. Many prior modulation spectrum approaches are based on uniform modulation frequency decomposition. The drawbacks of these approaches are high dimensionality and a lack of a connection to human perception of modulation. The paper presents multi-scale modulation frequency decomposition and shows an improvement over the standard modulation spectrum in a digital communication signal classification task. Features derived from this representation provide lower classification error rates than those from a constant-bandwidth modulation spectrum, whether used alone or in combination with short-term features.

Time-varying short-term spectral estimates have been successfully applied in many classification tasks. However, they are still insufficient for many non-stationary signals where time-varying information is useful. We propose to improve the deficiencies of current short-term feature analysis by adding information to describe the time-varying behavior of the signals. Our proposed method, which is motivated by the human auditory system, can be applied to several non-stationary signal types. Real world communication signals were used for experimental verification. These experimental results, assessed with a conventional probabilistic classifier, showed significant improvement when the new features were added to short-term spectral estimates.

This paper extends Thomson's (1982) multitaper spectrum estimation method to nonstationary signals. The method uses a newly-derived set of basis functions which generalize the concentration properties of the prolate spheroidal waveforms to the time-frequency case. We solve for the basis which diagonalizes the nonstationary spectrum generating operator over a finite region of the time-frequency plane. These eigenfunctions are maximally concentrated to and orthogonal over the specified time-frequency region, and are thus doubly orthogonal. Individual spectrograms computed with these eigenfunctions form direct time-frequency spectrum estimates. We next present a multitaper time-frequency spectrum estimation procedure using these time-frequency eigenestimates. Bias and variance expressions are derived, allowing for a statistical characterization of the accuracy of the estimate. The time-frequency concentration property of the basis functions yields an estimator with excellent bias properties, while the variance of the estimate is reduced through the use of multiple orthogonal windows.

This paper presents a new foundation for positive time-frequency distributions (TFDs). Based on an integral equation formulation of nonstationary systems, a positive TFD can be constructed from a decomposition of a signal over an orthonormal basis. This basis function definition of a positive TFD is used to obtain a relationship between the Wigner distribution and the positive TFD. The results are then generalized to derive positive joint distributions over arbitrary variables, following the approach of Baraniuk and Jones (1995). This general theory provides a common foundation for the two approaches of computing time-frequency representations: those based on linear decompositions of the signal (e.g. best basis methods) and those based on a quadratic, or bilinear, functional of the signal (i.e. Cohen's bilinear class).

Modern speech processing research may be categorized into three broad areas: statistical, physiological, and perceptual. Statistical research investigates the nature of the variability of the speech waveform from a signal processing viewpoint. This approach relates to the processing of speech in order to obtain measurements of speech characteristics which demonstrate manageable variabilities across a wide range of the talker population, in the presence of noise or competing speakers as well as the interaction of speech with the channel through which it is transmitted, and under the inherent interaction of the information content of speech itself (i.e., the contextual factor). Physiological research aims at constructing accurate models of the articulatory and auditory process, helping to limit the signal space for speech processing. In the perceptual realm, work focuses on understanding the psychoacoustic and possibly the psycholinguistic aspects of the speech communication process that the human so conveniently conducts. By studying this working analysis/recognition system, insights may be garnered that will lead to improved methods of speech processing. Conversely by studying the limitations of this system, particularly how it reduces the information rate of the received signal through, for example, masking and adaptation improvements may be made in the efficiency of speech coding schemes without impacting the quality of the reconstructed speech. Thus comprehension of speech production and perception impacts methods of speech processing, and vice-versa. This paper enunciates such a position, focusing on how modern time-frequency signal analysis methods could help expedite needed advances in these areas.

We present a general approach to approximating positive time-frequency distributions (TFDs) through nonlinear combinations of spectrograms. Closed-form solutions for the combinations are obtained via optimization of entropy functionals subject to an energy constraint. We apply two such combinations to generating approximate TFDs for whale sounds and speech. Through these applications, it can be seen that these methods give results superior to that achieved with individual spectrograms, and remarkably close to the positive TFDs obtained via computationally-intensive methods.

We demonstrate that two previously proposed methods for combining the information content from multiple spectrograms into a single, positive time-frequency function are optimal in a cross-entropy sense. The goal in combining the spectrograms is to obtain an improved approximation of the joint time-frequency signal density by overcoming limitations of any single spectrogram. An example of each method is provided, and results are compared with spectrograms and a Cohen-Posch (1985) time-frequency density (TFD) of a nonstationary pulsed tone signal. The proposed combinations are effective and can be efficiently computed.>

Much of our current knowledge and intuition of speech is derived from analyses involving assumptions of short-time stationarity (e.g., the speech spectrogram). Such methods are, by their very nature, incapable of revealing the true nonstationary nature of speech. A careful consideration of the theory of time-frequency distributions (TFDs), however, allows the construction of methods that reveal far more of the nonstationarities of speech, thereby highlighting just what it is that conventional approaches miss. We apply two iterative methods for generating positive time-frequency distributions (TFDs) to speech analysis. Both methods make use of multiple sources of information (e.g., multiple spectrograms) to yield a high-resolution estimate of the joint time-frequency energy density of speech. Plosive events and formant harmonic structure are simultaneously preserved in these TFDs. Rapidly time-varying formants are also resolved by these TFDs, and harmonic structure is revealed, independent of sweep rate; this result is quite different from that seen with conventional speech spectrograms. The speech features observed in these distributions demonstrate that conventional sliding window techniques lose or distort much of the rich nonstationary structure of speech. Examples for synthetic formants and real speech are provided. The differences between joint distributions and conditional distributions are also illustrated.>

The principle of minimum cross entropy (MCE) is used to generate positive distributions in the Cohen-Posch (1985) class of proper time-frequency distributions (TFDs). The MCE-TFDs are not only intuitively satisfying, but they also yield the correct marginals, have strong finite support, and are everywhere nonnegative. The usual cross-term artifacts that hinder interpretation of other time-frequency representations (most notably, the Wigner-Ville) are not a problem with the MCE-TFDs. Examples of speech and chirps are given and compared to the spectrogram. An interesting observation in the case of speech is that spectrograms more closely resemble time-conditional-frequency distributions (narrowband) or frequency-conditional-time distributions (wideband) than they do joint time-frequency distributions.>

The assumption of quasi-stationarity in the analysis of speech is questioned from the standpoint of the best resolution of global periodicity and vocal tract formant frequency locations. The generalized class of time-frequency representations is discussed from the standpoint of speech analysis and is integrated with a description of the spectrogram. The spectrogram is known to have minimal interference artifacts, and the truly nonstationary cone-kernel time-frequency representation (CK-TFR) is shown to be similarly free of interference. The CK-TFR is observed to give higher resolution of the time point of glottal closure, and the clarity of some formant frequencies, especially for a nasal consonant, is shown to be better than the spectrogram. It is shown that interference-free representations of speech with a higher simultaneous resolution in time and frequency than the spectrogram are possible, and that these new representations may be applicable to the better analysis and understanding of speech.>

Two properties are presented that constrain the cross-terms of Cohen-class time-frequency representations (TFRs) to appear only at signal frequencies, and only when the signal is nonzero. These properties thus guarantee strong finite support, i.e. the TFR is zero everywhere the signal or its spectrum is zero. When combined with cross-term attenuation, one can obtain TFRs with spectrogram-like interference suppression, but without the inherent time-frequency resolution tradeoff of the spectrogram.>