In this paper, we introduce an app-radio cross-layer framework for improving Quality of user Experience (QoE) of Over-The-Top (OTT) Internet applications in Mobile Broadband Networks, e.g. WiMAX, 3G, LTE, etc. We apply methods similar to the well-known network layer techniques: Explicit Congestion Notification (ECN) and Differentiated Services (DiffServ) to application layer and wireless link layer. We focus on UDP based delay-sensitive real-time video call applications such as Skype, and propose to exchange cross-layer information, e.g. congestion, packet priority, etc., through API (application programming interface) based control signals instead of the existing ECN and DiffServ fields in IP data packet header. We built a prototype system with Microsoft Windows XP OS and Intel Mobile WiMAX network adapter, and use POLQA Mean Opinion Score (MOS) to measure the audio QoE for the video call. Empirical results show that immediate congestion indication effectively speeds up Skype response to bandwidth variation, and intra-flow prioritization further reduces the delay of audio packets. Both significantly improve the audio MOS of a video call.

We consider linear predictive coding and noise shaping for coding and transmission of auto-regressive (AR) sources over lossy networks. We generalize an existing framework to arbitrary filter orders and propose use of fixed-lag smoothing at the decoder, in order to further reduce the impact of transmission failures. We show that fixed-lag smoothing up to a certain delay can be obtained without additional computational complexity by exploiting the state-space structure. We prove that the proposed smoothing strategy strictly improves performance under quite general conditions. Finally, we provide simulations on AR sources, and channels with correlated losses, and show that substantial improvements are possible.

Gaussian mixture model (GMM)-based Kalman predictive coders have been shown to perform better than baseline GMM recursive coders in predictive coding of line spectral frequencies (LSFs) for both clean and packet loss conditions However, these stationary GMM Kalman predictive coders were not specifically designed for operation in packet loss conditions. In this paper, we demonstrate an approach to the the design of GMM-based predictive coding for packet loss channels. In particular, we show how a stationary GMM Kalman predictive coder can be modified to obtain a set of encoding and decoding modes, each with different Kalman gains. This approach leads to more robust performance of predictive coding of LSFs in packet loss conditions, as the coder mismatch between the encoder and decoder are minimized. Simulation results show that this Robust GMM Kalman predictive coder performs better than other baseline GMM predictive coders with no increase in complexity. To the best of our knowledge, no previous work has specifically examined the design of GMM predictive coders for packet loss conditions.

Gaussian mixture model (GMM)-based predictive coding of line spectral frequencies (LSFs) has gained wide acceptance. In such coders, each mixture of a GMM can be interpreted as defining a linear predictive transform coder. In this paper, we use Kalman filtering principles to model each of these linear predictive transform coders to present GMM Kalman predictive coding. In particular, we show how suitable modeling of quantization noise leads to an adaptive a posteriori GMM that defines a signal-adaptive predictive coder that provides improved coding of LSFs in comparison with the baseline recursive GMM predictive coder. Moreover, we show how running the GMM Kalman predictive coders to convergence can be used to design a stationary GMM Kalman predictive coding system which again provides improved coding of LSFs but now with only a modest increase in run-time complexity over the baseline. In packet loss conditions, this stationary GMM Kalman predictive coder provides much better performance than the recursive GMM predictive coder, and in fact has comparable mean performance to a memoryless GMM coder. Finally, we illustrate how one can utilize Kalman filtering principles to design a postfilter which enhances decoded vectors from a recursive GMM predictive coder without any modifications to the encoding process.

Gaussian Mixture Model (GMM)-based predictive coding of line spectral frequencies (lsf’s) has gained wide acceptance. In such coders, each mixture of a GMM can be interpreted as defining a linear predictive transform coder. In this paper we optimize each of these linear predictive transform coders using Kalman predictive coding techniques to present GMM Kalman predictive coding. In particular, we show how suitable modeling of quantization noise leads to an adaptive a-posteriori GMM that defines a signal-adaptive predictive coder that provides superior coding of lsfs in comparison with the baseline GMM predictive coder. Moreover, we show how running the Kalman predictive coders to convergence can be used to design a stationary predictive coding system which again provides superior coding of lsfs but now with no increase in run-time complexity over the baseline.

We address the problem of rate adaptation at the source, given a congested packet based voice carrying network. We propose and analyze a novel method for perceptually based frame discarding. Thus, we propose a perceptually based classifier (PBC) to do the discarding and we combine the PBC with a method for state synchronization, which exploits the knowledge of frame discards to combat the error propagation normally following a frame loss for state dependent coders. We further propose to use a combination of a queue model and an empirical speech quality model to decide on the proper discard rate for a given congestion scenario. In this design, we particularly focus on the queue model's ability to capture the characteristics of bursty traffic from voice sources in discontinuous transmission mode. Frame discarding is evaluated in a network bottleneck scenario, where objective results, based on PESQ-LQO, show significant improvements due to perceptually based frame discarding. The improvement depends on the degree of congestion without rate adaptation, the number of incoming voice flows, and the bottleneck buffer.

In this paper, we propose a new statistical model for noise periodogram modeling and estimation. The proposed model is a hidden Markov model (HMM) with a Rayleigh mixture model (RMM) in each state. For this new model, we derive an expectation-maximization (EM) training algorithm and a minimum mean-square error (MMSE) noise periodogram estimator. It is shown that when compared to the Gaussian mixture model (GMM)-based HMM, the RMM-based HMM has less computationally complex EM iterations and gives a better fit of the noise periodograms when the mixture models has a low number of components. Furthermore, we propose a specialization of the proposed model, which is shown to provide better MMSE noise periodogram estimates than any other of the tested HMM initializations for cyclo-stationary noise types

The Internet low bit-rate coder (iLBC) is now widely used for voice over Internet protocol (VoIP) applications. Unlike speech coders such as those based on code excited linear prediction (CELP), the iLBC achieves superior robustness to packet loss by avoiding inter-frame coding dependencies. While robustness to packet loss is essential, a VoIP codec should also possess the flexibility to change its source coding rate in order to counter network congestion and facilitate joint source channel coding for wireless channels. Previously, we presented a new variation of the iLBC encoding procedure which yielded a more efficient, rate-flexible result. In an effort to improve performance at lower source rates, we present various improvements to the original framework. Specifically, we reallocate bits from the adaptive codebook procedure; reduce the length of the start state vector; utilize an adaptive pulse gain quantization scheme; and extend the use of entropy coding. Overall, the various combined improvements result in the modified iLBC (with entropy coding) achieving a rate reduction of 2.0 to 2.9 kbps when compared to the original fixed-rate iLBC without any loss in quality. In comparisons with adaptive multi-rate (AMR), the modified iLBC coder remarkably exhibits equivalent perceptual evaluation of speech quality (PESQ) scores as the AMR coder at 10.2 and 12.2 kbps, and out-performs AMR for all packet loss rates. This is a significant result as the modified iLBC performs equivalent to AMR without exploiting inter-frame redundancies

Speech signals, especially voiced speech, can be better modeled by non-Gaussian autoregressive (AR) models than by Gaussian ones. Non-Gaussian AR estimators are usually highly non-linear and computationally prohibitive. This paper presents an efficient algorithm that jointly estimates the AR parameters and the exci- tation statistics and dynamics of voiced speech signals. A model called the Hidden Markov-Autoregressive model (HMARM) is de- signed for this purpose. The HMARM models the excitation to the AR model using a Hidden Markov Model with two Gaussian states that have, respectively, a small and a large mean but identi- cal variances. This formulation enables a computationally efficient exact EM algorithm to learn all parameters jointly, instead of re- sorting to pure numerical optimization or relaxed EM algorithms. The algorithm converges in typically 3 to 5 iterations. Experimen- tal results show that the estimated AR parameters have much lower bias and variance than the conventional Least Squares solution. We also show that the new estimator has a very good shift-invariance property that is useful in many applications.

Packet loss concealment (PLC) at a receiver has a substantial effect on the speech quality in voice over IP. Most conventional PLC systems have largely relied upon variations of signal repetition and overlap-add interpolation which can produce speech signals that do not follow the larger overall statistical trends. in this paper, we demonstrate how hidden Markov models can be utilized to effect PLC based on statistical signal processing. In particular, we show how HMM-based PLC yields conditional density functions that can be utilized by various statistical estimation methods that produce signal parameter estimates that produce more natural variation than conventional PLC methods, thereby providing much better speech quality

We address the problem of optimizing resource allocation for Perceptually based unequal packet protection (PUPP) in a packet based voice carrying network. For that purpose, we design a novel real-time working perceptually based classifier (PBC) optimizing the assignment of voice packets to either a premium (Pch) or an ordinary (Och) transmission channel with regard to packet perceptual importance. In particular, our PBC is based on sliding window optimization (SWO) and implement PESQa, an improved method to real-time estimation of speech quality. Based on this PBC and a differentiated service (DS) implementation of the Pch/Och, objective results indicate that 70% premium packet assignments optimizes performance over a broad range of loss scenarios on a bottleneck link. Additionally, packet loss statistics gives a clear indication on criteria for optimizing PUPP Pch/Och

In this letter, we present a decomposition for sinusoidal coding of audio, based on an amplitude modulation of sinusoids via a linear combination of arbitrary basis vectors. The proposed method, which incorporates a perceptual distortion measure, is based on a relaxation of a nonlinear least-squares minimization. Rate-distortion curves and listening tests show that, compared to a constant-amplitude sinusoidal coder, the proposed decomposition offers perceptually significant improvements in critical transient signals

As voice over IP proliferates, packet loss concealment (PLC) at the receiver has emerged as an important factor in determining voice quality of service. Through the use of heuristic variations of signal and parameter repetition and overlap-add interpolation to handle packet loss, conventional PLC systems largely ignore the dynamics of the statistical evolution of the speech signal, possibly leading to perceptually annoying artifacts. To address this problem, we propose the use of hidden Markov models for PLC. With a hidden Markov model (HMM) tracking the evolution of speech signal parameters, we demonstrate how PLC is performed within a statistical signal processing framework. Moreover, we show how the HMM is used to index a specially designed PLC module for the particular signal context, leading to signal-contingent PLC. Simulation examples, objective tests, and subjective listening tests are provided showing the ability of an HMM-based PLC built with a sinusoidal analysis/synthesis model to provide better loss concealment than a conventional PLC based on the same sinusoidal model for all types of speech signals, including onsets and signal transitions

We present a novel decomposition for sinusoidal audio coding using amplitude modulation of sinusoids via a linear combination of arbitrary basis vectors. The proposed method, which incorporates a perceptual distortion measure, is based on a relaxation of a non-linear least squares minimization. It offers benefits in the modeling of transients in audio signals. We compare the decomposition to constant-amplitude sinusoidal coding using rate-distortion curves and listening tests. Both indicate that, at the same bit-rate, perceptually significant improvements can be achieved using the proposed decomposition.

The Internet low bit-rate coder (iLBC) has emerged as a candidate for VoIP applications. iLBC is able to achieve superior robustness to packet loss. A VoIP codec should also possess the agility to adjust its source coding rate in order to react to network congestion and to be amenable to joint source channel coding for wireless channels. Towards this end, we develop a new formulation of the iLBC encoding process that allows for a variable rate iLBC. In particular, we demonstrate how the LP excitation signal is constructed from a much shorter vector of 'start state' samples through a non-square synthesis matrix that captures the effects of the adaptive codebook operations. With this new framework, the search and quantization of the start state is re-formulated as an analysis by synthesis matching problem. We demonstrate how a multi-pulse (MP) approach can be utilized to effect a variable rate coding solution for this new framework. A variable rate coder with the MP start state achieves better performance than the adaptive multi-rate (AMR) coder at 12.2 and 10.2 kbps for packet loss rates greater than 4 %.

In this paper, we investigate the importance of taking frequency-dependent temporal phenomena into account in audio coding. We do this in the context of sinusoidal modeling of audio signals by applying amplitude modulation to the sinusoidal components. Traditionally, audio coders use a fixed time-segmentation for all frequencies despite the fact that it is well-known that the time-frequency resolution of the human auditory system is not constant. The well-known window switching is an example of this. We compare multiband amplitude modulated sinusoidal models to a singleband model using different audio excerpts. Based on both comparative listening tests and a psychoacoustical distortion measure it is concluded that an improvement is generally gained using multiband amplitude modulation, although specific single sources are well-modeled using a singleband model.

In many VoIP systems, the end-to-end voice QoS is largely dictated by the packet loss rate and the packet loss concealment (PLC) method at the receiver. Typical PLC algorithms are usually based on simple signal extrapolation methods that can produce perceptually annoying artifacts. We present a taxonomy of PLC artifacts that degrade voice quality, and demonstrate how this knowledge can be used to define a PLC-driven labeling of critical voice packets. In particular, we show how the packets whose loss will produce an annoying artifact can be labeled for transmission over a virtual premium channel effected through either a DiffServe method, or adaptive FEC on a best-effort ordinary network. With a PLC-driven labeling of critical packets transmitted over a premium channel, the VoIP application can achieve large potential QoS gains over random labeling approaches, and is more robust to packet loss over an ordinary channel. To automate the labeling of packets, we demonstrate the feasibility of a sender-based packet classifier to detect the packets whose loss will produce PLC artifacts.

We consider the problem of packet loss concealment for voice over IP (VoIP). The speech signal is compressed at the transmitter using a sinusoidal coding scheme working at 8 kbit/s. At the receiver, packet loss concealment is carried out working directly on the quantized sinusoidal parameters, based on time-scaling of the packets surrounding the missing ones. Subjective listening tests show promising results indicating the potential of sinusoidal speech coding for VoIP.

In this paper, we investigate the dependency between the spectral envelopes of speech in disjoint frequency bands, one covering the telephone bandwidth from 0.3 kHz to 3.4 kHz and one covering the frequencies from 3.7 kHz to 8 kHz. The spectral envelopes are jointly modeled with a Gaussian mixture model based on mel-frequency cepstral coefficients and the log-energy-ratio of the disjoint frequency bands. Using this model, we quantify the dependency between bands through their mutual information and the perceived entropy of the high frequency band. Our results indicate that the mutual information is only a small fraction of the perceived entropy of the high band. This suggests that speech bandwidth extension should not rely only on mutual information between narrow- and high-band spectra. Rather, such methods need to make use' of perceptual properties to ensure that the extended signal sounds pleasant.

We present a novel method for predictive coding with application to transmission of speech over packet-switched networks. Our method uses multiplexing to distribute a part of the information about a segment of each speech signal in several data packets while keeping the data packet rate and payload for that part of the information unchanged. We investigate three multiplexing schemes: a packet hopping, a Hadamard multiplexing, and an extension of the Hadamard multiplexing that exploits a nonlinear preprocessing and estimation method. We show by means of formal AB-preference tests that multiplexed predictive coding can lead to coders that are more robust to packet losses than scalar quantization and packet loss concealment according to the G.711 standard.

In this paper we investigate the mutual information in speech between the spectral envelope of the high frequency band and low frequency bands of various widths. Direct methods on the computation of the mutual information often result in an excessive amount of data required even for modest situations. We reduce the required amount of data by quantizing the low band leading to a lower bound expression on the mutual information. We indicate by simulation that this lower bound is in the same order of magnitude as the true mutual information. Simulations on speech show that we have no less than 0.1 bit of shared information between the slope of the high band and the low frequency band from 0-4 kHz. Performing the analogous simulation with the gain of the high band we obtained no less than 0.45 bit of mutual information.

In this paper, we elaborate on the issue of analysis-synthesis consistency in sinusoidal coding. Our analysis is based on windowed sinusoids, and uses the same amplitude-complementary window as is used in the overlap-add synthesis. Reconstructions of the neighboring segments are taken into account when forming a particular analysis segment. Sinusoidal estimation is based on a perceptual criterion. In our new procedure, when analyzing the current segment we take advantage of the forward masking effect due to estimated sinusoids in the previous segments (possibly overlapping with the current segment). Experimental results verify that the number of sinusoids can be reduced significantly with our time masking model, without introducing perceptual artifacts in the reconstructed signal.