Recent contrastive methods show significant improvement in self-supervised learning in several domains. In particular, contrastive methods are most effective where data augmentation can be easily constructed e.g. in computer vision. However, they are less successful in domains without established data transformations such as time series data. In this paper, we propose a novel self-supervised learning framework that combines contrastive learning with neural processes. It relies on recent advances in neural processes to perform time series forecasting. This allows to generate augmented versions of data by employing a set of various sampling functions and, hence, avoid manually designed augmentations. We extend conventional neural processes and propose a new contrastive loss to learn times series representations in a self-supervised setup. Therefore, unlike previous self-supervised methods, our augmentation pipeline is task-agnostic, enabling our method to perform well across various applications. In particular, a ResNet with a linear classifier trained using our approach is able to outperform state-of-the-art techniques across industrial, medical and audio datasets improving accuracy over 10% in ECG periodic data. We further demonstrate that our self-supervised representations are more efficient in the latent space, improving multiple clustering indexes and that fine-tuning our method on 10% of labels achieves results competitive to fully-supervised learning.

AutoAugment [4] has sparked an interest in automated augmentation methods for deep learning models. These methods estimate image transformation policies for train data that improve generalization to test data. While recent papers evolved in the direction of decreasing policy search complexity, we show that those methods are not robust when applied to biased and noisy data. To overcome these limitations, we reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task that minimizes the distribution shift between test data and distorted train dataset. In our AutoDO model, we explicitly estimate a set of per-point hyperparameters to flexibly change distribution of train data. In particular, we include hyperparameters for augmentation, loss weights, and softlabels that are jointly estimated using implicit differentiation. We develop a theoretical probabilistic interpretation of this framework using Fisher information and show that its complexity scales linearly with the dataset size. Our experiments on SVHN, CIFAR-10/100, and ImageNet classification show up to 9.3% improvement for biased datasets with label noise compared to prior methods and, importantly, up to 36.6% gain for underrepresented SVHN classes1.

Dense optical flow estimation is complex and time consuming, with state-of-the-art methods relying either on large synthetic data sets or on pipelines requiring up to a few minutes per frame pair. In this paper, we address the problem of optical flow estimation in the automotive scenario in a self-supervised manner. We argue that optical flow can be cast as a geometrical warping between two successive video frames and devise a deep architecture to estimate such transformation in two stages. First, a dense pixel-level flow is computed with a projective bootstrap on rigid surfaces. We show how such global transformation can be approximated with a homography and extend spatial transformer layers so that they can be employed to compute the flow field implied by such transformation. Subsequently, we refine the prediction by feeding a second, deeper network that accounts for moving objects. A final reconstruction loss compares the warping of frame Xtwith the subsequent frame Xt+1and guides both estimates. The model has the speed advantages of end-to-end deep architectures while achieving competitive performances, both outperforming recent unsupervised methods and showing good generalization capabilities on new automotive data sets.

This paper presents a generalized feature projection scheme which allows each feature dimension to be classified in a set of 1 to M classes, where M is the total number of classes. Our method is an extension of the classical full-space null-space approach where each dimension can only be classified in either M classes or 1 class. We believe that this more general formulation allows for a better trade-off of number of parameters versus model complexity, which in turn should provide better classification. We first tested GLDA on TIMIT and obtained an improvement up to 1% in phone classification rate over the best HLDA classifier. Preliminary results on Wall Street Journal 20K also show an improvement over the best HLDA system of about 0.2% absolute.

Our previous study on maximum relative margin estimation (MRME) of HMM ([7, 8]) demonstrated its advantage over the standard minimum classification error (MCE) training. In this paper, we report our recent improvement on MRME. Specifically, two novel approaches are proposed to handle recognition errors in training sets for the MRME. One is a new training criterion based on a combination of MRME and MCE objective functions. The other approach proposes to remove a strong constraint in the original MRME algorithm, so that MRME algorithm can be applied to all training data as opposed to only correctly recognized data in the original MRME approach. Both new approaches can take advantage of more training data during the large margin training and can bootstrap directly from MLE models without a separate MCE training step. Improvement on recognition accuracy has been achieved on a speaker independent connected digit strings recognition task using the TIDIGITS database.

The paper addresses the problem of noise robustness from the standpoint of the sensitivity to noise estimation errors. Since the noise is usually estimated in the power-spectral domain, we show that the implied error in the cepstral domain has interesting properties. These properties allow us to compare two key methods used in noise robust speech recognition: spectral subtraction and parallel model combination. We show that parallel model combination has an advantage over spectral subtraction because it is less sensitive to noise estimation errors. Experimental results on the Aurora2 database confirm our theoretical findings, with parallel model combination clearly outperforming spectral subtraction and other well-known signal-based robustness methods. Our Aurora2 results, with parallel model combination, a basic MFCC front-end and a simple noise estimation, are close to the best results obtained on this database with very complex signal processing schemes.

This paper presents experiments of noise robust ASR on the Aurora4 database. The database is designed to test large vocabulary systems in presence of noise and channel distortions. A number of different model-based and signal-based noise robustness techniques have been tested. Results show that it is difficult to design a technique that is superior in every condition. Because of this we combined different techniques to improve results. Best results have been obtained when short time compensation / normalization methods are combined with long term environmental adaptation and robust acoustic models. The best average error rate obtained over the 52 conditions is 30.8%. This represents a 40 % relative improvement compared to the baseline results [1]. 1.

In this paper we propose an enhancement of the Jacobian adaptation by estimating automatically a noise over estimation factor which yields to a closer approximation of Parallel model combination (PMC) than the traditional Jacobian adaptation. Noise over estimation factors are estimated at run-time for a set of clustered Gaussians obtained on the training set. Experiments conducted on a French natural number database show that similar performance as PMC can be obtained at the expense of a slight increase in computational complexity as compared to Jacobian adaptation.

Setting linear constraints on HMM model space appears to be very effective for speaker adaptation. In doing so, we assume that model parameters are jointly Gaussian. While this approach has proven reasonably successful, we question it accuracy in the case of very high dimensionality parameter spaces. To address this problem, we employ a hierarchical piecewise linear model. Gross speaker variations are modeled with a linear eigenspace, subsuming the joint Gaussian model, and finer residues are modeled using another eigenspace chosen depending on the location of the first values. We perform experiments on Wall Street Journal (WSJ) dictation task, and we observe a cumulative 1.3% WER improvement (11 % relative) when using self-adaptation.

Cepstral mean normalization is the standard technique for channel robustness. Despite its good performance, the effectiveness of cepstral mean normalization (CMN) for short sentences is argued. CMN underlying hypothesis that the speech cepstral mean is constant is not valid for short processing windows. This implies the removal of some phonetic information. In this paper we show that the speech correlation structure may be used to estimate the communication channel and we propose an efficient algorithm to compute this estimate. We argue that the resulting channel estimate is more accurate because the underlying hypothesis is better verified than the original CMN hypothesis. Results for the Kai-Fu Lee phone recognition task on NTIMIT, with acoustic models trained on TIMIT (mismatch conditions), show that our method provides an 8% relative error rate reduction as compared to CMN.

We propose an algorithm that compensates for both additive and convolutional noise. The goal of this method is to achieve an efficient environmental adaptation to realistic environments both in terms of computation time and memory. The algorithm described in this paper is an extension of an additive noise adaptation algorithm. Experimental results are given on a realistic database recorded in a car. This database is further filtered by a low pass filter to combine additive and channel noise. The proposed adaptation algorithm reduces the error rate by 75 % on this database, when compared to our baseline system without environmental adaptation.

The "eigenvoice" technique achieves rapid speaker adaptation by employing prior knowledge of speaker space obtained from reference speakers to place strong constraints on the initial model for each new speaker. It has previously been shown to yield very fast adaptation for a large-vocabulary system. In this paper, we describe a new way of applying the eigenvoice technique to context-dependent acoustic modeling, called the "eigencentroid plus delta trees" (EDT) model. Here, the context-dependent model is defined so that it consists of a speaker-dependent component with a small number of parameters linked to a speaker-independent component with far more parameters. The eigenvoice technique can then be applied to the speaker-dependent component alone to attain very fast adaptation of the entire context-dependent model (e.g., 10% relative reduction in error rate after 3 sentences). EDT requires only a small number of parameters to represent speaker space and works even if only a small amount of data is available per reference speaker.

The problem of clustering Gaussian distributions can be effectively solved by standard vector quantization algorithms where the metric is defined by the Bhattacharyya distance. This paper presents a novel algorithm for computing the optimal centroid for a cluster of Gaussian distributions according to the Bhattacharyya metric. We show that this centroid maximizes an upper bound on the probability of representing the population modeled by the distributions associated with the cluster. The proposed method is evaluated in clustering distributions of hidden Markov model speech recognizers to reduce the overall memory consumption and runtime complexity of the decoding. Experimental results show that, depending on the task, the number of distributions can be reduced by a factor of 2 to 6 with an increase in recognition accuracy. When compared to a maximum likelihood centroid, the Bhattacharyya centroid provides a 13% error rate reduction in a 2k word recognition task.

We present an optimized implementation of the Viterbi algorithm suitable for small to large vocabulary, and isolated or continuous speech recognition. The Viterbi algorithm is certainly the most popular dynamic programming algorithm used in speech recognition. In this paper we propose a new algorithm that outperforms the Viterbi algorithm in term of complexity and of memory requirements. It is based on the assumption of strictly left to right models and explores the lexical tree in an optimal way, such that book-keeping computation is minimized. The tree is encoded such that children of a node are placed contiguously and in increasing order of memory heap so that the proposed algorithm also optimizes cache usage. Even though the algorithm is asymptotically two times faster that the conventional Viterbi algorithm, in our experiments

Discriminative training is effective in enhancing robustness for recognition tasks characterized by high confusion rates. We apply discriminative training to different components of a spelled word recognizer to improve the recognition accuracy among confusable letters. First we weighted the HMM states to emphasize the letters' discriminant part. The training achieved a 17% decrease in unit (letter) error rate when the search was performed with an unconstrained grammar. Then we designed a new algorithm that relies on discriminative training to adapt the grammar transition probabilities and the language weight. This method uses acoustic information to provide a tight coupling between the acoustic and language models. Experimental results showed the state weighting followed by the adaptation of a bigram language model reduced by 11% the total unit errors and by 12% the unit errors among the E-Set of the English alphabet.