Contrastive representation learning has proven to be an effective self-supervised learning method. Most successful approaches are based on Noise Contrastive Estimation (NCE) and use different views of an instance as positives that should be contrasted with other instances, called negatives, that are considered as noise. However, several instances in a dataset are drawn from the same distribution and share underlying semantic information. A good data representation should contain relations, or semantic similarity, between the instances. Contrastive learning implicitly learns relations but considering all negatives as noise harms the quality of the learned relations. To circumvent this issue, we propose a novel formulation of contrastive learning using semantic similarity between instances called Similarity Contrastive Estimation (SCE). Our training objective is a soft contrastive learning one. Instead of hard classifying positives and negatives, we estimate from one view of a batch a continuous distribution to push or pull instances based on their semantic similarities. This target similarity distribution is sharpened to eliminate noisy relations. The model predicts for each instance, from another view, the target distribution while contrasting its positive with negatives. Experimental results show that SCE is Top-1 on the ImageNet linear evaluation protocol at 100 pretraining epochs with 72.1% accuracy and is competitive with state-of-the-art algorithms by reaching 75.4% for 200 epochs with multi-crop. We also show that SCE is able to generalize to several tasks. Source code is available here: https://github.com/CEA-LIST/SCE.

Road scene analysis is a fundamental task for both autonomous vehicles and ADAS systems. Nowadays, one can find autonomous vehicles that are able to properly detect objects in the scene in good weather conditions; however, some improvements still need to be done when the visibility is altered. People claim that using some non-conventional sensors such as, infra-red or Lidar, combined with classical vision, enhances road scene analysis in optimal weather conditions. In this work, we present the improvements achieved using polarimetric imaging in the complex situation of some adverse weather conditions. This rich modality is known for its ability to describe an object not only by its intensity information, even under poor illumination or strong reflection. The experimental results have shown that, using a new multimodal dataset, polarimetric imaging was able to provide generic features for both good weather conditions and adverse weather conditions, especially fog. By combining polarimetric images with an adapted learning model, the different detection tasks under fog were improved by about 15% to 44%.

Road scene analysis is a fundamental task for both autonomous vehicles and ADAS systems. Nowadays, most of autonomous vehicles are able to properly detect objects in good weather conditions; however, some improvements still need to be done when the visibility is altered. Polarimetric imaging is a rich modality that enables to describe an object by its physical information and has recently shown great performances in enhancing road scenes analysis under adverse weather conditions, especially under fog. Besides, it has been shown in a previous work that this modality could be complementary to classical color images especially for car detection. In this work, four different multimodal fusion schemes as well as different color and polarimetric features combinations are explored to achieve a robust scene analysis. The combination of both modalities could be a great asset to describe road scenes when the visibility is altered. Experimental results have shown that, using a well chosen fusion scheme with an adapted features combination, the detection of objects in road scenes under fog was reinforced. The different detection tasks show a significant improvement when using the adapted fusion scheme and features combination. Thanks to the architecture of the fusion scheme and to the properties of the selected features, these results could be extended to other adverse weather conditions.

Finding the exact robust test error is a good way to compare the robustness of neural networks, but it is a difficult task even on small networks and datasets like MNIST. However, finding reasonable lower and upper bounds is possible and can be done using either complete methods or attacks. On the one hand, complete methods such as Mixed Integer Program (MIP) give exact robust test accuracy but are time-consuming. On the other hand, attacks are usually fast but tend to perform badly against robust networks and underestimate the lower bound on the robust test error. The purpose of this paper is to present a novel attack method that is both fast and gives better lower bounds than previous attacks. This method exploits the algebraic properties of networks with piecewise linear activation functions to partition the input space in such a way that for each subset of that partition, finding the local optimal adversarial example is done by solving a linear program. Moving from one subset to another is done using classical gradient-based attack tools. To evaluate the quality of the produced adversarial examples, we compare our lower bound on the robust test error to the one previously found. The results found are satisfying in the sense that it does better than previous lower bounds on several models and finds adversarial examples that the MIP failed to expose before reaching its time limit.

Magnetic Resonance Imaging (MRI) is a widely used imaging technique to assess brain tumor. Accurately segmenting brain tumor from MR images is the key to clinical diagnostics and treatment planning. In addition, multi-modal MR images can provide complementary information for accurate brain tumor segmentation. However, it's common to miss some imaging modalities in clinical practice. In this paper, we present a novel brain tumor segmentation algorithm with missing modalities. Since it exists a strong correlation between multi-modalities, a correlation model is proposed to specially represent the latent multi-source correlation. Thanks to the obtained correlation representation, the segmentation becomes more robust in the case of missing modality. First, the individual representation produced by each encoder is used to estimate the modality independent parameter. Then, the correlation model transforms all the individual representations to the latent multi-source correlation representations. Finally, the correlation representations across modalities are fused via attention mechanism into a shared representation to emphasize the most important features for segmentation. We evaluate our model on BraTS 2018 and BraTS 2019 dataset, it outperforms the current state-of-the-art methods and produces robust results when one or more modalities are missing.

Person Re-Identification (re-ID) aims at retrieving images of the same person taken by different cameras. A challenge for re-ID is the performance preservation when a model is used on data of interest (target data) which belong to a different domain from the training data domain (source data). Unsupervised Domain Adaptation (UDA) is an interesting research direction for this challenge as it avoids a costly annotation of the target data. Pseudo-labeling methods achieve the best results in UDA-based re-ID. They incrementally learn with identity pseudo-labels which are initialized by clustering features in the source reID encoder space. Surprisingly, labeled source data are discarded after this initialization step. However, we believe that pseudo-labeling could further leverage the labeled source data in order to improve the post-initialization training steps. In order to improve robustness against erroneous pseudo-labels, we advocate the exploitation of both labeled source data and pseudo-labeled target data during all training iterations. To support our guideline, we introduce a framework which relies on a two-branch architecture optimizing classification and triplet loss based metric learning in source and target domains, respectively, in order to allow adaptability to the target domain while ensuring robustness to noisy pseudo-labels. Indeed, shared low and mid-level parameters benefit from the source classification and triplet loss signal while high-level parameters of the target branch learn domain-specific features. Our method is simple enough to be easily combined with existing pseudo-labeling UDA approaches. We show experimentally that it is efficient and improves performance when the base method has no mechanism to deal with pseudo-label noise. Our approach reaches state-of-the-art performance when evaluated on commonly used datasets, Market-1501 and DukeMTMC-reID, and outperforms the state of the art when targeting the bigger and more challenging dataset MSMT.

In the field of multimodal segmentation, the correlation between different modalities can be considered for improving the segmentation results. In this paper, we propose a multimodality segmentation network with a correlation constraint. Our network includes N model-independent encoding paths with N image sources, a correlation constrain block, a feature fusion block, and a decoding path. The model independent encoding path can capture modality-specific features from the N modalities. Since there exists a strong correlation between different modalities, we first propose a linear correlation block to learn the correlation between modalities, then a loss function is used to guide the network to learn the correlated features based on the linear correlation block. This block forces the network to learn the latent correlated features which are more relevant for segmentation. Considering that not all the features extracted from the encoders are useful for segmentation, we propose to use dual attention based fusion block to recalibrate the features along the modality and spatial paths, which can suppress less informative features and emphasize the useful ones. The fused feature representation is finally projected by the decoder to obtain the segmentation result. Our experiment results tested on BraTS-2018 dataset for brain tumor segmentation demonstrate the effectiveness of our proposed method.

In this paper, we propose a self-supervised method for video representation learning based on Contrastive Predictive Coding (CPC) [27]. Previously, CPC has been used to learn representations for different signals (audio, text or image). It benefits from the use of an autoregressive modeling and contrastive estimation to learn long-term relations inside raw signal while remaining robust to local noise. Our self-supervised task consists in predicting the latent representation of future segments of the video. As opposed to generative models, predicting directly in the feature space is easier and avoid incertitude problems for long-term predictions. Today, using CPC to learn representations for videos remains challenging due to the structure and the high dimensionality of the signal. We demonstrate experimentally that the representations learned by the network are useful for action recognition. We test it with different input types such as optical flows, image differences and raw images on different datasets (UCF-101 and HMDB51). It gives consistent results across the modalities. At last, we notice the utility of our pre-training method by achieving competitive results for action recognition using few labeled data.

We propose a fully automatic method for learning gestures on big touch devices in a potentially multi-user context. The goal is to learn general models capable of adapting to different gestures, user styles and hardware variations (e.g. device sizes, sampling frequencies and regularities). Based on deep neural networks, our method features a novel dynamic sampling and temporal normalization component, transforming variable length gestures into fixed length representations while preserving finger/surface contact transitions, that is, the topology of the signal. This sequential representation is then processed with a convolutional model capable, unlike recurrent networks, of learning hierarchical representations with different levels of abstraction. To demonstrate the interest of the proposed method, we introduce a new touch gestures dataset with 6591 gestures performed by 27 people, which is, up to our knowledge, the first of its kind: a publicly available multi-touch gesture dataset for interaction. We also tested our method on a standard dataset of symbolic touch gesture recognition, the MMG dataset, outperforming the state of the art and reporting close to perfect performance.

We present a new kernel on fuzzy sets: the Cross Product kernel on fuzzy sets. This kernel implements a similarity measure between fuzzy sets with a geometrical interpretation in Reproducing Kernel Hilbert Spaces. We prove that this kernel is a convolution kernel that generalizes the widely know kernel on sets towards the space of fuzzy sets. Moreover, we show that the Cross Product kernel on fuzzy sets performs an embedding of probability measures into a Reproduction Kernel Hilbert space. Finally, we validated the kernel performance through experiments on supervised classification on noisy datasets.

Machine learning is a very promising way of solving some image processing tasks. However, existing approaches fails at integrating feature selection within the learning task. This paper introduces a new two stage learning algorithm called near infinitely linear combination (NILC) that performs at the same time variable selection and error minimization. Empirical evidence reported on different document processing tasks shows that our approach significantly outperforms existing approaches.

In this paper (This is a combined and expanded version of previous conference papers Kadri et al., 2010, 2011c) we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of functions, and the label is also a function. We focus on the use of reproducing kernel Hilbert space theory to learn from such functional data. Basic concepts and properties of kernel-based learning are extended to include the estimation of function-valued functions. In this setting, the representer theorem is restated, a set of rigorously defined infinite-dimensional operator-valued kernels that can be valuably applied when the data are functions is described, and a learning algorithm for nonlinear functional data analysis is introduced. The methodology is illustrated through speech and audio signal processing experiments.

This paper presents a theoretical foundation for an SVM solver in Kreĭn spaces. Up to now, all methods are based either on the matrix correction, or on non-convex minimization, or on feature-space embedding. Here we justify and evaluate a solution that uses the original (indefinite) similarity measure, in the original Kreĭn space. This solution is the result of a stabilization procedure. We establish the correspondence between the stabilization problem (which has to be solved) and a classical SVM based on minimization (which is easy to solve). We provide simple equations to go from one to the other (in both directions). This link between stabilization and minimization problems is the key to obtain a solution in the original Kreĭn space. Using KSVM, one can solve SVM with usually troublesome kernels (large negative eigenvalues or large numbers of negative eigenvalues). We show experiments showing that our algorithm KSVM outperforms all previously proposed approaches to deal with indefinite matrices in SVM-like kernel methods.

Embedding non-vectorial data into a vector space is very common in machine learning, aiming to perform tasks such as classification, regression or clustering. Fuzzy datasets or datasets whose observations are fuzzy sets, are examples of non-vectorial data and, several fuzzy pattern recognition algorithms analyze them in the space formed by the set of fuzzy sets. However, the analysis of fuzzy data in such space has the limitation of not being a vector space. To overcome such limitation, we propose the embedding of fuzzy data into a proper Hilbert space of functions called the Reproducing Kernel Hilbert Space (RKHS). This embedding is possible by using a positive definite kernel function on fuzzy sets. We present a formulation of a real-valued kernels on fuzzy sets, in particular, we define the intersection kernel and the cross product kernel on fuzzy sets giving some examples of them using T-norm operators. Also, we analyze the nonsingleton TSK fuzzy kernel and, finally, we give some examples of kernels on fuzzy sets that can be easily constructed from the previous ones.

Building an accurate training database is challenging in supervised classification. For instance, in medical imaging, radiologists often delineate malignant and benign tissues without access to the histological ground truth, leading to uncertain data sets. This paper addresses the pattern classification problem arising when available target data include some uncertainty information. Target data considered here are both qualitative (a class label) or quantitative (an estimation of the posterior probability). In this context, usual discriminative methods, such as the support vector machine (SVM), fail either to learn a robust classifier or to predict accurate probability estimates. We generalize the regular SVM by introducing a new formulation of the learning problem to take into account class labels as well as class probability estimates. This original reformulation into a probabilistic SVM (P-SVM) can be efficiently solved by adapting existing flexible SVM solvers. Furthermore, this framework allows deriving a unique learned prediction function for both decision and posterior probability estimation providing qualitative and quantitative predictions. The method is first tested on synthetic data sets to evaluate its properties as compared with the classical SVM and fuzzy-SVM. It is then evaluated on a clinical data set of multiparametric prostate magnetic resonance images to assess its performances in discriminating benign from malignant tissues. P-SVM is shown to outperform classical SVM as well as the fuzzy-SVM in terms of probability predictions and classification performances, and demonstrates its potential for the design of an efficient computer-aided decision system for prostate cancer diagnosis based on multiparametric magnetic resonance (MR) imaging.

Feature selection in learning to rank has recently emerged as a crucial issue. Whereas several preprocessing approaches have been proposed, only a few have focused on integrating feature selection into the learning process. In this paper, we propose a general framework for feature selection in learning to rank using support vector machines with a sparse regularization term. We investigate both classical convex regularizations, such as ℓ1or weighted ℓ1, and nonconvex regularization terms, such as log penalty, minimax concave penalty, or ℓppseudo-norm with p1scheme to address nonconvex regularizations. We conduct intensive experiments on nine datasets from Letor 3.0 and Letor 4.0 corpora. Numerical results show that the use of nonconvex regularizations we propose leads to more sparsity in the resulting models while preserving the prediction performance. The number of features is decreased by up to a factor of 6 compared to the ℓ1regularization. In addition, the software is publicly available on the web.

The objective of the paper is to investigate the effect of subject's emotional states on Brain Computer Interface (BCI) performance. Two psycho-physiological experiments are designed and implemented. The first one induces subjects' emotion using video clips first, then involves subjects' in SSVEP task. The second one induces subjects' emotions and SSVEP simultaneously by flickering IAPS pictures in four directions. used to recognize the performed BCI tasks. Based on the performances of learned classifiers, we analyzed the influence of emotion using two statistical tests. The McNamara's test serves to assess if emotion has any influences on mental task performing while Wilcox on signed-rank test analyses if emotion has a positive or detrimental effect on ability to achieve a SSVEP task. Obtained results suggest influence of emotional states: the positive and neutral emotions influence BCI performance similarly, while the negative emotion tends to deteriorate classification accuracy.

Algorithms for supervised classification problems usually do not consider imprecise data, e.g., interval collections, histograms, list of values, fuzzy sets among others that represent observed data. Moreover, fuzzy set theory is a natural choice to model imprecision and kernel methods are the state of the art in learning machines. Previous works describe a link between both areas: the interaction between fuzzy rules of Takagi-Sugeno-Kang (TSK) fuzzy systems and singleton fuzzy inputs are equivalent to positive definite kernels (PDK). Current research in fuzzy systems shows that nonsingleton fuzzy sets can be used to model imprecise data. In this work, we study the relationship between positive definite kernels and TSK fuzzy systems with nonsingleton inputs. As a result, we define an extension of TSK fuzzy systems to deal with nonsingleton fuzzy input and we show that the interaction between fuzzy rules and nonsingleton fuzzy inputs induces a new class of PDK, the nonsingleton TSK kernel class, which are close related, but not equal, to Vapnik's vicinal kernels. Finally, based on nonsingleton TSK kernels and distance substitution kernels we give a general procedure to formulate PDKs for interval data. Experiments conducted with interval datasets show better performance than the state of the art approaches.

Recently, there has been much interest around multitask learning (MTL) problem with the constraints that tasks should share a common sparsity profile. Such a problem can be addressed through a regularization framework where the regularizer induces a joint-sparsity pattern between task decision functions. We follow this principled framework and focus on l(p)-l(q) (with 0 ≤ p ≤ 1 and 1 ≤ q ≤ 2) mixed norms as sparsity-inducing penalties. Our motivation for addressing such a larger class of penalty is to adapt the penalty to a problem at hand leading thus to better performances and better sparsity pattern. For solving the problem in the general multiple kernel case, we first derive a variational formulation of the l(1)-l(q) penalty which helps us in proposing an alternate optimization algorithm. Although very simple, the latter algorithm provably converges to the global minimum of the l(1)-l(q) penalized problem. For the linear case, we extend existing works considering accelerated proximal gradient to this penalty. Our contribution in this context is to provide an efficient scheme for computing the l(1)-l(q) proximal operator. Then, for the more general case, when , we solve the resulting nonconvex problem through a majorization-minimization approach. The resulting algorithm is an iterative scheme which, at each iteration, solves a weighted l(1)-l(q) sparse MTL problem. Empirical evidences from toy dataset and real-word datasets dealing with brain-computer interface single-trial electroencephalogram classification and protein subcellular localization show the benefit of the proposed approaches and algorithms.

We consider the problem of binary classification where the classifier may abstain instead of classifying each observation. The Bayes decision rule for this setup, known as Chow's rule, is defined by two thresholds on posterior probabilities. From simple desiderata, namely the consistency and the sparsity of the classifier, we derive the double hinge loss function that focuses on estimating conditional probabilities only in the vicinity of the threshold points of the optimal decision rule. We show that, for suitable kernel machines, our approach is universally consistent. We cast the problem of minimizing the double hinge loss as a quadratic program akin to the standard SVM optimization problem and propose an active set method to solve it efficiently. We finally provide preliminary experimental results illustrating the interest of our constructive approach to devising loss functions.

An efficient and general multiple kernel learning (MKL) algorithm has been recently proposed by Sonnenburg et al. (2006). This approach has opened new perspectives since it makes the MKL approach tractable for large-scale problems, by iteratively using existing support vector machine code. However, it turns out that this iterative algorithm needs several iterations before converging towards a reasonable solution. In this paper, we address the MKL problem through an adaptive 2-norm regularization formulation. Weights on each kernel matrix are included in the standard SVM empirical risk minimization problem with a l1 constraint to encourage sparsity. We propose an algorithm for solving this problem and provide an new insight on MKL algorithms based on block 1-norm regularization by showing that the two approaches are equivalent. Experimental results show that the resulting algorithm converges rapidly and its efficiency compares favorably to other MKL algorithms.

Using unlabeled data to unravel the structure of the data to leverage the learning process is the goal of semi supervised learning. A common way to represent this underlying structure is to use graphs. Flexibility of the maximum margin kernel framework allows to model graph smoothness and to build kernel machine for semi supervised learning such as Laplacian SVM [1]. But a common complaint of the practitioner is the long running time of these kernel algorithms for classification of new points. We provide an efficient way of alleviating this problem by using a L1 penalization term and a regularization path algorithm to efficiently compute the solution. Empirical evidence shows the benefit of the algorithm.

In a recently published paper in JMLR, Tsang et al. (2005) present an algorithm for SVM called Core Vector Machines (CVM) and illustrate its performances through comparisons with other SVM solvers. After reading the CVM paper we were surprised by some of the reported results. In order to clarify the matter, we decided to reproduce some of the experiments. It turns out that to some extent, our results contradict those reported. Reasons of these different behaviors are given through the analysis of the stopping criterion.

Minimum measure sets (MMSs) summarize the information of a (single-class) dataset. In many situations, they can be preferred to estimated probability density functions (pdfs): they are strongly related to pdf level sets while being much easier to estimate in large dimensions. The main contribution of this paper is a theoretical connection between MMSs and one class support vector machines. This justifies the use of one-class SVMs in the following applications: novelty detection (we give explicit convergence rate) and change detection

This paper presents an algorithm to estimate simultaneously both mean and variance of a non parametric regression problem. The key point is that we are able to estimate variance locally unlike standard Gaussian Process regression or SVMs. This means that our estimator adapts to the local noise. The problem is cast in the setting of maximum a posteriori estimation in exponential families. Unlike previous work, we obtain a convex optimization problem which can be solved via Newton's method.

Ontologies are widely used for organising and sharing knowledge. But elaborating these resources is a heavy and time-consuming task. This paper is two-fold: it describes EADS DCS text-mining platform, in particular, its service to annotate documents with semantic tags and it presents its extension for incremental learning of ontologies. Domain experts are assisted in the ontology population task by recent machine learning techniques (i.e. conditional random fields). Comparisons are made between annotations from the ontology and from a trained CRF model, so as to detect candidate instances. An iterative process controlled by the experts results in knowledge discovery and constitution of an accurate ontology.

This work deals with a method for building a reproducing kernel Hilbert space (RKHS) from a Hilbert space with frame elements having special properties. Conditions on existence and a method of construction are given. Then, these RKHS are used within the framework of regularization theory for function approximation. Implications on semiparametric estimation are discussed and a multiscale scheme of regularization is also proposed. Results on toy and real-world approximation problems illustrate the effectiveness of such methods.

In this paper we show that many kernel methods can be adapted to deal with indefinite kernels, that is, kernels which are not positive semidefinite. They do not satisfy Mercer's condition and they induce associated functional spaces called Reproducing Kernel Kreĭn Spaces (RKKS), a generalization of Reproducing Kernel Hilbert Spaces (RKHS).Machine learning in RKKS shares many "nice" properties of learning in RKHS, such as orthogonality and projection. However, since the kernels are indefinite, we can no longer minimize the loss, instead we stabilize it. We show a general representer theorem for constrained stabilization and prove generalization bounds by computing the Rademacher averages of the kernel class. We list several examples of indefinite kernels and investigate regularization methods to solve spline interpolation. Some preliminary experiments with indefinite kernels for spline smoothing are reported for truncated spectral factorization, Landweber-Fridman iterations, and MR-II.

This paper introduces an algorithm for the automatic relevance determi- nation of input variables in kernelized Support Vector Machines. Rele- vance is measured by scale factors defining the input space metric, and feature selection is performed by assigning zero weights to irrelevant variables. The metric is automatically tuned by the minimization of the standard SVM empirical risk, where scale factors are added to the usual set of parameters defining the classifier. Feature selection is achieved by constraints encouraging the sparsity of scale factors. The resulting algorithm compares favorably to state-of-the-art feature selection proce- dures and demonstrates its effectiveness on a demanding facial expres- sion recognition problem.

Noise injection consists of adding noise to the inputs during neural network training. Experimental results suggest that it might improve the generalization ability of the resulting neural network. A justification of this improvement remains elusive: describing analytically the average perturbed cost function is difficult, and controlling the fluctuations of the random perturbed cost function is hard. Hence, recent papers suggest replacing the random perturbed cost by a (deterministic) Taylor approximation of the average perturbed cost function. This article takes a different stance: when the injected noise is gaussian, noise injection is naturally connected to the action of the heat kernel. This provides indications on the relevance domain of traditional Taylor expansions and shows the dependence of the quality of Taylor approximations on global smoothness properties of neural networks under consideration. The connection between noise injection and heat kernel also enables controlling the fluctuations of the random perturbed cost function. Under the global smoothness assumption, tools from gaussian analysis provide bounds on the tail behavior of the perturbed cost. This finally suggests that mixing input perturbation with smoothness-based penalization might be profitable.

The generalization capacity of neural networks learning from examples is important. Several authors showed experimentally that training a neural network with noise injected inputs could improve its generalization abilities. In the original paper (ibid., vol. 22, no. 3. p. 436-40, 1992), Matsuoka explained this fact in a formal way, claiming that using noise injected inputs is equivalent to reduce the sensitivity of the network. However, the author states that an error in Matsuoka's calculations lead him to inadequate conclusions. This paper corrects these calculations and conclusions.>