We present a framework based on bilevel optimization for learning multilayer, deep data representations. On the one hand, the lower-level problem finds a representation by successively minimizing layer-wise objectives made of the sum of a prescribed regularizer as well as a fidelity term and some linear function both depending on the representation found at the previous layer. On the other hand, the upper-level problem optimizes over the linear functions to yield a linearly separable final representation. We show that, by choosing the fidelity term as the quadratic distance between two successive layer-wise representations, the bilevel problem reduces to the training of a feed-forward neural network. Instead, by elaborating on Bregman distances, we devise a novel neural network architecture additionally involving the inverse of the activation function reminiscent of the skip connection used in ResNets. Numerical experiments suggest that the proposed Bregman variant benefits from better learning properties and more robust prediction performance.

A common approach in multi-task learning is to encourage the tasks to share a low dimensional representation. This has led to the popular method of trace norm regularization, which has proved effective in many applications. In this paper, we extend this approach by allowing the tasks to partition into different groups, within which trace norm regularization is separately applied. We propose a continuous bilevel optimization framework to simultaneously identify groups of related tasks and learn a low dimensional representation within each group. Hinging on recent results on the derivative of generalized matrix functions, we devise a smooth approximation of the upper-level objective via a dual forward-backward algorithm with Bregman distances. This allows us to solve the bilevel problem by a gradient-based scheme. Numerical experiments on synthetic and benchmark datasets support the effectiveness of the proposed method.

Regression with group-sparsity penalty plays a central role in high-dimensional prediction problems. Most of existing methods require the group structure to be known a priori. In practice, this may be a too strong assumption, potentially hampering the effectiveness of the regularization method. To circumvent this issue, we present a method to estimate the group structure by means of a continuous bilevel optimization problem where the data is split into training and validation sets. Our approach relies on an approximation scheme where the lower level problem is replaced by a smooth dual forward-backward algorithm with Bregman distances. We provide guarantees regarding the convergence of the approximate procedure to the exact problem and demonstrate the well behaviour of the proposed method on synthetic experiments. Finally, a preliminary application to genes expression data is tackled with the purpose of unveiling functional groups.

This contribution focuses, within the ℓ1-Potts model, on the automated estimation of the regularization parameter balancing the ℓ1data fidelity term and the TVℓ0penalization. Variational approaches based on total variation gained considerable interest to solve piecewise constant denoising problems thanks to their deterministic setting and low computational cost. However, the quality of the achieved solution strongly depends on the tuning of the regularization parameter. While recent works have tailored various hierarchical Bayesian procedures to additionally estimate the regularization parameter for Gaussian noise, less attention has been granted to Laplacian noise, of interested in numerous applications. This contribution promotes a fast and parameter-free denoising procedure for piecewise constant signals corrupted by Laplacian noise, that includes automated selection of the regularization parameter. It relies on the minimization of a Bayesian-driven criterion whose similarities with the ℓ1-Potts model permit to derive a computationally efficient algorithm.

Fetal heart rate (FHR) monitoring is routinely used in clinical practice to help obstetricians assess fetal health status during delivery. However, early detection of fetal acidosis that allows relevant decisions for operative delivery remains a challenging task, receiving considerable attention. This contribution promotes sparse support vector machine classification that permits to select a small number of relevant features and to achieve efficient fetal acidosis detection. A comprehensive set of features is used for FHR description, including enhanced and computerized clinical features, frequency domain, and scaling and multifractal features, all computed on a large (1288 subjects) and well-documented database. The individual performance obtained for each feature independently is discussed first. Then, it is shown that the automatic selection of a sparse subset of features achieves satisfactory classification performance (sensitivity 0.73 and specificity 0.75, outperforming clinical practice). The subset of selected features (average depth of decelerations MADdtrd, baseline level β0, and variability H) receives simple interpretation in clinical practice. Intrapartum fetal acidosis detection is improved in several respects: A comprehensive set of features combining clinical, spectral, and scale-free dynamics is used; an original multivariate classification targeting both sparse feature selection and high performance is devised; state-of-the-art performance is obtained on a much larger database than that generally studied with description of common pitfalls in supervised classification performance assessments.

Internet traffic monitoring is a crucial task for network security. Self-similarity, a key property for a relevant description of internet traffic statistics, has already been massively and successfully involved in anomaly detection. Self-similar analysis was however so far applied either to byte or Packet count time series independently, while both signals are jointly collected and technically deeply related. The present contribution elaborates on a recently proposed multivariate self-similar model, Operator fractional Brownian Motion (OfBm), to analyze jointly self-similarity in bytes and packets. A non-linear regression procedure, based on an original Branch & Bound resolution procedure, is devised for the full identification of bivariate OfBm. The estimation performance is assessed by means of Monte Carlo simulations. Further, an Internet traffic anomaly detection procedure is proposed, that makes use of the vector of Hurst exponents underlying the OfBm based Internet data modeling. Applied to a large set of high quality and modern Internet data from the MAWI repository, proof-of-concept results in anomaly detection are detailed and discussed.

This work aims to segment a texture into different regions, each characterized by a priori unknown multifractal properties. The multifractal properties are quantified using the multiscale function C1, jthat quantifies the evolution along analysis scales 2jof the empirical mean of the log of the wavelet leaders. The segmentation procedure is applied to local estimate of C1, j. It involves a multivariate Mumford-Shah relaxation formulated as a convex optimization problem involving a structure tensor penalization and an efficient algorithmic solution based on primal-dual proximal algorithm. The performances are evaluated on synthetic textures.