Neural networks have been shown to be vulnerable against minor adversarial perturbations of their inputs, especially for high dimensional data under $\ell_\infty$ attacks.To combat this problem, techniques like adversarial training have been employed to obtain models that are robust on the training set.However, the robustness of such models against adversarial perturbations may not generalize to unseen data.To study how robustness generalizes, recent works assume that the inputs have bounded $\ell_2$-norm in order to bound the adversarial risk for $\ell_\infty$ attacks with no explicit dimension dependence.In this work, we focus on $\ell_\infty$ attacks with $\ell_\infty$ bounded inputs and prove margin-based bounds.Specifically, we use a compression-based approach that relies on efficiently compressing the set of tunable parameters without distorting the adversarial risk. To achieve this, we apply the concept of effective sparsity and effective joint sparsity on the weight matrices of neural networks.This leads to bounds with no explicit dependence on the input dimension, neither on the number of classes.Our results show that neural networks with approximately sparse weight matrices not only enjoy enhanced robustness but also better generalization. Finally, empirical simulations show that the notion of effective joint sparsity plays a significant role in generalizing robustness to $\ell_\infty$ attacks.

We consider the reconstruction of group-sparse vectors from sign measurements of random projections. In particular, we establish conditions on the number of measurements under which such signal ensembles can be recovered up to a prescribed accuracy. The results rely on a mixed restricted isometry property as first employed by Foucart in the context of 1-bit compressed sensing, as well as certain results on random hyperplane tessellations. The paper fills a gap in the literature by establishing that group-sparse signals can be estimated from 1-bit observations with the same number of measurements as required for the reconstruction of block-sparse signals from unquantized measurements. We confirm the correct behavior of the recovery schemes in a series of numerical experiments.

We address the problem of reconstructing group-sparse vectors from compressive measurements acquired via subgaussian block diagonal measurement operators. Such results can be obtained by establishing the so-called group restricted isometry property of the underlying measurement matrix. In particular, the problem is reduced to the task of bounding certain geometric objects associated with the suprema of a particular chaos process, which involves estimating Talagrand's γ2-functional via Dudley's metric entropy integral. As part of the proof, we generalize Maurey's empirical method to provide new bounds on the covering number of sets consisting of finite convex combinations of compact sets.

Optimization on manifolds is a class of methods for optimization of an objective function, subject to constraints which are smooth, in the sense that the set of points which satisfy the constraints admits the structure of a differentiable manifold. While many optimization problems are of the described form, technicalities of differential geometry and the laborious calculation of derivatives pose a significant barrier for experimenting with these methods. We introduce Pymanopt (available at pymanopt.github.io), a toolbox for optimization on manifolds, implemented in Python, that---similarly to the Manopt Matlab toolbox---implements several manifold geometries and optimization algorithms. Moreover, we lower the barriers to users further by using automated differentiation for calculating derivative information, saving users time and saving them from potential calculation and implementation errors.

In this paper, a novel noise-shaping method for Multi-Channel Linear Prediction (MCLP) is presented. Without special consideration, the quantization noise of the prediction error poses a serious problem in multi-channel prediction as each noise component distorts the reconstruction of every channel at the decoder.