Even though neural networks have become standard tools in many areas, many important statistical questions remain open. This paper studies the question of how much data are needed to train a ReLU feed-forward neural network. Our theoretical and empirical results suggest that the generalization error of ReLU feed-forward neural networks scales at the rate $1/\sqrt{n}$ in the sample size $n$-rather than the "parametric rate" $1/n$, which might be suggested by traditional statistical theories. Thus, broadly speaking, our results underpin the common belief that neural networks need "many" training samples. Along the way, we also establish new technical insights, such as the first lower bounds of the entropy of ReLU feed-forward networks.

Modern technologies are generating ever-increasing amounts of data. Making use of these data requires methods that are both statistically sound and computationally efficient. Typically, the statistical and computational aspects are treated separately. In this paper, we propose an approach to entangle these two aspects in the context of regularized estimation. Applying our approach to sparse and group-sparse regression, we show that it can improve on standard pipelines both statistically and computationally.

In this paper, a new unsupervised metric learning algorithm with real-world application in clustering is proposed. To have a desirable clustering, the separability among different classes of data needs to be improved. A common manner in accomplishing this objective is to utilize the advantages of metric learning in clustering and vice versa. Clustering provides an estimation of class labels and metric learning maximizes the separability among these different estimated classes of data. This procedure is performed in an iterative fashion, alternating between clustering and metric learning. Here, a new method is proposed, called discriminative fuzzy c-means (Dis-FCM), in which FCM and metric learning are integrated into one joint formulation. Unlike traditional approaches, which simply alternate between clustering and metric learning, Dis-FCM applies both simultaneously. Here, FCM provides an estimation of class labels. This can avoid the problem of fast convergence, which is common in previous algorithm. Moreover, Dis-FCM is able to handle not only numerical data, but also categorical data, which are not found in traditional methods. The experimental results indicate its superiority over other state-of-the-art algorithms in terms of extrinsic and intrinsic clustering measures.