We study the cost of overfitting in noisy kernel ridge regression (KRR), which we define as the ratio between the test error of the interpolating ridgeless model and the test error of the optimally-tuned model. We take an ``agnostic'' view in the following sense: we consider the cost as a function of sample size for any target function, even if the sample size is not large enough for consistency or the target is outside the RKHS. We analyze the cost of overfitting under a Gaussian universality ansatz using recently derived (non-rigorous) risk estimates in terms of the task eigenstructure. Our analysis provides a more refined characterization of benign, tempered and catastrophic overfitting (cf. Mallinar et al. 2022).

We establish generic uniform convergence guarantees for Gaussian data in terms of the Radamacher complexity of the hypothesis class and the Lipschitz constant of the square root of the scalar loss function. We show how these guarantees substantially generalize previous results based on smoothness (Lipschitz constant of the derivative), and allow us to handle the broader class of square-root-Lipschtz losses, which includes also non-smooth loss functions appropriate for studying phase retrieval and ReLU regression, as well as rederive and better understand “optimistic rate” and interpolation learning guarantees.

We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau envelopes of the loss $\ell$ . We use our finite-sample bound to directly recover the “optimistic rate” of Zhou et al. (2021) for linear regression with the square loss, which is known to be tight for minimal $\ell_2$-norm interpolation, but we also handle more general settings where the label is generated by a potentially misspecified multi-index model. The same argument can analyze noisy interpolation of max-margin classifiers through the squared hinge loss, and establishes consistency results in spiked-covariance settings. More generally, when the loss is only assumed to be Lipschitz, our bound effectively improves Talagrand’s well-known contraction lemma by a factor of two, and we prove uniform convergence of interpolators (Koehler et al. 2021) for all smooth, non-negative losses. Finally, we show that application of our generalization bound using localized Gaussian width will generally be sharp for empirical risk minimizers, establishing a non-asymptotic Moreau envelope theory for generalization that applies outside of proportional scaling regimes, handles model misspecification, and complements existing asymptotic Moreau envelope theories for M-estimation.

We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the class’s Gaussian width. Applying the generic bound to Euclidean norm balls recovers the consistency result of Bartlett et al. (2020) for minimum-norm interpolators, and confirms a prediction of Zhou et al. (2020) for near-minimal-norm interpolators in the special case of Gaussian data. We demonstrate the generality of the bound by applying it to the simplex, obtaining a novel consistency result for minimum $\ell_1$-norm interpolators (basis pursuit). Our results show how norm-based generalization bounds can explain and be used to analyze benign overfitting, at least in some settings.

We consider an underdetermined noisy linear regression model where the minimum-norm interpolating predictor is known to be consistent, and ask: can uniform convergence in a norm ball, or at least (following Nagarajan and Kolter) the subset of a norm ball that the algorithm selects on a typical input set, explain this success? We show that uniformly bounding the difference between empirical and population errors cannot show any learning in the norm ball, and cannot show consistency for any set, even one depending on the exact algorithm and distribution. But we argue we can explain the consistency of the minimal-norm interpolator with a slightly weaker, yet standard, notion: uniform convergence of zero-error predictors in a norm ball. We use this to bound the generalization error of low- (but not minimal-)norm interpolating predictors.

In modern petroleum extraction industry, Internet of Things (IoT) technology is widely used, and various of sensors constitute a typical cyber-physical systems (CPS). There are more requirements of interoperability and interaction between well site device systems. In such industrial critical domain, high-confidence is also important. This paper presents the requirements and running environment of developing high-confidence CPS. To satisfy the requirements of interoperability and data exchange among heterogeneous well site systems, a foundation framework with well defined interfaces and quality of service constraints was established. Based on this framework, designing high-confidence petroleum extraction CPS using Model-Driven Architecture approach is proposed, the long-lived models produced in the process that can be applied to any implementation technologies and platforms through model transformation which make the systems more portable. A case study shows the detail of the development process and using communication bridges to build relationships between models. At last some important problems which need to be paid more attention in model transformation process are discussed.

Image decomposition is an important issue in image processing. The existing approaches including the bi‐dimensional empirical mode decomposition (BEMD) still fail to separate monocomponents in multicomponents in many cases. To solve this problem, this study proposes a new image decomposition method based on the new derived combined bi‐dimensional Bedrosian's principle that has not been reported anywhere else for image processing. First, this study investigates a few bi‐dimensional Bedrosian's principles according to the bi‐dimensional Hilbert transforms. Second, based on the derived bi‐dimensional Bedrosian's principles and the original multicomponents, the authors provide the combined bi‐dimensional Bedrosian's principle and the assisted components obtained through projections via optimisation so that these monocomponents in multicomponents can be separated in the case that the existing methods fail. Third, an iterative image decomposition method is proposed via the above principles to decompose the multicomponent image into true monocomponents. The proposed method can solve the problems caused by the cross‐angle and amplitude ratio and frequency ratio between these components that BEMD fails to solve. Also, the phase and amplitude are estimated for texture analysis after the decomposition is demonstrated. Experiments are shown to support the proposed methods.

In this study, the authors investigated some new inequalities on sparse representation for pairs of bases and frames, which would enrich the theory ensemble. First, for fixed pairs of bases, frames and the signal to be represented, we presented the bounds (which can be used in practice directly) of l 0 ‐norm of signal coefficients by the max/min cross‐inner‐products, as is of much importance to bases and frames selections in sparse representation. Also, the error bounds associated with the given min cross‐inner‐products are achieved. Moreover, the equivalence condition between l 0 solution and l 1 solution was achieved via relations of cross‐inner‐products, as can be applied directly for selection of optimal bases. Finally, the estimation of the parameters of cross‐inner‐products was shown.