This paper addresses weak features learning (WFL), focusing on learning scenarios characterized by low-quality input features (weak features; WFs) that arise due to missingness, measurement errors, or ambiguous observations. We present a theoretical formalization and error analysis of WFL for continuous WFs (continuous WFL), which has been insufficiently explored in existing literature. A previous study established formalization and error analysis for WFL with discrete WFs (discrete WFL); however, this analysis does not extend to continuous WFs due to the inherent constraints of discreteness. To address this, we propose a theoretical framework specifically designed for continuous WFL, systematically capturing the interactions between feature estimation models for WFs and label prediction models for downstream tasks. Furthermore, we derive the theoretical conditions necessary for both sequential and iterative learning methods to achieve consistency. By integrating the findings of this study on continuous WFL with the existing theory of discrete WFL, we demonstrate that the WFL framework is universally applicable, providing a robust theoretical foundation for learning with low-quality features across diverse application domains.

In many real-world applications, predictive tasks inevitably involve low-quality input features (Weak Features; WFs) which arise due to factors such as misobservations, missingness, or partial observations. While several methods have been proposed to estimate the true values of specific types of WFs and to solve a downstream task, a unified theoretical framework that comprehensively addresses these methods remains underdeveloped. In this paper, we propose a unified framework called Weak Features Learning (WFL), which accommodates arbitrary discrete WFs and a broad range of learning algorithms, and we demonstrate its validity. Furthermore, we introduce a class of algorithms that learn both the estimation model for WFs and the predictive model for a downstream task and perform a generalization error analysis under finite-sample conditions. Our results elucidate the interdependencies between the estimation errors of WFs and the prediction error of a downstream task, as well as the theoretical conditions necessary for the learning approach to achieve consistency. This work establishes a unified theoretical foundation, providing generalization error analysis and performance guarantees, even in scenarios where WFs manifest in diverse forms.

Mixup is a data augmentation technique that gener-ates new samples using a convex combination of samples. Despite its remarkable simplicity, this method significantly improves the generalization performance of Deep Neural Networks (DNNs). Various studies have been conducted to understand Mixup, and it has been theoretically shown that Mixup involves Jacobian regularization. Furthermore, a relationship between Mixup and label smoothing has been suggested. However, these studies are limited to cases where the sample generated by Mixup is close to the original sample, and other cases have not yet been analyzed. In this study, we theoretically proved that applying Mixup to logistic regression results in Hessian regularization when the sample generated by Mixup is far from the original sample, a previously unexplored scenario. Additionally, through numerical experiments, we confirmed that a similar tendency holds true for DNNs as well. Our results provide a novel interpretation that Mixup is a method for collectively approximating several regularization methods, including Jacobian regularization, label smoothing, and Hessian regularization.