Both long-tailed and noisily labeled data frequently appear in real-world applications and impose significant challenges for learning. Most prior works treat either problem in an isolated way and do not explicitly consider the coupling effects of the two. Our empirical observation reveals that such solutions fail to consistently improve the learning when the dataset is long-tailed with label noise. Moreover, with the presence of label noise, existing methods do not observe universal improvements across different sub-populations; in other words, some sub-populations enjoyed the benefits of improved accuracy at the cost of hurting others. Based on these observations, we introduce the Fairness Regularizer (FR), inspired by regularizing the performance gap between any two sub-populations. We show that the introduced fairness regularizer improves the performances of sub-populations on the tail and the overall learning performance. Extensive experiments demonstrate the effectiveness of the proposed solution when complemented with certain existing popular robust or class-balanced methods.

Instruction tuning is critical for adapting large language models (LLMs) to downstream tasks, and recent studies have demonstrated that small amounts of human-curated data can outperform larger datasets, challenging traditional data scaling laws. While LLM-based data quality rating systems offer a cost-effective alternative to human annotation, they often suffer from inaccuracies and biases, even in powerful models like GPT-4. In this work, we introduce $DS^2$, a **D**iversity-aware **S**core curation method for **D**ata **S**election. By systematically modeling error patterns through a score transition matrix, $DS^2$ corrects LLM-based scores and promotes diversity in the selected data samples. Our approach shows that a curated subset (just 3.3\% of the original dataset) outperforms full-scale datasets (300k samples) across various machine-alignment benchmarks, and matches or surpasses human-aligned datasets such as LIMA with the same sample size (1k samples). These findings challenge conventional data scaling assumptions, highlighting that redundant, low-quality samples can degrade performance and reaffirming that ``more can be less''.

Recent studies show that in supervised fine-tuning (SFT) of large language models (LLMs), data quality matters more than quantity. While most data cleaning methods concentrate on filtering entire samples, the quality of individual tokens within a sample can vary significantly. After pre-training, even in high-quality samples, patterns or phrases that are not task-related can be redundant, uninformative, or even harmful. Continuing to fine-tune on these patterns may offer limited benefit and even degrade downstream task performance. In this paper, we investigate token quality from a noisy-label perspective and propose a generic token cleaning pipeline for SFT tasks. Our method filters out uninformative tokens while preserving those carrying key task-specific information. Specifically, we first evaluate token quality by examining the influence of model updates on each token, then apply a threshold-based separation. The token influence can be measured in a single pass with a fixed reference model or iteratively with self-evolving reference models. The benefits and limitations of both methods are analyzed theoretically by error upper bounds. Extensive experiments show that our framework consistently improves downstream performance. Code is available at https://github.com/UCSC-REAL/TokenCleaning.

Detecting anomalies in temporal data has gained significant attention across various real-world applications, aiming to identify unusual events and mitigate potential hazards. In practice, situations often involve a mix of segment-level labels (detected abnormal events with segments of time points) and unlabeled data (undetected events), while the ideal algorithmic outcome should be point-level predictions. Therefore, the huge label information gap between training data and targets makes the task challenging. In this study, we formulate the above imperfect information as noisy labels and propose NRdetector, a noise-resilient framework that incorporates confidence-based sample selection, robust segment-level learning, and data-centric point-level detection for multivariate time series anomaly detection. Particularly, to bridge the information gap between noisy segment-level labels and missing point-level labels, we develop a novel loss function that can effectively mitigate the label noise and consider the temporal features. It encourages the smoothness of consecutive points and the separability of points from segments with different labels. Extensive experiments on real-world multivariate time series datasets with 11 different evaluation metrics demonstrate that NRdetector consistently achieves robust results across multiple real-world datasets, outperforming various baselines adapted to operate in our setting.

The recent success of generative AI highlights the crucial role of high-quality human feedback in building trustworthy AI systems. However, the increasing use of large language models (LLMs) by crowdsourcing workers poses a significant challenge: datasets intended to reflect human input may be compromised by LLM-generated responses. Existing LLM detection approaches often rely on high-dimensional training data such as text, making them unsuitable for structured annotation tasks like multiple-choice labeling. In this work, we investigate the potential of peer prediction --- a mechanism that evaluates the information within workers' responses --- to mitigate LLM-assisted cheating in crowdsourcing with a focus on annotation tasks. Our method quantifies the correlations between worker answers while conditioning on (a subset of) LLM-generated labels available to the requester. Building on prior research, we propose a training-free scoring mechanism with theoretical guarantees under a novel model that accounts for LLM collusion. We establish conditions under which our method is effective and empirically demonstrate its robustness in detecting low-effort cheating on real-world crowdsourcing datasets.

Federated Learning (FL) heavily depends on label quality for its performance. However, the label distribution among individual clients is always both noisy and heterogeneous. The high loss incurred by client-specific samples in heterogeneous label noise poses challenges for distinguishing between client-specific and noisy label samples, impacting the effectiveness of existing label noise learning approaches. To tackle this issue, we propose FedFixer, where the personalized model is introduced to cooperate with the global model to effectively select clean client-specific samples. In the dual models, updating the personalized model solely at a local level can lead to overfitting on noisy data due to limited samples, consequently affecting both the local and global models’ performance. To mitigate overfitting, we address this concern from two perspectives. Firstly, we employ a confidence regularizer to alleviate the impact of unconfident predictions caused by label noise. Secondly, a distance regularizer is implemented to constrain the disparity between the personalized and global models. We validate the effectiveness of FedFixer through extensive experiments on benchmark datasets. The results demonstrate that FedFixer can perform well in filtering noisy label samples on different clients, especially in highly heterogeneous label noise scenarios.

Language models have shown promise in various tasks but can be affected by undesired data during training, fine-tuning, or alignment. For example, if some unsafe conversations are wrongly annotated as safe ones, the model fine-tuned on these samples may be harmful. Therefore, the correctness of annotations, i.e., the credibility of the dataset, is important. This study focuses on the credibility of real-world datasets, including the popular benchmarks Jigsaw Civil Comments, Anthropic Harmless & Red Team, PKU BeaverTails & SafeRLHF, that can be used for training a harmless language model. Given the cost and difficulty of cleaning these datasets by humans, we introduce a systematic framework for evaluating the credibility of datasets, identifying label errors, and evaluating the influence of noisy labels in the curated language data, specifically focusing on unsafe comments and conversation classification. With the framework, we find and fix an average of **6.16\%** label errors in **11** datasets constructed from the above benchmarks. The data credibility and downstream learning performance can be remarkably improved by directly fixing label errors, indicating the significance of cleaning existing real-world datasets. Code is available at [https://github.com/Docta-ai/docta](https://github.com/Docta-ai/docta).

Unsupervised timeseries anomaly detection (UTAD) aims to identify abnormal patterns within time series data and is of immense importance in extensive applications. Contrastive learning has been seen as an effective candidate for UTAD as it can learn invariants existing in two contrastive views. However, as most time series contain complex multi-periodic and non-periodic signals, the huge difference between sequences with different periodicity would make contrastive learning hard to learn representative temporal and/or spatial patterns that are essential for anomaly detection. To address this issue, we propose PACdetector, a periodicity association-based contrastive framework for UTAD. Specifically, we perceive time series as an aggregation of various periodic sequences, and for each point in the periodical sequences, we employ self-attention maps to calculate its association with points within the period (intraperiod association) and points at the same phase across different periods (interperiod association). We then perform contrastive learning between the two associations to preserve temporal consistency and obtain a distinguishable criterion between normal points and anomalies, which we refer to as Periodicity Association Discrepancy. Extensive experiments show that PACdetector outperforms various state-of-the-art algorithms, achieving the best performance across six benchmark datasets.

The pursuit of fairness in machine learning (ML), ensuring that the models do not exhibit biases toward protected demographic groups, typically results in a compromise scenario. This compromise can be explained by a Pareto frontier where given certain resources (e.g., data), reducing the fairness violations often comes at the cost of lowering the model accuracy. In this work, we aim to train models that mitigate group fairness disparity without causing harm to model accuracy. Intuitively, acquiring more data is a natural and promising approach to achieve this goal by reaching a better Pareto frontier of the fairness-accuracy tradeoff. The current data acquisition methods, such as fair active learning approaches, typically require annotating sensitive attributes. However, these sensitive attribute annotations should be protected due to privacy and safety concerns. In this paper, we propose a tractable active data sampling algorithm that does not rely on training group annotations, instead only requiring group annotations on a small validation set. Specifically, the algorithm first scores each new example by its influence on fairness and accuracy evaluated on the validation dataset, and then selects a certain number of examples for training. We theoretically analyze how acquiring more data can improve fairness without causing harm, and validate the possibility of our sampling approach in the context of risk disparity. We also provide the upper bound of generalization error and risk disparity as well as the corresponding connections. Extensive experiments on real-world data demonstrate the effectiveness of our proposed algorithm. Our code is available at [github.com/UCSC-REAL/FairnessWithoutHarm](https://github.com/UCSC-REAL/FairnessWithoutHarm).

Outlier detection identifies rare items, events, or observations in data analysis and has critical applications in many fields. In most such applications, datasets are high-dimensional. To reduce the impact of the “curse of dimension-ality”, many such applications decompose the entire feature space into different subspaces with two or more “relevant” features for deviations of interest. Those approaches often ignore the case for subspaces with a single feature. Due to the low dimension and high data density, it sometimes sufficient to identify univariate outliers. Thus, this paper proposes a multi-view outlier detection algorithm MVOD to ensemble outlier detection from three views: single feature view, local view, and global view. More specifically, we design a general outlier score function based on the quantities of information to evaluate the strength of the data distribution structure. Then, the outlier score for each point from different views is normalized and combined to reduce representational bias under different views. Extensive experiments are carried out on ten public benchmark datasets with ten state-of-art baselines. Experimental results show that MVOD is significantly better than those baselines in terms of AVC_ROC.

Evaluating fairness can be challenging in practice because the sensitive attributes of data are often inaccessible due to privacy constraints. The go-to approach that the industry frequently adopts is using off-the-shelf proxy models to predict the missing sensitive attributes, e.g. Meta (Alao et al., 2021) and Twitter (Belli et al., 2022). Despite its popularity, there are three important questions unanswered: (1) Is directly using proxies efficacious in measuring fairness? (2) If not, is it possible to accurately evaluate fairness using proxies only? (3) Given the ethical controversy over infer-ring user private information, is it possible to only use weak (i.e. inaccurate) proxies in order to protect privacy? Our theoretical analyses show that directly using proxy models can give a false sense of (un)fairness. Second, we develop an algorithm that is able to measure fairness (provably) accurately with only three properly identified proxies. Third, we show that our algorithm allows the use of only weak proxies (e.g. with only 68.85% accuracy on COMPAS), adding an extra layer of protection on user privacy. Experiments validate our theoretical analyses and show our algorithm can effectively measure and mitigate bias. Our results imply a set of practical guidelines for prac-titioners on how to use proxies properly. Code is available at https://github.com/UCSC-REAL/fair-eval.

The rawly collected training data often comes with separate noisy labels collected from multiple imperfect annotators (e.g., via crowdsourcing). A typical way of using these separate labels is to first aggregate them into one and apply standard training methods. The literature has also studied extensively on effective aggregation approaches. This paper revisits this choice and aims to provide an answer to the question of whether one should aggregate separate noisy labels into single ones or use them separately as given. We theoretically analyze the performance of both approaches under the empirical risk minimization framework for a number of popular loss functions, including the ones designed specifically for the problem of learning with noisy labels. Our theorems conclude that label separation is preferred over label aggregation when the noise rates are high, or the number of labelers/annotations is insufficient. Extensive empirical results validate our conclusions.

Label noise in real-world datasets encodes wrong correlation patterns and impairs the generalization of deep neural networks (DNNs). It is critical to find efficient ways to detect corrupted patterns. Current methods primarily focus on designing robust training techniques to prevent DNNs from memorizing corrupted patterns. These approaches often require customized training processes and may overfit corrupted patterns, leading to a performance drop in detection. In this paper, from a more data-centric perspective, we propose a training-free solution to detect corrupted labels. Intuitively, “closer” instances are more likely to share the same clean label. Based on the neighborhood information, we propose two methods: the first one uses “local voting" via checking the noisy label consensuses of nearby features. The second one is a ranking-based approach that scores each instance and filters out a guaranteed number of instances that are likely to be corrupted. We theoretically analyze how the quality of features affects the local voting and provide guidelines for tuning neighborhood size. We also prove the worst-case error bound for the ranking-based method. Experiments with both synthetic and real-world label noise demonstrate our training-free solutions consistently and significantly improve most of the training-based baselines. Code is available at github.com/UCSC-REAL/SimiFeat.

The label noise transition matrix, denoting the transition probabilities from clean labels to noisy labels, is crucial for designing statistically robust solutions. Existing estimators for noise transition matrices, e.g., using either anchor points or clusterability, focus on computer vision tasks that are relatively easier to obtain high-quality representations. We observe that tasks with lower-quality features fail to meet the anchor-point or clusterability condition, due to the coexistence of both uninformative and informative representations. To handle this issue, we propose a generic and practical information-theoretic approach to down-weight the less informative parts of the lower-quality features. This improvement is crucial to identifying and estimating the label noise transition matrix. The salient technical challenge is to compute the relevant information-theoretical metrics using only noisy labels instead of clean ones. We prove that the celebrated $f$-mutual information measure can often preserve the order when calculated using noisy labels. We then build our transition matrix estimator using this distilled version of features. The necessity and effectiveness of the proposed method are also demonstrated by evaluating the estimation error on a varied set of tabular data and text classification tasks with lower-quality features. Code is available at github.com/UCSC-REAL/BeyondImages.

The presence of label noise often misleads the training of deep neural networks. Departing from the recent literature which largely assumes the label noise rate is only determined by the true label class, the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks, resulting in settings with instance-dependent label noise. We first provide evidences that the heterogeneous instance-dependent label noise is effectively down-weighting the examples with higher noise rates in a non-uniform way and thus causes imbalances, rendering the strategy of directly applying methods for class-dependent label noise questionable. Built on a recent work peer loss [24], we then propose and study the potentials of a second-order approach that leverages the estimation of several covariance terms defined between the instance-dependent noise rates and the Bayes optimal label. We show that this set of second-order statistics successfully captures the induced imbalances. We further proceed to show that with the help of the estimated second-order statistics, we identify a new loss function whose expected risk of a classifier under instance-dependent label noise is equivalent to a new problem with only class-dependent label noise. This fact allows us to apply existing solutions to handle this better-studied setting. We provide an efficient procedure to estimate these second-order statistics without accessing either ground truth labels or prior knowledge of the noise rates. Experiments on CIFAR10 and CIFAR100 with synthetic instance-dependent label noise and Clothing1M with real-world human label noise verify our approach. Our implementation is available at https://github.com/UCSC-REAL/CAL.

The label noise transition matrix, characterizing the probabilities of a training instance being wrongly annotated, is crucial to designing popular solutions to learning with noisy labels. Existing works heavily rely on finding “anchor points” or their approximates, defined as instances belonging to a particular class almost surely. Nonetheless, finding anchor points remains a non-trivial task, and the estimation accuracy is also often throttled by the number of available anchor points. In this paper, we propose an alternative option to the above task. Our main contribution is the discovery of an efficient estimation procedure based on a clusterability condition. We prove that with clusterable representations of features, using up to third-order consensuses of noisy labels among neighbor representations is sufficient to estimate a unique transition matrix. Compared with methods using anchor points, our approach uses substantially more instances and benefits from a much better sample complexity. We demonstrate the estimation accuracy and advantages of our estimates using both synthetic noisy labels (on CIFAR-10/100) and real human-level noisy labels (on Clothing1M and our self-collected human-annotated CIFAR-10). Our code and human-level noisy CIFAR-10 labels are available at https://github.com/UCSC-REAL/HOC.

Most existing policy learning solutions require the learning agents to receive high-quality supervision signals, e.g., rewards in reinforcement learning (RL) or high-quality expert demonstrations in behavioral cloning (BC). These quality supervisions are either infeasible or prohibitively expensive to obtain in practice. We aim for a unified framework that leverages the available cheap weak supervisions to perform policy learning efficiently. To handle this problem, we treat the weak supervision'' as imperfect information coming from a peer agent, and evaluate the learning agent's policy based on a correlated agreement'' with the peer agent's policy (instead of simple agreements). Our approach explicitly punishes a policy for overfitting to the weak supervision. In addition to theoretical guarantees, extensive evaluations on tasks including RL with noisy reward, BC with weak demonstrations, and standard policy co-training (RL + BC) show that our method leads to substantial performance improvements, especially when the complexity or the noise of the learning environments is high.

Computation offloading is a promising technology in mobile edge computing (MEC) systems. In this paper, we take into account the system dynamics and the user mobility and formulate the mobile computation offloading as a stochastic optimal control problem. On the one hand, when the system information is fully known, we derive the optimal offloading policy. On the other hand, in the case of limited system information, we design a Q-learning algorithm which also gives optimal system performance yet with a slower converge rate. To speed up the convergence and deal with a more complex system, we further develop one more algorithm based on the deep-Q-network (DQN). Simulation results show our proposed DQN-based algorithm indeed converges at a much faster rate.

Task offloading is a promising technology to exploit the available computational resources in spatially distributed fog nodes efficiently in the era of fog computing. In this paper, we look for an online task offloading strategy to minimize the long-term cost, which factors in the latency, the energy consumption, and the switching cost. To this end, we formulate a stochastic programming problem and the expectations of the system parameters are allowed to change abruptly at unknown time instants. Meanwhile, we consider the fact that the queried nodes can only feed back the processing results after finishing the tasks. Then we put forth an effective bandit learning algorithm, i.e., the BLOT, to solve this challenging stochastic programming under the non-stationary bandit model. We also demonstrate that our proposed BLOT algorithm is asymptotically optimal in a non-stationary fog-enabled network. Numerical experiments further verify the superb performance of BLOT.

In-band relay nodes (RNs) can be utilized to enhance the coverage of heterogeneous networks (HetNets) in a cost effective way. However, the in-band RNs also consume the limited spectrum resources. Appropriate spectrum resource management/cooperation is necessary to ensure the balanced resource usages between the macro base stations (BSs) and the RNs. In this paper, we study the sparse spectrum reuse strategy in a HetNet with in-band RNs to maximize the overall proportional fairness metric. Although limiting the number of active reuse patterns will degrade the performance and render the resulting problem non-convex, we first show that there must exist one solution achieving the optimum when the upper bound on the number of active reuse patterns is not less than the total number of mobile stations (MSs) and RNs. We also put forth one active pattern identification scheme based on the re-weighted l1-norm algorithm to deal with the non-convex problem and refine the set of active reuse patterns in a soft manner. Furthermore, in order to offload the heavy computation burden from the central server, one distributed resource allocation algorithm based on the alternating direction method of multipliers (ADMM) algorithm is developed. Numerical simulations demonstrate the superiority and effectiveness of our proposed algorithm.

Task offloading is a promising technology to exploit the benefits of fog computing. An effective task offloading strategy is needed to utilize the computational resources efficiently. In this paper, we endeavor to seek an online task offloading strategy to minimize the long-term latency. In particular, we formulate a stochastic programming problem, where the expectations of the system parameters change abruptly at unknown time instants. Meanwhile, we consider the fact that the queried nodes can only feed back the processing results after finishing the tasks. We then put forward an effective algorithm to solve this challenging stochastic programming under the non-stationary bandit model. We further prove that our proposed algorithm is asymptotically optimal in a non-stationary fog-enabled network. Numerical simulations are carried out to corroborate our designs.

In time-division duplexing (TDD) systems, massive multiple-input multiple-output (MIMO) relies on the channel reciprocity to obtain the downlink (DL) channel state information (CSI) with the acquired uplink (UL) CSI at the base station (BS). However, the mismatches in the radio frequency (RF) analog circuits among different antennas at the BS break the end-to-end UL and DL channel reciprocity. To avoid the severe performance degradation with massive MIMO, it is necessary to calibrate all the M antennas at the BS to restore the end-to-end UL/DL channel reciprocity. In this paper, we examine the internal self-calibration scheme where different BS antennas are interconnected via hardware transmission lines. First, we study the resulting calibration performance for an arbitrary interconnection strategy. Next, we obtain closed-form Cramer-Rao lower bound (CRLB) expressions for each interconnection strategy at the BS with only (M-1) transmission lines. Basing on the derived results, we further prove that the star interconnection strategy is optimal for internal self-calibration due to its lowest CRLB. Numerical simulation results corroborate our theoretical analyses and results.

A new radio frequency fingerprint that incorporates the channel impulse response and Angle of Arrival was introduced to enhance the accuracy of the indoor positioning. To use the new fingerprint. We use the generalized distance metric, i.e., Jensen-Shannon Divergence and cluster center to compare the difference between data collected at the test point and given reference point. Based on the new RF fingerprint, we proposed a device free passive localization system that does not require the user to have any measurement devices. The user location is estimated by searching the reference points with the the smallest distance between the test data and the database. The experimental result shows that the proposed system has better location accuracy method comparing with the conventional fingerprint algorithms.

Device-to-device (D2D) communication has become one important part of next-generation mobile networks particularly due to the booming of proximity-based services, e.g. the ProSe standardized in LTE. However, D2D communications may create strong interference to nearby users that are sharing the same spectrum. Thus, interference management in a D2D-enabled cooperative network is critical. In this paper, the time reuse problem and its distributed solution in a D2D-enabled cooperative network are investigated. Even though the total number of possible time reuse patterns increases exponentially with the quantity of the users, the authors first show that the optimal network performance can be achieved by only activating a limited number of time reuse patterns. The effects of the incentive mechanism on the reuse pattern selection are also investigated. Meanwhile, the set of active reuse patterns are determined efficiently with the Frank-Wolfe method. For a specific set of active time reuse patterns, the authors further show that the optimal resource allocation problem can be formulated as one consensus-building problem. Based on the alternating-direction method of multipliers, one low-complexity algorithm is proposed to determine the optimal resource allocations in a distributed fashion. Numerical simulations are carried out to corroborate our designs.