Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly initialised backbones. However, there is limited discussion in the community for the reasons that pseudo-alignment occurs. In this work, we conduct a thorough investigation into the root causes of pseudo-alignment in LLM4TS and build a connection of pseudo-alignment to the cone effect in LLM. We demonstrate that pseudo-alignment arises from the interplay of cone effect within pretrained LLM components and the intrinsically low-dimensional manifold of time-series data. In addition, we also introduce TimeSUP, a novel technique designed to mitigate this issue and improve forecast performance in existing LLM4TS approaches. TimeSUP addresses this by increasing the time series manifold to more closely match the intrinsic dimension of language embeddings, allowing the model to distinguish temporal signals clearly while still capturing shared structures across modalities. As a result, representations for time and language tokens remain distinct yet exhibit high cosine similarity, signifying that the model preserves each modality's unique features while learning their commonalities in a unified embedding space. Empirically, TimeSUP consistently outperforms state-of-the-art LLM4TS methods and other lightweight baselines on long-term forecasting performance. Furthermore, it can be seamlessly integrated into four existing LLM4TS pipelines and delivers significant improvements in forecasting performance.

Multiple complementary-label learning (MCLL) is a machine learning task that involves learning a classifier from instances with multiple complementary labels (MCLs). MCLs are labels that indicate the incorrect labels of an instance. Previous methods for learning with ambiguous supervised information may not be effective because MCLs only make up a small proportion of all labels. In this article, we propose MulCo, a simple yet effective framework that uses contrastive learning to enhance the representation capability in MCLL. Contrastive learning involves contrasting semantically similar and dissimilar pairs of instances, with the goal of benefiting from negatives whose ground-truth labels differ from those of anchors. However, it is possible for dissimilar pairs to have the same label due to the random sampling of negatives from inaccurately labeled data. To solve this problem, we design a sifted contrastive loss for MulCo to correct the sampling of same-label negative pairs. We also provide theoretical evidence for the feasibility of the sifted contrastive loss by establishing an upper bound on the ideal contrastive loss. Correspondingly, we develop two progressive solutions using the properties of complementary labels to approximate the ideal contrastive loss through weighting. Our empirical study demonstrates the effectiveness of the proposed method. The code of this article is available at https://github.com/gaoyi439/MulCo .

Kolmogorov-Arnold Networks (KANs) are neural architectures inspired by the Kolmogorov-Arnold representation theorem that leverage B-spline parameterizations for flexible, locally adaptive function approximation. Although KANs can capture complex nonlinearities beyond those modeled by standard Multi-Layer Perceptrons (MLPs), they frequently exhibit miscalibrated confidence estimates-manifesting as overconfidence in dense data regions and underconfidence in sparse areas. In this work, we systematically examine the impact of four critical hyperparameters -- Layer Width, Grid Order, Shortcut Function, and Grid Range -- on the calibration of KANs. Furthermore, we introduce a novel Temperature-Scaled Loss (TSL) that integrates a temperature parameter directly into the training objective, dynamically adjusting the predictive distribution during learning. Both theoretical analysis and extensive empirical evaluations on standard benchmarks demonstrate that TSL significantly reduces calibration errors, thereby improving the reliability of probabilistic predictions. Overall, our study provides actionable insights into the design of spline-based neural networks and establishes TSL as a robust, loss-agnostic solution for enhancing calibration.

Classical overlap metrics such as Dice or IoU quantify where a medical-image segmentation falls short but say nothing about the confidence of each prediction. Over-confident errors are particularly dangerous in clinical practice, where a single false-positive voxel may trigger an unnecessary biopsy. We introduce three contributions that jointly address spatial precision and reliability. (i) Signed-Distance Calibration (SDC) loss couples cross-entropy, local calibration and a differentiable signed-distance penalty, enforcing boundary accuracy while moderating confidence. (ii) A Spatially Adaptive Margin (SAM) module applies lightweight morphological transforms to ground-truth masks before computing the local target, sharpening ambiguous edges. (iii) Pixel-wise Expected Calibration Error (pECE) extends ECE to millions of voxels and penalises high-confidence false positives. Across four public datasets (ACDC, FLARE, BraTS, PROSTATE) and two back-bones (U-Net, nnU-Net), SDC improves Dice by up to 4 percentage points and halves ECE compared with the state of the art, without sacrificing runtime. Code is available on: https://github.com/EagleAdelaide/SDC-Loss.

Kolmogorov-Arnold Neural Networks (KANs) have recently attracted significant attention in the machine learning community. However, their practical implementation often faces challenges such as poor training stability and a large number of trainable parameters. Moreover, the behavior of learnable activation functions based on B-splines remains insufficiently understood. In this work, we analyze KANs through the lens of spline knot behavior and derive lower and upper bounds on the number of knots in B-spline-based KANs. To address the existing limitations, we propose a novel KAN-based approach, which improves upon the original KAN by reducing the number of trainable parameters to match the scale of standard Multi-Layer Perceptrons (MLPs), while enhancing overall performance. Additionally, we introduce a new training strategy that enforces C2 continuity in the learnable splines, leading to smoother activation functions and improved training stability via range expansion. We evaluate our method across eight diverse datasets encompassing image, text, time series, multimodal, and function approximation tasks. The promising results demonstrate the feasibility of KAN-based architectures and the effectiveness of our proposed enhancements. The proposed method implementation is released in https://github.com/IcurasLW/FR-KAN.git

Large Language Models (LLMs) have demonstrated impressive performance in time series analysis and seems to understand the time temporal relationship well than traditional transformer-based approaches. However, since LLMs are not designed for time series tasks, simpler models-like linear regressions can often achieve comparable performance with far less complexity. In this study, we perform extensive experiments to assess the effectiveness of applying LLMs to key time series tasks, including forecasting, classification, imputation, and anomaly detection. We compare the performance of LLMs against simpler baseline models, such as single-layer linear models and randomly initialized LLMs. Our results reveal that LLMs offer minimal advantages for these core time series tasks and may even distort the temporal structure of the data. In contrast, simpler models consistently outperform LLMs while requiring far fewer parameters. Furthermore, we analyze existing reprogramming techniques and show, through data manifold analysis, that these methods fail to effectively align time series data with language and display ''pseudo-alignment'' behavior in embedding space. Our findings suggest that the performance of LLM-based methods in time series tasks arises from the intrinsic characteristics and structure of time series data, rather than any meaningful alignment with the language model architecture. We release the code for experiments here: https://github.com/IcurasLW/Official-Repository_Understanding_LLM_for_Time_Series_Analysis.git

Dynamic Learning-to-Rank (DLTR) is a method of updating a ranking policy in real time based on user feedback, which may not always be accurate. Although previous DLTR work has achieved fair and unbiased DLTR under inaccurate feedback, they face the tradeoff between fairness and user utility and also have limitations in the setting of feeding items. Existing DLTR works improve ranking utility by eliminating bias from inaccurate feedback on observed items, but the impact of another pervasive form of inaccurate feedback, overlooked or ignored interesting items, remains unclear. For example, users may browse the rankings too quickly to catch interesting items or miss interesting items because the snippets are not optimized enough. This phenomenon raises two questions: (i) Will overlooked interesting items affect the ranking results? and (ii) Is it possible to improve utility without sacrificing fairness if these effects are eliminated? These questions are particularly relevant for small and medium-sized retailers who are just starting out and may have limited data, leading to the use of inaccurate feedback to update their models. In this article, we find that inaccurate feedback in the form of overlooked interesting items has a negative impact on DLTR performance in terms of utility. To address this, we treat the overlooked interesting items as noise and propose a novel DLTR method, the Co-teaching Rank (CoTeR), that has good utility and fairness performance when inaccurate feedback is present in the form of overlooked interesting items. Our solution incorporates a co-teaching-based component with a customized loss function and data sampling strategy, as well as a mean pooling strategy to further accommodate newly added products without historical data. Through experiments, we demonstrate that CoTeR not only enhances utilities but also preserves ranking fairness and can smoothly handle newly introduced items.

Irregular Time Series Data (IRTS) has shown increasing prevalence in real-world applications. We observed that IRTS can be divided into two specialized types: Natural Irregular Time Series (NIRTS) and Accidental Irregular Time Series (AIRTS). Various existing methods either ignore the impacts of irregular patterns or statically learn the irregular dynamics of NIRTS and AIRTS data and suffer from limited data availability due to the sparsity of IRTS. We proposed a novel transformer-based framework for general irregular time series data that treats IRTS from four views: Locality, Time, Spatio and Irregularity to motivate the data usage to the highest potential. Moreover, we design a sophisticated irregularity-gate mechanism to adaptively select task-relevant information from irregularity, which improves the generalization ability to various IRTS data. We implement extensive experiments to demonstrate the resistance of our work to three highly missing ratio datasets (88.4%, 94.9%, 60% missing value) and investigate the significance of the irregularity information for both NIRTS and AIRTS by additional ablation study. We release our implementation in https://github.com/IcurasLW/MTSFormer-Irregular_Time_Series.git.

Machine unlearning requires removing the information of forgetting data while keeping the necessary information of remaining data. Despite recent advancements in this area, existing methodologies mainly focus on the effect removal of forgetting data without considering the negative impact this can have on the information of the remaining data, resulting in significant performance degradation after data removal. Although some methods try to repair the performance of remaining data after removal, the forgotten information can also return after repair. Such an issue is due to the intricate intertwining of the forgetting and remaining data. Without adequately differentiating the influence of these two kinds of data on the model, existing algorithms take the risk of either inadequate removal of the forgetting data or unnecessary loss of valuable information from the remaining data. To address this shortcoming, the present study undertakes a causal analysis of the unlearning and introduces a novel framework termed Causal Machine Unlearning (CaMU). This framework adds intervention on the information of remaining data to disentangle the causal effects between forgetting data and remaining data. Then CaMU eliminates the causal impact associated with forgetting data while concurrently preserving the causal relevance of the remaining data. Comprehensive empirical results on various datasets and models suggest that CaMU enhances performance on the remaining data and effectively minimizes the influences of forgetting data. Notably, this work is the first to interpret deep model unlearning tasks from a new perspective of causality and provide a solution based on causal analysis, which opens up new possibilities for future research in deep model unlearning.

Imbalanced learning, characterised by disproportionate class distributions, impedes the effectiveness of learning algorithms, particularly when available data is scarce. Although the utilisation of external data sources can alleviate these challenges, complete access to such resources is often hampered by privacy regulations or lack of annotations, further complicating the imbalanced learning problem. Additionally, exploiting all data from an external node may not be efficient due to data redundancy and computational constraints. To navigate these issues, this paper introduces an innovative solution for imbalanced learning with restricted data access. We propose a data selection method focused on selecting balanced data from the data-rich but restricted node, prioritising diversity, informativeness and balance. Our strategy mitigates the need for exhaustive data exploration and promotes efficient use of the available data.To further enhance the robustness of data selection, we present an iterative method that progressively selects balanced data. The iterative process, involving training a fully supervised model on the data-shortage node and a contrastive model on the data-rich node, incrementally refines the balance of selected data. Additionally, our method employs prediction entropy to automatically generate weights for training the contrastive models, a distinct improvement over manual weight specification. We validate the effectiveness of our approach through extensive experimentation and demonstrate that our proposed methodology addresses the challenges of imbalanced learning under restricted data access, leading to improved data utilisation, enhanced balance, and better representation in imbalanced learning scenarios. The code is available on GitHub at https://github.com/uqyqiu/CPSL.

Abstract Text classification is a widely used task in natural language processing. However, the presence of stereotype bias in text classification can lead to unfair and inaccurate predictions. Stereotype bias is particularly prevalent in words that are unevenly distributed across classes and are associated with specific categories. This bias can be further strengthened in pre-trained models on large natural language datasets. Prior works to remove stereotype bias have mainly focused on specific demographic groups or relied on specific thesauri without measuring the influence of stereotype words on predictions. In this work, we present a causal analysis of how stereotype bias occurs and affects text classification, and propose a framework to mitigate stereotype bias. Our framework detects potential stereotype bias words using SHAP values and alleviates bias in the prediction stage through a counterfactual approach. Unlike existing debiasing methods, our framework does not rely on existing stereotype word sets and can dynamically evaluate the influence of words on stereotype bias. Extensive experiments and ablation studies show that our approach effectively improves classification performance while mitigating stereotype bias.

Learning fair representations is an essential task to reduce bias in data-oriented decision making. It protects minority subgroups by requiring the learned representations to be independent of sensitive attributes. To achieve independence, the vast majority of the existing work primarily relaxes it to the minimization of the mutual information between sensitive attributes and learned representations. However, direct computation of mutual information is computationally intractable, and various upper bounds currently used either are still intractable or contradict the utility of the learned representations. In this paper, we introduce distance covariance as a new dependence measure into fair representation learning. By observing that sensitive attributes (e.g., gender, race, and age group) are typically categorical, the distance covariance can be converted to a tractable penalty term without contradicting the utility desideratum. Based on the tractable penalty, we propose FairDisCo, a variational method to learn fair representations. Experiments demonstrate that FairDisCo outperforms existing competitors for fair representation learning.

Medication recommendation aiming at accurate prescription is a significant clinical application that assists caregivers in professional practice of medicine, and obtaining informative patient representations plays an important role in building effective recommendation models. Meanwhile, conducting attentive multi-hop reading on Memory Neural Network (MemNN) that stores knowledge from previous admissions is widely applied to derive contextual patterns for accurate patient representations. However, regular attentive reading may repeatedly attend to the same slots of MemNN. Although the coverage mechanism is proposed to tackle the problem, it is based on the assumption that there is one-to-one alignment between source information and target outputs, which medical records do not follow. In pursuit of a valuable model for medication recommendation, we propose the Multi-hop Reading with Selective Coverage (MRSC). MRSC firstly conducts information selection on MemNN based on the coverage of each slot. Then the method involves coverage into the attention calculation during the multi-hop reading on MemNN, making sure that all important historical records is fully utilized by balancing attention within selected information. Experiments on real-world clinical dataset demonstrate that MRSC successfully derives informative patient representations for the recommendation by conducting selection on MemNN and limiting attention adjustment within selected information.

Multi-label learning methods assign multiple labels to one object. In practice, in addition to differentiating relevant labels from irrelevant ones, it is often desired to rank relevant labels for an object, whereas the ranking of irrelevant labels is not important. Thus, we require an algorithm to do classification and ranking of relevant labels simultaneously. Such a requirement, however, cannot be met because most existing methods were designed to optimize existing criteria, yet there is no criterion which encodes the aforementioned requirement. In this paper, we present a new criterion, PRO LOSS, concerning the prediction of all labels as well as the ranking of only relevant labels. We then propose ProSVM which optimizes PRO LOSS efficiently using alternating direction method of multipliers. We further improve its efficiency with an upper approximation that reduces the number of constraints from O(T2) to O(T), where T is the number of labels. We then notice that in real applications, it is difficult to get full supervised information for multi-label data. To make the proposed algorithm more robust to supervised information, we adapt ProSVM to deal with the multi-label learning with partial labels problem. Experiments show that our proposal is not only superior on PRO LOSS, but also highly competitive on existing evaluation criteria.

Feature missing is a serious problem in many applications, which may lead to low quality of training data and further significantly degrade the learning performance. While feature acquisition usually involves special devices or complex process, it is expensive to acquire all feature values for the whole dataset. On the other hand, features may be correlated with each other, and some values may be recovered from the others. It is thus important to decide which features are most informative for recovering the other features as well as improving the learning performance. In this paper, we try to train an effective classification model with least acquisition cost by jointly performing active feature querying and supervised matrix completion. When completing the feature matrix, a novel objective function is proposed to simultaneously minimize the reconstruction error on observed entries and the supervised loss on training data. When querying the feature value, the most uncertain entry is actively selected based on the variance of previous iterations. In addition, a bi-objective optimization method is presented for cost-aware active selection when features bear different acquisition costs. The effectiveness of the proposed approach is well validated by both theoretical analysis and experimental study.