Applying large vision models (LVMs) and large language models (LLMs) for item embedding is becoming cutting-edge for sequential recommendation, given their success in broad applications. Despite their advantages over traditional approaches, these models suffer more significant performance degradation on tail items against conventional ID-based solutions, which are largely overlooked by recent research. In this paper, we substantiate the above challenges as (1) all-in ground-truth, i.e., the standard cross-entropy (CE) loss focuses solely on the target items while treating all non-ground-truth equally, causing insufficient optimization for tail items, and (2) knowledge transfer tax, i.e., the knowledge encapsulated in LLMs and LVMs dominates the optimization process due to insufficient training for tail items. We propose Rewarding and reembedding, a simple yet efficient method to address the above challenges. Specifically, we reinitialize tail item embedding via a Gaussian distribution to alleviate knowledge transfer tax; besides, a rewarding function is incorporated in the CE loss, which adaptively adjusts item rewards during training to encourage the model to pay more attention to tail items rather than exclusively optimizing for ground-truth. Overall, our method enables a more nuanced optimization and is mathematically comparable to the direct preference optimization (DPO) in LLMs. Our extensive experiments on three public datasets show our method outperforms fourteen baselines in overall performance and improves the performance on tail items by a large margin. Our code is available at https://github.com/Yuhanleeee/R2Rec.

Abstract Large Language Models (LLMs) are widely applied in time series analysis. Yet, their utility in few-shot classification—a scenario with limited training data—remains unexplored. We aim to leverage the pre-trained knowledge in LLMs to overcome the data scarcity problem within multivariate time series. To this end, we propose LLMFew, an LLM-enhanced framework, to investigate the feasibility and capacity of LLMs for few-shot multivariate time series classification (MTSC). We first introduce a Patch-wise Temporal Convolution Encoder (PTCEnc) to align time series data with the textual embedding input of LLMs. Then, we fine-tune the pre-trained LLM decoder with Low-rank Adaptations (LoRA) to enable effective representation learning from time series data. Experimental results show our model consistently outperforms state-of-the-art baselines by a large margin, achieving 125.2% and 50.2% improvement in classification accuracy on Handwriting and EthanolConcentration datasets, respectively. Our results also show LLM-based methods achieve comparable performance to traditional models across various datasets in few-shot MTSC, paving the way for applying LLMs in practical scenarios where labeled data are limited. Our code is available at https://github.com/junekchen/llm-fewshot-mtsc .

Abstract As a significant application of machine learning in financial scenarios, loan default risk prediction aims to evaluate the client’s default probability. However, most existing deep learning solutions treat each application as an independent individual, neglecting the explicit connections among different application records. Besides, these attempts suffer from the problem of missing data and imbalanced distribution (i.e., the default records are small samples against all the applications). We believe similar records could provide some auxiliary signals, which are of critical importance to alleviate the data missing issue and facilitate data argumentation. To this end, we propose multi-view loan application graphs, dubbed MLAGs. By evaluating the similarity between the records, a loan application graph can be constructed. Furthermore, we arrange different similarity thresholds to organize various graph structures for multi-graph constructions; thus, a variety of representations can be generated via information propagation and aggregation for small sample argumentation. Consequently, the imbalanced data distribution and missing values issues can be alleviated effectively. We conduct experiments on three public datasets from real-world home credit and P2P lending platforms, which show that MGCN outperforms both conventional and deep learning models. Ablation studies also illustrated the validity of each module design.