Typical financial risk management involves distinct phases for pre-service risk assessment and in-service default detection, often modeled separately. This paper proposes a novel framework, Multi-Granularity Knowledge Distillation (abbreviated as MGKD), aimed at improving pre-service risk prediction through the integration of in-service user behavior data. MGKD follows the idea of knowledge distillation, where the teacher model, trained on historical in-service data, guides the student model, which is trained on pre-service data. By using soft labels derived from in-service data, the teacher model helps the student model improve its risk prediction prior to service activation. Meanwhile, a multi-granularity distillation strategy is introduced, including coarse-grained, fine-grained, and self-distillation, to align the representations and predictions of the teacher and student models. This approach not only reinforces the representation of default cases but also enables the transfer of key behavioral patterns associated with defaulters from the teacher to the student model, thereby improving the overall performance of pre-service risk assessment. Moreover, we adopt a re-weighting strategy to mitigate the model's bias towards the minority class. Experimental results on large-scale real-world datasets from Tencent Mobile Payment demonstrate the effectiveness of our proposed approach in both offline and online scenarios.

In this paper, we reveal that existing Differentially Private Graph Neural Networks (DP-GNNs) are not effective against Graph Reconstruction Attack (GRA). We further attribute the ineffectiveness of existing DP-GNNs against GRA to their unstructured perturbation mechanism, which only induces unidirectional shift in the embedding similarity distribution. Specifically, this perturbation mechanism tends to decrease the embedding similarity of all node pairs without significantly disrupting the relative ranking, thus allowing GRA to still reconstruct the original graph structure by leveraging the relative ranking of similarities. To address this, we propose a novel Differentially Private Graph Neural Network based on Structured Perturbation (GRASP). Specifically, we observe that independent noise tends to decrease the embedding similarity, while identical noise tends to increase it. By integrating these two types of noise using a Bernoulli technique, we introduce a simple yet effective structured perturbation mechanism, which promotes bidirectional shift in the embedding similarity distribution, thereby effectively disrupting the relative ranking and defending against GRA. Extensive experiments on eight benchmark datasets demonstrate that GRASP effectively defends against GRA. Furthermore, GRASP achieves a superior privacy-utility trade-off compared to existing graph structure protection methods. The implementation of GRASP is available at https://github.com/ZhiyuZone/GRASP/.

Scaling the context size of large language models (LLMs) enables them to perform various new tasks, e.g., book summarization.However, the memory cost of the Key and Value (KV) cache in attention significantly limits the practical applications of LLMs.Recent works have explored token pruning for KV cache reduction in LLMs, relying solely on attention scores as a token importance indicator.However, our investigation into value vector norms revealed a notably non-uniform pattern questioning their reliance only on attention scores.Inspired by this, we propose a new method: Value-Aware Token Pruning (VATP) which uses both attention scores and the ℓ 1 norm of value vectors to evaluate token importance.Extensive experiments on LLaMA2-7B-chat and Vicuna-v1.5-7Bacross 16 LongBench tasks demonstrate that VATP outperforms attention-score-only baselines in over 12 tasks, confirming the effectiveness of incorporating value vector norms into token importance evaluation of LLMs. 1

Credit scoring is an essential technique for credit risk management in the financial industry. However, most credit scoring models face the challenge of reject inference, which refers to the lack of post-loan performance data for rejected applicants, leading to sample selection bias and inaccurate credit assessment. Traditional credit scoring methods tackle this issue by assuming that the missing labels for rejected samples are missing at random (MAR) and by measuring sample similarity directly in the original feature space. Nevertheless, these strategies are not suitable for real-world business scenarios. Inspired by metric learning and transductive learning, we propose a novel credit scoring model called transductive semi-supervised metric network (TSSMN), which formalizes reject inference as a semi-supervised binary classification problem with the prior assumption of missing not at random (MNAR). TSSMN consists of two interconnected modules: the embedding metric network (EMN) that maps samples from the original feature space to the metric space for similarity measurement, and the transductive propagation network (TPN) that performs label propagation based on sample similarity. We evaluate TSSMN on a real-world credit dataset and compare it with traditional credit scoring methods. The results indicate that TSSMN can overcome sample selection bias and more accurately classify credit applicants. Therefore, TSSMN has the potential to enhance credit risk assessment in real-world business scenarios.