Constructing historical language models (LMs) plays a crucial role in aiding archaeological provenance studies and understanding ancient cultures. However, existing resources present major challenges for training effective LMs on historical texts. First, the scarcity of historical language samples renders unsupervised learning approaches based on large text corpora highly inefficient, hindering effective pre-training. Moreover, due to the considerable temporal gap and complex evolution of ancient scripts, the absence of comprehensive character encoding schemes limits the digitization and computational processing of ancient texts, particularly in early Chinese writing. To address these challenges, we introduce InteChar, a unified and extensible character list that integrates unencoded oracle bone characters with traditional and modern Chinese. InteChar enables consistent digitization and representation of historical texts, providing a foundation for robust modeling of ancient scripts. To evaluate the effectiveness of InteChar, we construct the Oracle Corpus Set (OracleCS), an ancient Chinese corpus that combines expert-annotated samples with LLM-assisted data augmentation, centered on Chinese oracle bone inscriptions. Extensive experiments show that models trained with InteChar on OracleCS achieve substantial improvements across various historical language understanding tasks, confirming the effectiveness of our approach and establishing a solid foundation for future research in ancient Chinese NLP.

Comprehension of ancient texts plays an important role in archaeology and understanding of Chinese history and civilization. The rapid development of large language models needs benchmarks that can evaluate their comprehension of ancient characters. Existing Chinese benchmarks are mostly targeted at modern Chinese and transmitted documents in ancient Chinese, but the part of excavated documents in ancient Chinese is not covered. To meet this need, we propose the AncientBench, which aims to evaluate the comprehension of ancient characters, especially in the scenario of excavated documents. The AncientBench is divided into four dimensions, which correspond to the four competencies of ancient character comprehension: glyph comprehension, pronunciation comprehension, meaning comprehension, and contextual comprehension. The benchmark also contains ten tasks, including radical, phonetic radical, homophone, cloze, translation, and more, providing a comprehensive framework for evaluation. We convened archaeological researchers to conduct experimental evaluations, proposed an ancient model as baseline, and conducted extensive experiments on the currently best-performing large language models. The experimental results reveal the great potential of large language models in ancient textual scenarios as well as the gap with humans. Our research aims to promote the development and application of large language models in the field of archaeology and ancient Chinese language.

In recent years, rapid advances in Multimodal Large Language Models (MLLMs) have increasingly stimulated research on ancient Chinese scripts.As the evolution of written characters constitutes a fundamental pathway for understanding cultural transformation and historical continuity, how MLLMs can be systematically leveraged to support and advance text evolution analysis remains an open and largely underexplored problem.To bridge this gap, we construct a comprehensive benchmark comprising 11 tasks and over 130,000 instances, specifically designed to evaluate the capability of MLLMs in analyzing the evolution of ancient Chinese scripts.We conduct extensive evaluations across multiple widely used MLLMs and observe that, while existing models demonstrate a limited ability in glyph-level comparison, their performance on core tasks-such as character recognition and evolutionary reasoning-remains substantially constrained.Motivated by these findings, we propose a glyph-driven fine-tuning framework (GEVO) that explicitly encourages models to capture evolutionary consistency in glyph transformations and enhances their understanding of text evolution.Experimental results show that even models at the 2B scale achieve consistent and comprehensive performance improvements across all evaluated tasks.To facilitate future research, we publicly release both the benchmark and the trained models 1 .

Although studies have demonstrated that Large Language Models (LLMs) can perform well on Out-of-Distribution (OOD) tasks, their advantage tends to diminish as the distribution shift becomes more severe. Consequently, researchers aim to retrieve distributionally similar and informative demonstrations from the available source domain to boost the inference capabilities of LLMs. However, in practical scenarios where the target domain is inaccessible, evaluating the unknown distribution is challenging, which indirectly impacts the quality of the selected demonstrations. To address this problem, we propose DOPA, a demonstration search framework that incorporates an OOD proxy to approximate the inaccessible target domain and guide the retrieval process. Building on proxy-based evaluation, DOPA further introduces a Mahalanobis distance-based global diversity constraint to ensure sufficient diversity among the retrieved demonstrations. Experimental results on multiple LLMs and tasks demonstrate that DOPA effectively enhances robustness in OOD settings.

Self-regulated learning significantly influences students’ academic behavior and performance. Traditional SRL assessment heavily relies on subjective and self-evaluation methods, which are susceptible to personal biases and cognitive limitations. In this study, we leverage mobile devices with GPS and sensors to collect self-reported and passive mobile sensing data from 211 college students over a year. Our aim is to conduct a passive assessment of self-regulated learning. To achieve this, we apply four deep learning models to analyze behavioral features associated with self-regulated learning using students’ life record data. Our findings demonstrate a precise assessment of student self-regulated learning, encompassing the following subscales. Environment structuring (MAE = 2.88, r = 0.54) represents participation in the organization and construction of the learning environment. Time management (MAE = 2.82, r = 0.57) represents planning study time and balancing study with other activities; Help seeking (MAE = 4.86, r = 0.53) represents the tendency and frequency of students to seek help during the learning process. Our study helps inspire new forms of education to assess self-regulated learning and paves the way for individualized interventions in future studies. • We develop and implement the iSense system for SRL assessment. • Collecting mobile sensing and self-reported data from 211 students over a year. • Establishing an SRL assessment model using mobile sensing data and deep learning. • We observe that SRL highly correlates with app usage patterns and sensing data. • Our study can achieve longitudinal monitoring and early intervention of SRL.

Negotiation agents need to influence the attitudes or intentions of users to reach a consensus. Strategy planning and expressive optimization are crucial aspects of effective negotiations. However, previous studies have typically focused on only one of these aspects, neglecting the fact that their combined synergistic effect can lead to better performance. Inspired by the dual-process theory in human cognition, we propose a Dual-Mind Negotiation Agent (DMNA) framework. This framework integrates an intuitive module for rapid, experience-based response and a deliberative module for slow, expression optimization. The intuitive module is trained using Monte Carlo Tree Search (MCTS) and Direct Preference Optimization (DPO), enabling it to make suitable strategic planning and expression. The deliberative module employs a multifaceted reflexion mechanism to enhance the quality of expression. Experiments conducted on negotiation datasets confirm that DMNA achieves state-of-the-art results, demonstrating an enhancement in the negotiation ability of agents.

Recent advances in oracle bone inscriptions (OBI) classification have explored various strategies such as zero-shot learning, augmentation, and complex convolution architectures. These strategies ultimately represent samples as global feature vectors in various forms but often fail to effectively capture the inherent multi-scale and region-specific features of OBI. The reliance on global feature vectors ignores the subtle differences between different scales and the uneven importance of different regions in the inscriptions. To address this issue, we propose a dual-pyramid attention collaborative network that enables classification models to learn OBI multi-scale attention. The dual-pyramid structure covers the convolutional and skeletonized features of pyramids. The spatial collaborative attention between level pairs corrects the bias produced by the pre-trained convolutional feature extractor. Experiments show that our model reduces training parameters by an average of 38% and improves accuracy by an average of 3.89% compared to state-of-the-art models.

In knowledge distillation, the performance of teacher models often serves as an upper limit for student models. For a long time, deeper and more accurate supervised learning algorithms have been the first choice for teacher models in image classification tasks where unsupervised models typically underperform. Therefore, the value of unsupervised teachers for distillation has not been explored. This paper demonstrates an effective path to distill unsupervised teacher clustering knowledge to students. Unlike traditional distillation methods where the teacher model guides the student, we use an annealing strategy to progressively decrease the teacher model’s influence and increase the student model’s own contribution to the distillation loss. Experiments show that, in transferring unsupervised knowledge, the proposed method (AD) improves the student’s accuracy by an average of 9.38% compared to the state-of-the-art. In transferring supervised knowledge, the proposed method performs slightly worse than the state-of-the-art but converges faster during the early epochs.

Multimodal sarcasm detection (MSD) leverages multimodal data, including both images and text, to detect whether the input content contains sarcastic information. Despite recent advances, existing MSD approaches often overlook the imbalance in sarcastic content between text and image modalities, where text typically carries more sarcastic cues conveyed through complex semantic relationships. To address this issue, we propose a dependency graph-enhanced joint attention network that integrates the flat representations extracted from the joint attention mechanisms and the graph-based representations learned from the dependency graph. Specifically, we design a joint attention module to capture the rich sarcastic cues and semantic relationships within the text, and use the graph-based representations to enhance the flat representations. Evaluated on the HFM dataset, our method achieved 0.54% improvement in F1 score and 0.81% in accuracy, demonstrating its effectiveness.

Understanding and identifying the nature of learner confusion is important for online learning platforms. In this study, we address this problem by analyzing forum posts from large-scale online courses. However, due to the large volume of comments and frequent interactions, confusion posts are often overlooked. Existing methods and models, while capable of detecting confusion, typically rely on linguistic features of posts and community factors (e.g. votes, views) but ignore personalized contexts, such as the specific causes and types of confusion. To address this problem, we create the first deep learning dataset focused on confusion types and develop a BERT-based network to model personalized features and identify confusion types. Considering the highly imbalanced distribution of different types of confusion, we further design a novel loss function that adaptively optimizes the training weights for each type. Our method’s effectiveness is confirmed through extensive experimentation.

Hierarchical Text Classification (HTC) organizes candidate labels into a hierarchical structure and uses one or more paths within the hierarchy as the ground-truth labels, which has been applied to various downstream tasks, e.g., sentiment analysis and harmful text detection. Existing works often involve data-driven models that are trained on large-scale datasets. However, creating annotated datasets is labor-intensive and time-consuming. To address this issue, recent work has focused on the few-shot HTC task, where each class has only a few samples, e.g., 5. These approaches perform classification at each layer separately and leverage the prompt learning capability of pre-trained models like BERT. However, we find that these methods always neglect the inter-layer relationships. To solve this problem, we propose a new model called Bidirectional Path Constraint by Label Weighting ( Bpc-lw ). Its basic idea is to use a pre-defined label embedding matrix and a feed-forward neural network for information propagation between layers, while also designing a bidirectional label weighting method to constrain the predictions of each layer to be along the same path in the label hierarchy. In addition, we employ a contrastive learning-based method to enhance the discriminative capacity of the hierarchical embeddings. We compare our proposed method with recent few-shot HTC baseline models across 3 benchmark datasets, and the experimental results demonstrate the effectiveness of Bpc-lw . • We argue that current methods neglect inter-layer information propagation. • We design label embedding matrices and feed-forward neural networks to transmit the label features. • Extensive experiments are conducted.

Cross-domain text classification aims to transfer models from label-rich source domains to label-poor target domains, giving it a wide range of practical applications. Many approaches promote cross-domain generalization by capturing domaininvariant features. However, these methods rely on unlabeled samples provided by the target domains, which renders the model ineffective when the target domain is agnostic. Furthermore, the models are easily disturbed by shortcut learning in the source domain, which also hinders the improvement of domain generalization ability. To solve the aforementioned issues, this paper proposes TACIT, a target domain agnostic feature disentanglement framework which adaptively decouples robust and unrobust features by Variational Auto-Encoders. Additionally, to encourage the separation of unrobust features from robust features, we design a feature distillation task that compels unrobust features to approximate the output of the teacher. The teacher model is trained with a few easy samples that are easy to carry potential unknown shortcuts. Experimental results verify that our framework achieves comparable results to state-of-the-art baselines while utilizing only source domain data.

Emotion-cause pair extraction (ECPE) main focus is on extracting all potential emotion clauses and corresponding cause clauses from unannotated documents. Existing methods achieve promising results with the help of fine-tuning and prompt paradigms, but they present three downsides. First, most approaches cannot distinguish between the emotion-cause pairs that belong to different types of emotions, limiting the existing approaches’ applicability. Second, existing prompt methods utilize a one-to-one mapping relation to achieve label words to category mapping, which brings considerable bias to the results. Third, existing methods achieve the cause extraction task supported by explicit semantic understanding or basic prompt templates, ignoring the implicit information contained in the cause clauses themselves. To solve these issues, we propose an Emotion knowledge-aware Prompt-tuning for Emotion-Cause Pair Extraction (EmoPrompt-ECPE) method, which integrate the knowledge of emotion categories in the ECPE task and mine the implicit knowledge of cause clauses. Specifically, we inject the latent knowledge of the cause clauses and the emotion types into the prompt template. Besides, we extend the emotion labels for many-to-one mapping of label words to categories with an external emotion word base. Furthermore, we utilize the cosine similarity filtering of the label word base to reduce the noise caused by knowledge introduction. Experiments on both Chinese and English benchmark datasets show that our approach can achieve state-of-the-art results. Our code and data can be found at: https://github.com/xy-xiaotudou/EmoPrompt-ECPE.

Human activity recognition (HAR) is a task designed to identify and classify physical activities or behaviors in people’s daily lives. This field relies on data collected from various sensors. Due to differences in environment, user posture and habits, as well as equipment placement, the data exhibits severe heterogeneity. The channel information from different sensors exhibits more complex and uncertain characteristics in terms of shape, noise level, and data quality. These properties can lead to poor generalization of deep learning models, posing challenges for the effectiveness of deep learning algorithms and the widespread use of specific embedded devices in this field. Therefore, this article proposes an adaptive channel signal scaling method to calibrate channel characteristics from the perspective of sensor channel information. Additionally, we propose a stable feature completion strategy to enrich the data feature information using different fusion strategies at the data level. Extensive experiments were conducted on three publicly available HAR datasets, and the experimental results demonstrate that our proposed method significantly improves the performance of state-of-the-art deep learning methods.

Deep Neural Networks (DNNs) are rather restrictive in long-tailed data, since they commonly exhibit an under-representation for minority classes. Various remedies have been proposed to tackle this problem from different perspectives, but they ignore the impact of the density of Backbone Features (BFs) on this issue. Through representation learning, DNNs can map BFs into dense clusters in feature space, while the features of minority classes often show sparse clusters. In practical applications, these features are discretely mapped or even cross the decision boundary resulting in misclassification. Inspired by this observation, we propose a simple and generic method, namely Feature Clusters Compression (FCC), to increase the density of BFs by compressing backbone feature clusters. The proposed FCC can be easily achieved by only multiplying original BFs by a scaling factor in training phase, which establishes a linear compression relationship between the original and multiplied features, and forces DNNs to map the former into denser clusters. In test phase, we directly feed original features without multiplying the factor to the classifier, such that BFs of test samples are mapped closer together and do not easily cross the decision boundary. Meanwhile, FCC can be friendly combined with existing long-tailed methods and further boost them. We apply FCC to numerous state-of-the-art methods and evaluate them on widely used long-tailed benchmark datasets. Extensive experiments fully verify the effectiveness and generality of our method. Code is available at https://github.com/lijian16/FCC.

Deep learning algorithms perform poorly on long-tailed datasets because there is insufficient data in the tail classes to recover its original distribution, resulting in an under-representation of the tail classes in the model. In this work, we propose H2T-FAST, a Head-to-Tail Feature Augmentation method by Style Transfer to improve the performance of the tail. H2T-FAST has the following advantages: (1) It is a fast and universal method that acts on the feature space and so, it can be applied to different backbone networks as well as easily integrated into various imbalanced algorithms with stable performance gains; and (2) it is used only in the training phase and therefore, imposes no additional burden on the deep neural network in the testing phase. In particular, we firstly and randomly select the same number of head samples as the tail ones in each training mini batch. Secondly, the style of the head is transferred to the tail to generate new tail data containing the head style, as a way to increase the number of the tail and get better feature representations. We test our methods on several benchmark vision tasks with state-of-the-art performances.

Ontology serves as the foundation for the underlying representation of knowledge. In order to achieve the sharing of knowledge across languages, ontologies that are typically only represented in English must be translated into different languages. Building a domain-specific translation system is necessary for ontology due to the extremely specialized vocabulary and absence of contextual information. In this paper, we introduce cross-lingual agreement to alleviate the aforementioned issues. We propose a method for representing ontology labels that constructs agreement constraint objects by minimizing the distances between source and target sides. We also integrate with adversarial learning to reduce the difference between the ontology label and the hypothesis generated by the translation model during the fine-tuning. Finally, the agreement modeling strategy is incorporated into the decoding phase to guide the generation of translation candidates. Experiments on the four domains English-to-German ontology show that the proposed method achieves significant improvements over the baselines.

The long-tail effect is a common issue that limits the performance of deep learning models on real-world datasets. Character image datasets are also affected by such unbalanced data distribution due to differences in character usage frequency. Thus, current character recognition methods are limited when applied in the real world, especially for the categories in the tail that lack training samples, e.g., uncommon characters. In this paper, we propose a zero-shot character recognition framework via radical-based reasoning, called RZCR, to improve the recognition performance of few-sample character categories in the tail. Specifically, we exploit radicals, the graphical units of characters, by decomposing and reconstructing characters according to orthography. RZCR consists of a visual semantic fusion-based radical information extractor (RIE) and a knowledge graph character reasoner (KGR). RIE aims to recognize candidate radicals and their possible structural relations from character images in parallel. The results are then fed into KGR to recognize the target character by reasoning with a knowledge graph. We validate our method on multiple datasets, and RZCR shows promising experimental results, especially on few-sample character datasets.

Optical character recognition (OCR) methods have been applied to diverse tasks, e.g., street view text recognition and document analysis. Recently, zero-shot OCR has piqued the interest of the research community because it considers a practical OCR scenario with unbalanced data distribution. However, there is a lack of benchmarks for evaluating such zero-shot methods that apply a divide-and-conquer recognition strategy by decomposing characters into radicals. Meanwhile, radical recognition, as another important OCR task, also lacks radical-level annotation for model training. In this paper, we construct an ancient Chinese character image dataset that contains both radical-level and character-level annotations to satisfy the requirements of the above-mentioned methods, namely, ACCID, where radical-level annotations include radical categories, radical locations, and structural relations. To increase the adaptability of ACCID, we propose a splicing-based synthetic character algorithm to augment the training samples and apply an image denoising method to improve the image quality. By introducing character decomposition and recombination, we propose a baseline method for zero-shot OCR. The experimental results demonstrate the validity of ACCID and the baseline model quantitatively and qualitatively.

Interactive argument pair identification is an emerging research task for argument mining, aiming to identify whether two arguments are interactively related.It is pointed out that the context of the argument is essential to improve identification performance.However, current context-based methods achieve limited improvements since the entire context typically contains much irrelevant information.In this paper, we propose a simple contrastive learning framework to solve this problem by extracting valuable information from the context.This framework can construct hard argumentcontext samples and obtain a robust and uniform representation by introducing contrastive learning.We also propose an argument-context extraction module to enhance information extraction by discarding irrelevant blocks.The experimental results show that our method achieves the state-of-the-art performance on the benchmark dataset.Further analysis demonstrates the effectiveness of our proposed modules and visually displays more compact semantic representations.The code is available at GitHub 1 .

Federated Learning (FL) is an emerging privacy-aware machine learning technique that applies successfully to the collaborative learning of global models for Human Activity Recognition (HAR). As of now, the applications of FL for HAR assume that the data associated with diverse individuals follow the same distribution. However, this assumption is impractical in real-world scenarios where the same activity is frequently performed differently by different individuals. To tackle this issue, we propose FedMAT, a Federated Multi-task ATtention framework for HAR, which extracts and fuses shared as well as individual-specific multi-modal sensor data features. Specifically, we treat the HAR problem associated with each individual as a different task and train a federated multi-task model, composed of a shared feature representation network in a central server plus multiple individual-specific networks with attention modules stored in decentralized nodes. In this architecture, the attention module operates as a mask that allows to learn individual-specific features from the global model, whilst simultaneously allowing for features to be shared among different individuals. We conduct extensive experiments based on publicly available HAR datasets, which are collected in both controlled environments and real-world scenarios. Numeric results verify that our proposed FedMAT significantly outperforms baselines not only in generalizing to existing individuals but also in adapting to new individuals.

Degraded images commonly exist in the general sources of character images, leading to unsatisfactory character recognition results. Existing methods have dedicated efforts to restoring degraded character images. However, the denoising results obtained by these methods do not appear to improve character recognition performance. This is mainly because current methods only focus on pixel-level information and ignore critical features of a character, such as its glyph, resulting in character-glyph damage during the denoising process. In this paper, we introduce a novel generic framework based on glyph fusion and attention mechanisms, i.e., CharFormer, for precisely recovering character images without changing their inherent glyphs. Unlike existing frameworks, CharFormer introduces a parallel target task for capturing additional information and injecting it into the image denoising backbone, which will maintain the consistency of character glyphs during character image denoising. Moreover, we utilize attention-based networks for global-local feature interaction, which will help to deal with blind denoising and enhance denoising performance. We compare CharFormer with state-of-the-art methods on multiple datasets. The experimental results show the superiority of CharFormer quantitatively and qualitatively.

Constructing high-quality character image datasets is challenging because real-world images are often affected by image degradation. There are limitations when applying current image restoration methods to such real-world character images, since (i) the categories of noise in character images are different from those in general images; (ii) real-world character images usually contain more complex image degradation, e.g., mixed noise at different noise levels. To address these problems, we propose a real-world character restoration network (RCRN) to effectively restore degraded character images, where character skeleton information and scale-ensemble feature extraction are utilized to obtain better restoration performance. The proposed method consists of a skeleton extractor (SENet) and a character image restorer (CiRNet). SENet aims to preserve the structural consistency of the character and normalize complex noise. Then, CiRNet reconstructs clean images from degraded character images and their skeletons. Due to the lack of benchmarks for real-world character image restoration, we constructed a dataset containing 1,606 character images with real-world degradation to evaluate the validity of the proposed method. The experimental results demonstrate that RCRN outperforms state-of-the-art methods quantitatively and qualitatively.

Few-shot classification aims at recognizing novel categories from low data regimes based on prior knowledge. However, the existing methods for few-shot scene classification have limitations on using few annotated data and do not fully consider the intra-class samples with classification targets in different sizes, which lead to poor feature representation. To address these problems, this study introduces an end-to-end framework called self-supervised contrastive learning-based metric learning network (SCL-MLNet) for few-shot remote sensing (RS) scene classification. On one hand, we weave self-supervised contrastive learning into few-shot classification algorithms through multi-task learning, enabling feature extractors to learn representative image features from few annotated samples. Moreover, we devise a new loss function to train the proposed model end-to-end and speed up the convergence of the model. On the other hand, considering the differences between intra-class samples, we introduce a novel attention module embedded in the feature extractor to fuse multi-scale spatial features from the classification targets in different sizes. In our experiments, SCL-MLNet is evaluated on three public benchmark datasets. The results demonstrate that SCL-MLNet achieves state-of-the-art performance for few-shot remote sensing scene classification.