Direct Preference Optimization (DPO) simplifies reinforcement learning from human feedback (RLHF) for large language models (LLMs) by directly training on offline preference data to align with human preferences. During DPO training, the reference model serves as a data weight adjuster. However, the common practice of initializing the policy and reference models identically in DPO can lead to inefficient data utilization and impose a performance ceiling. Meanwhile, the absence of a reference model in Simple Preference Optimization (SimPO) reduces training robustness and requires stricter conditions to prevent catastrophic forgetting. In this work, we propose Pre-DPO, a simple yet effective DPO-based training paradigm that improves preference optimization by introducing a guiding reference model. This reference model provides foresight into the desired policy state achievable through the training preference data, serving as a guiding mechanism that adaptively assigns higher weights to samples more suitable for the model and lower weights to those less suitable. Extensive experiments on the AlpacaEval 2 and Arena-Hard v0.1 benchmarks demonstrate that Pre-DPO consistently improves the performance of both DPO and SimPO, without relying on external models or additional data.

Many contemporary data-driven research efforts in the natural sciences, such as chemistry and materials science, require large-scale, highperformance entity recognition from scientific datasets.Large language models (LLMs) have increasingly been adopted to solve the entity recognition task, with the same trend being observed on all-spectrum NLP tasks.The prevailing entity recognition LLMs rely on finetuned technology, yet the fine-tuning process often incurs significant cost.To achieve a best performance-cost trade-off, we propose ALLabel, a three-stage framework designed to select the most informative and representative samples in preparing the demonstrations for LLM modeling.The annotated examples are used to construct a ground-truth retrieval corpus for LLM in-context learning.By sequentially employing three distinct active learning strategies, ALLabel consistently outperforms all baselines under the same annotation budget across three specialized domain datasets.Experimental results also demonstrate that selectively annotating only 5%-10% of the dataset with ALLabel can achieve performance comparable to the method annotating the entire dataset.Further analyses and ablation studies verify the effectiveness and generalizability of our proposal.

Aspect-based sentiment classification is a popular task aimed at identifying the corresponding emotion of a specific aspect.One sentence may contain various sentiments for different aspects.Many sophisticated methods such as attention mechanism and Convolutional Neural Networks (CNN) have been widely employed for handling this challenge.Recently, semantic dependency tree implemented by Graph Convolutional Networks (GCN) is introduced to describe the inner connection between aspects and the associated emotion words.But the improvement is limited due to the noise and instability of dependency trees.To this end, we propose a dependency graph enhanced dual-transformer network (named DGEDT) by jointly considering the flat representations learnt from Transformer and graphbased representations learnt from the corresponding dependency graph in an iterative interaction manner.Specifically, a dualtransformer structure is devised in DGEDT to support mutual reinforcement between the flat representation learning and graph-based representation learning.The idea is to allow the dependency graph to guide the representation learning of the transformer encoder and vice versa.The results on five datasets demonstrate that the proposed DGEDT outperforms all state-of-the-art alternatives with a large margin.

Meaning Representations (AMRs) capture sentence-level semantics structural representations to broad-coverage natural sentences.We investigate parsing AMR with explicit dependency structures and interpretable latent structures.We generate the latent soft structure without additional annotations, and fuse both dependency and latent structure via an extended graph neural networks.The fused structural information helps our experiments results to achieve the best reported results on both AMR 2.0 (77.5% Smatch F1 on LDC2017T10) and AMR 1.0 (71.8% Smatch F1 on LDC2014T12).