Yao Xiao, Hai Ye, Linyao Chen, Hwee Tou Ng, Lidong Bing, Xiaoli Li, Roy Ka-Wei Lee. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Iterative data generation and model re-training can effectively align large language models (LLMs) to human preferences.The process of data sampling is crucial, as it significantly influences the success of policy improvement.Repeated random sampling is a widely used method that independently queries the model multiple times to generate outputs.In this work, we propose a more effective sampling method, named Preference-Guided Reflective Sampling (PRS).Unlike random sampling, PRS employs a tree-based generation framework to enable more efficient sampling.It leverages adaptive self-refinement techniques to better explore the sampling space.By specifying user preferences in natural language, PRS can further optimize response generation according to these preferences.As a result, PRS can align models to diverse user preferences.Our experiments demonstrate that PRS generates higher-quality responses with significantly higher rewards.On AlpacaEval and Arena-Hard, PRS substantially outperforms repeated random sampling in bestof-N sampling.Moreover, PRS shows strong performance when applied in iterative offline RL training 1 .

Puncture robots pave a new way for stable, accurate and safe percutaneous liver tumor puncture operation. However, affected by respiratory motion, intraoperative accurate location of the tumor and its surrounding anatomical structures remains a difficult problem in existing robot-assisted puncture operations. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D ultrasound (US) and preoperative 3D computed tomography (CT) fusion is proposed, addressing the shortcomings of existing puncture robots. To deal with the challenge of cross-modal and cross-dimensional registration between 2D US and 3D CT, a decoupled two-stage registration approach combining initial vessel structure-based 3D US – 3D CT registration with intraoperative intensity-based 2D US -3D US registration is proposed. To achieve fast and robust ultrasound probe calibration, a method based on an improved N-wire phantom is proposed. Twenty puncture experiments are performed in different breath-holding positions on a respiratory motion simulation platform, and experimental results show that the mean puncture error is 2.48 mm, which can meet the requirements in a wide of clinical scenariosNote to Practitioners—In clinical percutaneous liver tumor puncture operation, due to the lack of real-time and clear image guidance, it is difficult to locate the tumor and its surrounding vital anatomical structures. In addition, the stability and accuracy of manual operation are poor. The development of a puncture robot is an effective solution for these problems. However, existing CT and magnetic resonance imaging (MRI) guided robots do not consider the tumor localization errors caused by inconsistent breath-holding positions between preoperative scan period and intraoperative puncture period, and US guided robots are limited by the poor image quality and the narrow field of vision. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D US and preoperative 3D CT fusion is proposed. This system can take advantage of the real-time ultrasound and clear CT images at the same time, and can provide real-time, clear and all-round guidance for percutaneous liver tumor puncture operation, which has obvious advantages over the existing puncture robots. Phantom experiments have been completed and animal experiments will be carried out in the future.

In this work, we study multi-source test-time model adaptation from user feedback, where K distinct models are established for adaptation.To allow efficient adaptation, we cast the problem as a stochastic decision-making process, aiming to determine the best adapted model after adaptation.We discuss two frameworks: multi-armed bandit learning and multi-armed dueling bandits.Compared to multi-armed bandit learning, the dueling framework allows pairwise collaboration among K models, which is solved by a novel method named Co-UCB proposed in this work.Experiments on six datasets of extractive question answering (QA) show that the dueling framework using Co-UCB is more effective than other strong baselines for our studied problem 1 .

Although pre-trained language models (PLM) have achieved great success in question answering (QA), their robustness is still insufficient to support their practical applications, especially in the face of distribution shifts.Recently, testtime adaptation (TTA) has shown great potential for solving this problem, which adapts the model to fit the test samples at test time.However, TTA sometimes causes model collapse, making almost all the model outputs incorrect, which has raised concerns about its stability and reliability.In this paper, we delve into why TTA causes model collapse and find that the imbalanced label distribution inherent in QA is the reason for it.To address this problem, we propose Anti-Collapse Fast test-time adaptation (Anti-CF), which utilizes the source model's output to regularize the update of the adapted model during test time.We further design an efficient side block to reduce its inference time.Extensive experiments on various distribution shift scenarios and pre-trained language models (e.g., XLM-RoBERTa, BLOOM) demonstrate that our method can achieve comparable or better results than previous TTA methods at a speed close to vanilla forward propagation, which is 1.8× to 4.4× speedup compared to previous TTA methods.Our code is available at https://github.com/yisunlp/Anti-CF.

In conversational question answering (CQA), the task of question rewriting (QR) in context aims to rewrite a context-dependent question into an equivalent self-contained question that gives the same answer.In this paper, we are interested in the robustness of a QR system to questions varying in rewriting hardness or difficulty.Since there is a lack of questions classified based on their rewriting hardness, we first propose a heuristic method to automatically classify questions into subsets of varying hardness, by measuring the discrepancy between a question and its rewrite.To find out what makes questions hard or easy for rewriting, we then conduct a human evaluation to annotate the rewriting hardness of questions.Finally, to enhance the robustness of QR systems to questions of varying hardness, we propose a novel learning framework for QR that first trains a QR model independently on each subset of questions of a certain level of hardness, then combines these QR models as one joint model for inference.Experimental results on two datasets show that our framework improves the overall performance compared to the baselines 1 .

Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, Luo Si. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Adapting pre-trained language models (PrLMs) (e.g., BERT) to new domains has gained much attention recently.Instead of fine-tuning PrLMs as done in most previous work, we investigate how to adapt the features of PrLMs to new domains without fine-tuning.We explore unsupervised domain adaptation (UDA) in this paper.With the features from PrLMs, we adapt the models trained with labeled data from the source domain to the unlabeled target domain.Self-training is widely used for UDA, and it predicts pseudo labels on the target domain data for training.However, the predicted pseudo labels inevitably include noise, which will negatively affect training a robust model.To improve the robustness of self-training, in this paper we present class-aware feature self-distillation (CFd) to learn discriminative features from PrLMs, in which PrLM features are self-distilled into a feature adaptation module and the features from the same class are more tightly clustered.We further extend CFd to a cross-language setting, in which language discrepancy is studied.Experiments on two monolingual and multilingual Amazon review datasets show that CFd can consistently improve the performance of self-training in cross-domain and cross-language settings.

Recent research indicates that pretraining cross-lingual language models on large-scale unlabeled texts yields significant performance improvements over various cross-lingual and low-resource tasks. Through training on one hundred languages and terabytes of texts, cross-lingual language models have proven to be effective in leveraging high-resource languages to enhance low-resource language processing and outperform monolingual models. In this paper, we further investigate the cross-lingual and cross-domain (CLCD) setting when a pretrained cross-lingual language model needs to adapt to new domains. Specifically, we propose a novel unsupervised feature decomposition method that can automatically extract domain-specific features and domain-invariant features from the entangled pretrained cross-lingual representations, given unlabeled raw texts in the source language. Our proposed model leverages mutual information estimation to decompose the representations computed by a cross-lingual model into domain-invariant and domain-specific parts. Experimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained cross-lingual language model in the CLCD setting.

Semantic parsing aims to transform natural language (NL) utterances into formal meaning representations (MRs), whereas an NL generator achieves the reverse: producing a NL description for some given MRs.Despite this intrinsic connection, the two tasks are often studied separately in prior work.In this paper, we model the duality of these two tasks via a joint learning framework, and demonstrate its effectiveness of boosting the performance on both tasks.Concretely, we propose the method of dual information maximization (DIM) to regularize the learning process, where DIM empirically maximizes the variational lower bounds of expected joint distributions of NL and MRs.We further extend DIM to a semisupervision setup (SEMIDIM), which leverages unlabeled data of both tasks.Experiments on three datasets of dialogue management and code generation (and summarization) show that performance on both semantic parsing and NL generation can be consistently improved by DIM, in both supervised and semi-supervised setups 1 .

We study the problem of generating keyphrases that summarize the key points for a given document.While sequence-to-sequence (seq2seq) models have achieved remarkable performance on this task (Meng et al., 2017), model training often relies on large amounts of labeled data, which is only applicable to resource-rich domains.In this paper, we propose semi-supervised keyphrase generation methods by leveraging both labeled data and large-scale unlabeled samples for learning.Two strategies are proposed.First, unlabeled documents are first tagged with synthetic keyphrases obtained from unsupervised keyphrase extraction methods or a selflearning algorithm, and then combined with labeled samples for training.Furthermore, we investigate a multi-task learning framework to jointly learn to generate keyphrases as well as the titles of the articles.Experimental results show that our semi-supervised learning-based methods outperform a state-of-the-art model trained with labeled data only.

Hai Ye, Xin Jiang, Zhunchen Luo, Wenhan Chao. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Connections between relations in relation extraction, which we call class ties, are common.In distantly supervised scenario, one entity tuple may have multiple relation facts.Exploiting class ties between relations of one entity tuple will be promising for distantly supervised relation extraction.However, previous models are not effective or ignore to model this property.In this work, to effectively leverage class ties, we propose to make joint relation extraction with a unified model that integrates convolutional neural network (CNN) with a general pairwise ranking framework, in which three novel ranking loss functions are introduced.Additionally, an effective method is presented to relieve the severe class imbalance problem from NR (not relation) for model training.Experiments on a widely used dataset show that leveraging class ties will enhance extraction and demonstrate the effectiveness of our model to learn class ties.Our model outperforms the baselines significantly, achieving stateof-the-art performance.