Joint multilingual instruction tuning is a widely adopted approach to improve the multilingual instruction-following ability and downstream performance of large language models (LLMs), but the resulting multilingual capability remains highly sensitive to the composition and selection of the training data. Existing selection methods, often based on features like text quality, diversity, or task relevance, typically overlook the intrinsic linguistic structure of multilingual data. In this paper, we propose LangGPS, a lightweight two-stage pre-selection framework guided by language separability—a signal that quantifies how well samples in different languages can be distinguished in the model’s representation space. LangGPS first filters training data based on separability scores and then refines the subset using existing selection methods. Extensive experiments across six benchmarks and 22 languages demonstrate that applying LangGPS on top of existing selection methods improves their effectiveness and generalizability in multilingual training, especially for understanding tasks and low-resource languages. Further analysis reveals that highly separable samples facilitate the formation of clearer language boundaries and support faster adaptation, while low-separability samples tend to function as bridges for cross-lingual alignment. Besides, we also find that language separability can serves as an effective signal for multilingual curriculum learning, where interleaving samples with diverse separability levels yields stable and generalizable gains. Together, we hope our work offers a new perspective on data utility in multilingual contexts and support the development of more linguistically informed LLMs.

Large language models (LLMs) have demonstrated significant progress in multilingual language understanding and generation. However, due to the imbalance in training data, their capabilities in non-English languages are limited. Recent studies revealed the English-pivot multilingual mechanism of LLMs, where LLMs implicitly convert non-English queries into English ones at the bottom layers and adopt English for thinking at the middle layers. However, due to the absence of explicit supervision for cross-lingual alignment in the intermediate layers of LLMs, the internal representations during these stages may become inaccurate. In this work, we introduce a deep supervision fine-tuning method (DFT) that incorporates additional supervision in the internal layers of the model to guide its workflow. Specifically, we introduce two training objectives on different layers of LLMs: one at the bottom layers to constrain the conversion of the target language into English, and another at the middle layers to constrain reasoning in English. To effectively achieve the guiding purpose, we designed two types of supervision signals: logits and feature, which represent a stricter constraint and a relatively more relaxed guidance. Our method guides the model to not only consider the final generated result when processing non-English inputs but also ensure the accuracy of internal representations. We conducted extensive experiments on typical English-centric large models, LLaMA-2 and Gemma-2, and the results on multiple multilingual datasets show that our method significantly outperforms traditional fine-tuning methods.

Recent Large Vision-Language Models (LVLMs) have shown promising reasoning capabilities on text-rich images from charts, tables, and documents. However, the abundant text within such images may increase the model's sensitivity to language. This raises the need to evaluate LVLM performance on cross-lingual text-rich visual inputs, where the language in the image differs from the language of the instructions. To address this, we introduce XT-VQA (Cross-Lingual Text-Rich Visual Question Answering), a benchmark designed to assess how LVLMs handle language inconsistency between image text and questions. XT-VQA integrates five existing text-rich VQA datasets and a newly collected dataset, XPaperQA, covering diverse scenarios that require faithful recognition and comprehension of visual information despite language inconsistency. Our evaluation of prominent LVLMs on XT-VQA reveals a significant drop in performance for cross-lingual scenarios, even for models with multilingual capabilities. A mutual information analysis suggests that this performance gap stems from cross-lingual questions failing to adequately activate relevant visual information. To mitigate this issue, we propose MVCL-MI (Maximization of Vision-Language Cross-Lingual Mutual Information), where a visual-text cross-lingual alignment is built by maximizing mutual information between the model's outputs and visual information. This is achieved by distilling knowledge from monolingual to cross-lingual settings through KL divergence minimization, where monolingual output logits serve as a teacher. Experimental results on the XT-VQA demonstrate that MVCL-MI effectively reduces the visual-text cross-lingual performance disparity while preserving the inherent capabilities of LVLMs, shedding new light on the potential practice for improving LVLMs.

Yangfan Ye, Xiaocheng Feng, Zekun Yuan, Xiachong Feng, Libo Qin, Lei Huang, Weitao Ma, Yichong Huang, Zhirui Zhang, Yunfei Lu, Xiaohui Yan, Duyu Tang, Dandan Tu, Bing Qin. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal abilities but remain prone to multilingual object hallucination, with a higher likelihood of generating responses inconsistent with the visual input when utilizing queries in non-English languages compared to English. Most existing approaches to address these rely on pretraining or fine-tuning, which are resource-intensive. In this paper, inspired by observing the disparities in cross-modal attention patterns across languages, we propose Cross-Lingual Attention Intervention for Mitigating multilingual object hallucination (CLAIM) in LVLMs, a novel near training-free method by aligning attention patterns. CLAIM first identifies language-specific cross-modal attention heads, then estimates language shift vectors from English to the target language, and finally intervenes in the attention outputs during inference to facilitate cross-lingual visual perception capability alignment. Extensive experiments demonstrate that CLAIM achieves an average improvement of 13.56% (up to 30% in Spanish) on the POPE and 21.75% on the hallucination subsets of the MME benchmark across various languages. Further analysis reveals that multilingual attention divergence is most prominent in intermediate layers, highlighting their critical role in multilingual scenarios.

Yirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu, Yutai Hou, Wu Ning, Xu Huang, Duyu Tang, Dandan Tu, Bing Qin, Ting Liu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Function calling significantly extends the application boundary of large language models (LLMs), where high-quality and diverse training data is critical for unlocking this capability. However, collecting and annotating real function-calling data is challenging, while synthetic data from existing pipelines often lack coverage and accuracy. In this paper, we present ToolACE, an automatic agentic pipeline designed to generate accurate, complex, and diverse tool-learning data, specifically tailored to the capabilities of LLMs. ToolACE leverages a novel self-evolution synthesis process to curate a comprehensive API pool of 26,507 diverse APIs. Dialogs are further generated through the interplay among multiple agents, under the guidance of a complexity evaluator. To ensure data accuracy, we implement a dual-layer verification system combining rule-based and model-based checks. We demonstrate that models trained on our synthesized data---even with only 8B parameters---achieve state-of-the-art performance, comparable to the latest GPT-4 models. Our model and a subset of the data are publicly available at https://huggingface.co/Team-ACE.

Mixture-of-Experts (MoE) activates only a subset of experts during inference, allowing the model to maintain low inference FLOPs and latency even as the parameter count scales up. However, since MoE dynamically selects the experts, all the experts need to be loaded into VRAM. Their large parameter size still limits deployment, and offloading, which load experts into VRAM only when needed, significantly increase inference latency. To address this, we propose Mixture of Lookup Experts (MoLE), a new MoE architecture that is efficient in both communication and VRAM usage. In MoLE, the experts are Feed-Forward Networks (FFNs) during training, taking the output of the embedding layer as input. Before inference, these experts can be re-parameterized as lookup tables (LUTs) that retrieves expert outputs based on input ids, and offloaded to storage devices. Therefore, we do not need to perform expert computations during inference. Instead, we directly retrieve the expert’s computation results based on input ids and load them into VRAM, and thus the resulting communication overhead is negligible. Experiments show that, with the same FLOPs and VRAM usage, MoLE achieves inference speeds comparable to dense models and significantly faster than MoE with experts offloading, while maintaining performance on par with MoE. Code: https://github.com/JieShibo/MoLE.

The development of reasoning capabilities represents a critical frontier in large language models (LLMs) research, where reinforcement learning (RL) and process reward models (PRMs) have emerged as predominant methodological frameworks. Contrary to conventional wisdom, empirical evidence from DeepSeek-R1 demonstrates that pure RL training focused on mathematical problem-solving can progressively enhance reasoning abilities without PRM integration, challenging the perceived necessity of process supervision. In this study, we conduct a systematic investigation of the relationship between RL training and PRM capabilities. Our findings demonstrate that problem-solving proficiency and process supervision capabilities represent complementary dimensions of reasoning that co-evolve synergistically during pure RL training. In particular, current PRMs underperform simple baselines like majority voting when applied to state-of-the-art models such as DeepSeek-R1 and QwQ-32B. To address this limitation, we propose Self-PRM, an introspective framework in which models autonomously evaluate and rerank their generated solutions through self-reward mechanisms. Although Self-PRM consistently improves the accuracy of the benchmark (particularly with larger sample sizes), analysis exposes persistent challenges: The approach exhibits low precision (<10\%) on difficult problems, frequently misclassifying flawed solutions as valid. These analyses underscore the need for combined training with process supervision and continued RL scaling to enhance reward alignment and introspective accuracy. We hope these findings provide actionable insights for building more reliable and self-aware complex reasoning models.

Traditional multitask learning methods typically can only leverage shared knowledge within specific tasks or languages, resulting in a loss of either cross-language or cross-task knowledge. This paper proposes a general multilingual multitask model, named SkillNet-X, which enables a single model to tackle many different tasks from different languages. To this end, we define several language-specific skills and task-specific skills, each of which corresponds to a skill module. SkillNet-X sparsely activates parts of the skill modules which are relevant to eitherthe target task or the target language. Acting as knowledge transit hubs, skill modules are capable of absorbing task-related knowledge and language-related knowledge consecutively. We evaluate SkillNet-X on eleven natural language understanding datasets in four languages. Results show that SkillNet-X performs better than task-specific and two multitask learning baselines.To investigate the generalization of our model, we conduct experiments on two new tasks and find that SkillNet-X significantly outperforms baselines.

Speculative decoding has demonstrated its effectiveness in accelerating the inference of large language models (LLMs) while maintaining an identical sampling distribution. However, the conventional approach of training separate draft model to achieve a satisfactory token acceptance rate can be costly and impractical. In this paper, we propose a novel self-speculative decoding framework \emph{Kangaroo} with \emph{double} early exiting strategy, which leverages the shallow sub-network and the \texttt{LM Head} of the well-trained target LLM to construct a self-drafting model. Then, the self-verification stage only requires computing the remaining layers over the \emph{early-exited} hidden states in parallel. To bridge the representation gap between the sub-network and the full model, we train a lightweight and efficient adapter module on top of the sub-network. One significant challenge that comes with the proposed method is that the inference latency of the self-draft model may no longer be negligible compared to the big model. To boost the token acceptance rate while minimizing the latency of the self-drafting model, we introduce an additional \emph{early exiting} mechanism for both single-sequence and the tree decoding scenarios. Specifically, we dynamically halt the small model's subsequent prediction during the drafting phase once the confidence level for the current step falls below a certain threshold. This approach reduces unnecessary computations and improves overall efficiency. Extensive experiments on multiple benchmarks demonstrate our effectiveness, where Kangaroo achieves walltime speedups up to 2.04$\times$, outperforming Medusa-1 with 88.7\% fewer additional parameters. The code for Kangaroo is available at https://github.com/Equationliu/Kangaroo.

Although large-scale video-language pre-training models, which usually build a global alignment between the video and the text, have achieved remarkable progress on various downstream tasks, the idea of adopting fine-grained information during the pre-training stage is not well explored. In this work, we propose STOA-VLP, a pre-training framework that jointly models object and action information across spatial and temporal dimensions. More specifically, the model regards object trajectories across frames and multiple action features from the video as fine-grained features. Besides, We design two auxiliary tasks to better incorporate both kinds of information into the pre-training process of the video-language model. The first is the dynamic object-text alignment task, which builds a better connection between object trajectories and the relevant noun tokens. The second is the spatial-temporal action set prediction, which guides the model to generate consistent action features by predicting actions found in the text. Extensive experiments on three downstream tasks (video captioning, text-video retrieval, and video question answering) demonstrate the effectiveness of our proposed STOA-VLP (e.g. 3.7 Rouge-L improvements on MSR-VTT video captioning benchmark, 2.9% accuracy improvements on MSVD video question answering benchmark, compared to previous approaches).

We present SkillNet-NLG, a sparsely activated approach that handles many natural language generation tasks with one model. Different from traditional dense models that always activate all the parameters, SkillNet-NLG selectively activates relevant parts of the parameters to accomplish a task, where the relevance is controlled by a set of predefined skills. The strength of such model design is that it provides an opportunity to precisely adapt relevant skills to learn new tasks effectively. We evaluate on Chinese natural language generation tasks. Results show that, with only one model file, SkillNet-NLG outperforms previous best performance methods on four of five tasks. SkillNet-NLG performs better than two multitask learning baselines (a dense model and a Mixture-of-Expert model) and achieves comparable performance to task-specific models. Lastly, SkillNet-NLG surpasses baseline systems when adapted to new tasks.

Minghuan Tan, Yong Dai, Duyu Tang, Zhangyin Feng, Guoping Huang, Jing Jiang, Jiwei Li, Shuming Shi. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi, Nan Duan, Ming Zhou. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Junjie Huang, Duyu Tang, Linjun Shou, Ming Gong, Ke Xu, Daxin Jiang, Ming Zhou, Nan Duan. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, Nan Duan. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evidence from external knowledge and make predictions based on the evidence. Recent studies either learn to generate evidence from human-annotated evidence which is expensive to collect, or extract evidence from either structured or unstructured knowledge bases which fails to take advantages of both sources simultaneously. In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence. Specifically, we extract evidence from both structured knowledge base (i.e. ConceptNet) and Wikipedia plain texts. We construct graphs for both sources to obtain the relational structures of evidence. Based on these graphs, we propose a graph-based approach consisting of a graph-based contextual word representation learning module and a graph-based inference module. The first module utilizes graph structural information to re-define the distance between words for learning better contextual word representations. The second module adopts graph convolutional network to encode neighbor information into the representations of nodes, and aggregates evidence with graph attention mechanism for predicting the final answer. Experimental results on CommonsenseQA dataset illustrate that our graph-based approach over both knowledge sources brings improvement over strong baselines. Our approach achieves the state-of-the-art accuracy (75.3%) on the CommonsenseQA dataset.

Neural semantic parsing has achieved impressive results in recent years, yet its success relies on the availability of large amounts of supervised data. Our goal is to learn a neural semantic parser when only prior knowledge about a limited number of simple rules is available, without access to either annotated programs or execution results. Our approach is initialized by rules, and improved in a back-translation paradigm using generated question-program pairs from the semantic parser and the question generator. A phrase table with frequent mapping patterns is automatically derived, also updated as training progresses, to measure the quality of generated instances. We train the model with model-agnostic meta-learning to guarantee the accuracy and stability on examples covered by rules, and meanwhile acquire the versatility to generalize well on examples uncovered by rules. Results on three benchmark datasets with different domains and programs show that our approach incrementally improves the accuracy. On WikiSQL, our best model is comparable to the state-of-the-art system learned from denotations.

Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs.Existing works usually ignore the context that is not explicitly provided, resulting in a context-independent semantic representation that struggles to support the generation.To address this, we propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts.Our approach works in an encoderdecoder manner and is equipped with a Vector Quantised-Variational Autoencoder, where the encoder outputs representations from a distribution over discrete variables.Such discrete representations enable automatically selecting relevant evidence, which not only facilitates evidence-aware generation, but also provides a natural way to uncover rationales behind the generation.Our approach provides state-ofthe-art performance on both Event2Mind and ATOMIC datasets.More importantly, we find that with discrete representations, our model selectively uses evidence to generate different inferential texts.

Wanjun Zhong, Duyu Tang, Zhangyin Feng, Nan Duan, Ming Zhou, Ming Gong, Linjun Shou, Daxin Jiang, Jiahai Wang, Jian Yin. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020.

Fact checking is a challenging task because verifying the truthfulness of a claim requires reasoning about multiple retrievable evidence.In this work, we present a method suitable for reasoning about the semantic-level structure of evidence.Unlike most previous works, which typically represent evidence sentences with either string concatenation or fusing the features of isolated evidence sentences, our approach operates on rich semantic structures of evidence obtained by semantic role labeling.We propose two mechanisms to exploit the structure of evidence while leveraging the advances of pre-trained models like BERT, GPT or XLNet.Specifically, using XLNet as the backbone, we first utilize the graph structure to re-define the relative distances of words, with the intuition that semantically related words should have short distances.Then, we adopt graph convolutional network and graph attention network to propagate and aggregate information from neighboring nodes on the graph.We evaluate our system on FEVER, a benchmark dataset for fact checking, and find that rich structural information is helpful and both our graph-based mechanisms improve the accuracy.Our model is the state-of-the-art system in terms of both official evaluation metrics, namely claim verification accuracy and FEVER score.

Ruize Wang, Duyu Tang, Nan Duan, Wanjun Zhong, Zhongyu Wei, Xuanjing Huang, Daxin Jiang, Ming Zhou. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

Deepfake detection, the task of automatically discriminating machine-generated text, is increasingly critical with recent advances in natural language generative models.Existing approaches to deepfake detection typically represent documents with coarse-grained representations.However, they struggle to capture factual structures of documents, which is a discriminative factor between machinegenerated and human-written text according to our statistical analysis.To address this, we propose a graph-based model that utilizes the factual structure of a document for deepfake detection of text.Our approach represents the factual structure of a given document as an entity graph, which is further utilized to learn sentence representations with a graph neural network.Sentence representations are then composed to a document representation for making predictions, where consistent relations between neighboring sentences are sequentially modeled.Results of experiments on two public deepfake datasets show that our approach significantly improves strong base models built with RoBERTa.Model analysis further indicates that our model can distinguish the difference in the factual structure between machine-generated text and humanwritten text.

Question answering (QA) and question generation (QG) are closely related tasks that could improve each other; however, the connection of these two tasks is not well explored in the literature. In this paper, we present two training algorithms for learning better QA and QG models through leveraging one another. The first algorithm extends Generative Adversarial Network (GAN), which selectively incorporates artificially generated instances as additional QA training data. The second algorithm is an extension of dual learning, which incorporates the probabilistic correlation of QA and QG as additional regularization in training objectives. To test the scalability of our algorithms, we conduct experiments on both document based and table based question answering tasks. Results show that both algorithms improve a QA model in terms of accuracy and QG model in terms of BLEU score. Moreover, we find that the performance of a QG model could be easily improved by a QA model via policy gradient, however, directly applying GAN that regards all the generated questions as negative instances could not improve the accuracy of the QA model. Our algorithm that selectively assigns labels to generated questions would bring a performance boost.

5 9 469 9 65 67 1 016 9 93969696 5 5 67 21 22 67 !"626 ! 5 67#$12 !6%5 6&5 6 '()*+(,,-,./0100234,5671 )8*+9 )2+):*72;01 < = )2>29 ?)8 = 9 1 @A B702C3,2CD)@E0F,8 01 ,8 @,.G9 C/01 0H20-@= 9 =023I8 ,+)= = 9 2C:B702CJ(,7:IA KA 4(9 20 !L9 +8 ,= ,. 1K)= )08 +(H= 9 0:G)9 M 9 2C:4(9 20 NOPQRSTUVWXYZ[X\\]SX^UVWXY_`\S\P`aRP`b^ NRPcW^O[^W^RPW^[VX^OdeQP_UVXbfQ\Qgc`bQV hi j 21 (9 =606)8 :k)68 )= )21020668 ,0+(1 ,9 2+,8 < 6,8 01 )8 )1 8 9 )?)3301 06,9 21 =0== 766,8 1 9 2C)?9 < 3)2+).,8+,21 )l1 < 3)6)23)21= )5021 9 +608 = 9 2C: = 7+(0=C)2)8 01 9 2C= ,78 +)+,3)+,239 1 9 ,2)3,2 1 ()+-0= =)2?9 8 ,25)21 Am780668 ,0+(201 78 0--@ +,5F9 2)=08 )1 8 9 )?0-5,3)-02305)1 0< -)08 2)8 : k()8 )1 ().,8 5)8-)08 2=1 ,n23= 9 59 -08301 < 06,9 21 =. 8 ,51 ()1 8 09 29 2C301 0:0231 ()-01 1 )8 +,2= 9 3)8 =8 )1 8 9 )?)3301 06,9 21 =0=06= )73,1 0= o .,8. 0= 103061 01 9 ,2A*6)+9 n+0--@:,788 )1 8 9 )?< )89 =0+,21 )l1 < 0k08 ))2+,3)8 < 3)+,3)85,3)-k9 1 (0-01 )21?08 9 0F-)k(9 +(1 0o)=+,21 )l1)2< ?9 8 ,25)219 21 ,+,2= 9 3)8 01 9 ,2:023,785)1 0< -)08 2)8-)08 2=1 ,71 9 -9 J)8 )1 8 9 )?)3301 06,9 21 = 9 205,3)-< 0C2,= 1 9 +5)1 0< -)08 29 2C608 039 C5 .,8. 0= 103061 01 9 ,2Ap)+,237+1)l6)8 9 5)21 = ,24mq4m/r0234*sH 301 0= )1 = :k()8 ) 1 ()+,21 )l18 ).)8 =1 ,+-0= =)2?9 8 ,25)219 2t H< uH+,3)=023+,2?)8 = 01 9 ,20-(9 = 1 ,8 @:8 )= 6)+< 1 9 ?)-@Ap)7=)=)v7)2+)< 1 ,< 0+1 9 ,25,3)-0= 1 ()F0= )=)5021 9 +608 = )8 :k(9 +(6)8 .,8 5=1 () = 1 01 )< ,. < 1 ()< 08 10++78 0+@,2F,1 (301 0= )1 = AK)< = 7-1 == (,k1 (01F,1 (1 ()+,21 )l1 < 0k08 )8 )1 8 9 )?< )80231 ()5)1 0< -)08 29 2C=1 8 01 )C@9 568 ,?)0+< +78 0+@:023,780668 ,0+(6)8 .,8 5=F)1 1 )81 (02 8 )1 8 9 )?)< 023< )39 1F0= )-9 2)= A w x 6 12 5 16 4,21 )l1 < 3)6)23)21= )5021 9 +608 = 9 2C09 5=1 ,506 0201 78 0--02C70C)71 1 )8 02+)1 ,0=1 8 7+1 78 0--,C9 < +0-.,8 5y )A CA= ,78 +)+,3)z+,239 1 9 ,2)3,20C9 ?< )2+,21 )l1y )A CA+-0= =)2?9 8 ,25)21 zy E9 2C)10-A : {|}~E,2C)10-A :{|}~j @@)8)10-A :{|}j @< )8)10-A :{|}*7(8)10-A :{|}*7(8023H8 1 J9 : {|}z A*1 02308 30668 ,0+()=1 @69 +0--@-)08 20,2)< = 9 J)< n1 = < 0--5,3)-,21 ())21 9 8 )1 8 09 29 2C301 0= )1 : k(9 +(9 =. )3k9 1 ()0+()l056-)9 239 ?9 370--@9 21 () 1 8 09 29 2C6(0= )02350o)=68 )39 +1 9 ,2=.,8)0+(1 )= 1 )l056-)9 21 ()9 2. )8 )2+)6(0= )A,k)?)8 :1 0o9 2C p,8 o3,2)k(9 -)1 (9 =071 (,8k0=029 21 )8 201L9 +8 ,= ,. 1 K)= )08 +(A +,3)C)2)8 01 9 ,20=02)l056-):68 ,C8 055)8 =7= 7< 0--@3,2,1k8 9 1 )+,3)=.8 ,5= +8 01 +(9 21 ()8 )0k,8 -3Ap()21 ()@k8 9 1 )069 )+),.+,3)920608 < 1 9 +7-08)2?9 8 ,25)21 :1 ()@1 @69 +0--@-)?)8 0C)60= 1 )l6)8 9 )2+),2k8 9 1 9 2C,88 )039 2C+,3)=9 21 ()= 9 59 < -08= 9 1 701 9 ,20=0C79 302+)AL)02k(9 -):301 06,9 21 = .,801 0= o50@?08 @k9 3)-@y 702C)10-A :{|}0z : 1 (7=9 19 =3)= 9 8 0F-)1 ,-)08 206)8 = ,20-9 J)35,3< )-.,81 ()1 08 C)1301 06,9 21 Aj 21 (9 =k,8 o:k)= 1 73@ (,k1 ,071 ,501 9 +0--@8 )1 8 9 )?)= 9 59 -08301 06,9 21 =9 2 0+,21 )l1 < 3)6)23)21= +)208 9 ,0237= )1 ()50=1 () = 766,8 1 9 2C)?9 3)2+)1 ,. 0+9 -9 1 01 )= )5021 9 +608 = 9 2CA '()8 )08 )8 )+)2101 1 )561 =01)l6-,9 1 9 2C8 )< 1 8 9 )?)3)l056-)=1 ,9 568 ,?)1 ()C)2)8 01 9 ,2,.-,C< 9 +0-.,8 50231 )l1 AK)1 8 9 )?)< 023< )39 10668 ,0+()= y 0= (9 5,1 ,)10-A :{|}702C)10-A :{|}Fp7 )10-A :{|}B7)10-A :{|}z1 @69 +0--@n8 = 17= )0 +,21 )l1 < 9 23)6)23)218 )1 8 9 )?)81 ,n231 ()5,= 18 )-< )?021301 06,9 21 :0231 ()27= )9 10=020339 1 9 ,20-9 2671,.1 ())39 1 9 2C5,3)-A,k)?)8:0+,21 )l1 < 0k08 )8 )1 8 9 )?)89 =?)8 @9 56,8 1 021.,81 ()1 0= o,.+,21 )l1 < 3)6)23)21= )5021 9 +608 = 9 2CA,8)l05< 6-)= :0== (,k29 29 C78 )}:+-0= =)2?9 8 ,25)21+02 ()-61 ()8 )1 8 9 )?)83)+9 3)k()1 ()81 ()3)= 9 8 )3+,3) ,. 9 =C)2)8 01 )3F@39 8 )+1 -@ +0--9 2C ,89 1 )8 01 9 2C1 () ¡08 8 0@ 1 ,9 2+8 )5)21)0+()-)5)21 A78 1 ()8 5,8 ):8 )1 8 9 )?)< 023< )39 10668 ,0+()=1 @69 +0--@+,2= 9 3)8,2-@,2) = 9 59 -08)l056-)1 ,)39 1 Aj 2=)5021 9 +608 = 9 2C:1 () 601 1 )8 2,.0= 1 8 7+1 78 0-,71 67150@+,5).8 ,539 .< .)8 )218 )1 8 9 )?)3)l056-)= A'()8 )0-= ,)l9 = 1k,8 o= 1 ,71 9 -9 J)57-1 9 6-))l056-)=1 ,C79 3)1 ()= )5021 9 + 608 = )8y 0@01 9)10-A :{|}702C)10-A :{|}0z : (,k)?)8 :1 ()= )0668 ,0+()=)9 1 ()87= )0()78 9 = 1 9 + k0@1 ,)l6-,9 11 ()8 )1 8 9 )?)3-,C9 +0-.,8 5= 7+(0= 9 2+8 )0= 9 2C1 ()68 ,F0F9 -9 1 @,.0+1 9 ,2=y 0@01 9)10-A : {|}z,87= )08 )-)?02+).72+1 9 ,23)= 9 C2)3023 -)08 2)3F0= )3,2)l6)8 1 9 = )0F,711 ()1 08 C)1-,C9 < +0-.,8 5y 702C)10-A :{|}0z Ap()2k)+,2= 9 3)8 1 ()+,21 )l1)2?9 8 ,25)21 :9 1 ¢ =2,21 8 9 ?9 0-1 ,3)= 9 C2

Tao Shen, Xiubo Geng, Tao Qin, Daya Guo, Duyu Tang, Nan Duan, Guodong Long, Daxin Jiang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, Xu Sun. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

This paper proposes a neural generative model, namely Table2Seq, to generate a natural language sentence based on a table. Specifically, the model maps a table to continuous vectors and then generates a natural language sentence by leveraging the semantics of a table. Since rare words, e.g., entities and values, usually appear in a table, we develop a flexible copying mechanism that selectively replicates contents from the table to the output sequence. We conduct extensive experiments to demonstrate the effectiveness of our Table2Seq model and the utility of the designed copying mechanism. On the WIKIBIO and SIMPLEQUESTIONS datasets, the Table2Seq model improves the state-of-the-art results from 34.70 to 40.26 and from 33.32 to 39.12 in terms of BLEU-4 scores, respectively. Moreover, we construct an open-domain dataset WIKITABLETEXT that includes 13 318 descriptive sentences for 4962 tables. Our Table2Seq model achieves a BLEU-4 score of 38.23 on WIKITABLETEXT outperforming template-based and language model based approaches. Furthermore, through experiments on 1 M table-query pairs from a search engine, our Table2Seq model considering the structured part of a table, i.e., table attributes and table cells, as additional information outperforms a sequence-to-sequence model considering only the sequential part of a table, i.e., table caption.

In this paper, we present a generative model to generate a natural language sentence describing a table region, e.g., a row. The model maps a row from a table to a continuous vector and then generates a natural language sentence by leveraging the semantics of a table. To deal with rare words appearing in a table, we develop a flexible copying mechanism that selectively replicates contents from the table in the output sequence. Extensive experiments demonstrate the accuracy of the model and the power of the copying mechanism. On two synthetic datasets, WIKIBIO and SIMPLEQUESTIONS, our model improves the current state-of-the-art BLEU-4 score from 34.70 to 40.26 and from 33.32 to 39.12, respectively. Furthermore, we introduce an open-domain dataset WIKITABLETEXT including 13,318 explanatory sentences for 4,962 tables. Our model achieves a BLEU-4 score of 38.23, which outperforms template based and language model based approaches.

We present assertion based question answering (ABQA), an open domain question answering task that takes a question and a passage as inputs, and outputs a semi-structured assertion consisting of a subject, a predicate and a list of arguments. An assertion conveys more evidences than a short answer span in reading comprehension, and it is more concise than a tedious passage in passage-based QA. These advantages make ABQA more suitable for human-computer interaction scenarios such as voice-controlled speakers. Further progress towards improving ABQA requires richer supervised dataset and powerful models of text understanding. To remedy this, we introduce a new dataset called WebAssertions, which includes hand-annotated QA labels for 358,427 assertions in 55,960 web passages. To address ABQA, we develop both generative and extractive approaches. The backbone of our generative approach is sequence to sequence learning. In order to capture the structure of the output assertion, we introduce a hierarchical decoder that first generates the structure of the assertion and then generates the words of each field. The extractive approach is based on learning to rank. Features at different levels of granularity are designed to measure the semantic relevance between a question and an assertion. Experimental results show that our approaches have the ability to infer question-aware assertions from a passage. We further evaluate our approaches by incorporating the ABQA results as additional features in passage-based QA. Results on two datasets show that ABQA features significantly improve the accuracy on passage-based QA.

Yibo Sun, Duyu Tang, Nan Duan, Jianshu Ji, Guihong Cao, Xiaocheng Feng, Bing Qin, Ting Liu, Ming Zhou. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.

We study how to learn a semantic parser of state-of-the-art accuracy with less supervised training data.We conduct our study on WikiSQL, the largest hand-annotated semantic parsing dataset to date.First, we demonstrate that question generation is an effective method that empowers us to learn a state-ofthe-art neural network based semantic parser with thirty percent of the supervised training data.Second, we show that applying question generation to the full supervised training data further improves the state-of-the-art model.In addition, we observe that there is a logarithmic relationship between the accuracy of a semantic parser and the amount of training data.

Duyu Tang, Nan Duan, Zhao Yan, Zhirui Zhang, Yibo Sun, Shujie Liu, Yuanhua Lv, Ming Zhou. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

We present an approach to map utterances in conversation to logical forms, which will be executed on a large-scale knowledge base. To handle enormous ellipsis phenomena in conversation, we introduce dialog memory management to manipulate historical entities, predicates, and logical forms when inferring the logical form of current utterances. Dialog memory management is embodied in a generative model, in which a logical form is interpreted in a top-down manner following a small and flexible grammar. We learn the model from denotations without explicit annotation of logical forms, and evaluate it on a large-scale dataset consisting of 200K dialogs over 12.8M entities. Results verify the benefits of modeling dialog memory, and show that our semantic parsing-based approach outperforms a memory network based encoder-decoder model by a huge margin.

This paper presents how to generate questions from given passages using neural networks, where large scale QA pairs are automatically crawled and processed from Community-QA website, and used as training data.The contribution of the paper is 2-fold: First, two types of question generation approaches are proposed, one is a retrieval-based method using convolution neural network (CNN), the other is a generation-based method using recurrent neural network (RNN); Second, we show how to leverage the generated questions to improve existing question answering systems.We evaluate our question generation method for the answer sentence selection task on three benchmark datasets, including SQuAD, MS MARCO, and WikiQA.Experimental results show that, by using generated questions as an extra signal, significant QA improvement can be achieved.

Knowledge base (KB) such as Freebase plays an important role for many natural language processing tasks. English knowledge base is obviously larger and of higher quality than low resource language like Chinese. To expand Chinese KB by leveraging English KB resources, an effective way is to translate English KB (source) into Chinese (target). In this direction, two major challenges are to model triple semantics and to build a robust KB translator. We address these challenges by presenting a neural network approach, which learns continuous triple representation with a gated neural network. Accordingly, source triples and target triples are mapped in the same semantic vector space. We build a new dataset for English-Chinese KB translation from Freebase, and compare with several baselines on it. Experimental results show that the proposed method improves translation accuracy compared with baseline methods. We show that adaptive composition model improves standard solution such as neural tensor network in terms of translation accuracy.

Target-dependent sentiment classification remains a challenge: modeling the semantic relatedness of a target with its context words in a sentence. Different context words have different influences on determining the sentiment polarity of a sentence towards the target. Therefore, it is desirable to integrate the connections between target word and context words when building a learning system. In this paper, we develop two target dependent long short-term memory (LSTM) models, where target information is automatically taken into account. We evaluate our methods on a benchmark dataset from Twitter. Empirical results show that modeling sentence representation with standard LSTM does not perform well. Incorporating target information into LSTM can significantly boost the classification accuracy. The target-dependent LSTM models achieve state-of-the-art performances without using syntactic parser or external sentiment lexicons.

We introduce a deep memory network for aspect level sentiment classification.Unlike feature-based SVM and sequential neural models such as LSTM, this approach explicitly captures the importance of each context word when inferring the sentiment polarity of an aspect.Such importance degree and text representation are calculated with multiple computational layers, each of which is a neural attention model over an external memory.Experiments on laptop and restaurant datasets demonstrate that our approach performs comparable to state-of-art feature based SVM system, and substantially better than LSTM and attention-based LSTM architectures.On both datasets we show that multiple computational layers could improve the performance.Moreover, our approach is also fast.The deep memory network with 9 layers is 15 times faster than LSTM with a CPU implementation.

We propose learning sentiment-specific word embeddings dubbed sentiment embeddings in this paper. Existing word embedding learning algorithms typically only use the contexts of words but ignore the sentiment of texts. It is problematic for sentiment analysis because the words with similar contexts but opposite sentiment polarity, such asgoodandbad, are mapped to neighboring word vectors. We address this issue by encoding sentiment information of texts (e.g., sentences and words) together with contexts of words in sentiment embeddings. By combining context and sentiment level evidences, the nearest neighbors in sentiment embedding space are semantically similar and it favors words with the same sentiment polarity. In order to learn sentiment embeddings effectively, we develop a number of neural networks with tailoring loss functions, and collect massive texts automatically with sentiment signals like emoticons as the training data. Sentiment embeddings can be naturally used as word features for a variety of sentiment analysis tasks without feature engineering. We apply sentiment embeddings to word-level sentiment analysis, sentence level sentiment classification, and building sentiment lexicons. Experimental results show that sentiment embeddings consistently outperform context-based embeddings on several benchmark datasets of these tasks. This work provides insights on the design of neural networks for learning task-specific word embeddings in other natural language processing tasks.

Duyu Tang, Bing Qin, Ting Liu. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Document level sentiment classification remains a challenge: encoding the intrin-sic relations between sentences in the se-mantic meaning of a document. To ad-dress this, we introduce a neural network model to learn vector-based document rep-resentation in a unified, bottom-up fash-ion. The model first learns sentence rep-resentation with convolutional neural net-work or long short-term memory. After-wards, semantics of sentences and their relations are adaptively encoded in docu-ment representation with gated recurren-t neural network. We conduct documen-t level sentiment classification on four large-scale review datasets from IMDB and Yelp Dataset Challenge. Experimen-tal results show that: (1) our neural mod-el shows superior performances over sev-eral state-of-the-art algorithms; (2) gat-ed recurrent neural network dramatically outperforms standard recurrent neural net-work in document modeling for sentiment classification.1 1

In this paper, we propose a representation learning research framework for document-level sentiment analysis. Given a document as the input, document-level sentiment analysis aims to automatically classify its sentiment/opinion (such as thumbs up or thumbs down) based on the textural information. Despite the success of feature engineering in many previous studies, the hand-coded features do not well capture the semantics of texts. In this research, we argue that learning sentiment-specific semantic representations of documents is crucial for document-level sentiment analysis. We decompose the document semantics into four cascaded constitutes: (1) word representation, (2) sentence structure, (3) sentence composition and (4) document composition. Specifically, we learn sentiment-specific word representations, which simultaneously encode the contexts of words and the sentiment supervisions of texts into the continuous representation space. According to the principle of compositionality, we learn sentiment-specific sentence structures and sentence-level composition functions to produce the representation of each sentence based on the representations of the words it contains. The semantic representations of documents are obtained through document composition, which leverages the sentiment-sensitive discourse relations and sentence representations.

In this paper, we propose a joint segmentation and classification framework for sentence-level sentiment classification. It is widely recognized that phrasal information is crucial for sentiment classification. However, existing sentiment classification algorithms typically split a sentence as a word sequence, which does not effectively handle the inconsistent sentiment polarity between a phrase and the words it contains, such as {“not bad,” “bad”} and {“a great deal of,” “great”}. We address this issue by developing a joint framework for sentence-level sentiment classification. It simultaneously generates useful segmentations and predicts sentence-level polarity based on the segmentation results. Specifically, we develop a candidate generation model to produce segmentation candidates of a sentence; a segmentation ranking model to score the usefulness of a segmentation candidate for sentiment classification; and a classification model for predicting the sentiment polarity of a segmentation. We train the joint framework directly from sentences annotated with only sentiment polarity, without using any syntactic or sentiment annotations in segmentation level. We conduct experiments for sentiment classification on two benchmark datasets: a tweet dataset and a review dataset. Experimental results show that: 1) our method performs comparably with state-of-the-art methods on both datasets; 2) joint modeling segmentation and classification outperforms pipelined baseline methods in various experimental settings.

We present a method that learns word embedding for Twitter sentiment classification in this paper.Most existing algorithms for learning continuous word representations typically only model the syntactic context of words but ignore the sentiment of text.This is problematic for sentiment analysis as they usually map words with similar syntactic context but opposite sentiment polarity, such as good and bad, to neighboring word vectors.We address this issue by learning sentimentspecific word embedding (SSWE), which encodes sentiment information in the continuous representation of words.Specifically, we develop three neural networks to effectively incorporate the supervision from sentiment polarity of text (e.g.sentences or tweets) in their loss functions.To obtain large scale training corpora, we learn the sentiment-specific word embedding from massive distant-supervised tweets collected by positive and negative emoticons.Experiments on applying SS-WE to a benchmark Twitter sentiment classification dataset in SemEval 2013 show that (1) the SSWE feature performs comparably with hand-crafted features in the top-performed system; (2) the performance is further improved by concatenating SSWE with existing feature set.

In this paper, we propose a joint segmentation and classification framework for sentiment analysis.Existing sentiment classification algorithms typically split a sentence as a word sequence, which does not effectively handle the inconsistent sentiment polarity between a phrase and the words it contains, such as "not bad" and "a great deal of ".We address this issue by developing a joint segmentation and classification framework (JSC), which simultaneously conducts sentence segmentation and sentence-level sentiment classification.Specifically, we use a log-linear model to score each segmentation candidate, and exploit the phrasal information of top-ranked segmentations as features to build the sentiment classifier.A marginal log-likelihood objective function is devised for the segmentation model, which is optimized for enhancing the sentiment classification performance.The joint model is trained only based on the annotated sentiment polarity of sentences, without any segmentation annotations.Experiments on a benchmark Twitter sentiment classification dataset in SemEval 2013 show that, our joint model performs comparably with the state-of-the-art methods.