Demo Data from SIGIR 2026 / reviewers in the wild / expert
Xuanang Chen
dblp:260/2156
· DBLP profile ↗
shortlisted
← Back
26ranked-venue papers
4as first author
25since 2021 · last 2026
0000-0003-0377-8451ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 17 · 1 first-author · 17 since 2021Databases, data management, data science and information retrieval · 9 · 3 first-author · 9 since 2021Graphics, computer vision, multimedia, augmented reality and games · 3 · 1 first-author · 2 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Databases, data mining, and information retrieval
14 papers |
Information retrieval · 79% Knowledge graphs · 5% Indexing and storage engines · 5% | |
| Artificial intelligence
10 papers |
Language models and text generation · 45% Question answering and dialogue systems · 15% Information extraction and text analysis · 15% |
Topics — the 30 heaviest of 41, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Information retrieval
retrieval-augmented generation |
3.6 | 4 | 2026 | All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG · ACL (1) 2026 Multi-Agent Proactive Information Seeking with Adaptive LLM Orchestration for Non-Factoid Question Answering · KDD (2) 2025 StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization · ICLR 2025 |
Information retrieval
retrieval evaluation |
2.6 | 3 | 2026 | Answer First, Evidence Second? Uncovering Hidden Risks in Well-Structured AI Search Summaries · SIGIR 2026 SURE or Not? Investigating Semantic Understanding in Dense Retrieval Models · ACL (1) 2026 Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective · SIGIR 2022 |
Information retrieval › retrieval models › neural retrieval
dense retrieval |
2.2 | 3 | 2026 | SURE or Not? Investigating Semantic Understanding in Dense Retrieval Models · ACL (1) 2026 Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrieval · SIGIR 2023 Towards Robust Dense Retrieval via Local Ranking Alignment · IJCAI 2022 |
Natural language and speech › Language models and text generation
large language model |
1.7 | 2 | 2026 | AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing · AAAI 2026 Contextual Interaction for Argument Post Quality Assessment · EMNLP 2023 |
Machine learning › Trustworthy machine learning
fairness |
1.0 | 1 | 2026 | All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG · ACL (1) 2026 |
Machine learning › Trustworthy machine learning › fairness › bias in language models
language bias |
1.0 | 1 | 2026 | All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG · ACL (1) 2026 |
Natural language and speech › Language models and text generation
machine-generated text detection |
1.0 | 1 | 2026 | CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection · ACL (1) 2026 |
Natural language and speech › Question answering and dialogue systems › conversational agents
persuasive dialogue |
1.0 | 1 | 2026 | AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing · AAAI 2026 |
Indexing and storage engines
hierarchical index |
1.0 | 1 | 2026 | PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register Indexing · ACL (1) 2026 |
Information retrieval
indexing |
1.0 | 1 | 2026 | PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register Indexing · ACL (1) 2026 |
Information retrieval › evaluation › test collection
retrieval benchmark |
1.0 | 1 | 2026 | SURE or Not? Investigating Semantic Understanding in Dense Retrieval Models · ACL (1) 2026 |
Information retrieval
search engines |
1.0 | 1 | 2026 | PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register Indexing · ACL (1) 2026 |
Empirical software engineering
peer review |
1.0 | 1 | 2026 | CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection · ACL (1) 2026 |
Natural language and speech › Language models and text generation
in-context learning |
0.9 | 2 | 2025 | Contextual Interaction for Argument Post Quality Assessment · EMNLP 2023 prompt4vis: prompting large language models with example mining for tabular data visualization · VLDB J. 2025 |
Knowledge, reasoning and agents › Knowledge representation and reasoning › knowledge-based systems
knowledge-intensive reasoning |
0.9 | 1 | 2025 | StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization · ICLR 2025 |
Natural language and speech › Question answering and dialogue systems › long-form question answering
non-factoid question answering |
0.9 | 1 | 2025 | Multi-Agent Proactive Information Seeking with Adaptive LLM Orchestration for Non-Factoid Question Answering · KDD (2) 2025 |
Natural language and speech › Language models and text generation
retrieval-augmented generation |
0.9 | 1 | 2025 | StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization · ICLR 2025 |
Data models and query languages › natural language interface
natural language interface to database |
0.9 | 1 | 2025 | prompt4vis: prompting large language models with example mining for tabular data visualization · VLDB J. 2025 |
Information retrieval › query reformulation
query expansion |
0.9 | 1 | 2025 | Not All Terms Matter: Recall-Oriented Adaptive Learning for PLM-aided Query Expansion in Open-Domain Question Answering · ACL (1) 2025 |
Natural language and speech › Language models and text generation › LLM agents
action generation |
0.8 | 1 | 2024 | Seg2Act: Global Context-aware Action Generation for Document Logical Structuring · EMNLP 2024 |
Graph data management › graph analytics
graph aggregation |
0.8 | 1 | 2024 | PRP-Graph: Pairwise Ranking Prompting to LLMs with Graph Aggregation for Effective Text Re-ranking · ACL (1) 2024 |
Information retrieval
reranking |
0.8 | 1 | 2024 | PRP-Graph: Pairwise Ranking Prompting to LLMs with Graph Aggregation for Effective Text Re-ranking · ACL (1) 2024 |
Information retrieval
retrieval models |
0.8 | 1 | 2024 | PRP-Graph: Pairwise Ranking Prompting to LLMs with Graph Aggregation for Effective Text Re-ranking · ACL (1) 2024 |
Natural language and speech › Information extraction and text analysis
argument mining |
0.7 | 1 | 2023 | Contextual Interaction for Argument Post Quality Assessment · EMNLP 2023 |
Natural language and speech › Information extraction and text analysis › argument mining
argument quality assessment |
0.7 | 1 | 2023 | Contextual Interaction for Argument Post Quality Assessment · EMNLP 2023 |
Information retrieval › relevance feedback
pseudo-relevance feedback |
0.7 | 1 | 2023 | Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrieval · SIGIR 2023 |
Information retrieval
relevance feedback |
0.7 | 1 | 2023 | Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrieval · SIGIR 2023 |
Knowledge graphs
link prediction |
0.6 | 1 | 2022 | Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective · SIGIR 2022 |
Knowledge graphs › link prediction
link prediction evaluation |
0.6 | 1 | 2022 | Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective · SIGIR 2022 |
Information retrieval › retrieval models › neural retrieval › dense retrieval
robust dense retrieval |
0.6 | 1 | 2022 | Towards Robust Dense Retrieval via Local Ranking Alignment · IJCAI 2022 |
Methods — techniques the papers use, named apart from their topics
large language model · 6.5retrieval-augmented generation · 5.5content-based detection · 2.0recall-oriented learning · 1.7pre-trained language model · 1.7in-context learning · 1.5BERT · 1.1register indexing · 1.0query log analysis · 1.0dynamic outline-guided agent · 1.0dense retrieval models · 1.0bayesian-supervised reinforcement learning · 1.0LLM-as-a-judge · 1.0sparse latents · 0.9information structurization · 0.9bi-point thinking · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | AI-Salesman: Towards Reliable Large Language Model Driven TelemarketingabstractGoal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a significant challenge for even state-of-the-art Large Language Models (LLMs). A lack of task-specific data often limits previous works, and direct LLM application suffers from strategic brittleness and factual hallucination. In this paper, we first construct and release TeleSalesCorpus, the first real-world-grounded dialogue dataset for this domain. We then propose AI-Salesman, a novel framework featuring a dual-stage architecture. For the training stage, we design a Bayesian-supervised reinforcement learning algorithm that learns robust sales strategies from noisy dialogues. For the inference stage, we introduce the Dynamic Outline-Guided Agent (DOGA), which leverages a pre-built script library to provide dynamic, turn-by-turn strategic guidance. Moreover, we design a comprehensive evaluation framework that combines fine-grained metrics for key sales skills with the LLM-as-a-Judge paradigm. Experimental results demonstrate that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, showcasing its effectiveness in complex persuasive scenarios. Chunlei Xin, Xuanang Chen, Yaojie Lu 0001, Xianpei Han, Le Sun 0001, Qianlong Xie |
AAAI | 3 |
| 2026 | CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review DetectionabstractThe growing use of large language models (LLMs) in peer review threatens scholarly integrity.Recent conference policies allow AI tools for language polishing but prohibit their use for generating substantive content.However, existing detectors mainly rely on stylistic cues, making it difficult to distinguish between surface-level language refinement and genuine content generation.To address this, we advocate a content-based detection paradigm and introduce CoCoNUTS, a comprehensive benchmark containing 315,535 reviews covering leading AI conferences and six human-AI collaboration modes.Our evaluation shows that current detectors struggle to handle these nuanced settings.Consequently, we propose CoCoDet, an AI review detector designed to identify substantive AI-generation.Experiments demonstrate that CoCoDet achieves a macro F1-score of 98.24%.Crucially, on permissible machinepolished reviews, it maintains a low false positive rate of 3.89%, substantially outperforming the strongest baseline (7.84%).Examination on real-world reviews using CoCoDet reveals an escalating trend of substantive AI generation.Our work exposes the inadequacy of current detectors, underscoring the importance of domainspecific solutions.Our code is available at https://github.com/icip-cas/CoCoNUTS. Jiawei Chen 0011, Guozhao Mo, Xuanang Chen, Ben He 0001, Xianpei Han, Le Sun 0001 |
ACL (1) | 4 |
| 2026 | SURE or Not? Investigating Semantic Understanding in Dense Retrieval ModelsabstractDense retrieval has become a core technique in applications like web search and retrievalaugmented generation.Despite their empirical success, it remains unclear whether these models truly understand semantics, and to what degree they can represent semantic consistency and distinguish subtle semantic differences.To address this gap, this paper conducts a systematic investigation by introducing SURE, a benchmark for Semantic Understanding in dense REtrieval built upon the MSMARCO, NQ, and FiQA datasets.SURE characterizes semantic understanding in dense retrieval along three dimensions: semantic precision, semantic abstraction, and semantic equivalence.We evaluate ten representative models ranging from 110M to 8B parameters, including both generalpurpose and domain-specific models.Results show that current dense retrievers struggle to distinguish fine-grained semantic differences across texts with varying information density, and to recognize semantic consistency under lexical paraphrasing.Moreover, larger models do not necessarily exhibit stronger semantic understanding, and diverse training data generally enhances semantic understanding on challenging retrieval tasks. Lingdi Kong, Xuanang Chen, Ben He 0001, Le Sun 0001 |
ACL (1) | 2 |
| 2026 | PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register IndexingabstractZhuoqun Li, Xuanang Chen, Hongyu Lin, Yaojie Lu, Xianpei Han, Shanshan Jiang, Bin Dong, Le Sun. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026. Xuanang Chen, Yaojie Lu 0001, Xianpei Han, Shanshan Jiang 0001, Bin Dong 0003, Le Sun 0001 |
ACL (1) | 2 |
| 2026 | All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAGabstractDan Wang, Guozhao Mo, Yafei Shi, Cheng Zhang, Bo Zheng, Boxi Cao, Xuanang Chen, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, Le Sun. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026. Guozhao Mo, Bo Zheng 0007, Boxi Cao, Xuanang Chen, Yaojie Lu 0001, Ben He 0001, Xianpei Han, Le Sun 0001 |
ACL (1) | 7 |
| 2026 | VisPoison: An Effective Backdoor Attack Framework for Tabular Data Visualization ModelsabstractText-to-visualization (text-to-vis) models for tabular data have become essential tools in the era of big data, enabling users to generate visualizations and make data-driven decisions through natural language queries (NLQs). Despite their growing adoption, the security vulnerabilities of these models remain largely unexplored. To address this gap, we propose VisPoison, a backdoor attack framework that realistically simulates three types of attacks on text-to-vis models via data poisoning: data exposure, misleading visualizations, and denial-of-service (DoS). Specifically, VisPoison introduces two types of stealthy triggers to enable both proactive and passive backdoor activations. Proactive triggers are deliberately inserted by attackers using rare-word patterns to extract sensitive information, whereas passive triggers are unintentionally activated by users through first-word prompts, resulting in visualization errors or DoS failures. To support these triggers, we craft specialized payloads for visualization queries that allow compromised models to function normally on benign inputs while producing malicious outputs in the presence of triggers. Extensive evaluations on both trainable and in-context learning (ICL)-based text-to-vis models show that VisPoison achieves attack success rates exceeding 90\%, exposing serious vulnerabilities. Additionally, existing defense strategies reveal limited effectiveness against VisPoison, underscoring the urgent need for more robust and security-aware text-to-vis systems to safeguard human-data interaction. Shuaimin Li, Chen Zhang 0013, Xuanang Chen, Anni Peng, Zhuoyue Wan, Yuanfeng Song, Shiwen Ni, Min Yang 0007, Raymond Chi-Wing Wong |
ICDE | 3 |
| 2026 | Answer First, Evidence Second? Uncovering Hidden Risks in Well-Structured AI Search SummariesabstractAs search engines increasingly adopt answer-centric interfaces, AI-generated summaries are often consumed as final answers. These summaries are typically well-structured and citation-rich, creating a strong appearance of reliability that can encourage user trust, even when evidential grounding is uncertain. Motivated by this gap between appearance and evidence, we analyze 14,175 real-world queries from MS MARCO by examining Google Search AI summaries and their cited sources. We find that reliable-looking summaries mask substantial evidential failures. Despite their credible appearance, 32.31% of summaries are incorrect, and 56.16% of these errors arise even when supporting evidence exists in the cited sources. Moreover, 31.08% of summaries exhibit citation inconsistencies, with conflict risk increasing as more sources are cited and as citations appear in more prominent positions. Our findings reveal systematic, user-facing risks in answer-centric search, highlighting the need for evaluation and design practices beyond surface-level reliability signals. https://github.com/icip-cas/AISummary. Jinman Li, Xuanang Chen, Ruoxi Xu, Yaojie Lu 0001, Zecheng Fan, Xianpei Han, Le Sun 0001 |
SIGIR | 2 |
| 2025 | Not All Terms Matter: Recall-Oriented Adaptive Learning for PLM-aided Query Expansion in Open-Domain Question AnsweringabstractThe effectiveness of open-domain question answering (ODQA), particularly those employing a retriever-reader architecture, depends on the ability to recall relevant documents - a critical step that enables the reader to accurately extract answers. To enhance this retrieval phase, current query expansion (QE) techniques leverage pre-trained language models (PLM) to mitigate word mismatches and improve the recall of relevant documents. Despite their advancements, these techniques often treat all expanded terms uniformly, which can lead to less-than-optimal retrieval outcomes. In response, we propose a novel Recall-oriented Adaptive Learning (ReAL) method, which iteratively adjusts the importance weights of QE terms based on their relevance, thereby refining term distinction and enhancing the separation of relevant terms. Specifically, ReAL employs a similarity-based model to classify documents into pseudo-relevant and pseudo-irrelevant sets, and then optimizes term weights via two tailored loss functions to maximize the scoring gap between them. Experiments on four ODQA datasets and five QE methods show that ReAL consistently enhances retrieval accuracy and overall end-to-end QA performance, providing a robust and efficient solution for improving QE strategies in ODQA scenarios. Ben He 0001, Xuanang Chen, Le Sun 0001 |
ACL (1) | 3 |
| 2025 | DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point ThinkingabstractZhuoqun Li, Haiyang Yu, Xuanang Chen, Hongyu Lin, Yaojie Lu, Fei Huang, Xianpei Han, Yongbin Li, Le Sun. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. Haiyang Yu 0003, Xuanang Chen, Yaojie Lu 0001, Fei Huang 0002, Xianpei Han, Yongbin Li 0001, Le Sun 0001 |
ACL (1) | 3 |
| 2025 | Sparse Latents Steer Retrieval-Augmented GenerationabstractChunlei Xin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Xuanang Chen, Xinyan Guan, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. Chunlei Xin, Shuheng Zhou 0001, Huijia Zhu, Weiqiang Wang 0002, Xuanang Chen, Xinyan Guan, Yaojie Lu 0001, Xianpei Han, Le Sun 0001 |
ACL (1) | 5 |
| 2025 | Match, Compare, or Select? An Investigation of Large Language Models for Entity MatchingabstractEntity matching (EM) is a critical step in entity resolution (ER). Recently, entity matching based on large language models (LLMs) has shown great promise. However, current LLM-based entity matching approaches typically follow a binary matching paradigm that ignores the global consistency among record relationships. In this paper, we investigate various methodologies for LLM-based entity matching that incorporate record interactions from different perspectives. Specifically, we comprehensively compare three representative strategies: matching, comparing, and selecting, and analyze their respective advantages and challenges in diverse scenarios. Based on our findings, we further design a compound entity matching framework (ComEM) that leverages the composition of multiple strategies and LLMs. ComEM benefits from the advantages of different sides and achieves improvements in both effectiveness and efficiency. Experimental results on 8 ER datasets and 10 LLMs verify the superiority of incorporating record interactions through the selecting strategy, as well as the further cost-effectiveness brought by ComEM. Tianshu Wang 0002, Xiaoyang Chen 0001, Xuanang Chen, Xianpei Han, Le Sun 0001, Zhenyu Zeng |
COLING | 4 |
| 2025 | Can LLMs Clarify? Investigation and Enhancement of Large Language Models on Argument Claim OptimizationabstractIn argumentation, the claim is the foundational proposition that underpins the argument, serving as the central pillar upon which the argument is constructed. It guides the subsequent presentation of evidence, reasoning, and analysis, thereby facilitating the audience’s understanding of the core issue. Therefore, ensuring that the claim is precise and unambiguous is crucial for constructing a coherent and persuasive argument. While Large Language Models (LLMs) have demonstrated proficiency in text rewriting tasks such as style transfer and query rewriting, their application to claim optimization remains unexplored. Unlike other rewriting tasks, claim clarification requires the model to rewrite ambiguous or unclear segments of the claim, enhance the content by adding omitted key details, and eliminate redundant or verbose elements. Addressing this gap, this paper evaluates the performance of LLMs on the claim clarification task across various settings. While popular rewriting evaluation methods such as BLEU and Rouge rely on exact word matching, this paper introduces a novel semantic evaluation approach based on a sliding window mechanism. Three distinct LLMs, including LLama2, Mistral, and Qwen2, are assessed for their ability to clarify arguments through zero-shot or few-shot prompting, and supervised fine-tuning (SFT). Additionally, we propose a reinforcement learning-based clarification approach that optimally balances content preservation with claim clarity, thereby augmenting the performance of LLMs on the claim clarification task. Ben He 0001, Xuanang Chen, Le Sun 0001 |
COLING | 3 |
| 2025 | Aligning Retrieval with Reader Needs: Reader-Centered Passage Selection for Open-Domain Question AnsweringabstractOpen-Domain Question Answering (ODQA) systems often struggle with the quality of retrieved passages, which may contain conflicting information and be misaligned with the reader’s needs. Existing retrieval methods aim to gather relevant passages but often fail to prioritize consistent and useful information for the reader. In this paper, we introduce a novel Reader-Centered Passage Selection (R-CPS) method, which enhances the performance of the retrieve-then-read pipeline by re-ranking and clustering passages from the reader’s perspective. Our method re-ranks passages based on the reader’s prediction probability distribution and clusters passages according to the predicted answers, prioritizing more useful and relevant passages to the top and reducing inconsistent information. Experiments on ODQA datasets demonstrate the effectiveness of our approach in improving the quality of evidence passages under zero-shot settings. Chunlei Xin, Shuheng Zhou 0001, Xuanang Chen, Yaojie Lu 0001, Huijia Zhu, Weiqiang Wang 0002, Xianpei Han, Le Sun 0001 |
COLING | 3 |
| 2025 | StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information StructurizationabstractRetrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) in many knowledge-based tasks.
However, existing RAG methods struggle with knowledge-intensive reasoning tasks, because useful information required to these tasks are badly scattered.
This characteristic makes it difficult for existing RAG methods to accurately identify key information and perform global reasoning with such noisy augmentation.
In this paper, motivated by the cognitive theories that humans convert raw information into various structured knowledge when tackling knowledge-intensive reasoning, we proposes a new framework, StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure.
Extensive experiments across various knowledge-intensive tasks show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios, demonstrating its potential as an effective solution for enhancing LLMs in complex real-world applications. Xuanang Chen, Haiyang Yu 0003, Yaojie Lu 0001, Qiaoyu Tang, Fei Huang 0002, Xianpei Han, Le Sun 0001, Yongbin Li 0001 |
ICLR | 2 |
| 2025 | Multi-Agent Proactive Information Seeking with Adaptive LLM Orchestration for Non-Factoid Question AnsweringabstractThe proliferation of complex non-factoid questions in modern information seeking (IS) systems exposes critical limitations in conventional Retrieval-Augmented Generation (RAG) approaches, particularly their static search strategies and the lack of systematic multi-source information integration capabilities. Facing these limitations, we present PASS (Proactive Agent-driven Search System), a novel multi-agent framework that operationalizes human-like proactive search strategies through five specialized agents: Revealer for intent analysis, Navigator for search planning, Seeker/Reader for adaptive retrieval, and Writer for response synthesis, systematically expanding the search space through iterative query refinement and multi-perspective knowledge integration. Crucially, our framework demonstrates remarkable adaptability to mid-sized LLMs, demonstrating its scalability in resource-constrained environments. To comprehensively assess the effectiveness of the proposed framework, we carry out extensive experiments on both mid-sized and proprietary large-scale LLMs, evaluating response quality for complex non-factoid questions using a newly introduced nugget-based assessment. Experimental results from offline nugget-based evaluation and online A/B Tests confirm substantial improvements in answer quality, advancing proactive information seeking methodologies and offering practical pathways for democratizing complex reasoning capabilities to resource-constrained environments. Yuchen Li 0006, Hengyi Cai, Xuanang Chen, Haoyi Xiong, Shuaiqiang Wang, Ben He 0001, Le Sun 0001, Dawei Yin 0001 |
KDD (2) | 5 |
| 2025 | prompt4vis: prompting large language models with example mining for tabular data visualizationabstractAbstract We are currently in the epoch of Large Language Models (LLMs), which have transformed numerous technological domains within the database community. In this paper, we examine the application of LLMs in text-to-visualization (text-to-vis). The advancement of natural language processing technologies has made natural language interfaces more accessible and intuitive for visualizing tabular data. However, despite utilizing advanced neural network architectures, current methods such as Seq2Vis, ncNet, and RGVisNet for transforming natural language queries into DV commands still underperform, indicating significant room for improvement. In this paper, we introduce Prompt4Vis , a novel framework that leverages LLMs and In-context learning to enhance the generation of data visualizations from natural language. Given that In-context learning’s effectiveness is highly dependent on the selection of examples, it is critical to optimize this aspect. Additionally, encoding the full database schema of a query is not only costly but can also lead to inaccuracies. This framework includes two main components: (1) an example mining module that identifies highly effective examples to enhance In-context learning capabilities for text-to-vis applications, and (2) a schema filtering module designed to streamline database schemas. Comprehensive testing on the NVBench dataset has shown that Prompt4Vis significantly outperforms the current state-of-the-art model, RGVisNet, by approximately 35.9% on development sets and 71.3% on test sets. To the best of our knowledge, Prompt4Vis is the first framework to incorporate In-context learning for enhancing text-to-vis, marking a pioneering step in the domain. Shuaimin Li, Xuanang Chen, Yuanfeng Song, Yunze Song, Chen Zhang 0013, Lei Chen 0002 |
VLDB J. | 2 |
| 2024 | PRP-Graph: Pairwise Ranking Prompting to LLMs with Graph Aggregation for Effective Text Re-rankingabstractPairwise Ranking Prompting (PRP) demonstrates impressive effectiveness in zero-shot document re-ranking tasks with large language models (LLMs).However, in the existing methods, PRP only outputs the same label for the comparison results of different confidence intervals without considering the uncertainty of pairwise comparison, which implies an underutilization of the generation probability information of LLMs.To bridge this gap, we propose PRP-Graph, a novel pairwise re-ranking approach, based on a refined scoring PRP unit that exploits the output probabilities of target labels to capture the degree of certainty of the comparison results.Specifically, the PRP-Graph consists of two stages, namely ranking graph construction and ranking graph aggregation.Extensive experiments conducted on the BEIR benchmark demonstrate the superiority of our approach over existing PRP-based methods.Comprehensive analysis reveals that the PRP-Graph displays strong robustness towards the initial ranking order and delivers exceptional re-ranking results with acceptable efficiency. Xuanang Chen, Ben He 0001, Le Sun 0001 |
ACL (1) | 2 |
| 2024 | Seg2Act: Global Context-aware Action Generation for Document Logical StructuringabstractDocument logical structuring aims to extract the underlying hierarchical structure of documents, which is crucial for document intelligence.Traditional approaches often fall short in handling the complexity and the variability of lengthy documents.To address these issues, we introduce SEG2ACT, an end-to-end, generation-based method for document logical structuring, revisiting logical structure extraction as an action generation task.Specifically, given the text segments of a document, SEG2ACT iteratively generates the action sequence via a global context-aware generative model, and simultaneously updates its global context and current logical structure based on the generated actions.Experiments on ChCa-tExt and HierDoc datasets demonstrate the superior performance of SEG2ACT in both supervised and transfer learning settings 1 . Shaojie He, Meng Liao, Xuanang Chen, Yaojie Lu 0001, Yanxiong Lu, Xianpei Han, Le Sun 0001 |
EMNLP | 4 |
| 2023 | Contextual Interaction for Argument Post Quality AssessmentabstractRecently, there has been an increased emphasis on assessing the quality of natural language arguments.Existing approaches primarily focus on evaluating the quality of individual argument posts.However, they often fall short when it comes to effectively distinguishing arguments that possess a narrow quality margin.To address this limitation, this paper delves into two alternative methods for modeling the relative quality of different arguments: 1) Supervised contrastive learning that captures the intricate interactions between arguments.By incorporating this approach, we aim to enhance the assessment of argument quality by effectively distinguishing between arguments with subtle differences.2) Large language models (LLMs) with in-context examples that harness the power of LLMs and enrich them with incontext demonstration.Through extensive evaluation and analysis on the publicly available IBM-Rank-30k dataset, we demonstrate the superiority of our contrastive interaction approach over state-of-the-art baselines.On the other hand, while LLMs with in-context examples demonstrate a commendable ability to identify high-quality argument posts, they exhibit relatively limited effectiveness in quantifying argument quality and distinguishing between arguments with a narrow quality gap.Code is available at https://github.com/ucasYW/Contextual- Interaction-for-AQA. Xuanang Chen, Ben He 0001, Le Sun 0001 |
EMNLP | 2 |
| 2023 | Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrieval
Xueru Wen, Xiaoyang Chen 0001, Xuanang Chen, Ben He 0001, Le Sun 0001 |
SIGIR | 3 |
| 2023 | Dealing with textual noise for robust and effective BERT re-ranking
Xuanang Chen, Ben He 0001, Kai Hui 0001, Le Sun 0001, Yingfei Sun |
Inf. Process. Manag. | 1 |
| 2022 | Towards Robust Dense Retrieval via Local Ranking AlignmentabstractDense retrieval (DR) has extended the employment of pre-trained language models, like BERT, for text ranking. However, recent studies have raised the robustness issue of DR model against query variations, like query with typos, along with non-trivial performance losses. Herein, we argue that it would be beneficial to allow the DR model to learn to align the relative positions of query-passage pairs in the representation space, as query variations cause the query vector to drift away from its original position, affecting the subsequent DR effectiveness. To this end, we propose RoDR, a novel robust DR model that learns to calibrate the in-batch local ranking of query variation to that of original query for the DR space alignment. Extensive experiments on MS MARCO and ANTIQUE datasets show that RoDR significantly improves the retrieval results on both the original queries and different types of query variations. Meanwhile, RoDR provides a general query noise-tolerate learning framework that boosts the robustness and effectiveness of various existing DR models. Our code and models are openly available at https://github.com/cxa-unique/RoDR. Xuanang Chen, Ben He 0001, Le Sun 0001, Yingfei Sun |
IJCAI | 1 |
| 2022 | Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval PerspectiveabstractKnowledge graph completion (KGC) aims to infer missing knowledge triples based on known facts in a knowledge graph. Current KGC research mostly follows an entity ranking protocol, wherein the effectiveness is measured by the predicted rank of a masked entity in a test triple. The overall performance is then given by a micro(-average) metric over all individual answer entities. Due to the incomplete nature of the large-scale knowledge bases, such an entity ranking setting is likely affected by unlabelled top-ranked positive examples, raising questions on whether the current evaluation protocol is sufficient to guarantee a fair comparison of KGC systems. To this end, this paper presents a systematic study on whether and how the label sparsity affects the current KGC evaluation with the popular micro metrics. Specifically, inspired by the TREC paradigm for large-scale information retrieval (IR) experimentation, we create a relatively "complete" judgment set based on a sample from the popular FB15k-237 dataset following the TREC pooling method. According to our analysis, it comes as a surprise that switching from the original labels to our "complete" labels results in a drastic change of system ranking of a variety of 13 popular KGC models in terms of micro metrics. Further investigation indicates that the IR-like macro(-average) metrics are more stable and discriminative under different settings, meanwhile, less affected by label sparsity. Thus, for KGC evaluation, we recommend conducting TREC-style pooling to balance between human efforts and label completeness, and reporting also the IR-like macro metrics to reflect the ranking nature of the KGC task. Xuanang Chen, Ben He 0001, Le Sun 0001 |
SIGIR | 2 |
| 2021 | Simplified TinyBERT: Knowledge Distillation for Document Retrieval
Xuanang Chen, Ben He 0001, Kai Hui 0001, Le Sun 0001, Yingfei Sun |
ECIR (2) | 1 |
| 2021 | Contextualized Offline Relevance Weighting for Efficient and Effective Neural RetrievalabstractOnline search latency is a major bottleneck in deploying large-scale pre-trained language models, e.g. BERT, in retrieval applications. Inspired by the recent advances in transformer-based document expansion technique, we propose to trade offline relevance weighting for online retrieval efficiency by utilizing the powerful BERT ranker to weight the neighbour documents collected by generated pseudo-queries for each document. In the online retrieval stage, the traditional query-document matching is reduced to the much less expensive query to pseudo-query matching, and a document rank list is quickly recalled according to the pre-computed neighbour documents. Extensive experiments on the standard MS MARCO dataset with both passage and document ranking tasks demonstrate promising results of our method in terms of both online efficiency and effectiveness. Xuanang Chen, Ben He 0001, Kai Hui 0001, Le Sun 0001, Yingfei Sun |
SIGIR | 1 |
| 2020 | Multi-dimensional data modelling of video image action recognition and motion capture in deep learning frameworkabstractIn order to improve the accuracy of small‐range human motion recognition in video and the computational efficiency of large‐scale data sets, a multi‐dimensional data model of motion recognition and motion capture in video image based on deep‐learning framework was proposed. First, the moving foreground of the target is extracted by the Gauss mixture model, and the human body is recognised by the gradient histogram. At the second level, the dense trajectory feature and the deep learning feature are fused, according to the integration of global encoding algorithm and convolutional neural network. In the deep learning feature, the fusion of the deep video feature and the video RGB tricolour feature is taken as the feature of deep learning. Finally, the classification is based on the deep learning network model. The simulation experiments based on large‐scale real data sets and small‐scale gesture data sets show that the algorithm has high recognition accuracy for large‐scale data sets and small‐scale gesture actions. In addition, Imperial Computer Vision & Learning Lab human behaviour data set is used to classify the experimental data. The average classification accuracy is 85.79%. The algorithm can run at a speed of about 20 frames per second. Peijun Gao, Xuanang Chen |
IET Image Process. | 3 |