Demo Data from SIGIR 2026 / reviewers in the wild / expert
Yingfei Sun
dblp:08/6724
· DBLP profile ↗
38ranked-venue papers
1as first author
30since 2021 · last 2026
0000-0003-0615-2569ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 24 · 1 first-author · 21 since 2021Databases, data management, data science and information retrieval · 8 · 7 since 2021Graphics, computer vision, multimedia, augmented reality and games · 8 · 8 since 2021Applied, interdisciplinary, general and emerging computing · 5 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
14 papers |
Trustworthy machine learning · 25% Segmentation and scene understanding · 19% Language models and text generation · 18% | |
| Databases, data mining, and information retrieval
10 papers |
Information retrieval · 98% Recommender systems · 2% | |
| Network and information security
2 papers |
Security and privacy of machine learning · 100% |
Topics — the 30 heaviest of 60, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Machine learning › Trustworthy machine learning › robustness › adversarial robustness › test-time defense
adversarial purification |
1.7 | 2 | 2025 | Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain · ICML 2025 Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification · CVPR 2025 |
Machine learning › Trustworthy machine learning › robustness
adversarial robustness |
1.7 | 2 | 2025 | Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain · ICML 2025 Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification · CVPR 2025 |
Computer vision › Segmentation and scene understanding › semantic segmentation
open-vocabulary segmentation |
1.7 | 2 | 2025 | SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation · ICLR 2025 Boosting Segment Anything Model Towards Open-Vocabulary Learning · AAAI 2025 |
Machine learning › Trustworthy machine learning
robustness |
1.4 | 2 | 2024 | Sequential Manipulation Against Rank Aggregation: Theory and Algorithm · IEEE Trans. Pattern Anal. Mach. Intell. 2024 Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection · EMNLP 2023 |
Natural language and speech › Language models and text generation
large language model |
1.3 | 2 | 2026 | TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking · AAAI 2026 Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning · ACL (1) 2025 |
Information retrieval
ranking |
1.2 | 2 | 2026 | TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking · AAAI 2026 Towards Robust Dense Retrieval via Local Ranking Alignment · IJCAI 2022 |
Computer vision › Segmentation and scene understanding
semantic segmentation |
1.1 | 2 | 2025 | SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation · ICLR 2025 ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts · IEEE Trans. Image Process. 2025 |
Machine learning › Representation and self-supervised learning › representation learning › visual representation learning
vision foundation model |
1.1 | 2 | 2025 | Boosting Segment Anything Model Towards Open-Vocabulary Learning · AAAI 2025 SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation · ICLR 2025 |
Security and privacy of machine learning
adversarial attack |
1.1 | 2 | 2025 | Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs · ICML 2025 Exploring Query Efficient Data Generation Towards Data-Free Model Stealing in Hard Label Setting · AAAI 2025 |
Information retrieval
retrieval models |
1.1 | 2 | 2026 | Spiral of Silence: How is Large Language Model Killing Information Retrieval? - A Case Study on Open Domain Question Answering · ACL (1) 2024 Breaking the Spiral: A Utility-Driven Optimization Framework for Balanced Information Retrieval in the LLM Era · ACM Trans. Inf. Syst. 2026 |
Information retrieval › ranking
learning to rank |
1.0 | 1 | 2026 | TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking · AAAI 2026 |
Information retrieval › ranking › learning to rank
pointwise ranking |
1.0 | 1 | 2026 | TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking · AAAI 2026 |
Information retrieval
retrieval-augmented generation |
1.0 | 1 | 2026 | Breaking the Spiral: A Utility-Driven Optimization Framework for Balanced Information Retrieval in the LLM Era · ACM Trans. Inf. Syst. 2026 |
Information retrieval
retrieval evaluation |
1.0 | 1 | 2026 | Breaking the Spiral: A Utility-Driven Optimization Framework for Balanced Information Retrieval in the LLM Era · ACM Trans. Inf. Syst. 2026 |
Machine learning › Generative modeling › diffusion model
diffusion-based purification |
0.9 | 1 | 2025 | Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification · CVPR 2025 |
Machine learning › Generative modeling
diffusion model |
0.9 | 1 | 2025 | Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification · CVPR 2025 |
Computer vision › Segmentation and scene understanding
instance segmentation |
0.9 | 1 | 2025 | SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation · ICLR 2025 |
Natural language and speech › Language models and text generation › large language model › knowledge in language models
knowledge injection |
0.9 | 1 | 2025 | Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning · ACL (1) 2025 |
Natural language and speech › Language models and text generation
large language model evaluation |
0.9 | 1 | 2025 | Large Language Models Often Say One Thing and Do Another · ICLR 2025 |
Natural language and speech › Language models and text generation › trustworthy language model
large language model reliability |
0.9 | 1 | 2025 | Large Language Models Often Say One Thing and Do Another · ICLR 2025 |
Machine learning › Deep learning architectures and training
mixture of experts |
0.9 | 1 | 2025 | ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts · IEEE Trans. Image Process. 2025 |
Machine learning › Efficient and distributed learning › large-scale learning
model scaling |
0.9 | 1 | 2025 | ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts · IEEE Trans. Image Process. 2025 |
Computer vision › Image recognition and object detection › object detection
open-vocabulary object detection |
0.9 | 1 | 2025 | Boosting Segment Anything Model Towards Open-Vocabulary Learning · AAAI 2025 |
Computer vision › Segmentation and scene understanding
panoptic segmentation |
0.9 | 1 | 2025 | SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation · ICLR 2025 |
Machine learning › Deep learning architectures and training › transformer
vision transformer |
0.9 | 1 | 2025 | ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts · IEEE Trans. Image Process. 2025 |
Image and video processing › frequency domain analysis › frequency-domain image processing
frequency-domain image analysis |
0.9 | 1 | 2025 | Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain · ICML 2025 |
Security and privacy of machine learning › model stealing
data-free model stealing |
0.9 | 1 | 2025 | Exploring Query Efficient Data Generation Towards Data-Free Model Stealing in Hard Label Setting · AAAI 2025 |
Security and privacy of machine learning › adversarial attack
jailbreak attack |
0.9 | 1 | 2025 | Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs · ICML 2025 |
Security and privacy of machine learning
model stealing |
0.9 | 1 | 2025 | Exploring Query Efficient Data Generation Towards Data-Free Model Stealing in Hard Label Setting · AAAI 2025 |
Information retrieval › retrieval models
neural retrieval |
0.8 | 2 | 2021 | Contextualized Offline Relevance Weighting for Efficient and Effective Neural Retrieval · SIGIR 2021 NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval · EMNLP 2018 |
Methods — techniques the papers use, named apart from their topics
multi-task training · 2.0fine-grained score supervision · 2.0chain-of-thought · 2.0distributionally robust estimator · 1.5bayesian decision framework · 1.5BERT · 1.1multi-objective optimization · 1.0NSGA-II · 1.0single-step resampling · 0.9side adapter · 0.9segment anything model · 0.9relevance bias · 0.9reasoning-based knowledge injection · 0.9rank centrality · 0.9query-efficient learning · 0.9open-set RPN · 0.9hodgerank · 0.9heterogeneous noise injection · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | TFRank: Think-Free Reasoning Enables Practical Pointwise LLM RankingabstractReasoning-intensive ranking models built on Large Language Models (LLMs) have made notable progress. However, existing approaches often rely on large-scale LLMs and explicit Chain-of-Thought (CoT) reasoning, resulting in high computational cost and latency that limit real-world use. To address this, we propose TFRank, an efficient pointwise reasoning ranker based on small-scale LLMs. To improve ranking performance, TFRank effectively integrates CoT data, fine-grained score supervision, and multi-task training. Furthermore, it achieves an efficient "Think-Free" reasoning capability by employing a "think-mode switch" and pointwise format constraints. Specifically, this allows the model to leverage explicit reasoning during training while delivering precise relevance scores for complex queries at inference without generating any reasoning chains. Experiments show that TFRank achieves performance comparable to models with four times more parameters on the BRIGHT benchmark, and demonstrates strong competitiveness on the BEIR benchmark. Further analysis shows that TFRank achieves an effective balance between performance and efficiency, providing a practical solution for integrating advanced reasoning into real-world systems. Yongqi Fan, Xiaoyang Chen 0001, Dezhi Ye, Jie Liu 0075, Haijin Liang, Jin Ma 0003, Ben He 0001, Yingfei Sun, Tong Ruan |
AAAI | 8 |
| 2026 | Breaking the Spiral: A Utility-Driven Optimization Framework for Balanced Information Retrieval in the LLM EraabstractThe widespread adoption of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems is reshaping the landscape of information retrieval. However, the long-term effects of LLM-generated texts on retrieval systems remain underexplored, creating challenges for mitigating their impact. The effects are examined in this study, with a particular focus on the “Spiral of Silence” phenomenon, which refers to the marginalization of diverse information as certain types of content dominate, leading to a homogenized information ecosystem. To investigate this, a simulation pipeline is constructed to model the iterative introduction of LLM-generated texts into retrieval systems. Experimental results across multiple iterations reveal that as the presence of LLM-generated texts within the system grows, retrieval systems exhibit a stronger tendency to retrieve these texts. This trend, in turn, reduces the visibility of human-generated content, diminishes diversity, propagates errors, and results in a notable decline in retrieval performance. To address these challenges, we propose a Utility-Driven Multi-Objective Optimization (UMO) framework to effectively mitigate the “Spiral of Silence.” This framework employs a two-phase approach: an optimization phase, leveraging the NSGA-II algorithm to derive optimal preference weights for multiple objectives, and a memorization phase, which directly integrates these weights into the retrieval vector space without requiring additional model retraining. Experimental results demonstrate that this framework maintains stable retrieval effectiveness, improves the retrieval proportion of human-generated content, reduces the excessive influence of LLM-generated texts, and preserves information diversity, effectively mitigating the “Spiral of Silence.” Xiaoyang Chen 0001, Ben He 0001, Xianpei Han, Tianshu Wang 0002, Boxi Cao, Le Sun 0001, Yingfei Sun |
ACM Trans. Inf. Syst. | 8 |
| 2025 | Boosting Segment Anything Model Towards Open-Vocabulary LearningabstractThe recent Segment Anything Model (SAM) has emerged as a new paradigmatic vision foundation model, showcasing potent zero-shot generalization and flexible prompting. Despite SAM finding applications and adaptations in various domains, its primary limitation lies in the inability to grasp object semantics. In this paper, we present Sambor to seamlessly integrate SAM with the open-vocabulary object detector in an end-to-end framework. While retaining all the remarkable capabilities inherent to SAM, we boost it to detect arbitrary objects from human inputs like category names or reference expressions. Building upon the SAM image encoder, we introduce a novel SideFormer module designed to acquire SAM features adept at perceiving objects and inject comprehensive semantic information for recognition. In addition, we devise an Open-set RPN that leverages SAM proposals to assist in finding potential objects. Consequently, Sambor enables the open-vocabulary detector to equally focus on generalizing both localization and classification sub-tasks. Our approach demonstrates superior zero-shot performance across benchmarks, including COCO and LVIS, proving highly competitive against previous state-of-the-art methods. We aspire for this work to serve as a meaningful endeavor in endowing SAM to recognize diverse object categories and advancing open-vocabulary learning with the support of vision foundation models. Xumeng Han, Longhui Wei, Xuehui Yu, Zhiyang Dou, Kuiran Wang, Yingfei Sun, Zhenjun Han, Qi Tian 0001 |
AAAI | 7 |
| 2025 | Exploring Query Efficient Data Generation Towards Data-Free Model Stealing in Hard Label SettingabstractData-free model stealing involves replicating the functionality of a target model into a substitute model without accessing the target model's structure, parameters, or training data. Instead, the adversary can only access the target model's predictions for generated samples. Once the substitute model closely approximates the behavior of the target model, attackers can exploit its white-box characteristics for subsequent malicious activities, such as adversarial attacks. Existing methods within cooperative game frameworks often produce samples with high confidence for the prediction of the substitute model, which makes it difficult for the substitute model to replicate the behavior of the target model. This paper presents a new data-free model stealing approach called Query Efficient Data Generation (QEDG). We introduce two distinct loss functions to ensure the generation of sufficient samples that closely and uniformly align with the target model's decision boundary across multiple classes. Building on the limitation of current methods, which typically yield only one piece of supervised information per query, we propose the query-free sample augmentation that enables the acquisition of additional supervised information without increasing the number of queries. Motivated by theoretical analysis, we adopt the consistency rate metric, which more accurately evaluates the similarity between the substitute and target models. We conducted extensive experiments to verify the effectiveness of our proposed method, which achieved better performance with fewer queries compared to the state-of-the-art methods on the real MLaaS scenario and five datasets. Gaozheng Pei, Shaojie Lyu, Ke Ma 0001, Pinci Yang, Qianqian Xu 0001, Yingfei Sun |
AAAI | 6 |
| 2025 | Memorizing is Not Enough: Deep Knowledge Injection Through ReasoningabstractRuoxi Xu, Yunjie Ji, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Ben He, Yingfei Sun, Xiangang Li, Le Sun. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. Ruoxi Xu, Yunjie Ji, Boxi Cao, Yaojie Lu 0001, Xianpei Han, Ben He 0001, Yingfei Sun, Xiangang Li, Le Sun 0001 |
ACL (1) | 8 |
| 2025 | Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial PurificationabstractExisting diffusion-based purification methods aim to disrupt adversarial perturbations by introducing a certain amount of noise through a forward diffusion process, followed by a reverse process to recover clean examples. However, this approach is fundamentally flawed: the uniform operation of the forward process across all pixels compromises normal pixels while attempting to combat adversarial perturbations, resulting in the target model producing incorrect predictions. Simply relying on low-intensity noise is insufficient for effective defense. To address this critical issue, we implement a heterogeneous purification strategy grounded in the interpretability of neural networks. Our method decisively applies higher-intensity noise to specific pixels that the target model focuses on while the remaining pixels are subjected to only low-intensity noise. This requirement motivates us to redesign the sampling process of the diffusion model, allowing for the effective removal of varying noise levels. Furthermore, to evaluate our method against strong adaptative attack, our proposed method sharply reduces time cost and memory usage through a single-step resampling. The empirical evidence from extensive experiments across three datasets demonstrates that our method outperforms most current adversarial training and purification techniques by a substantial margin. Code is available at https://github.com/GaozhengPei/Purification. Gaozheng Pei, Shaojie Lyu, Ke Ma 0001, Qianqian Xu 0001, Yingfei Sun, Qingming Huang |
CVPR | 6 |
| 2025 | SAM-CP: Marrying SAM with Composable Prompts for Versatile SegmentationabstractThe Segment Anything model (SAM) has shown a generalized ability to group image pixels into patches, but applying it to semantic-aware segmentation still faces major challenges. This paper presents SAM-CP, a simple approach that establishes two types of composable prompts beyond SAM and composes them for versatile segmentation. Specifically, given a set of classes (in texts) and a set of SAM patches, the Type-I prompt judges whether a SAM patch aligns with a text label, and the Type-II prompt judges whether two SAM patches with the same text label also belong to the same instance. To decrease the complexity in dealing with a large number of semantic classes and patches, we establish a unified framework that calculates the affinity between (semantic and instance) queries and SAM patches, and then merges patches with high affinity to the query. Experiments show that SAM-CP achieves semantic, instance, and panoptic segmentation in both open and closed domains. In particular, it achieves state-of-the-art performance in open-vocabulary segmentation. Our research offers a novel and generalized methodology for equipping vision foundation models like SAM with multi-grained semantic perception abilities. Codes are released on https://github.com/ucas-vg/SAM-CP. Pengfei Chen 0004, Lingxi Xie, Xinyue Huo, Xuehui Yu, Xiaopeng Zhang 0008, Yingfei Sun, Zhenjun Han, Qi Tian 0001 |
ICLR | 6 |
| 2025 | Large Language Models Often Say One Thing and Do AnotherabstractAs large language models (LLMs) increasingly become central to various applications and interact with diverse user populations, ensuring their reliable and consistent performance is becoming more important. This paper explores a critical issue in assessing the reliability of LLMs: the consistency between their words and deeds. To quantitatively explore this consistency, we developed a novel evaluation benchmark called the Words and Deeds Consistency Test (WDCT). The benchmark establishes a strict correspondence between word-based and deed-based questions across different domains, including opinion vs. action, non-ethical value vs. action, ethical value vs. action, and theory vs. application. The evaluation results reveal a widespread inconsistency between words and deeds across different LLMs and domains. Subsequently, we conducted experiments with either word alignment or deed alignment to observe their impact on the other aspect. The experimental results indicate that alignment only on words or deeds poorly and unpredictably influences the other aspect. This supports our hypothesis that the underlying knowledge guiding LLMs' word or deed choices is not contained within a unified space. Dataset and code are available at [https://github.com/icip-cas/Word-Deed-Consistency-Test](https://github.com/icip-cas/Word-Deed-Consistency-Test). Ruoxi Xu, Xianpei Han, Jia Zheng 0009, Weixiang Zhou, Le Sun 0001, Yingfei Sun |
ICLR | 7 |
| 2025 | Diffusion-based Adversarial Purification from the Perspective of the Frequency DomainabstractThe diffusion-based adversarial purification methods attempt to drown adversarial perturbations into a part of isotropic noise through the forward process, and then recover the clean images through the reverse process. Due to the lack of distribution information about adversarial perturbations in the pixel domain, it is often unavoidable to damage normal semantics. We turn to the frequency domain perspective, decomposing the image into amplitude spectrum and phase spectrum. We find that for both spectra, the damage caused by adversarial perturbations tends to increase monotonically with frequency. This means that we can extract the content and structural information of the original clean sample from the frequency components that are less damaged. Meanwhile, theoretical analysis indicates that existing purification methods indiscriminately damage all frequency components, leading to excessive damage to the image. Therefore, we propose a purification method that can eliminate adversarial perturbations while maximizing the preservation of the content and structure of the original image. Specifically, at each time step during the reverse process, for the amplitude spectrum, we replace the low-frequency components of the estimated image’s amplitude spectrum with the corresponding parts of the adversarial image. For the phase spectrum, we project the phase of the estimated image into a designated range of the adversarial image’s phase spectrum, focusing on the low frequencies. Empirical evidence from extensive experiments demonstrates that our method significantly outperforms most current defense methods. Gaozheng Pei, Ke Ma 0001, Yingfei Sun, Qianqian Xu 0001, Qingming Huang |
ICML | 3 |
| 2025 | Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMsabstractDespite the remarkable performance of Large Language Models (LLMs), they remain vulnerable to jailbreak attacks, which can compromise their safety mechanisms. Existing studies often rely on brute-force optimization or manual design, failing to uncover potential risks in real-world scenarios. To address this, we propose a novel jailbreak attack framework, ICRT, inspired by heuristics and biases in human cognition. Leveraging the simplicity effect, we employ cognitive decomposition to reduce the complexity of malicious prompts. Simultaneously, relevance bias is utilized to reorganize prompts, enhancing semantic alignment and inducing harmful outputs effectively. Furthermore, we introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm by employing ranking aggregation methods such as Elo, HodgeRank, and Rank Centrality to comprehensively quantify the harmfulness of generated content. Experimental results show that our approach consistently bypasses mainstream LLMs’ safety mechanisms and generates high-risk content. Haoming Yang, Ke Ma 0001, Xiaojun Jia, Yingfei Sun, Qianqian Xu 0001, Qingming Huang |
ICML | 4 |
| 2025 | In-Orbit Assessment of the Synchronization Performance of LuTan-1 Bistatic SARabstractTime synchronization and phase synchronization are indispensable for spaceborne bi-/multi-static SAR systems. The LuTan-1 (LT-1) bistatic SAR system employs a time synchronization scheme combining pulse-per-second (PPS) signals with GNSS-disciplined oscillators (GDOs), along with a novel non-interrupted alternate pulse (NIAP) scheme for phase synchronization. Furthermore, the time synchronization accuracy can be further enhanced by applying the two-way time transfer (TWTT) technique based on synchronization data from the NIAP scheme. To accurately evaluate the synchronization performance of the LT-1 system during in-orbit operation, an assessment method based on the two-dimensional (2D) positional offset of calibration targets is proposed. The method compares the theoretical offsets derived from imaging geometry with the measured offsets extracted from monostatic and bistatic SAR images to determine the system’s time and frequency deviations. Using 16 repeat-pass acquisitions over the calibration field in Hami, Xinjiang, during LT-1’s bistatic operation phase, the in-orbit synchronization performance of LT-1 is demonstrated. The PPS+GDO scheme achieves time synchronization accuracy under 100 ns, while the TWTT algorithm enhances this metric by three orders of magnitude, which is better than 0.10 ns. After NIAP phase synchronization, the LT-1 system exhibits a frequency deviation of approximately −2.17 × 10−3Hz, corresponding to frequency stability of 2 × 10−12. Yonghua Cai, Zongbiao Chen, Yachao Wang, Bo Li 0129, Yuesheng Chen, Pingping Lu, Yingfei Sun, Robert Wang 0001 |
IEEE Trans. Geosci. Remote. Sens. | 7 |
| 2025 | ViMoE: An Empirical Study of Designing Vision Mixture-of-ExpertsabstractMixture-of-Experts (MoE) models embody the divide-and-conquer concept and are a promising approach for increasing model capacity, demonstrating excellent scalability across multiple domains. In this paper, we integrate the MoE structure into the classic Vision Transformer (ViT), naming it ViMoE, and explore the potential of applying MoE to vision through a comprehensive study on image classification and semantic segmentation. However, we observe that the performance is sensitive to the configuration of MoE layers, making it challenging to obtain optimal results without careful design. The underlying cause is that inappropriate MoE layers lead to unreliable routing and hinder experts from effectively acquiring helpful information. To address this, we introduce a shared expert to learn and capture common knowledge, serving as an effective way to construct a stable ViMoE. Furthermore, we demonstrate how to analyze expert routing behavior, revealing which MoE layers are capable of specializing in handling specific information and which are not. This provides guidance for retaining the critical layers while removing redundancies, thereby advancing ViMoE to be more efficient without sacrificing accuracy. We aspire for this work to offer new insights into the design of vision MoE models and provide valuable empirical guidance for future research. Xumeng Han, Longhui Wei, Zhiyang Dou, Yingfei Sun, Zhenjun Han, Qi Tian 0001 |
IEEE Trans. Image Process. | 4 |
| 2024 | Spiral of Silence: How is Large Language Model Killing Information Retrieval? - A Case Study on Open Domain Question AnsweringabstractXiaoyang Chen, Ben He, Hongyu Lin, Xianpei Han, Tianshu Wang, Boxi Cao, Le Sun, Yingfei Sun. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. Xiaoyang Chen 0001, Ben He 0001, Xianpei Han, Tianshu Wang 0002, Boxi Cao, Le Sun 0001, Yingfei Sun |
ACL (1) | 8 |
| 2024 | CLIP-Driven Low-Cost Image CaptioningabstractImage captioning, as a typical multi-modal task, has received increasing attention and has made significant progress. Recently, the proposal of CLIP has shown strong application potential in various tasks, including image captioning. However, the current approaches simply employ CLIP as a visual feature extractor without fully utilizing its potential benefits from the contrastive learning paradigm. Furthermore, most of them are resource-expensive due to the need for additional large-scale image-text datasets for incremental pre-training or additional pre-trained language models for captioning decoder initialization and finetuning. In this paper, we propose a simple and effective approach that fully utilizes CLIP’s properties and advantages to transfer it to an image captioning model. Specifically, following the obtained visual features, we design a memory retrieval module to introduce more information and design a joint representation module for feature enhancement. Furthermore, our captioning decoder is constructed based on the CLIP text encoder with the reuse of its parameters, which aims to leverage its priori textual embedding knowledge. Experimental results on MSCOCO dataset verify the effectiveness of our proposal. Jungang Xu, Yingfei Sun |
IJCNN | 3 |
| 2024 | Improving Image Captioning with Image Concepts of Words
Xunzhi Xiang, Kun Jing, Jungang Xu, Yingfei Sun |
KSEM (2) | 5 |
| 2024 | NLPCC2024 Shared Task 3 Technical Report
Yingfei Sun, Wei Ji 0008 |
NLPCC (5) | 1 |
| 2024 | AI for social science and social science of AI: A survey
Ruoxi Xu, Yingfei Sun, Mengjie Ren, Shiguang Guo, Ruotong Pan, Le Sun 0001, Xianpei Han |
Inf. Process. Manag. | 2 |
| 2024 | Sequential Manipulation Against Rank Aggregation: Theory and AlgorithmabstractRank aggregation with pairwise comparisons is widely encountered in sociology, politics, economics, psychology, sports, etc. Given the enormous social impact and the consequent incentives, the potential adversary has a strong motivation to manipulate the ranking list. However, the ideal attack opportunity and the excessive adversarial capability cause the existing methods to be impractical. To fully explore the potential risks, we leverage an online attack on the vulnerable data collection process. Since it is independent of rank aggregation and lacks effective protection mechanisms, we disrupt the data collection process by fabricating pairwise comparisons without knowledge of the future data or the true distribution. From the game-theoretic perspective, the confrontation scenario between the online manipulator and the ranker who takes control of the original data source is formulated as a distributionally robust game that deals with the uncertainty of knowledge. Then we demonstrate that the equilibrium in the above game is potentially favorable to the adversary by analyzing the vulnerability of the sampling algorithms such as Bernoulli and reservoir methods. According to the above theoretical analysis, different sequential manipulation policies are proposed under a Bayesian decision framework and a large class of parametric pairwise comparison models. For attackers with complete knowledge, we establish the asymptotic optimality of the proposed policies. To increase the success rate of the sequential manipulation with incomplete knowledge, a distributionally robust estimator, which replaces the maximum likelihood estimation in a saddle point problem, provides a conservative data generation solution. Finally, the corroborating empirical evidence shows that the proposed method manipulates the results of rank aggregation methods in a sequential manner. Ke Ma 0001, Qianqian Xu 0001, Jinshan Zeng, Wei Liu 0005, Xiaochun Cao, Yingfei Sun, Qingming Huang |
IEEE Trans. Pattern Anal. Mach. Intell. | 6 |
| 2024 | Enhancing Sample Utilization in Noise-Robust Deep Metric Learning With Subgroup-Based Positive-Pair SelectionabstractThe existence of noisy labels in real-world data negatively impacts the performance of deep learning models. Although much research effort has been devoted to improving the robustness towards noisy labels in classification tasks, the problem of noisy labels in deep metric learning (DML) remains under-explored. Existing noisy label learning methods designed for DML mainly discard suspicious noisy samples, resulting in a waste of the training data. To address this issue, we propose a noise-robust DML framework with SubGroup-based Positive-pair Selection (SGPS), which constructs reliable positive pairs for noisy samples to enhance the sample utilization. Specifically, SGPS first effectively identifies clean and noisy samples by a probability-based clean sample selectionstrategy. To further utilize the remaining noisy samples, we discover their potential similar samples based on the subgroup information given by a subgroup generation module and then aggregate them into informative positive prototypes for each noisy sample via a positive prototype generation module. Afterward, a new contrastive loss is tailored for the noisy samples with their selected positive pairs. SGPS can be easily integrated into the training process of existing pair-wise DML tasks, like image retrieval and face recognition. Extensive experiments on multiple synthetic and real-world large-scale label noise datasets demonstrate the effectiveness of our proposed method. Without any bells and whistles, our SGPS framework outperforms the state-of-the-art noisy label DML methods. Qianqian Xu 0001, Yangbangyan Jiang, Yingfei Sun, Qingming Huang |
IEEE Trans. Image Process. | 4 |
| 2024 | PARADE: Passage Representation Aggregation forDocument RerankingabstractPre-trained transformer models, such as BERT and T5, have shown to be highly effective at ad hoc passage and document ranking. Due to the inherent sequence length limits of these models, they need to process document passages one at a time rather than processing the entire document sequence at once. Although several approaches for aggregating passage-level signals into a document-level relevance score have been proposed, there has yet to be an extensive comparison of these techniques. In this work, we explore strategies for aggregating relevance signals from a document’s passages into a final ranking score. We find that passage representation aggregation techniques can significantly improve over score aggregation techniques proposed in prior work, such as taking the maximum passage score. We call this new approach PARADE. In particular, PARADE can significantly improve results on collections with broad information needs where relevance signals can be spread throughout the document (such as TREC Robust04 and GOV2). Meanwhile, less complex aggregation techniques may work better on collections with an information need that can often be pinpointed to a single passage (such as TREC DL and TREC Genomics). We also conduct efficiency analyses and highlight several strategies for improving transformer-based aggregation. Canjia Li, Andrew Yates, Sean MacAvaney, Ben He 0001, Yingfei Sun |
ACM Trans. Inf. Syst. | 5 |
| 2023 | Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay DetectionabstractLarge language models (LLMs) have exhibited remarkable capabilities in text generation tasks.However, the utilization of these models carries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues in educational exercises.Although several detectors have been proposed to address these concerns, their effectiveness against adversarial perturbations, specifically in the context of student essay writing, remains largely unexplored.This paper aims to bridge this gap by constructing AIG-ASAP, an AI-generated student essay dataset, employing a range of text perturbation methods that are expected to generate high-quality essays while evading detection.Through empirical experiments, we assess the performance of current AIGC detectors on the AIG-ASAP dataset.The results reveal that the existing detectors can be easily circumvented using straightforward automatic adversarial attacks.Specifically, we explore word substitution and sentence substitution perturbation methods that effectively evade detection while maintaining the quality of the generated essays.This highlights the urgent need for more accurate and robust methods to detect AI-generated student essays in the education domain.Code and data are released for public use 1 . Xinlin Peng, Ben He 0001, Le Sun 0001, Yingfei Sun |
EMNLP | 5 |
| 2023 | Dealing with textual noise for robust and effective BERT re-ranking
Xuanang Chen, Ben He 0001, Kai Hui 0001, Le Sun 0001, Yingfei Sun |
Inf. Process. Manag. | 5 |
| 2022 | End-to-End Transformer Based Model for Image CaptioningabstractCNN-LSTM based architectures have played an important role in image captioning, but limited by the training efficiency and expression ability, researchers began to explore the CNN-Transformer based models and achieved great success. Meanwhile, almost all recent works adopt Faster R-CNN as the backbone encoder to extract region-level features from given images. However, Faster R-CNN needs a pre-training on an additional dataset, which divides the image captioning task into two stages and limits its potential applications. In this paper, we build a pure Transformer-based model, which integrates image captioning into one stage and realizes end-to-end training. Firstly, we adopt SwinTransformer to replace Faster R-CNN as the backbone encoder to extract grid-level features from given images; Then, referring to Transformer, we build a refining encoder and a decoder. The refining encoder refines the grid features by capturing the intra-relationship between them, and the decoder decodes the refined features into captions word by word. Furthermore, in order to increase the interaction between multi-modal (vision and language) features to enhance the modeling capability, we calculate the mean pooling of grid features as the global feature, then introduce it into refining encoder to refine with grid features together, and add a pre-fusion process of refined global feature and generated words in decoder. To validate the effectiveness of our proposed model, we conduct experiments on MSCOCO dataset. The experimental results compared to existing published works demonstrate that our model achieves new state-of-the-art performances of 138.2% (single model) and 141.0% (ensemble of 4 models) CIDEr scores on 'Karpathy' offline test split and 136.0% (c5) and 138.3% (c40) CIDEr scores on the official online test server. Trained models and source code will be released. Jungang Xu, Yingfei Sun |
AAAI | 3 |
| 2022 | Towards Robust Dense Retrieval via Local Ranking AlignmentabstractDense retrieval (DR) has extended the employment of pre-trained language models, like BERT, for text ranking. However, recent studies have raised the robustness issue of DR model against query variations, like query with typos, along with non-trivial performance losses. Herein, we argue that it would be beneficial to allow the DR model to learn to align the relative positions of query-passage pairs in the representation space, as query variations cause the query vector to drift away from its original position, affecting the subsequent DR effectiveness. To this end, we propose RoDR, a novel robust DR model that learns to calibrate the in-batch local ranking of query variation to that of original query for the DR space alignment. Extensive experiments on MS MARCO and ANTIQUE datasets show that RoDR significantly improves the retrieval results on both the original queries and different types of query variations. Meanwhile, RoDR provides a general query noise-tolerate learning framework that boosts the robustness and effectiveness of various existing DR models. Our code and models are openly available at https://github.com/cxa-unique/RoDR. Xuanang Chen, Ben He 0001, Le Sun 0001, Yingfei Sun |
IJCAI | 5 |
| 2022 | A Graph Architecture Search Method Based On Grouped OperationsabstractGraph data is ubiquitous in the real world and graph neural networks (GNNs) are effective for modeling the complex relationships and dependencies between the entities. However, it's difficult to design data-specific GNNs. Recently, researchers have started to apply neural architecture search (NAS) to design GNNs. In this work, we propose a graph architecture search method to decrease the instability with a large number of candidate operations. Following SANE(Search to Aggregate NEighborhood), we focus on searching to aggregate neighbourhoods but we divide the candidate operations into groups. We use a continuous relaxation of our search space and optimize the hyper-networks with a gradient-based algorithm. Extensive experiments on several node-level and graph-level tasks demonstrate that our method achieves a promising performance. Luoyu Chen, Jungang Xu, Kun Jing, Yingfei Sun |
IJCNN | 4 |
| 2022 | A visual persistence model for image captioning
Jungang Xu, Yingfei Sun |
Neurocomputing | 3 |
| 2021 | Simplified TinyBERT: Knowledge Distillation for Document Retrieval
Xuanang Chen, Ben He 0001, Kai Hui 0001, Le Sun 0001, Yingfei Sun |
ECIR (2) | 5 |
| 2021 | A Light Ranker for Open-Domain Question AnsweringabstractOpen-domain question answering aims to extract answers from some unlabeled texts. Existing approaches usually follow the retrieve-then-read pipeline, which depends heavily on the quality of the retriever. Therefore, a ranker is significant to denoise those irrelevant paragraphs by giving each candidate paragraph a confidence score. In this paper, we design a light paragraph ranker, which utilizes an iterative approach to combine the paragraph-paragraph relevance and the paragraph-question relevance. Our ranker can fully mine global paragraph information and is lighter than that in most other Open-domain QA models. Furthermore, we explore two types of readers based on whether they sample candidate paragraphs or retain all. Experiments on public datasets show that our model achieves improvements compared to baselines. Boyu Qiu, Jungang Xu, Yingfei Sun |
IJCNN | 4 |
| 2021 | Hierarchical Dialogue State Tracking with Machine Reading ComprehensionabstractIn task-oriented dialogue systems, dialogue state tracking (DST) is responsible for estimating the current belief state of a dialogue. Recent research tends to utilize historical information to predict the states. However, most of them lack an efficient attention mechanism to comprehend the utterances well. Besides, existing methods usually ignore the relevance between the current utterances with the earlier ones, which determines whether or not the states change. In this paper, we propose an HDST-MRC (Hierarchical Dialogue State Tracking with Machine Reading Comprehension) model to tackle these issues. Within HSDT-MRC, we introduce bi-directional attention flow to extract a context span as the evidence of the ground truth and leverage another copy-augmented generator to predict the states. Experimental results on MultiWoz 2.0 and MultiWoz 2.1 demonstrate that our model achieves significant improvement compared with the baselines. Boyu Qiu, Jungang Xu, Yingfei Sun |
IJCNN | 3 |
| 2021 | Contextualized Offline Relevance Weighting for Efficient and Effective Neural RetrievalabstractOnline search latency is a major bottleneck in deploying large-scale pre-trained language models, e.g. BERT, in retrieval applications. Inspired by the recent advances in transformer-based document expansion technique, we propose to trade offline relevance weighting for online retrieval efficiency by utilizing the powerful BERT ranker to weight the neighbour documents collected by generated pseudo-queries for each document. In the online retrieval stage, the traditional query-document matching is reduced to the much less expensive query to pseudo-query matching, and a document rank list is quickly recalled according to the pre-computed neighbour documents. Extensive experiments on the standard MS MARCO dataset with both passage and document ranking tasks demonstrate promising results of our method in terms of both online efficiency and effectiveness. Xuanang Chen, Ben He 0001, Kai Hui 0001, Le Sun 0001, Yingfei Sun |
SIGIR | 6 |
| 2020 | An end-to-end pseudo relevance feedback framework for neural document retrieval
Ze Luo, Canjia Li, Ben He 0001, Le Sun 0001, Yingfei Sun |
Inf. Process. Manag. | 7 |
| 2019 | Upper Limb Muscle Force Estimation During Table Tennis StrokesabstractBased on an EMG-adjusted method in neuromusculoskeletal model, this study aims to predict the individual muscle force in shoulder and elbow during table tennis strokes. Muscle force estimation makes muscle activation analysis more physiological in sports. Twenty subjects, divided into professional group and amateur group, were adopted in this study. They were asked to do a basic stoke motion: backhand block. Surface electromyography (sEMG) of nine muscles was recorded, as well as the motion data collected by three inertial sensors. A Hill-type musculotendon model was then adopted to estimate individual muscle force by combining adjusted sEMG and motion data. The result shows that the method can estimate individual muscle force during table tennis strokes accurately, and the two groups show significant difference in muscle force of shoulders and elbows. Yingfei Sun, Zhipei Huang, Jian-Kang Wu, Zhiqiang Zhang 0001 |
BSN | 2 |
| 2019 | A Novel Image Captioning Method Based on Generative Adversarial Networks
Jungang Xu, Yingfei Sun |
ICANN (4) | 3 |
| 2018 | Neural Precision Medicine by Mining Implicit Treatment Concepts
Canjia Li, Ben He 0001, Le Sun 0001, Yingfei Sun |
BIBM | 4 |
| 2018 | NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information RetrievalabstractPseudo relevance feedback (PRF) is commonly used to boost the performance of traditional information retrieval (IR) models by using top-ranked documents to identify and weight new query terms, thereby reducing the effect of query-document vocabulary mismatches.While neural retrieval models have recently demonstrated strong results for adhoc retrieval, combining them with PRF is not straightforward due to incompatibilities between existing PRF approaches and neural architectures.To bridge this gap, we propose an end-to-end neural PRF framework that can be used with existing neural IR models by embedding different neural models as building blocks.Extensive experiments on two standard test collections confirm the effectiveness of the proposed NPRF framework in improving the performance of two state-of-theart neural IR models. Canjia Li, Yingfei Sun, Ben He 0001, Kai Hui 0001, Andrew Yates, Le Sun 0001, Jungang Xu |
EMNLP | 2 |
| 2018 | Long-Term Recurrent Merge Network Model for Image CaptioningabstractLanguage models based on Recurrent Neural Networks, e.g. Long Short Term Memory Network (LSTM), have shown strong ability in generating captions from image. However, in previous LSTM-based image captioning models, the image information is input to LSTM at 0th time step, and the network gradually forgets the image information, and only uses the language model to generate a simple description, leaving the potential in generating a better description. To address this challenge, in this paper, a Long-term Recurrent Merge Network (LRMN) model is proposed to merge the image feature at each step via a language model, which not only can improve the accuracy of image captioning, but also can describe the image better. Experimental results show that the proposed LRMN model has a promising improvement in image captioning. Jungang Xu, Yingfei Sun |
ICTAI | 3 |
| 2017 | An Adaptive Multilook Approach for Small Sets of Multitemporal SAR Data Based on Adaptive Joint Data VectorabstractThe multitemporal interferometric synthetic aperture radar (InSAR) technique is a potential tool for measuring digital elevation models and surface deformation. It has the advantage of high precision, competitive spatial resolution, and wide coverage. To improve the accuracy of the final results, some adaptive multilook strategies have been proposed in which the identification of statistically homogeneous pixels (SHPs) is the key task. However, these methods are not always reliable in the case of small data sets. To improve this reliability, SHPs are identified based on the adaptive joint data vector comprising of temporal sample and spatial information in this letter. Additionally, the formulation of adaptive joint data vector is combined with local spatial features of SAR images. The presented adaptive multilook approach can be used in many interferometric applications, such as InSAR data filtering and coherence estimation. Experiments on six TerraSAR-X stripmap images of Tianjin in China validate the feasibility and effectiveness of the proposed approach. Huina Song, Yingfei Sun, Robert Wang 0001, Ning Li 0002, Yingjie Wang 0008, Wenbo Fei |
IEEE Geosci. Remote. Sens. Lett. | 2 |
| 2016 | Phase estimation of distributed scatterer for high resolution data stacks in nonurban areasabstractIt has been proven that SqueeSAR technique has validated the potential to increase the density of measure points (MP) involving persistent scatter (PS) and distributed scatter (DS) candidate in nonurban areas. In SqueeSAR, DS candidate exhibiting high extended temporal coherence can be processed jointly with PS for deformation estimation after phase estimation by the phase triangulation algorithm (PTA). The PTA extracts phase values of DS by using all possible interferograms under the Gaussian scattering assumption. However, the statistics hypothesis, Gaussian random variable, is no more applicable, with radar resolution increasing. More precisely, it has been shown that clutter in high resolution SAR images can be modeled as a compound Gaussian process. In this letter, a modified approach to phase estimation of DS named MPTA is proposed based on the compound Gaussian model. Experiments on real data are presented to demonstrate the effectiveness of the proposed method. Huina Song, Yingfei Sun, Robert Wang 0001, Wenbo Fei, Yingjie Wang 0008, Jili Wang |
IGARSS | 2 |