Large language models (LLMs) have shown outstanding performance across a variety of tasks, partly due to advanced prompting techniques. However, these techniques often require lengthy prompts, which increase computational costs and can hinder performance because of the limited context windows of LLMs. While prompt compression is a straightforward solution, existing methods confront the challenges of retaining essential information, adapting to context changes, and remaining effective across different tasks. To tackle these issues, we propose a task-agnostic method called Dynamic Prompt Compression (LLM-DPC). Our method reduces the number of prompt tokens while minimizing any degradation in LLM performance. We model prompt compression as a Markov Decision Process (MDP), enabling the DPC-Agent to sequentially remove redundant tokens by adapting to dynamic contexts and retaining crucial content. We develop a reward function for training the DPC-Agent that balances the compression ratio, the quality of the LLM output, and the retention of key information. This allows for prompt token reduction without needing an external black-box LLM. Inspired by the progressive difficulty adjustment in curriculum learning, we introduce a Hierarchical Prompt Compression (HPC) training strategy that gradually increases the compression difficulty, enabling the DPC-Agent to learn an effective compression method that maintains information integrity. Experiments demonstrate that our method outperforms state-of-the-art techniques, especially at higher compression ratio.

Open-domain dialogue systems aim to generate natural and engaging conversations, providing significant practical value in real applications such as social robotics and personal assistants. The advent of large language models (LLMs) has greatly advanced this field by improving context understanding and conversational fluency. However, existing LLM-based dialogue systems often fall short in proactively understanding the user's chatting preferences and guiding conversations toward user-centered topics. This lack of user-oriented proactivity can lead users to feel unappreciated, reducing their satisfaction and willingness to continue the conversation in human-computer interactions. To address this issue, we propose a User-oriented Proactive Chatbot (UPC) to enhance the user-oriented proactivity. Specifically, we first construct a critic to evaluate this proactivity inspired by the LLM-as-a-judge strategy. Given the scarcity of high-quality training data, we then employ the critic to guide dialogues between the chatbot and user agents, generating a corpus with enhanced user-oriented proactivity. To ensure the diversity of the user backgrounds, we introduce the ISCO-800, a diverse user background dataset for constructing user agents. Moreover, considering the communication difficulty varies among users, we propose an iterative curriculum learning method that trains the chatbot from easy-to-communicate users to more challenging ones, thereby gradually enhancing its performance. Experiments demonstrate that our proposed training method is applicable to different LLMs, improving user-oriented proactivity and attractiveness in open-domain dialogues. Code and appendix are available at github.com/wang678/LLM-UPC.

LLMs have demonstrated impressive performance across various language tasks. However, the strengths of LLMs can vary due to different architectures, model sizes, areas of training data, etc. Therefore, ensemble reasoning for the strengths of different LLM experts is critical to achieving consistent and satisfactory performance on diverse inputs across a wide range of tasks. However, existing LLM ensemble methods are either computationally intensive or incapable of leveraging complementary knowledge among LLM experts for various inputs. In this paper, we propose an efficient Dynamic Ensemble Reasoning paradigm, called DER to integrate the strengths of multiple LLM experts conditioned on dynamic inputs. Specifically, we model the LLM ensemble reasoning problem as a Markov Decision Process, wherein an agent sequentially takes inputs to request knowledge from an LLM candidate and passes the output to a subsequent LLM candidate. Moreover, we devise a reward function to train a DER-Agent to dynamically select an optimal answering route given the input questions, aiming to achieve the highest performance with as few computational resources as possible. Last, to fully transfer the expert knowledge from the prior LLMs, we develop a Knowledge Transfer Prompt that enables the subsequent LLM candidates to transfer complementary knowledge effectively. Experiments demonstrate that our method uses fewer computational resources to achieve better performance compared to state-of-the-art baselines. Code and appendix are available at https://github.com/Fhujinwu/DER.

Qianyue Wang, Jinwu Hu, Zhengping Li, Yufeng Wang, Daiyuan Li, Yu Hu, Mingkui Tan. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Traditional clustering methods rely on pairwise affinity to divide samples into different subgroups. However, high-dimensional small-sample (HDLSS) data are affected by the concentration effects, rendering traditional pairwise metrics unable to accurately describe relationships between samples, leading to suboptimal clustering results. This article advances the proposition of employing high-order affinities to characterize multiple sample relationships as a strategic means to circumnavigate the concentration effects. We establish a nexus between different order affinities by constructing specialized decomposable high-order affinities, thereby formulating a uniform mathematical framework. Building upon this insight, a novel clustering method named uniform tensor clustering (UTC) is proposed, which learns a consensus low-dimensional embedding for clustering by the synergistic exploitation of multiple-order affinities. Extensive experiments on synthetic and real-world datasets demonstrate two findings: 1) high-order affinities are better suited for characterizing sample relationships in complex data and 2) reasonable use of different order affinities can enhance clustering effectiveness, especially in handling high-dimensional data.

Tensor spectral clustering (TSC) is a recently proposed approach to robustly group data into underlying clusters. Unlike the traditional spectral clustering (SC), which merely uses pairwise similarities of data in an affinity matrix, TSC aims at exploring their multiwise similarities in an affinity tensor to achieve better performance. However, the performance of TSC highly relies on the design of multiwise similarities, and it remains unclear especially for high-dimension-low-sample-size (HDLSS) data. To this end, this article has proposed a discriminating TSC (DTSC) for HDLSS data. Specifically, DTSC uses the proposed discriminating affinity tensor that encodes the pair-to-pair similarities, which are particularly constructed by the anchor-based distance. HDLSS asymptotic analysis shows that the proposed affinity tensor can explicitly differentiate samples from different clusters when the feature dimension is large. This theoretical property allows DTSC to improve the clustering performance on HDLSS data. Experimental results on synthetic and benchmark datasets demonstrate the effectiveness and robustness of the proposed method in comparison to several baseline methods.

Tensor spectral clustering (TSC) is an emerging approach that explores multi-wise similarities to boost learning. However, two key challenges have yet to be well addressed in the existing TSC methods: (1) The construction and storage of high-order affinity tensors to encode the multi-wise similarities are memory-intensive and hampers their applicability, and (2) they mostly employ a two-stage approach that integrates multiple affinity tensors of different orders to learn a consensus tensor spectral embedding, thus often leading to a suboptimal clustering result. To this end, this paper proposes a tensor spectral clustering network (TSC-Net) to achieve one-stage learning of a consensus tensor spectral embedding, while reducing the memory cost. TSC-Net employs a deep neural network that learns to map the input samples to the consensus tensor spectral embedding, guided by a TSC objective with multiple affinity tensors. It uses stochastic optimization to calculate a small part of the affinity tensors, thereby avoiding loading the whole affinity tensors for computation, thus significantly reducing the memory cost. Through using an ensemble of multiple affinity tensors, the TSC can dramatically improve clustering performance. Empirical studies on benchmark datasets demonstrate that TSC-Net outperforms the recent baseline methods.

Unsupervised graph embedding aims to extract highly discriminative node representations that facilitate the subsequent analysis. Converging evidence shows that a multiview graph provides a more comprehensive relationship between nodes than a single-view graph to capture the intrinsic topology. However, little attention has been paid to excavating discriminative representations of each node from multiview heterogeneous networks in an unsupervised manner. To that end, we propose a novel unsupervised multiview graph embedding method, called multiview deep graph infomax (MVDGI). The backbone of our proposed model sought to maximize the mutual information between the view-dependent node representations and the fused unified representation via contrastive learning. Specifically, the MVDGI first uses an encoder to extract view-dependent node representations from each single-view graph. Next, an aggregator is applied to fuse the view-dependent node representations into the view-independent node representations. Finally, a discriminator is adopted to extract highly discriminative representations via contrastive learning. Extensive experiments demonstrate that the MVDGI achieves better performance than the benchmark methods on five real-world datasets, indicating that the obtained node representations by our proposed approach are more discriminative than by its competitors for classification and clustering tasks.

Multi-view clustering aims at integrating information from different views to improve clustering performance. Recent methods integrate multiple view-specific partition matrices to seek a consensus one and have demonstrated promising clustering performance in various applications. However, the clustering performance of such methods heavily relies on the consensus partition matrix estimated by the arithmetic mean in euclidean space and thus is highly susceptible to noise corruption. To this end, this article proposes to learn a consensus partition matrix through the geometric mean on the manifold to achieve robust clustering. Specifically, the multiple view-specific partition matrices can be regarded as points residing in the Stiefel manifold and enable a manifold-based integration. Consequently, the view-specific partition matrices are integrated by estimating a consensus partition matrix as the center point on the Stiefel manifold. Such a partition integration boils down to the Fréchet mean problem on a manifold, which is solved by the intrinsic manifold-based optimization and proves effective in providing a more robust estimation against noise. Experimental results on seven benchmark datasets demonstrate the effectiveness and noise-robustness of our proposed method in comparison to eight competitive methods.

Multi-graph clustering aims at integrating complementary information across multiple graphs to partition multi-view data into underlying clusters. Most current methods rely on pairwise graphs to characterize each view and then employ popular Euclidean averaging to integrate multiple graphs. How-ever, operations of the pairwise graphs on Euclidean space result in insufficient robustness to noise. To address the issue, we propose a method called multi-hypergraph clustering on the Stiefel manifold. First, a hypergraph for each view is constructed to extract high-order relations, which are more resistant to the noise than pairwise graphs. Second, a consensus partition matrix is derived through integrating the multiple hypergraphs on the Stiefel manifold. Such integration is completely driven by the manifold-based operation and enables an effective fusion to mitigate noise contamination, thus improving multi-view clustering performance. Empirical evaluations on five benchmark datasets have demonstrated that our method achieves consistent performance improvement compared with six baseline methods.

The performance of most clustering methods hinges on the used pairwise affinity, which is usually denoted by a similarity matrix. However, the pairwise similarity is notoriously known for its vulnerability of noise contamination or the imbalance in samples or features, and thus hinders accurate clustering. To tackle this issue, we propose to use information among samples to boost the clustering performance. We proved that a simplified similarity for pairs, denoted by a fourth order tensor, equals to the Kronecker product of pairwise similarity matrices under decomposable assumption, or provide complementary information for which the pairwise similarity missed under indecomposable assumption. Then a high order similarity matrix is obtained from the tensor similarity via eigenvalue decomposition. The high order similarity capturing spatial information serves as a robust complement for the pairwise similarity. It is further integrated with the popular pairwise similarity, named by IPS2, to boost the clustering performance. Extensive experiments demonstrated that the proposed IPS2 significantly outperformed previous similarity-based methods on real-world datasets and it was capable of handling the clustering task over under-sampled and noisy datasets.

Phase recovery (PR) of a signal from its amplitude measurements is one challenging task in signal processing. The key is suppressing the noise while rectifying the phase of the signal during the inversion process. This letter proposes a deep model-aware approach for PR by unrolling an optimization model regularized with image priors defined in the complex domain. A complex-valued (CV) deep neural network is then introduced to implement effective plug-and-play image priors that enjoy the benefits of CV operations for PR, such as sophisticated operations on local phases and regularization by compact convolution. As a result, the proposed approach can handle the noise well at each iteration in the unrolled process and improve the recovery accuracy. In experiments, the proposed approach shows superior performance to recent methods.

Multiple modality clustering seeks to partition objects via leveraging cross-modality relations to provide comprehensive descriptions of the same objects. Current clustering methods rely heavily on accurate affinity measurements among samples. The samplewise affinity is costive to be constructed yet easy to corrupt by the heterogeneous gap. In the era of big data, fast and accurate clustering of multiple modality data remains challenging. To fill the gap, we propose a novel approach to achieve the clustering by focusing on feature matching across different modalities instead of samplewise affinity. First, a feature matching matrix is calculated by measuring the potential featurewise correlations. The obtained matching matrix is decomposed into two bases corresponding to the column and row spaces of feature matching, acting as coded bases within feature spaces of the different modalities. Then, the sample assignment is obtained by jointly reconstructing the samples by the two bases. The feature matching potential and sample assignment are collaboratively learned by an alternating optimization scheme. The proposed method dramatically reduces the computational cost by avoiding the costive samplewise affinity estimation, without sacrificing accuracy. Extensive experiments on the synthetic and real-world datasets demonstrate its superior speed and high accuracy.

Clustering objects remains challenges in seeking an under-lying partition by exploiting multiple views. Popular clustering algorithms focus on designing various constraints to handle particular representation tasks, all of which rely on a predefined pairwise similarity (sample-to-sample). However, the pairwise similarity is notoriously vulnerable to noise or outliers contaminations, resulting in sub-optimal clustering performances. To tackle the issue, this paper proposes to enhance multi-view clustering by exploring varieties of high-order statistics within multi-view data, named by HIgh-order Similarity and essential Tensor clustering method (HIST). The HIST incorporates both high-order similarity (samples-to-samples) and high-order correlation (view-to-view) into an adaptive learning model to comprehensively exploit the inherent clustering structure. Experimental results on six real datasets show the superiority of our approach over the ten popular methods.

The human retinal surface resembles to a sphere while it is captured by two-dimensional (2D) planar imaging to have a stereo sequence in clinical practice. Reconstructing its three-dimensional (3D) structure from the 2D planar retinal images is crucial for analyzing the relationship between the topological morphology and clinical implication. In this regard, we propose to reconstruct the 3D retina structure from 2D stereo fundus images via dynamic registration. The fundus images from different viewpoints are first co-registrated by using multi-scale deep convolutional feature and geometric structure feature by building their transformation function. The aligned images are then mosaicked together and a 3D reconstruction is obtained by a learned weighted smoothing project the registered images onto 3D coordinates. We compare the proposed registration method with five state-of-the-art methods. Extensive experimental results demonstrate that the proposed framework achieves superior performances, even with challenging scenarios in which the tested images are severely degraded by illness, large eyeball rotation and low resolutions.

COVID-19 causes burdens to the ICU. Evidence-based planning and optimal allocation of the scarce ICU resources is urgently needed but remains unaddressed. This study aims to identify variables and test the accuracy to predict the need for ICU admission, death despite ICU care, and among survivors, length of ICU stay, before patients were admitted to ICU. Retrospective data from 733 in-patients confirmed with COVD-19 in Wuhan, China, as of March 18, 2020. Demographic, clinical and laboratory were collected and analyzed using machine learning to build the predictive models. The built machine learning model can accurately assess ICU admission, length of ICU stay, and mortality in COVID-19 patients toward optimal allocation of ICU resources. The prediction can be done by using the clinical data collected within 1-15 days before the actual ICU admission. Lymphocyte absolute value involved in all prediction tasks with a higher AUC. The online predictive system is freely available to the public (http://212.64.70.65:8000/).

Accurate nasopharyngeal carcinoma (NPC) segmentation in magnetic resonance image (MRI) is crucial for diagnosis and treatment. However, most existing deep learning methods performed unsatisfactorily, since NPC is infiltrative and typically has a small or even tiny volume with indistinguishable boundary, making it indiscernible from tightly connected surrounding tissue in immense and complex background. To address the background dominant problem, this paper proposes a coarse-to-fine deep model. The proposed model starts with predicting a coarse mask with a well-designed segmentation module, followed by a boundary rendering module, which exploits semantic information from different layers of feature maps to refine the boundary of the coarse mask. The designed rendering module is shown to achieve superior performance with dramatically fewer parameters by operating only on the segmented mask, rather than on the whole feature maps as the popular methods do. Extensive experiments are conducted on a collected dataset consisting of 2000 MRI slices from 596 patients. Experimental results demonstrate that the proposed model not only outperforms six popular segmentation models but also has a considerable generalization capability on existing models.

Multi-view clustering aims to partition the data into their underlying clusters via leveraging multiple views information. To exploit cross-view information, existed approaches in tensor-based subspace learning attract much attention. In order to explore essential tensor, the most recent work mainly focuses on capturing representation tensor with sparse and low-rank constraints. However, one shortcoming is that this process may suffer from instability since it did not consider retaining local structure between samples. To tackle the issue, we introduce a novel self-expressive tensor learning method considering both global and local constraints to promote the learning of representation tensor. In particular, we construct a tensor-based subspace representation that joint low-rank and graph-regularized tensor learning to a united optimization problem. The essential global structure and high-order correlations can be naturally captured through low-rank self-expressive tensor learning. Meanwhile, the local structures can be preserved by introducing graph regularized terms on representation tensor, thus bring benefits to subsequent clustering task. An effective optimization procedure for solving the proposed model is presented. We conduct extensive experiments on text, object, and gene expression datasets. The experimental results well demonstrate that the proposed method, named by TLGRL, achieves superiority over benchmark methods.