Multimodal Retrieval-Augmented Generation (MRAG) has recently been explored to empower Large Vision Language Models (LVLMs) with more comprehensive and up-to-date contextual knowledge, aiming to compensate for their limited and coarse-grained parametric knowledge in knowledge-intensive tasks. However, the retrieved contextual knowledge is usually not aligned with LVLMs’ internal parametric knowledge, leading to knowledge conflicts and further unreliable responses. To tackle this issue, we design KCM, a training-free and plug-and-play framework that can effectively mitigate knowledge conflicts while incorporating MRAG for more accurate LVLM responses. KCM enhances contextual knowledge utilization by modifying the LVLM architecture from three key perspectives. First, KCM adaptively adjusts attention distributions among multiple attention heads, encouraging LVLMs to focus on contextual knowledge with reduced distraction. Second, KCM identifies and prunes knowledge-centric LVLM neurons that encode coarse-grained parametric knowledge, thereby suppressing interferences and enabling more effective integration of contextual knowledge. Third, KCM amplifies the information flow from the input context by injecting supplementary context logits, reinforcing its contribution to the final output. Extensive experiments over multiple LVLMs and benchmarks show that KCM outperforms the state-of-the-art consistently by large margins, incurring neither extra training nor external tools.

Sequential Recommender Systems (SRS), which model a user's interaction history to predict the next item of interest, are widely used in various applications. However, existing SRS often struggle with low-popularity items, a challenge known as the long-tail problem. This issue leads to reduced serendipity for users and diminished profits for sellers, ultimately harming the overall system. Large Language Model (LLM) has the ability to capture semantic relationships between items, independent of their popularity, making them a promising solution to this problem. In this paper, we introduce LLMEmb, a novel method leveraging LLM to generate item embeddings that enhance SRS performance. To bridge the gap between general-purpose LLM and the recommendation domain, we propose a Supervised Contrastive Fine-Tuning (SCFT) approach. This approach includes attribute-level data augmentation and a tailored contrastive loss to make LLM more recommendation-friendly. Additionally, we emphasize the importance of integrating collaborative signals into LLM-generated embeddings, for which we propose Recommendation Adaptation Training (RAT). This further refines the embeddings for optimal use in SRS. The LLMEmb-derived embeddings can be seamlessly integrated with any SRS model, underscoring the practical value. Comprehensive experiments conducted on three real-world datasets demonstrate that LLMEmb significantly outperforms existing methods across multiple SRS models.

Generalized Category Discovery is a significant and complex task that aims to identify both known and undefined novel categories from a set of unlabeled data, leveraging another labeled dataset containing only known categories. The primary challenges stem from model bias induced by pre-training on only known categories and the lack of precise supervision for novel ones, leading to category bias towards known categories and category confusion among different novel categories, which hinders models' ability to identify novel categories effectively. To address these challenges, we propose a novel framework named Self-Debiasing Calibration (SDC). Unlike prior methods that regard model bias towards known categories as an obstacle to novel category identification, SDC provides a novel insight into unleashing the potential of the bias to facilitate novel category learning. Specifically, we utilize the biased pre-trained model to guide the subsequent learning process on unlabeled data. The output of the biased model serves two key purposes. First, it provides an accurate modeling of category bias, which can be utilized to measure the degree of bias and debias the output of the current training model. Second, it offers valuable insights for distinguishing different novel categories by transferring knowledge between similar categories. Based on these insights, SDC dynamically adjusts the output logits of the current training model using the output of the biased model. This approach produces less biased logits to effectively address the issue of category bias towards known categories, and generates more accurate pseudo labels for unlabeled data, thereby mitigating category confusion for novel categories. Experiments on three benchmark datasets show that SDC outperforms SOTA methods, especially in the identification of novel categories.

Despite great success across various multimodal tasks, Large Vision-Language Models (LVLMs) often encounter object hallucinations with generated textual responses being inconsistent with the actual objects in images. We examine different LVLMs and pinpoint that one root cause of object hallucinations lies with deficient attention on discriminative image features. Specifically, LVLMs often predominantly attend to prompt-irrelevant global features instead of prompt-relevant local features, undermining their visual grounding capacity and leading to object hallucinations. We propose Assembly of Global and Local Attention (AGLA), a training-free and plug-and-play approach that mitigates hallucinations by assembling global features for response generation and local features for visual discrimination simultaneously. Specifically, we introduce an image-prompt matching scheme that captures prompt-relevant local features from images, leading to an augmented view of the input image where prompt-relevant content is highlighted while irrelevant distractions are suppressed. Hallucinations can thus be mitigated with a calibrated logit distribution that is from generative global features of the original image and discriminative local features of the augmented image. Extensive experiments show the superiority of AGLA in LVLM hallucination mitigation, demonstrating its wide applicability across both discriminative and generative tasks. Our code is available at https://github.com/Lackel/AGLA.

Dual-view gaze target estimation in classroom environments has not been thoroughly explored. Existing methods lack consideration of depth information, primarily focusing on 2D image information and neglecting the latent 3D spatial context, which could lead to suboptimal transformation and cause the gaze cone to intersect with an incorrect object. This paper introduces a novel dual-view gaze target estimation method tailored for classroom settings, leveraging depth-enhanced spatial transformations. By formulating a depth-enhanced 2D space, our method uses depth-enhanced spatial transformation to accurately project students’ gaze cones to the teacher-oriented image. Additionally, we collected a dataset named DVSGE, specifically for student gaze target estimation in dual-view classroom images. Experimental results demonstrate significant performance improvements of 9.8% in AUC and 19.9% in L2-Distance for our method, surpassing existing methods.

Previous methods often utilize CLIP semantic classifiers with class names for zero-shot object counting. However, they ignore crucial density and depth knowledge for counting tasks. Thus, we propose a density-aware and depth-aware prompt counting model, which captures density information via learning density-aware prompts based on density-aware contrastive loss and incorporates depth guidance with predefined depth-aware prompts. To facilitate the training process, we design two strategies for standard counting loss and the contrastive loss, where the former prioritizes larger and sparser objects initially, gradually focusing on smaller and denser objects, and the latter adopts coarse-to-fine density learning. Besides, we construct a dataset named LVIS-372 with more real-world scenarios and balanced instance distribution compared to existing ones. Finally, the experimental results demonstrate the effectiveness of our proposed method.

Current zero-shot human-object interaction detection methods often follow a two-phase pipeline, which uses a pre-trained detector to detect instances and then adopts CLIP to perform interaction prediction. During the second phase, they either obtain pairwise representations by directly performing RoI-Align on CLIP features or designing additional queries and decoders to fuse CLIP features. However, CLIP visual features often lack fine-grained information, thus being detrimental to capturing complex HOI interactions. Besides, extra decoders might increase computation costs. Thus, we propose a triple knowledge cues exploration model without extra decoders to explore various knowledge guidance for improving CLIP representations. First, we incorporate position distribution and semantic priors to delineate a layout from the predicted boxes and inject semantics by using the CLIP text embeddings. Next, we explore object priors by leveraging predefined class names and the text encoder to obtain saliency maps for humans and objects. Then, we design three types of holistic tokens to capture diverse attribute cues for human, object, and interaction, respectively. The above cues are finally integrated into a vanilla two-stage CLIP-based baseline. The experimental results on HICO-DET demonstrate the effectiveness of our proposed model.

Explainable Artificial Intelligence (XAI) for visual tasks remains a critical and challenging problem due to the lack of ground truth (GT) datasets, which limits the development of XAI evaluation methodologies. To address this issue, and inspired by two approaches, using either the model’s output data (including model predictions and interpretable outputs) or utilizing Large Vision Language Models (LVLMs) to generate pseudo-ground truth, we propose a GT-free XAI evaluation method to assess visual task interpretations without the need for ground truth. We have developed a framework called GT-free XAI, which provides unique XAI evaluation capabilities for visual task. Preliminary experimental results demonstrate that the GT-free XAI evaluation method correctly evaluates the interpretations of different XAI methods, reduces the workload associated with manual labeling, and opens new directions for XAI evaluation.

Due to exceptional reasoning and understanding abilities, the Large Language Model (LLM) has revolutionized the pattern of many fields, including recommender systems (RS). There has been a handful of research that focuses on empowering the RS by LLM. Recently, considering the latency and memory costs in real-world applications, LLM-Enhanced RS (LLMERS) is highlighted. This direction pushes the LLM into the online system with a large step by eliminating the utilization of LLM during inference. As a cutting-edge field, there is a clear need for a comprehensive survey to summarize this direction. In this survey, we systematically investigate the most up-to-date works of LLM-enhanced RS to boost this direction. Based on the component of an RS model that the LLM aims to augment, the basic taxonomy includes Knowledge Enhancement, Interaction Enhancement and Model Enhancement. Additionally, we identify several promising research directions. To facilitate access to the surveyed papers, we release a repository.

Despite their recent progress, Multimodal Large Language Models (MLLMs) often struggle in knowledge-intensive tasks due to the limited and outdated parametric knowledge acquired during training. Multimodal Retrieval Augmented Generation addresses this issue by retrieving contextual knowledge from external databases, thereby enhancing MLLMs with expanded knowledge sources. However, existing MLLMs often fail to fully leverage the retrieved contextual knowledge for response generation. We examine representative MLLMs and identify two major causes, namely, attention bias toward different tokens and knowledge conflicts between parametric and contextual knowledge. To this end, we design Adaptive Logits Fusion and Attention Reallocation (ALFAR), a training-free and plug-and-play approach that improves MLLM responses by maximizing the utility of the retrieved knowledge. Specifically, ALFAR tackles the challenges from two perspectives. First, it alleviates attention bias by adaptively shifting attention from visual tokens to relevant context tokens according to query-context relevance. Second, it decouples and weights parametric and contextual knowledge at output logits, mitigating conflicts between the two types of knowledge. As a plug-and-play method, ALFAR achieves superior performance across diverse datasets without requiring additional training or external tools. Extensive experiments over multiple MLLMs and benchmarks show that ALFAR consistently outperforms the state-of-the-art by large margins. Our code and data are available at https://github.com/Lackel/ALFAR.

Cross-domain Sequential Recommendation (CDSR) aims to extract the preference from the user's historical interactions across various domains. Despite some progress in CDSR, two problems set the barrier for further advancements, i.e., overlap dilemma and transition complexity. The former means existing CDSR methods severely rely on users who own interactions on all domains to learn cross-domain item relationships, compromising the practicability. The latter refers to the difficulties in learning the complex transition patterns from the mixed behavior sequences. With powerful representation and reasoning abilities, Large Language Models (LLMs) are promising to address these two problems by bridging the items and capturing the user's preferences from a semantic view. Therefore, we propose an LLMs Enhanced Cross-domain Sequential Recommendation model (LLM4CDSR). To obtain the semantic item relationships, we first propose an LLM-based unified representation module to represent items. Then, a trainable adapter with contrastive regularization is designed to adapt the CDSR task. Besides, a hierarchical LLMs profiling module is designed to summarize user cross-domain preferences. Finally, these two modules are integrated into the proposed tri-thread framework to derive recommendations. We have conducted extensive experiments on three public cross-domain datasets, validating the effectiveness of LLM4CDSR. We have released the code online.

Medication recommendation is one of the most critical health-related applications, which has attracted extensive research interest recently. Most existing works focus on a single hospital with abundant medical data. However, many small hospitals only have a few records, which hinders applying existing medication recommendation works to the real world. Thus, we seek to explore a more practical setting, i.e., multi-center medication recommendation. In this setting, most hospitals have few records, but the total number of records is large. Though small hospitals may benefit from total affluent records, it is also faced with the challenge that the data distributions between various hospitals are much different. In this work, we introduce a novel Contrastive Pretrain Model with Prompt Tuning (TEMPT) for multi-center medication recommendation, which includes two stages of pretraining and finetuning. We first design two self-supervised tasks for the pretraining stage to learn general medical knowledge. They are mask prediction and contrastive tasks, which extract the intra- and inter-relationships of input diagnosis and procedures. Furthermore, we devise a novel prompt tuning method to capture the specific information of each hospital rather than adopting the common finetuning. On the one hand, the proposed prompt tuning can better learn the heterogeneity of each hospital to fit various distributions. On the other hand, it can also relieve the catastrophic forgetting problem of finetuning. To validate the proposed model, we conduct extensive experiments on the public eICU, a multi-center medical dataset. The experimental results illustrate the effectiveness of our model. The implementation code is available to ease the reproducibility. 1

Generalized Category Discovery (GCD) is a crucial real-world task that aims to recognize both known and novel categories from an unlabeled dataset by leveraging another labeled dataset with only known categories. Despite the improved performance on known categories, current methods perform poorly on novel categories. We attribute the poor performance to two reasons: biased knowledge transfer between labeled and unlabeled data and noisy representation learning on the unlabeled data. The former leads to unreliable estimation of learning targets for novel categories and the latter hinders models from learning discriminative features. To mitigate these two issues, we propose a Transfer and Alignment Network (TAN), which incorporates two knowledge transfer mechanisms to calibrate the biased knowledge and two feature alignment mechanisms to learn discriminative features. Specifically, we model different categories with prototypes and transfer the prototypes in labeled data to correct model bias towards known categories. On the one hand, we pull instances with known categories in unlabeled data closer to these prototypes to form more compact clusters and avoid boundary overlap between known and novel categories. On the other hand, we use these prototypes to calibrate noisy prototypes estimated from unlabeled data based on category similarities, which allows for more accurate estimation of prototypes for novel categories that can be used as reliable learning targets later. After knowledge transfer, we further propose two feature alignment mechanisms to acquire both instance- and category-level knowledge from unlabeled data by aligning instance features with both augmented features and the calibrated prototypes, which can boost model performance on both known and novel categories with less noise. Experiments on three benchmark datasets show that our model outperforms SOTA methods, especially on novel categories. Theoretical analysis is provided for an in-depth understanding of our model in general. Our code and data are available at https://github.com/Lackel/TAN.

Generalized Category Discovery (GCD) aims to recognize both known and novel categories in an unlabeled dataset by leveraging another labeled dataset with only known categories. Without considering knowledge transfer from known to novel categories, current methods usually perform poorly on novel categories due to the lack of corresponding supervision. To mitigate this issue, we propose a unified Knowledge Transfer Network (KTN), which solves two obstacles to knowledge transfer in GCD. First, the mixture of known and novel categories in unlabeled data makes it difficult to identify transfer candidates (i.e., samples with novel categories). For this, we propose an entropy-based method that leverages knowledge in the pre-trained classifier to differentiate known and novel categories without requiring extra data or parameters. Second, the lack of prior knowledge of novel categories presents challenges in quantifying semantic relationships between categories to decide the transfer weights. For this, we model different categories with prototypes and treat their similarities as transfer weights to measure the semantic similarities between categories. On the basis of two treatments, we transfer knowledge from known to novel categories by conducting pre-adjustment of logits and post-adjustment of labels for transfer candidates based on the transfer weights between different categories. With the weighted adjustment, KTN can generate more accurate pseudo-labels for unlabeled data, which helps to learn more discriminative features and boost model performance on novel categories. Extensive experiments show that our method outperforms state-of-the-art models on all evaluation metrics across multiple benchmark datasets. Furthermore, different from previous clustering-based methods that can only work offline with abundant data, KTN can be deployed online conveniently with faster inference speed. Code and data are available at https://github.com/yibai-shi/KTN.

In this paper, we propose a novel task, Universal Category Discovery (UCD), to address the challenge of partial overlap between source and target domain categories. Different from previous tasks that assume all known categories exist in the target domain, UCD introduces "private-known" categories that only exist in the source domain and aims to classify unlabeled data as "common" or "novel" categories while avoiding misclassifying them into "private-known" categories. For this task, we propose a Tri-branch network with bidirectional Prototype-aware Matching (TriPM). TriPM effectively transfers knowledge from labeled to unlabeled data by bidirectionally matching similar data pairs, while a prototype matching strategy reduces the negative transfer risk from "private-known" categories. Finally, we propose a tri-branch network to decouple knowledge acquisition from labeled data, unlabeled data, and their interactions, which can avoid knowledge forgetting, explore novel patterns, and transfer common knowledge, respectively. Experiments demonstrate our model’s superiority over SOTA methods.

Text-guided diffusion models have significantly advanced image editing, enabling high-quality and diverse modifications driven by text prompts. However, effective editing requires inverting the source image into a latent space, a process often hindered by prediction errors inherent in DDIM inversion. These errors accumulate during the diffusion process, resulting in inferior content preservation and edit fidelity, especially with conditional inputs. We address these challenges by investigating the primary contributors to error accumulation in DDIM inversion and identify the singularity problem in traditional noise schedules as a key issue. To resolve this, we introduce the *Logistic Schedule*, a novel noise schedule designed to eliminate singularities, improve inversion stability, and provide a better noise space for image editing. This schedule reduces noise prediction errors, enabling more faithful editing that preserves the original content of the source image. Our approach requires no additional retraining and is compatible with various existing editing methods. Experiments across eight editing tasks demonstrate the Logistic Schedule's superior performance in content preservation and edit fidelity compared to traditional noise schedules, highlighting its adaptability and effectiveness. The project page is available at https://lonelvino.github.io/SYE/.

Recent advancements have shown promise in applying traditional Semi-Supervised Learning strategies to the task of Generalized Category Discovery (GCD). Typically, this involves a teacher-student framework in which the teacher imparts knowledge to the student to classify categories, even in the absence of explicit labels. Nevertheless, GCD presents unique challenges, particularly the absence of priors for new classes, which can lead to the teacher's misguidance and unsynchronized learning with the student, culminating in suboptimal outcomes. In our work, we delve into why traditional teacher-student designs falter in generalized category discovery as compared to their success in closed-world semi-supervised learning. We identify inconsistent pattern learning as the crux of this issue and introduce FlipClass—a method that dynamically updates the teacher to align with the student's attention, instead of maintaining a static teacher reference. Our teacher-attention-update strategy refines the teacher's focus based on student feedback, promoting consistent pattern recognition and synchronized learning across old and new classes. Extensive experiments on a spectrum of benchmarks affirm that FlipClass significantly surpasses contemporary GCD methods, establishing new standards for the field.

Sequential recommender systems (SRS) aim to predict users' subsequent choices based on their historical interactions and have found applications in diverse fields such as e-commerce and social media. However, in real-world systems, most users interact with only a handful of items, while the majority of items are seldom consumed. These two issues, known as the long-tail user and long-tail item challenges, often pose difficulties for existing SRS. These challenges can adversely affect user experience and seller benefits, making them crucial to address. Though a few works have addressed the challenges, they still struggle with the seesaw or noisy issues due to the intrinsic scarcity of interactions. The advancements in large language models (LLMs) present a promising solution to these problems from a semantic perspective. As one of the pioneers in this field, we propose the Large Language Models Enhancement framework for Sequential Recommendation (LLM-ESR). This framework utilizes semantic embeddings derived from LLMs to enhance SRS without adding extra inference load. To address the long-tail item challenge, we design a dual-view modeling framework that combines semantics from LLMs and collaborative signals from conventional SRS. For the long-tail user challenge, we propose a retrieval augmented self-distillation method to enhance user preference representation using more informative interactions from similar users. To verify the effectiveness and versatility of our proposed enhancement framework, we conduct extensive experiments on three real-world datasets using three popular SRS models. The results consistently show that our method surpasses existing baselines. The implementation code is available in Supplementary Material.

The recent surge in Large Language Models (LLMs) has garnered significant attention across numerous fields. Fine-tuning is often required to fit general LLMs for a specific domain, like the web-based healthcare system. However, two problems arise during fine-tuning LLMs for medical applications. One is the task variety problem, which involves distinct tasks in real-world medical scenarios. The variety often leads to sub-optimal fine-tuning for data imbalance and seesaw problems. Besides, the large amount of parameters in LLMs leads to huge time and computation consumption by fine-tuning. To address these two problems, we propose a novel parameter efficient fine-tuning framework for multi-task medical applications, dubbed as MOELoRA. The designed framework aims to absorb both the benefits of mixture-of-expert (MOE) for multi-task learning and low-rank adaptation (LoRA) for parameter efficient fine-tuning. For unifying MOE and LoRA, we devise multiple experts as the trainable parameters, where each expert consists of a pair of low-rank matrices to retain the small size of trainable parameters. Then, a task-motivated gate function for all MOELoRA layers is proposed, which can control the contributions of each expert and produce distinct parameters for various tasks. We conduct experiments on a multi-task medical dataset, indicating MOELoRA outperforms the existing parameter efficient fine-tuning methods. The code is available online.

Recently, video recognition is emerging with the help of multi-modal learning, which focuses on integrating distinct modalities to improve the performance or robustness of the model. Although various multi-modal learning methods have been proposed and offer remarkable recognition results, almost all of these methods rely on high-quality manual annotations and assume that modalities among multi-modal data provide semantically relevant information. Unfortunately, the widely used video datasets are usually coarse-annotated or collected from the Internet. Thus, it inevitably contains a portion of noisy labels and noisy correspondence. To address this challenge, we use the audio-visual action recognition task as a proxy and propose a noise-tolerant learning framework to find anti-interference model parameters against both noisy labels and noisy correspondence. Specifically, our method consists of two phases that aim to rectify noise by the inherent correlation between modalities. First, a noise-tolerant contrastive training phase is performed to make the model immune to the possible noisy-labeled data. Despite the benefits brought by contrastive training, it would overfit the noisy correspondence and thus provide false supervision. To alleviate the influence of noisy correspondence, we propose a cross-modal noise estimation component to adjust the consistency between different modalities. As the noisy correspondence existed at the instance level, we further propose a category-level contrastive loss to reduce its interference. Second, in the hybrid-supervised training phase, we calculate the distance metric among features to obtain corrected labels, which are used as complementary supervision to guide the training. Furthermore, due to the lack of suitable datasets, we establish a benchmark of real-world noisy correspondence in audio-visual data by relabeling the Kinetics dataset. Extensive experiments on a wide range of noisy levels demonstrate that our method significantly improves the robustness of the action recognition model and surpasses the baselines by a clear margin.

Generalized Category Discovery (GCD) aims to recognize both known and novel categories from a set of unlabeled data, based on another dataset labeled with only known categories. Without considering differences between known and novel categories, current methods learn about them in a coupled manner, which can hurt model's generalization and discriminative ability. Furthermore, the coupled training approach prevents these models transferring category-specific knowledge explicitly from labeled data to unlabeled data, which can lose high-level semantic information and impair model performance. To mitigate above limitations, we present a novel model called Decoupled Prototypical Network (DPN). By formulating a bipartite matching problem for category prototypes, DPN can not only decouple known and novel categories to achieve different training targets effectively, but also align known categories in labeled and unlabeled data to transfer category-specific knowledge explicitly and capture high-level semantics. Furthermore, DPN can learn more discriminative features for both known and novel categories through our proposed Semantic-aware Prototypical Learning (SPL). Besides capturing meaningful semantic information, SPL can also alleviate the noise of hard pseudo labels through semantic-weighted soft assignment. Extensive experiments show that DPN outperforms state-of-the-art models by a large margin on all evaluation metrics across multiple benchmark datasets. Code and data are available at https://github.com/Lackel/DPN.

Sequential recommendation (SRS) has become the technical foundation in many applications recently, which aims to recommend the next item based on the user's historical interactions. However, sequential recommendation often faces the problem of data sparsity, which widely exists in recommender systems. Besides, most users only interact with a few items, but existing SRS models often underperform these users. Such a problem, named the long-tail user problem, is still to be resolved. Data augmentation is a distinct way to alleviate these two problems, but they often need fabricated training strategies or are hindered by poor-quality generated interactions. To address these problems, we propose a Diffusion Augmentation for Sequential Recommendation (DiffuASR) for a higher quality generation. The augmented dataset by DiffuASR can be used to train the sequential recommendation models directly, free from complex training procedures. To make the best of the generation ability of the diffusion model, we first propose a diffusion-based pseudo sequence generation framework to fill the gap between image and sequence generation. Then, a sequential U-Net is designed to adapt the diffusion noise prediction model U-Net to the discrete sequence generation task. At last, we develop two guide strategies to assimilate the preference between generated and origin sequences. To validate the proposed DiffuASR, we conduct extensive experiments on three real-world datasets with three sequential recommendation models. The experimental results illustrate the effectiveness of DiffuASR. As far as we know, DiffuASR is one pioneer that introduce the diffusion model to the recommendation.The implementation code is available online.

Discovering fine-grained categories from coarsely labeled data is a practical and challenging task, which can bridge the gap between the demand for fine-grained analysis and the high annotation cost.Previous works mainly focus on instance-level discrimination to learn low-level features, but ignore semantic similarities between data, which may prevent these models learning compact cluster representations.In this paper, we propose Denoised Neighborhood Aggregation (DNA), a self-supervised framework that encodes semantic structures of data into the embedding space.Specifically, we retrieve k-nearest neighbors of a query as its positive keys to capture semantic similarities between data and then aggregate information from the neighbors to learn compact cluster representations, which can make fine-grained categories more separatable.However, the retrieved neighbors can be noisy and contain many false-positive keys, which can degrade the quality of learned embeddings.To cope with this challenge, we propose three principles to filter out these false neighbors for better representation learning.Furthermore, we theoretically justify that the learning objective of our framework is equivalent to a clustering loss, which can capture semantic similarities between data to form compact fine-grained clusters.Extensive experiments on three benchmark datasets show that our method can retrieve more accurate neighbors (21.31% accuracy improvement) and outperform state-of-the-art models by a large margin (average 9.96% improvement on three metrics).Our code and data are available at https://github.com/Lackel/DNA.

New Intent Discovery (NID) aims to recognize both new and known intents from unlabeled data with the aid of limited labeled data containing only known intents.Without considering structure relationships between samples, previous methods generate noisy supervisory signals which cannot strike a balance between quantity and quality, hindering the formation of new intent clusters and effective transfer of the pre-training knowledge.To mitigate this limitation, we propose a novel Diffusion Weighted Graph Framework (DWGF) to capture both semantic similarities and structure relationships inherent in data, enabling more sufficient and reliable supervisory signals.Specifically, for each sample, we diffuse neighborhood relationships along semantic paths guided by the nearest neighbors for multiple hops to characterize its local structure discriminately.Then, we sample its positive keys and weigh them based on semantic similarities and local structures for contrastive learning.During inference, we further propose Graph Smoothing Filter (GSF) to explicitly utilize the structure relationships to filter high-frequency noise embodied in semantically ambiguous samples on the cluster boundary.Extensive experiments show that our method outperforms state-of-the-art models on all evaluation metrics across multiple benchmark datasets.

Aspect-based sentiment analysis aims to predict sentiment polarities of given aspects in text. Most current approaches employ attention-based neural methods to capture semantic relationships between aspects and words in one sentence. However, these methods ignore the fact that sentences with the same aspect and sentiment polarity often share the structure and semantic information in a domain, which leads to lower model performance. To mitigate this problem, we propose a heterogeneous aspect graph neural network (HAGNN) to learn the structure and semantic knowledge from intersentence relationships. Our model is a heterogeneous graph neural network since it contains three different kinds of nodes: word nodes, aspect nodes, and sentence nodes. These nodes can pass structure and semantic information between each other and update their embeddings to improve the performance of our model. To the best of our knowledge, we are the first to use a heterogeneous graph to capture relationships between sentences and aspects. The experimental results on five public datasets show the effectiveness of our model outperforming some state-of-the-art models.

Novel category discovery aims at adapting models trained on known categories to novel categories.Previous works only focus on the scenario where known and novel categories are of the same granularity.In this paper, we investigate a new practical scenario called Fine-grained Category Discovery under Coarsegrained supervision (FCDC).FCDC aims at discovering fine-grained categories with only coarse-grained labeled data, which can adapt models to categories of different granularity from known ones and reduce significant labeling cost.It is also a challenging task since supervised training on coarse-grained categories tends to focus on inter-class distance (distance between coarse-grained classes) but ignore intra-class distance (distance between fine-grained sub-classes) which is essential for separating fine-grained categories.Considering most current methods cannot transfer knowledge from coarse-grained level to fine-grained level, we propose a hierarchical weighted self-contrastive network by building a novel weighted self-contrastive module and combining it with supervised learning in a hierarchical manner.Extensive experiments on public datasets show both effectiveness and efficiency of our model over compared methods.Code and data are available at https://github.com/Lackel/ Hierarchical_Weighted_SCL.

Facial Expression Recognition (FER) for various low-resolution images is an important task and need in applications of analyzing crowd scenes (station, classroom, etc.). Due to the discriminative feature loss caused by reduced resolution, classifying various low-resolution facial images into the right category is still a challenging task. In this work, we proposed a novel generative adversarial network-based feature level super-resolution method for robust facial expression recognition (FSR-FER), which can reduce the chance of privacy leaking without restoring high-resolution facial images. In particular, a pre-trained FER model was employed as a feature extractor, and a generator network G and a discriminator network D are trained with features extracted from low-resolution and corresponding high-resolution images. Generator network G tries to transform features of low-resolution images to more discriminative ones by making them closer to the ones of corresponding high-resolution images. For better classification performance, we also proposed an effective classification-aware loss reweighting strategy based on the classification probability calculated by a fixed FER model to make our model focus more on samples that are prone to misclassification. Experimental results on the Real-World Affective Faces (RAF) Database and Static Facial Expressions in the Wild (SFEW) 2.0 dataset demonstrate that our method achieves satisfying results on various down-sample factors with a single model and has better performance on low-resolution images compared with methods using image super-resolution and expression recognition separately.

This article focuses on the problem of deadlock for sequential automated manufacturing systems (AMSs) that allow for the general resource allocation and flexible routings. A class of Petri nets, systems of sequential systems with shared resources (S4PR), are used to model these considered AMSs. Our previous work has showed that deadlocks in S4PR are characterized by saturated perfect activity-circuit (PA-circuit). In this article, we divide all saturable PA-circuits into two categories: 1) dependent and 2) independent. An algorithm is proposed to compute all independent saturable PA-circuits. We prove that by adding a monitor for each independent PA-circuit to ensure that it is not saturated, all dependent PA-circuits cannot be saturated either and deadlocks in S4PR are successfully prevented. The presented method simplifies the structure of the deadlock controller without imposing tight constraints on the system. Finally, the proposed controller is illustrated by some examples.

Individual treatment effect (ITE) estimation is widely used in many essential fields, such as medical and education. But two problems, unknown counterfactual outcome and confounder, are the barriers for making a good ITE estimation. Although some representation learning methods based on potential outcome framework have been proposed to solve the problems, we find that most of previous works assume all features (also named covariate) of a unit are confounders. However, this assumption is not easy to become true, because instrument variables, adjustment variables and irrelevant variables can also be included in features. Therefore, this paper proposes a simple method to split covariates, and then a network, Split Covariate Representation Network (SCRNet), is mentioned, which is used to estimate ITE by different kinds of variables. Experiment results show that our method outperforms other state-of-arts methods on IHDP, a semi-synthetic dataset, and Jobs, a real-world dataset.

This paper presents a liveness analysis method for sequential automated manufacturing systems (AMSs), which can be modeled by a class of Petri nets named systems of sequential systems with shared resources (S4PR). We show that deadlocks in S4PR can be characterized by the saturation of its structural object named a perfect activity circuit (PA-circuits). Thus, S4PR is live if and only if no PA-circuits in it is saturated at all reachable states. A PA-circuits of an S4PR may not be saturated at any state; hence, we propose an integer linear program (ILP) to determine whether a PA-circuits can be saturated or not. Then an algorithm is proposed to compute the set of PA-circuits that may be saturated. This presented method nontrivially generalizes deadlock characterization and liveness condition of ordinary Petri nets to a broader class of nonordinary ones.

It is difficult for e-learners to make decisions on how to learn when they are facing with a large amount of learning resources, especially when they have to balance available limited learning time and multiple learning objectives in various learning scenarios. This research presented in this paper addresses this challenge by proposing a new multi-constraint learning path recommendation algorithm based on knowledge map . The main contributions of the paper are as follows. Firstly, two hypotheses on e-learners’ different learning path preferences for four different learning scenarios (initial learning, usual review, pre-exam learning and pre-exam review) are verified through questionnaire-based statistical analysis. Secondly, according to learning behavior characteristics of four types of the learning scenarios, a multi-constraint learning path recommendation model is proposed, in which the variables and their weighted coefficients considers different learning path preferences of the learners in different learning scenarios as well as learning resource organization and fragmented time. Thirdly, based on the proposed model and knowledge map , the design and implementation of a multi-constraint learning path recommendation algorithm is described. Finally, it is shown that the questionnaire results from over 110 e-learners verify the effectiveness of the proposed algorithm and show the similarity between the learners’ self-organized learning paths and the recommended learning paths.

Abstract It is a challenging task to discover information from a large amount of data in an open domain.1In this paper, an event network framework is proposed to address this challenge. It is in fact an empirical construct for exploring open information, composed of three steps: document event detection, event network construction and event network analysis. First, documents are clustered into document events for reducing the impact of noisy and heterogeneous resources. Secondly, linguistic units (e.g., named entities or entity relations) are extracted from each document event and combined into an event network, which enables content-oriented retrieval. Then, in the final step, techniques such as social network or complex network can be applied to analyze the event network for exploring open information. In the implementation section, we provide examples of exploring open information via event network.

There is evidence that an increasing number of enterprises plot together to evade tax in an unperceived way. At the same time, the taxation information related data is a classic kind of big data. These issues challenge the effectiveness of traditional data mining-based tax evasion detection methods. To address this problem, we first investigate the classic tax evasion cases, and employ a graph-based method to characterize their property that describes two suspicious relationship trails with a same antecedent node behind an Interest-Affiliated Transaction (IAT). Next, we propose a Colored Network-Based Model (CNBM) for characterizing economic behaviors, social relationships and the IATs between taxpayers, and generating a Taxpayer Interest Interacted Network (TPIIN). To accomplish the tax evasion detection task by discovering suspicious groups in a TPIIN, methods for building a patterns tree and matching component patterns are introduced and the completeness of the methods based on graph theory is presented. Then, we describe an experiment based on real data and a simulated network. The experimental results show that our proposed method greatly improves the efficiency of tax evasion detection, as well as provides a clear explanation of the tax evasion behaviors of taxpayer groups.

Ranking similar structure groups/subgraphs can contribute to importantly practical applications, such as finding the most suspicious tax evasion group within a heterogeneous information network. This problem, however lacks attentions in the current research community. Inspired by social network analysis, this paper coins a definition of a taxpayer interest interacted network (TPIIN), in which persons or companies act as nodes as well as the arcs act as the relationships between persons and/or companies, and the weight of an arc is equal to the interest affiliated degree (IAD) of a direct tie or link according to various economic behaviors between persons and/or companies. Moreover, we adopted four operations, Maximum, Minimum, Addition, and Multiply, to calculate the IAD of a trail, then propose a voting algorithm based on the four operations to compare the suspicion degrees of all groups containing the same trading relationship arc. Experiments are carried out and their results verify the proposed method.

There is evidence that an increasing number of enterprises plot together to evade tax in an unperceived way. At the same time, the taxation information related data is a classic kind of big data. These issues challenge the effectiveness of traditional data mining-based tax evasion detection methods. To address this problem, we first investigate the classic tax evasion cases, and employ a graph-based method to characterize their property that describes two suspicious relationship trails with a same antecedent node behind an Interest-Affiliated Transaction (IAT). Next, we propose a Colored Network-Based Model (CNBM) for characterizing economic behaviors, social relationships, and the IATs between taxpayers, and generating a Taxpayer Interest Interacted Network (TPIIN). To accomplish the tax evasion detection task by discovering suspicious groups in a TPIIN, methods for building a patterns tree and matching component patterns are introduced and the completeness of the methods based on graph theory is presented. Then, we describe an experiment based on real data and a simulated network. The experimental results show that our proposed method greatly improves the efficiency of tax evasion detection, as well as provides a clear explanation of the tax evasion behaviors of taxpayer groups.

Cloud computing is a promising technology having ability to influence the way of the provision of computing and storage resources through virtual machine (VM). VM Consolidation is an efficient way to improve power efficiency and quality guarantee for on-demand services. However, it is an integer programming problem and as well as a NP-hard problem to find optimal solutions within polynomial time. In this paper, the VM consolidation problem is formulated as a multi-objective optimization problem, which has three conflicting objectives, i.e., reducing power consumption, achieving good load balancing and shortening VM migration time. We propose a multi-objective optimization algorithm based on biogeography-based optimization (BBO) for the VM consolidation problem, which is named as MBBO/DE: Multi-objective Biogeography-Based Optimization algorithm hybrid with Differential Evolution. It utilizes cosine migration model, differential strategies and Gaussian mutation model to improve the quality of habitats and the ability of finding optimal solutions. Experiments have been conducted to evaluate the effectiveness of MBBO/DE using synthetic and real-world instances. Experimental results show that MBBO/DE obtains a better performance while simultaneously reducing power consumption and achieving good load balancing within a satisfactory time as compared to genetic algorithm (GA), differential evolution (DE), ant colony optimization (ACO) and BBO.

Topic detection is a hot topic in the field of text mining. In this paper, focusing on the Chinese interactive text, we explored a novel topic detection method, named SDD-PLSA, which integrates Semantic Dependency Distance (SDD) and PLSA. It not only has the advantages of PLSA, which is an efficient, effective method and is widely used in text mining, but also considers the semantic and syntax information. Thus, the problem of lacking semantic information in PLSA can be avoided. SDD-PLSA has two main steps. The first is using SDD to classify the sentences that have a high similarity in semantics into several groups according to semantic feature extraction of the interactive text. Then, a PLSA classifier is used upon the result of the first step. The experiments show that the accuracy of detection on `love' topic has been improved to 64.8% when using SDD-PLSA, better than 55.4% when using PLSA.

Technology develops rapidly and information floods. In the information explosion era, what people lack is not the scale of information but how to obtain the needed information quickly and accurately. Personalized search and service emerges. And the key problem is to make clear the needs of the users. In this paper, we combined user interest and collaborative filtering to reorder the search results and implemented the approach using multi-agent technology. Finally, we set up an experiment to check the effect of this method.

To solve the problem of emotional illiteracy in current e-Learning environment, researches on sentiment analysis now get more attentions. This paper focuses on recognizing emotion from interactive Chinese texts (ICTs). Through observation, firstly, characteristics of ICTs are discussed. Then two kinds of feature sets, frequency based feature set and interaction related feature set, are presented. Finally, the corresponding feature extraction and selection for ICTs are presented. To validate the feature sets and choose the best method of sentiment analysis, we carry out a number of experiments. The experiments' results show that, combining with syntax based feature set, frequency based feature set and interaction related feature set can improve algorithm classification performance, and multi-class classifier and the tree based methods perform better than others.

An architecture of a group oriented intelligent tutoring system for rocket engineers and researchers is proposed, which combines the technologies of multi-agent and virtual reality with HLA. At the same time, a group oriented joint intention is introduced into the description of multi-agent cooperation in order to teach or guide trainers/beginners to learn right operation procedures or resolve faults. Then, an intelligent tutoring system for measuring and controlling rocket launch is implemented to verify the proposed method, which operates well at Jiuquan Satellite Launch Center (JSLC).

Aiming at compensating the lack of affect interaction between teachers and students in e-Learning systems/environments, this paper presents an architecture of interactive text-oriented affect compensation Mechanism in e-Learning. Based on which, a hierarchical emotion regulation agent (ERA) is proposed to perceive the emotion state of an individual or group from sentences, make decisions on how to adjust their emotions in a way of on-line discussion group, and control the compensation behaviors. In which, inspired by the strategies that have been widely used in the fields of psychology and sociology, the emotion regulation polices are designed for textual interaction applications. Moreover, two curves for emotion intensity change over time are introduced into predict the emotion intensity for individual and when a group effort takes, respectively. Finally, an emotion regulation chatroom is designed and implemented to validate and verify the proposed model and mechanism. Different from the video- and audio-data based emotion classification, our method is suitable for interactive texts that widely exist in e- Learning. The emotion regulation strategies can especially benefit the interacting sides in the text-based communication.

E-learning provides an unprecedented flexibility and convenience for learners via breaking the limitation of space-time. Most researchers are only concerned about the learner's cognitive and construct a great amount of substantive digital learning resources, however they neglect of the learners' affect in current e-learning systems. In this paper, we focus primarily on the negative affect of learners, and propose an emotion compensation mechanism associated with trust and personality traits in traditional recommender technology. First, we analyze the difference between emotion compensation and traditional recommender. Next, the score of trust is calculated with historical behavior, otherwise depend on similarity of personality traits without historical experience. We use trustworthiness to replace similarity as prediction weight in trust filtering process. At last we do experiments with data collected in previous system named emotion-chatting. Compared with results of experiments between traditional recommender and trust-personality recommender, the average of accuracy is improved 4 points in percentage.

To help teachers know class/group members better in the case of large scale on-line textual interaction, this paper tried to display e-Learner's emotion combined with topics and group structure. For achieving this goal, a color palette of emotions based on Plutchik's color palette was presented, an extended cascaded PLSI algorithm using sliding window technique was proposed to detect and track topics in Chinese interactive texts, and multiple star-field variants were introduced to display the group structure.

The resource-transition circuit ( RTC) and siphon are two different structural objects of Petri nets and used to develop deadlock control policies for automated manufacturing systems. They are related to the liveness property of Petri net models and thus used to characterize and avoid deadlocks. Based on them, there are two kinds of methods for developing deadlock controllers. Such methods rely on the computation of all maximal perfect RTCs and strict minimal siphons (SMSs), respectively. This paper concentrates on a class of Petri nets called a system of simple sequential processes with resources, establishes the relation between two kinds of control methods, and identifies maximal perfect RTCs and SMSs. A graph-based technique is used to find all elementary RTC structures. They are then used to derive all RTCs. Next, an iterative method is developed to recursively construct all maximal perfect RTCs from elementary ones. Finally, a one-to-one correspondence between SMSs and maximal perfect RTCs and, hence, an equivalence between two deadlock control methods are established.

There is lack of researches on discovering the emotion communication patterns in turn level. It is believe that can give a new view to build emotion computing model and improve the emotion prediction performance for interactive texts based applications. In this paper, a hierarchical emotion category is given. Then a hierarchical sequential pattern mining method based on SPAM is proposed and used to mining the emotion communication patterns in turn level of Chinese interactive texts. Some interesting sequences are found and discussed.

Large cross-language lexicons involving hybrid terms emerge in various fields like e-learning educational materials. Some of their common properties make many traditional lexicon management mechanisms less efficient. In this paper, we propose a new method for the cross-language lexicon based on collaborative work of hash family and double-array trie and a prototype is implemented. Our experiments on four lexicons of different styles show that the presented method demonstrates a better overall performance on not only cross-language lexicons but also some traditional lexicons.

Aiming at emotional illiteracy in current e-learning environment, this paper discussed the reason why this situation happens. A research and application framework of recognizing emotion based text interaction is presented. An emotion category model for e-Learners is defined. Many Chinese metaphors are abstracted from corpus according to the sentence semantic and syntax. A method is applied to identify e-learner emotion. Good results are achieved in our initial experiment. A case based reasoning sentiment adjustment strategy recommendation is proposed to enhance emotion communication between two interacted sides.

As technology advanced, e-learning becomes increasingly important and popular in our society. Determining how to care about the e-learnerpsilas emotional state is a vital issue. Our research concerns offering a better service to e-learners and making their experience in online learning and communication enjoyable. In this paper, which is based on literature search, we find the main emotional categories; then bring forward a novel discrete-dimensions duality emotion (DDE) model for e-learners and present the 3 characteristics of it. Finally, through statistic emotional words from the 2000 topics of BBS, we prove that the DDE model is reasonable. All those are to serve as the basis for our subsequent studies.

Even for a simple automated manufacturing system (AMS), such as a general single-unit resource allocation system, the computation of an optimal or maximally permissive deadlock-avoidance policy (DAP) is NP-hard. Based on its Petri-net model, this paper addresses the deadlock-avoidance problem in AMSs, which can be modeled by systems of simple sequential processes with resources. First, deadlock is characterized as a perfect resource-transition circuit that is saturated at a reachable state. Second, for AMSs that do not have one-unit resources shared by two or more perfect resource-transition circuits that do not contain each other, it is proved that there are only two kinds of reachable states: safe states and deadlock. An algorithm for determining the safety of a new state resulting from a safe one is then presented, which has polynomial complexity. Hence, the optimal DAP with polynomial complexity can be obtained by a one-step look-ahead method, and the deadlock-avoidance problem is polynomially solved with Petri nets for the first time. Finally, by reducing a Petri-net model and applying the design of optimal DAP to the reduced one, a suboptimal DAP for a general AMS is synthesized, and its computation is of polynomial complexity.

Many clustering methods have been adopted by personalized e-learning system to find interested groups or common characteristics of members within the same group. However, hard boundary during discretization on collected data or subjective influences was introduced, and corresponding methods were utilized. Aiming at this problem, a fuzzy clustering method based on fuzzy statistic is proposed to cluster the learners according to their personality and learning strategy data collected from an online system. Then, an analysis method based on frequent pattern is introduced to testify the result of the proposed unsupervised clustering methods. The clustering results correspond with viewpoints of pedagogy.

With the rapid development of e-learning, stand-alone e-learning information and resource management systems show disadvantages obviously in integration and collaboration. To address the problem, this paper proposes a service-oriented e-learning reference architecture (SOELA), and a federate prototype system SOELS by using SOELA, Web services and portlet. Particularly, the network model and interaction model for collaboration of SOELS are introduced. SOELS verifies the effectiveness of SOELA and meets the integration requirements on e-learning information and resource management systems.

In the last few years, several Web-based conference management systems have been developed and used by many international conferences. However, almost all of them were built on stand-alone Web servers. Their fault-tolerance, scalability and ability of responding to dispersed users are limited. Aimed at addressing these problems, this paper presents a collaborative conference management system, whose fault-tolerance, scalability and ability of responding to dispersed users are greatly enhanced by collaboration technologies. A prototype system has been implemented and used to facilitate the management of submissions and paper reviews of the CSCWD2008 conference.

This paper focuses on the design and implementation of a domain ontology-based navigation learning system, which can guide the user to learn more efficiently. Aiming at improving users' conceptual understanding of course material, a navigation structure was built among concepts, and each concept is presumed to have at least one concept instance which is provided by a Web answer system based on natural language which was integrated in this system. Then users would browse through the linked resources under those particular concepts as well as related concepts and get a better understanding of the material in the course. The kernel of such function was a domain knowledge base which was composed of concepts and relationship between them. Ontology was introduced as a mechanism to guide the establishment of this base. At last, a method of extracting concepts and relationship between them by using machine learning algorithms were presented.

Networked education is developing increasingly toward the direction of personalization. Inspired by the personality analysis process based on a learner model, a method for customizing personalized learning strategies was presented. Its process is as follows, firstly, aiming at the weakness of the personality description method and the lack of effective personality analysis method in current researches of learning strategies personalization, a learner personality model which can support learner personality analysis by static and dynamic methods in an intelligent e-learning environment was proposed. Secondly, a personality analysis process using data mining technologies based on the learner model was introduced. Finally, a prototype system of learning strategies personalization was designed and implemented according to the learner model and process. The experiment results showed the process's availability and functionality, such as acquiring learners' personalities by static and dynamic methods, and customizing individual learning strategies. Keywords:

The construction of learner model is one of the important aspects when designing personalized distance learning/training system. Most of which usually take the cognitive aspects of the learning into account only, for examples, show the right content, correct mistakes and provide explanations. However, besides cognition, learning strategies are increasingly recognized as the important aspect for personalized learning/training, and correlate with learner's individual differences, which include personality traits, learning styles and conceptions etc. In this paper, the relationships between learners' individual differences and learning strategies are investigated by using two different methods - the rough set theory based method and the correlation analysis method to analyze the same data set and compare the results. At last, some conclusions are made. The results help to construct an optimized learner model in a personalized distance learning/training environment.

Learner grouping is a key step to build both personalized e-learning system and adaptive cooperative learning environment. Clustering analysis has been widely adopted in many researches, while the validity assessments of clustering results were largely ignored. In the study, validity assessment for e-learner clustering was emphasized and a new assessing index based on label information was proposed. Experiment results on the real dataset indicated that precise and reliable learner partitions could be obtained by using clustering validation indices. In addition, by visualizing the distribution of labeled clusters, we confirmed the underlying hypothesis of learning strategies intelligent recommendation that learners with similar personality would be likely to employ similar learning strategies.

The temporal uncertainty of activities and limited resources are important factors for successful operation and performance evaluation on collaborative design. In the paper a method of modelling well-define collaboration process based on workflow technology, probability theory and fuzzy-timing high-level Petri nets is presented. Then while an approach for identifying the fuzzy critical path is given, a policy of solving resource conflict and resource scheduling with limited resources is introduced. These approaches offer theoretic basis and available means of time management, resource scheduling and conflict solving in collaborative work processes. Finally, the validity of those proposed methods is proven by an example

At present, it is difficult to handle unexpected exceptions in collaborative design. Ad-hoc methods are important ones, but how to utilize the expert experience efficiently, especially, store those in structured format and not to lose any information and interact under more user-friendly interface during handling exceptions, seem a mission impossible. Aiming at those, we proposed a question answering mechanism based approach for handling unexpected exceptions in collaborative design, which have some good characteristics, 1) unexpected exception handling is treated as a question answering mechanism so as to be more user-friendly, 2) in which exception handler can use natural language to define an exception; 3) expert experience are stored in natural language, there is no information loss; 4) higher efficiency, extendability and precision. In this paper, two approaches for mining similar records are proposed, and their performance comparison is shown