Current zero-shot human-object interaction detection methods often follow a two-phase pipeline, which uses a pre-trained detector to detect instances and then adopts CLIP to perform interaction prediction. During the second phase, they either obtain pairwise representations by directly performing RoI-Align on CLIP features or designing additional queries and decoders to fuse CLIP features. However, CLIP visual features often lack fine-grained information, thus being detrimental to capturing complex HOI interactions. Besides, extra decoders might increase computation costs. Thus, we propose a triple knowledge cues exploration model without extra decoders to explore various knowledge guidance for improving CLIP representations. First, we incorporate position distribution and semantic priors to delineate a layout from the predicted boxes and inject semantics by using the CLIP text embeddings. Next, we explore object priors by leveraging predefined class names and the text encoder to obtain saliency maps for humans and objects. Then, we design three types of holistic tokens to capture diverse attribute cues for human, object, and interaction, respectively. The above cues are finally integrated into a vanilla two-stage CLIP-based baseline. The experimental results on HICO-DET demonstrate the effectiveness of our proposed model.

Generalized Category Discovery (GCD) is a crucial real-world task that aims to recognize both known and novel categories from an unlabeled dataset by leveraging another labeled dataset with only known categories. Despite the improved performance on known categories, current methods perform poorly on novel categories. We attribute the poor performance to two reasons: biased knowledge transfer between labeled and unlabeled data and noisy representation learning on the unlabeled data. The former leads to unreliable estimation of learning targets for novel categories and the latter hinders models from learning discriminative features. To mitigate these two issues, we propose a Transfer and Alignment Network (TAN), which incorporates two knowledge transfer mechanisms to calibrate the biased knowledge and two feature alignment mechanisms to learn discriminative features. Specifically, we model different categories with prototypes and transfer the prototypes in labeled data to correct model bias towards known categories. On the one hand, we pull instances with known categories in unlabeled data closer to these prototypes to form more compact clusters and avoid boundary overlap between known and novel categories. On the other hand, we use these prototypes to calibrate noisy prototypes estimated from unlabeled data based on category similarities, which allows for more accurate estimation of prototypes for novel categories that can be used as reliable learning targets later. After knowledge transfer, we further propose two feature alignment mechanisms to acquire both instance- and category-level knowledge from unlabeled data by aligning instance features with both augmented features and the calibrated prototypes, which can boost model performance on both known and novel categories with less noise. Experiments on three benchmark datasets show that our model outperforms SOTA methods, especially on novel categories. Theoretical analysis is provided for an in-depth understanding of our model in general. Our code and data are available at https://github.com/Lackel/TAN.

Generalized Category Discovery (GCD) aims to recognize both known and novel categories in an unlabeled dataset by leveraging another labeled dataset with only known categories. Without considering knowledge transfer from known to novel categories, current methods usually perform poorly on novel categories due to the lack of corresponding supervision. To mitigate this issue, we propose a unified Knowledge Transfer Network (KTN), which solves two obstacles to knowledge transfer in GCD. First, the mixture of known and novel categories in unlabeled data makes it difficult to identify transfer candidates (i.e., samples with novel categories). For this, we propose an entropy-based method that leverages knowledge in the pre-trained classifier to differentiate known and novel categories without requiring extra data or parameters. Second, the lack of prior knowledge of novel categories presents challenges in quantifying semantic relationships between categories to decide the transfer weights. For this, we model different categories with prototypes and treat their similarities as transfer weights to measure the semantic similarities between categories. On the basis of two treatments, we transfer knowledge from known to novel categories by conducting pre-adjustment of logits and post-adjustment of labels for transfer candidates based on the transfer weights between different categories. With the weighted adjustment, KTN can generate more accurate pseudo-labels for unlabeled data, which helps to learn more discriminative features and boost model performance on novel categories. Extensive experiments show that our method outperforms state-of-the-art models on all evaluation metrics across multiple benchmark datasets. Furthermore, different from previous clustering-based methods that can only work offline with abundant data, KTN can be deployed online conveniently with faster inference speed. Code and data are available at https://github.com/yibai-shi/KTN.

In this paper, we propose a novel task, Universal Category Discovery (UCD), to address the challenge of partial overlap between source and target domain categories. Different from previous tasks that assume all known categories exist in the target domain, UCD introduces "private-known" categories that only exist in the source domain and aims to classify unlabeled data as "common" or "novel" categories while avoiding misclassifying them into "private-known" categories. For this task, we propose a Tri-branch network with bidirectional Prototype-aware Matching (TriPM). TriPM effectively transfers knowledge from labeled to unlabeled data by bidirectionally matching similar data pairs, while a prototype matching strategy reduces the negative transfer risk from "private-known" categories. Finally, we propose a tri-branch network to decouple knowledge acquisition from labeled data, unlabeled data, and their interactions, which can avoid knowledge forgetting, explore novel patterns, and transfer common knowledge, respectively. Experiments demonstrate our model’s superiority over SOTA methods.

Text-guided diffusion models have significantly advanced image editing, enabling high-quality and diverse modifications driven by text prompts. However, effective editing requires inverting the source image into a latent space, a process often hindered by prediction errors inherent in DDIM inversion. These errors accumulate during the diffusion process, resulting in inferior content preservation and edit fidelity, especially with conditional inputs. We address these challenges by investigating the primary contributors to error accumulation in DDIM inversion and identify the singularity problem in traditional noise schedules as a key issue. To resolve this, we introduce the *Logistic Schedule*, a novel noise schedule designed to eliminate singularities, improve inversion stability, and provide a better noise space for image editing. This schedule reduces noise prediction errors, enabling more faithful editing that preserves the original content of the source image. Our approach requires no additional retraining and is compatible with various existing editing methods. Experiments across eight editing tasks demonstrate the Logistic Schedule's superior performance in content preservation and edit fidelity compared to traditional noise schedules, highlighting its adaptability and effectiveness. The project page is available at https://lonelvino.github.io/SYE/.

Recent advancements have shown promise in applying traditional Semi-Supervised Learning strategies to the task of Generalized Category Discovery (GCD). Typically, this involves a teacher-student framework in which the teacher imparts knowledge to the student to classify categories, even in the absence of explicit labels. Nevertheless, GCD presents unique challenges, particularly the absence of priors for new classes, which can lead to the teacher's misguidance and unsynchronized learning with the student, culminating in suboptimal outcomes. In our work, we delve into why traditional teacher-student designs falter in generalized category discovery as compared to their success in closed-world semi-supervised learning. We identify inconsistent pattern learning as the crux of this issue and introduce FlipClass—a method that dynamically updates the teacher to align with the student's attention, instead of maintaining a static teacher reference. Our teacher-attention-update strategy refines the teacher's focus based on student feedback, promoting consistent pattern recognition and synchronized learning across old and new classes. Extensive experiments on a spectrum of benchmarks affirm that FlipClass significantly surpasses contemporary GCD methods, establishing new standards for the field.

Recently, video recognition is emerging with the help of multi-modal learning, which focuses on integrating distinct modalities to improve the performance or robustness of the model. Although various multi-modal learning methods have been proposed and offer remarkable recognition results, almost all of these methods rely on high-quality manual annotations and assume that modalities among multi-modal data provide semantically relevant information. Unfortunately, the widely used video datasets are usually coarse-annotated or collected from the Internet. Thus, it inevitably contains a portion of noisy labels and noisy correspondence. To address this challenge, we use the audio-visual action recognition task as a proxy and propose a noise-tolerant learning framework to find anti-interference model parameters against both noisy labels and noisy correspondence. Specifically, our method consists of two phases that aim to rectify noise by the inherent correlation between modalities. First, a noise-tolerant contrastive training phase is performed to make the model immune to the possible noisy-labeled data. Despite the benefits brought by contrastive training, it would overfit the noisy correspondence and thus provide false supervision. To alleviate the influence of noisy correspondence, we propose a cross-modal noise estimation component to adjust the consistency between different modalities. As the noisy correspondence existed at the instance level, we further propose a category-level contrastive loss to reduce its interference. Second, in the hybrid-supervised training phase, we calculate the distance metric among features to obtain corrected labels, which are used as complementary supervision to guide the training. Furthermore, due to the lack of suitable datasets, we establish a benchmark of real-world noisy correspondence in audio-visual data by relabeling the Kinetics dataset. Extensive experiments on a wide range of noisy levels demonstrate that our method significantly improves the robustness of the action recognition model and surpasses the baselines by a clear margin.

Discovering fine-grained categories from coarsely labeled data is a practical and challenging task, which can bridge the gap between the demand for fine-grained analysis and the high annotation cost.Previous works mainly focus on instance-level discrimination to learn low-level features, but ignore semantic similarities between data, which may prevent these models learning compact cluster representations.In this paper, we propose Denoised Neighborhood Aggregation (DNA), a self-supervised framework that encodes semantic structures of data into the embedding space.Specifically, we retrieve k-nearest neighbors of a query as its positive keys to capture semantic similarities between data and then aggregate information from the neighbors to learn compact cluster representations, which can make fine-grained categories more separatable.However, the retrieved neighbors can be noisy and contain many false-positive keys, which can degrade the quality of learned embeddings.To cope with this challenge, we propose three principles to filter out these false neighbors for better representation learning.Furthermore, we theoretically justify that the learning objective of our framework is equivalent to a clustering loss, which can capture semantic similarities between data to form compact fine-grained clusters.Extensive experiments on three benchmark datasets show that our method can retrieve more accurate neighbors (21.31% accuracy improvement) and outperform state-of-the-art models by a large margin (average 9.96% improvement on three metrics).Our code and data are available at https://github.com/Lackel/DNA.

Tax evasion refers to the illegal act of taxpayers using deception and concealment to avoid paying taxes. How to detect tax evasion effectively is always an important topic for the government and academic researchers. Recent research has proposed using machine learning technologies to detect tax evasion and has achieved good results in some specific conditions. However, recent methods have three shortcomings. First, recent methods mainly use the basic features extracted based on expert experience. Second, recent methods do not make full use of the edge features of the transaction network. Third, recent methods cannot adapt to a dynamic transaction network. To overcome these challenges, we propose a novel tax evasion detection method, the temporal edge enhanced graph attention network (T-EGAT), which combines the edge enhanced graph attention network (EGAT) and the recurrent weighted average unit (RWA). Specifically, the EGAT is used to learn complex topological structures for capturing spatial dependence and the RWA is used to learn the dynamic changes of transaction data for capturing temporal dependence. Experimental tests using real-world tax data demonstrate that our method achieves better performance at detecting tax evaders than existing methods.

The curse of dimensionality is a fundamental difficulty in anomaly detection for high dimensional data. To deal with this problem, the autoencoder based approach is an elegant solution. However, existing works require a clean training dataset that is not always guaranteed in real scenarios. In this paper, we propose a novel anomaly detection method named RVAE-ABFA (robust variational autoencoder with attention based feature adaptation for high dimensional data anomaly detection), which significantly improves the anomaly detection performance when training data is contaminated. Rather than only utilize reconstruction error, we take the learned low dimensional embeddings generated by variational autoencoder into consideration. In RVAE-ABFA, the learned low dimensional embeddings are helpful to detect anomalies in contaminated data because of the ability of variational inference. We also propose an ABFA (attention based feature adaptation) mechanism to adjust the weights of low dimensional embeddings and reconstruction error. Furthermore, we adopt the adversarial training criterion to perform variational inference by the adversarial network named RAAE-ABFA (robust adversarial autoencoder with attention based feature adaptation for high dimensional data anomaly detection) in which we can generate extra samples when training data is not enough. Experimental results on several benchmark datasets show that the proposed method significantly outperforms state-of-the-art unsupervised anomaly detection methods and is more robust when training data is contaminated.

It is difficult for e-learners to make decisions on how to learn when they are facing with a large amount of learning resources, especially when they have to balance available limited learning time and multiple learning objectives in various learning scenarios. This research presented in this paper addresses this challenge by proposing a new multi-constraint learning path recommendation algorithm based on knowledge map . The main contributions of the paper are as follows. Firstly, two hypotheses on e-learners’ different learning path preferences for four different learning scenarios (initial learning, usual review, pre-exam learning and pre-exam review) are verified through questionnaire-based statistical analysis. Secondly, according to learning behavior characteristics of four types of the learning scenarios, a multi-constraint learning path recommendation model is proposed, in which the variables and their weighted coefficients considers different learning path preferences of the learners in different learning scenarios as well as learning resource organization and fragmented time. Thirdly, based on the proposed model and knowledge map , the design and implementation of a multi-constraint learning path recommendation algorithm is described. Finally, it is shown that the questionnaire results from over 110 e-learners verify the effectiveness of the proposed algorithm and show the similarity between the learners’ self-organized learning paths and the recommended learning paths.

Topic detection is a hot topic in the field of text mining. In this paper, focusing on the Chinese interactive text, we explored a novel topic detection method, named SDD-PLSA, which integrates Semantic Dependency Distance (SDD) and PLSA. It not only has the advantages of PLSA, which is an efficient, effective method and is widely used in text mining, but also considers the semantic and syntax information. Thus, the problem of lacking semantic information in PLSA can be avoided. SDD-PLSA has two main steps. The first is using SDD to classify the sentences that have a high similarity in semantics into several groups according to semantic feature extraction of the interactive text. Then, a PLSA classifier is used upon the result of the first step. The experiments show that the accuracy of detection on `love' topic has been improved to 64.8% when using SDD-PLSA, better than 55.4% when using PLSA.

Knowledge element relation extraction is to find predefined relations between pairs of knowledge elements from text documents. As a novel form for organization and management of knowledge resources, knowledge element relation can be utilized to establish knowledge navigation system, knowledge retrieval system and collaborative knowledge construction system. In this paper, we employ conditional random fields (CRFs) to extract relations between knowledge elements from natural language documents by treating the relation extraction task as a sequence labeling problem. We first introduce three rules to generate candidate relation instances, and then incorporate various features including terms, semantic type, distance and context information to represent candidate relation instances. Experimental evaluation shows that our method achieves better performance than previous work. It also indicates that CRFs outperform other probabilistic models i.e. hidden Markov model and maximum entropy, and show effective in knowledge element relation extraction.