Contrastive learning has been widely applied into sentence representation to obtain high quality embedding vectors through pulling close positive samples and pushing apart irrelevant negative samples. Subsequently, various knowledge distillation methods have been employed to enhance the performance of contrastive learning framework. However, current distillation techniques focus on transferring knowledge through similarity of embedding vectors in a mini-batch, thus ignoring the relational structural information of them. To address this, we propose a novel framework, RKDCSE(Relational Knowledge Distillation Contrastive learning of Sentence Embedding) for sentence representation learning, designed to transferring structural information between sentences from teacher model to student model. We also introduce a random features masking mechanism, which mitigates the issues of overfitting via masking features in embedding vectors randomly. Finally, we conduct extensive experiments on Semantic Textual Similarity(STS) and Transfer Tasks(TR). The experimental results demonstrate that our proposed RKDCSE is effective and outperforms baseline method.

The proliferation and advancement of generative AI tools blur the boundaries between authentic and AI-synthesized imagery, sparking widespread public concern about visual content authenticity. While existing AI-generated image detection approaches have demonstrated notable achievements, Multimodal Large Language Models (MLLMs) remain relatively underutilized in this field. As MLLMs show strong abilities in understanding image features and can provide high-quality textual analysis, substantial potential within MLLM itself remains well explored. In this work, we propose a novel triple-feature fusion prompt learning framework to effectively stimulate the potential of the MLLMs in detecting AI-generated images. We build a prompt structure that allows MLLM to analyze the inauthenticity of AI-generated images from different perspectives, both locally and globally, and ask questions on the generated image dataset to obtain open-ended answers. We then design a triple fusion encoder that combines semantic features, structured knowledge representation and fine-grained modifications. Through a hierarchical feature induction mechanism, we were able to construct multidimensional feature sets with interpretive properties. Then, we compose a set of prompts based on the obtained feature sets to guide the MLLM to discriminate the input images under global and partial strategies. The test result shows prominent improvement compared to direct inquiry approaches, suggesting that MLLMs demonstrate substantial potential in no need for fine-tuning in AI-generated image detection. This work will drive future research toward prompt-based methods, further expanding the capabilities of MLLMs.

The application of smart contracts in electric power systems is widespread. However, vulnerabilities in smart contracts can cause significant economic losses and require careful attention. Smart contracts in electric power systems have domain-specific characteristics that differ from traditional public blockchain applications. As a result, existing vulnerability detection tools cannot be directly applied to these systems. To address this challenge, we design a vulnerability detection tool called E-Guard specifically for smart contracts in electric power systems. E-Guard uses a tailored intermediate representation (IR) known as EIR, which provides control flow and data flow information more suited to the business logic of electric power systems than traditional static analysis tools. We identify and summarize three types of vulnerabilities unique to electric power systems based on expert knowledge. Experimental results show that E-Guard significantly outperforms traditional static analysis tools in detecting these three types of vulnerabilities. Additionally, the extra overhead generated by using EIR is minimal and negligible. This demonstrates that E-Guard is an effective and efficient tool for enhancing the security of smart contracts in electric power systems.

Contrastive Learning and Feature Space Tactics: A Dual Approach to Strengthen Backdoor Attacks

Learning 3D representation plays a critical role in masked autoencoder (MAE) based pre-training methods for point cloud, including single-modal and cross-modal based MAE. Specifically, although cross-modal MAE methods learn strong 3D representations via the auxiliary of other modal knowledge, they often suffer from heavy computational burdens and heavily rely on massive cross-modal data pairs that are often unavailable, which hinders their applications in practice. Instead, single-modal methods with solely point clouds as input are preferred in real applications due to their simplicity and efficiency. However, such methods easily suffer from limited 3D representations with global random mask input. To learn compact 3D representations, we propose a simple yet effective Point Feature Enhancement Masked Autoencoders (Point-FEMAE), which mainly consists of a global branch and a local branch to capture latent semantic features. Specifically, to learn more compact features, a share-parameter Transformer encoder is introduced to extract point features from the global and local unmasked patches obtained by global random and local block mask strategies, followed by a specific decoder to reconstruct. Meanwhile, to further enhance features in the local branch, we propose a Local Enhancement Module with local patch convolution to perceive fine-grained local context at larger scales. Our method significantly improves the pre-training efficiency compared to cross-modal alternatives, and extensive downstream experiments underscore the state-of-the-art effectiveness, particularly outperforming our baseline (Point-MAE) by 5.16%, 5.00%, and 5.04% in three variants of ScanObjectNN, respectively. Code is available at https://github.com/zyh16143998882/AAAI24-PointFEMAE.

Continual learning is an emerging area of machine learning that deals with the issue where models adapt well to the latest data but lose the ability to remember past data due to changes in the data source. A widely adopted solution is by keeping a small memory of previous learned data that use replay. Most of the previous studies on continual learning focused on classification tasks, such as image classification and text classification, where the model needs only to categorize the input data. Inspired by the human ability to incrementally learn knowledge and solve different problems using learned knowledge, we considered a more pratical scenario, knowledge based quesiton answering about continual learning. In this scenario, each single question is different from others(means different fact trippes to answer them) while classification tasks only need to find feature boundaries of different categories, which are the curves or surfaces that separate different categories in the feature space. To address this issue, we proposed a depth aware hierarchical replay framework which include a tree structure classfier to have a sense of knowledge distribution and fill the gap between text classfication tasks and question-answering tasks for continual learning, a local sampler to grasp these critical samples and a depth aware learning network to reconstructe the feature space of a single learning round. In our experiments, we have demonstrated that our proposed model outperforms previous continual learning methods in mitigating the issue of catastrophic forgetting.

Meta-learning has achieved promising results in low-resource text classification, which aims to identify target classes by transferring knowledge from source classes through a series of small tasks called episodes. However, the current meta-learning algorithms that solely rely on learning from meta-training tasks may struggle to generalize well to meta-testing tasks. To address this problem, we propose a method called Retrieval-Augmented Meta Learning (RAML) that utilizes external knowledge to compensate for the performance degradation when meta-training tasks do not adequately support meta-testing tasks. RAML first utilizes a retriever to retrieve knowledge relevant to the query from an external corpus, and then employs the Multi-View Passages Fusion Network to integrate the retrieved knowledge for performing few-shot classification. This network can effectively combine the probability distributions of classifications obtained from multiple messages by considering the importance of different messages. Furthermore, inspired by knowledge distillation, we iteratively train the retriever model using the synthetic labels generated by the aforementioned network. Extensive experiments demonstrate that RAML significantly outperforms current state-of-the-art baselines(e.g., ChatGPT).

In this paper, we focus on the challenging and realistic Generalized Few-Shot Intent Detection (GFSID), which requires to categorize both seen and novel intents simultaneously. Moreover, there are only few training samples for novel intents. Generalized few-shot intent detection has to deal with two major challenges: learning novel intents from only few samples and preventing forgetting knowledge of seen intents. To address the dilemma, we propose to convert the GFSID task into the class incremental learning paradigm. Specifically, we propose a two-phase learning framework based on prompt learning, which sequentially training the model on the data of seen intents and novel intents. Furthermore, to alleviate the forgetting of knowledge related to seen intents, we introduce prompt-based intra-class relation knowledge distillation. To the best of our knowledge, this is the first study to simultaneously address both aspects in the context of GFSID. Extensive experiments and detailed analyses conducted on two widely used datasets demonstrate that our proposed framework achieves promising performance.

Metaphor detection is a challenging task for natural language processing (NLP) systems.Previous works failed to sufficiently utilize the internal and external semantic relationships between target words and their context.Furthermore, they have faced challenges in tackling the problem of data sparseness due to the very limited available training data.To address these two challenges, we propose a novel model called MiceCL.By leveraging the difference between the literal meaning of the target word and the meaning of the sentence as the sentence external difference, MiceCL can better handle the semantic relationships.Additionally, we propose a curriculum learning framework for automatically assessing difficulty of the sentence with a pre-trained model.By starting from easy examples and gradually progressing to more difficult ones, we can ensure that the model will not deal with complex data when its ability is weak so that to avoid wasting limited data.Experimental results demonstrate that MiceCL achieves competitive performance across multiple datasets, with a significantly improved convergence speed compared to other models.Our model is available at https: //github.com/Evilxya/MiceCL.git.

Abstract A large number of norms, which express constraints on people's behaviour within a specific range, are contained in normative documents. In writing and revising normative documents, conflicts between two norms often arise. The task Norm Conflict Identification (NCI) aims to identify such conflicts. The existing NCI methods based on statistical learning are all pipelines, which cause errors to accumulate, and cannot sufficiently extract helpful information. According to the characteristics of NCI, we propose a neural network model called NeuralConflict. This end‐to‐end model can avoid the accumulation of errors and makes it easier to obtain the optimal global solution. The model sets up an auxiliary task to predict whether two norms are semantically related and shares part of the information with the task NCI. In addition, the model uses a convolutional neural network with differently sized convolution kernels to extract local semantic information from the norms. Finally, the model inputs the shared and local semantic information into a fully connected neural network to predict whether the two norms conflict. We construct a Chinese dataset and use it with an existing English dataset for the experiments. Experimental results show that NeuralConflict achieves optimum results on both datasets.

Multi-view projection methods have demonstrated their ability to reach state-of-the-art performance in point cloud downstream tasks(e.g., classification and retrieval). These methods first require rendering the point cloud into 2D multi-view images. However, conventional methods only project the geometry of the point cloud, and such projections inevitably suffer from a loss of point cloud semantic information due to dimensionality reduction. We propose a semantic-aware and task-oriented differentiable feature rendering (SFR), which reduces the information loss during projection by generating rendered images with more point cloud semantic information for downstream tasks. Our SFR method can be applied as a plug-and-play module added to any multi-view-based backbone network for end-to-end training. Extensive experiments on benchmark datasets show that our SFR method reaches state-of-the-art performance and brings general improvements to point cloud classification and retrieval tasks.

Abstract Organizing and managing policy documents (PDs) issued to the public in a good way can improve the efficiency of government employees and make it easier for the public to find the needed policy information. However, existing PDs are organized only by dates and manually defined categories; besides, PDs issued by different government branches are isolated from each other. These problems make it challenging and time‐consuming for the public to find the needed policy information. We argue that implicit links should be established between PDs based on their relevance, thus helping the public find the needed policy information efficiently. To this end, we propose an unsupervised relevance scoring method for PDs consist six modules, taking Chinese social security policies as the application case. The method combines the TextRank algorithm, TF‐IDF representation, mutual information and left–right information entropy algorithm, and BERT. The method can decrease the interference of noisy words in PDs to relevance scoring. In addition, the method can consider multiple features of PDs simultaneously so that the measure of relevance can be more comprehensive. The method is not driven by domain‐specific labelled data, hence can be easily generalized to PDs in various domains. We construct a dataset containing 5000 Chinese social security policies and then conduct experiments on it to evaluate our method. Experimental results show that our method is feasible and can bring convenience to government agencies and the public to a certain extent. Furthermore, our method achieves more than a 3% improvement in evaluation results on test tasks than the methods with a similar purpose in the legal AI community.

In this paper, a more accurate outage performance analysis is obtained by employing order statistics for multiple-input multiple-out two-way relay system with joint transmit/receive zero-forcing. Furthermore, closed-form upper and lower bounds are first derived for the overall outage probability when there exist spatial correlations at the relay. Analysis and simulation results indicate that the upper bound derived with order statistics is tight under various spatial correlations at the relay, and it is also tighter than that derived by eigenvalues of Wishart matrices over independent identically distributed Rayleigh fading channel. For example, when the relay is equipped with 4, 6 and 8 antennas, the upper bounds derived with order statistics are 2dB, 3dB and 4dB tighter than those with eigenvalues, respectively. In particular, when the numbers of antennas equipped at the users are greater than that equipped at the relay, the derived upper bound is nearly identical to the exact results.