The evolving worldly dynamics necessitate continuous revision and updating of knowledge within Large Language Models (LLMs), driving the development of Knowledge Editing (KE) techniques. Recently, a novel paradigm of Temporal Knowledge Editing (TKE) has been proposed, emphasizing that models deployed in dynamic environments should integrate new information while retaining historical knowledge. However, we observe that current definitions and methods for TKE are insufficient, as they do not effectively capture or adapt to the fine-grained temporal dynamics inherent in real-world knowledge evolution. In this paper, we introduce the notion of multi-granularity TKE, encompassing temporal knowledge across yearly, monthly, and daily granularities, and propose a corresponding dataset, named MTKE. We argue that comprehending and retaining knowledge across different temporal granularities is crucial for LLMs to accurately reflect real-world changes. The key challenge lies in integrating new temporal knowledge at various granularities while also preserving relevant historical knowledge, thus ensuring LLMs maintain a consistent and accurate understanding over time. To achieve this, we propose a Sparse Parameter-Injected Knowledge Editing method, dubbed SPIKE, which anchors both temporal knowledge and subject positions within the model. Experiments demonstrate that our method effectively preserves historical knowledge performance while accurately incorporating dynamic temporal knowledge across multi-granularity temporal scenarios.

This paper investigates constrained multi-autonomous underwater vehicle (AUV) systems with hysteresis output. First, two novel shift functions are proposed to construct new state variables, addressing the issue of initial position and velocity variables of AUVs exceeding predefined boundaries. Based on these new state variables and combined with a coordinate transformation method, asymmetric time-varying full-state constraints independent of the initial conditions are achieved. Moreover, an innovative predefined-time convergence criterion is introduced. Based on this criterion, the proposed strategy ensures robust consensus within a predefined time under asymmetric full-state constraints, while effectively handling external disturbances, hysteresis, and saturation issues. A novel scaling inequality related to the hyperbolic tangent function is also proposed. Based on this scaling inequality, the sign function is replaced with the hyperbolic tangent function in the controller, and the proposed control scheme completely avoids issues of singularity and chattering. By leveraging neural networks (NN) and adaptive parameters to manage uncertainties and complex terms, the proposed control scheme successfully avoids the explosion of complexity. Notably, through adaptive parameter estimation, the negative impacts on system stability caused by deviations of the neural network weight matrix from its optimal value and approximation errors introduced by the neural network are mitigated. The closed-loop system is proven to be predefined-time stable. In addition, the sensitivity of the system to measurement noise is analyzed, and a NN-based observer is proposed to significantly mitigate the adverse effects caused by noise. Finally, the effectiveness of the proposed control scheme is validated through numerical simulation.

Continual Few-shot Relation Extraction (CFRE) aims to continually learn new relations from limited labeled data while preserving knowledge about previously learned relations. Facing the inherent issue of catastrophic forgetting, previous approaches predominantly rely on memory replay strategies. However, they often overlook task interference in continual learning and the varying memory requirements for different relations. To address these shortcomings, we propose a novel framework, DPC-FT, which features: 1) a lightweight relation encoder for each task to mitigate negative knowledge transfer across tasks; 2) a dynamic prototype module to allocate less memory for easier relations and more memory for harder relations. Additionally, we introduce the None-Of-The-Above (NOTA) detection in CFRE and propose a threshold criterion to identify relations that have never been learned. Extensive experiments demonstrate the effectiveness and efficiency of our method in CFRE, making our approach more practical and comprehensive for real-world scenarios.

Subspace clustering is an effective way to analyze high-dimensional data. The main problems of the conventional subspace clustering techniques are as follows: first, conventional clustering methods can not describe categorical attribute space in more detail; second, most subspace-clustering techniques failure to process dynamic data effectively; finally, lack of effective noise recognition leads to the decline of the efficiency of incremental subspace-clustering analysis. We address the above problems by an incremental subspace-clustering algorithm — called ICE. With attribute subspace constructed by a rough set-based weight computing method, ICE obtains clustering results through initial and incremental clustering stage. Utilizing the original cluster results generated from initial clustering stage, we adopt merging and splitting operation to dynamic adjust cluster-structure in incremental clustering stage. Before achieving the final results, a polymerization-based noise recognition technique is employed to automatically identify noise from sparse clusters without human threshold intervention. We implement ICE on synthetic and real-world datasets. The experimental results reveal that incremental subspace-clustering method can achieves satisfactory performance on extensibility, accuracy and robustness.

Relation classification plays a crucial role in detecting semantic relations between annotated entities within text data, serving as a fundamental tool for knowledge structurization. In recent years, federated learning has emerged as a promising approach for training relation classification models in decentralized settings. Existing methods have focused on developing a robust server model by decoupling model training at the server from direct access to client-side text data, while taking advantage of distributed data sources. However, a significant challenge arises from the heterogeneous nature of client texts, characterized by diverse and skewed distributions of relations, which has limited the practicality of current approaches. In response to this challenge, this study introduces the concept of personalized federated relation classification, aiming to tailor strong client models to adapt to their individual data distributions. To further address the issues stemming from heterogeneous texts, a novel framework, referred to as${\sf pFedRC}$, is proposed with several optimized designs. This framework incorporates a knowledge fusion method that leverages a relation-wise weighting mechanism, and a feature augmentation approach utilizing prototypes to adaptively enhance the representations of instances associated with long-tail relations. Although federated learning can safely ensure private data unexposed, it is important to recognize that, from an information theory standpoint, there still exists a possibility for a curious server to deduce private information by analyzing the shared knowledge uploaded by clients. To enhance the privacy guarantees of the personalized federated relation classification system, this work integrates client-level differential privacy mechanism into the federated training process. According to our theoretical analysis,${\sf pFedRC}$with client-level differential privacy can realize rigorous privacy guarantees. Experimental evaluations demonstrate the superiority of the proposed${\sf pFedRC}$framework over competing baselines in various settings, illustrating that the tailored techniques effectively mitigate the challenges posed by heterogeneous text data while preserving privacy guarantees. This research contributes to the advancement of learning privacy-preserving personalized relation classification models via taking advantages of data from multiple sources.

The challenge of the exploration-exploitation dilemma persists in off-policy reinforcement learning (RL) algorithms, impeding the improvement of policy performance and sample efficiency. To tackle this challenge, a novel historical decision-making regularized maximum entropy (HDMRME) RL algorithm is developed to strike the balance between exploration and exploitation. Built upon the maximum entropy RL framework, the historical decision-making regularization method is proposed to enhance the exploitation capability of RL policies. The theoretical analysis involves proving the convergence of HDMRME, investigating the tradeoff between exploration and exploitation of HDMRME, examining the disparity between the Q-function learned through HDMRME and the classic one, and analyzing the suboptimality of the trained policy. The performance of HDMRME is evaluated across various continuous-action control tasks from Mujoco and OpenAI Gym platforms. Comparative experiments demonstrate that HDMRME exhibits superior sample efficiency and achieves more competitive performance compared with other state-of-the-art RL algorithms.

Relation classification (RC) aims to detect the semantic relation between two annotated entities in a piece of sentence, serving as an essential task in automatic knowledge graph construction. Due to the emergence of new relations, there is a recent trend to train RC models in continual settings. To overcome the catastrophic forgetting problem in continual learning, existing research is devoted in a two-stage training paradigm, fast adaptation to novel relations, and memory replay for all historical relations. These memory-replay-based methods explore different techniques to mitigate the forgetting problem of continual RC (CRC) models during the memory replay stage. However, we find that the representation space undergoes distortion due to the incoming of fresh relations in the fast adaptation phase. To address this issue, we propose using a knowledge distillation strategy and designing a margin loss, aiming to maintain the stability of the RC model during adaptation to new relations. In addition, in the second stage, with a limited number of typical memory instances available, we introduce a self-contrastive learning objective to facilitate learning a balanced decision boundary for RC. Through training in two stages, our objective is to acquire a stable representation space to encode instances for CRC. We experimentally demonstrate the superiority of our model over competing methods in various settings, and the results suggest that our tailored designs can achieve better performance in CRC.

Multimodal sentiment analysis (MSA) detects human sentiments by understanding data from multiple modalities, such as text and images. Existing research primarily strives for an effective multimodal fusion framework to derive informative representations. However, these methods neglect the necessity of exploiting external knowledge to aid in analyzing sentiments. As a result, the lack of external commonsense embarrasses these models when the opinion cues come in an implicit and obscure manner. To address the limitation, in this paper, we propose an Auxiliary Rationale Knowledge enhanced framework, namely ARK, which improves MSA models via learning from a multimodal large language model (MLLM). Specifically, based on text-image pairs, we employ Chain-of-Thought prompting to generate image descriptions and rationales from the MLLM as auxiliary knowledge, thus enriching the original samples with commonsense knowledge encoded within the MLLM. By combining the source text with image descriptions, we are able to effectively handle MSA through a Text+Text paradigm. In this paradigm, smaller pre-trained language models (LMs) can be tasked for sentiment classification via prompt-tuning. Besides, rationales are leveraged as additional supervision to facilitate the learning of reasoning abilities by LMs. Experimental results demonstrate that our proposed method outperforms current state-of-the-art approaches across four datasets. Our data and code are available at https://github.com/ningpang/ArkMSA.

Abstract Relation extraction has evolved from supervised relation extraction to zero-shot setting due to the continuous emergence of newly generated relations. Some pioneering works handle zero-shot relation extraction by reformulating it into proxy tasks, such as reading comprehension and textual entailment. Nonetheless, the divergence in proxy task formulations from relation extraction hinders the acquisition of informative semantic representations, leading to subpar performance. Therefore, in this paper, we take a data-driven view to handle zero-shot relation extraction under a three-step paradigm, including encoder training, relation clustering, and summarization. Specifically, to train a discriminative relational encoder, we propose a novel selective contrastive learning framework, namely, SCL, where selective importance scores are assigned to distinguish the importance of different negative contrastive instances. During testing, the prompt-based encoder is employed to map test samples into representation vectors, which are then clustered into several groups. Typical samples closest to the cluster centroid are selected for summarization to generate the predicted relation for all samples in the cluster. Moreover, we design a simple non-parametric threshold plugin to reduce false-positive errors in inference on unseen relation representations. Our experiments demonstrate that SCL outperforms the current state-of-the-art method by over 3% across all metrics.

The neural-approximation-based adaptive nonlinear containment control issue for multiagent systems with full-state constraints is studied by invoking the backstepping approach. First, the barrier Lyapunov functions are established to deal with the state constraining issue in the multiple leaders/followers control scenarios. Then, by introducing the first-order filter, the system communication burden is substantially reduced. Moreover, the event-triggered controller is constructed by utilizing the switching-based mechanism so that the system security, control accuracy, resource consumption, and imposed state constraints are neatly balanced. We prove the output of each follower can converge to the desired hull formulated by leaders under the premise that the imposed state constraints are never violated. Besides, the considered closed-loop signals are uniformly bounded. We finally present a simulation example to show the validity of the developed approach.

Relation classification detects the semantic relation between two annotated entities from a piece of text, which is a useful tool for structurization of knowledge. Recently, federated learning has been introduced to train relation classification models in decentralized settings. Current methods strive for a strong server model by decoupling the model training at server from direct access to texts at clients while taking advantage of them. Nevertheless, they overlook the fact that clients have heterogeneous texts (i.e., texts with diversely skewed distribution of relations), which renders existing methods less practical. In this paper, we propose to investigate personalized federated relation classification, in which strong client models adapted to their own data are desired. To further meet the challenges brought by heterogeneous texts, we present a novel framework, namely pf-RC, with several optimized designs. It features a knowledge aggregation method that exploits a relation-wise weighting mechanism, and a feature augmentation method that leverages prototypes to adaptively enhance the representations of instances of long-tail relations. We experimentally validate the superiority of pf-RC against competing baselines in various settings, and the results suggest that the tailored techniques mitigate the challenges.

Abstract In recent years, Coronavirus disease 2019 (COVID-19) has become a global epidemic, and some efforts have been devoted to tracking and controlling its spread. Extracting structured knowledge from involved epidemic case reports can inform the surveillance system, which is important for controlling the spread of outbreaks. Therefore, in this paper, we focus on the task of Chinese epidemic event extraction (EE), which is defined as the detection of epidemic-related events and corresponding arguments in the texts of epidemic case reports. To facilitate the research of this task, we first define the epidemic-related event types and argument roles. Then we manually annotate a Chinese COVID-19 epidemic dataset, named COVID-19 Case Report (CCR). We also propose a novel hierarchical EE architecture, named multi-model fusion-based hierarchical event extraction (MFHEE). In MFHEE, we introduce a multi-model fusion strategy to tackle the issue of recognition bias of previous EE models. The experimental results on CCR dataset show that our method can effectively extract epidemic events and outperforms other baselines on this dataset. The comparative experiments results on other generic datasets show that our method has good scalability and portability. The ablation studies also show that the proposed hierarchical structure and multi-model fusion strategy contribute to the precision of our model.

Based on radial basis function neural networks (RBF NNs) and backstepping techniques, this brief considers the consensus tracking problem for nonlinear semi-strict-feedback multiagent systems with unknown states and disturbances. The adaptive event-triggered control scheme is introduced to decrease the update times of the controller so as to save the limited communication resources. To detect the unknown state, external disturbance, and reduce calculation workload, the state observer and disturbance observer as well as the first-order filter are first jointly constructed. It is shown that all the output signals of followers can uniformly track the reference signal of the leader and all the error signals are uniformly bounded. A simulation example is carried out to further prove the effectiveness of the proposed control scheme.

This article focuses on the adaptive control issue for uncertain nonlinear systems with time-varying full-state constraints. First, a novel integral barrier Lyapunov functions (IBLFs)-based neural backstepping control approach is designed, which circumvents the trouble of conversion in the traditional used BLFs. And then, the sliding-mode disturbance observers (SMDOs) are established to deal with the immeasurable disturbances in each order of the state-constrained uncertain nonlinear systems. Besides, the dynamic threshold-based event-sampling mechanism is constructed to deal with the sparsity of resources and system controlling burden. Finally, according to the given design approach, an event-triggered adaptive controller is developed and ensures disturbance observation errors uniformly converge to the origin in finite time, and all the signals in the closed-loop system are semiglobally uniformly ultimately bounded. A developed numerical simulation case verifies the validity of the proposed approach.

Community Q&A forum is a special type of social media that provides a platform to raise questions and to answer them (both by forum participants), to facilitate online information sharing. Currently, community Q&A forums in professional domains have attracted a large number of users by offering professional knowledge. To support information access and save users’ efforts of raising new questions, they usually come with a question retrieval function, which retrieves similar existing questions (and their answers) to a user’s query. However, it can be difficult for community Q&A forums to cover all domains, especially those emerging lately with little labeled data but great discrepancy from existing domains. We refer to this scenario as cross-domain question retrieval. To handle the unique challenges of cross-domain question retrieval, we design a model based on adversarial training, namely, X-QR , which consists of two modules—a domain discriminator and a sentence matcher. The domain discriminator aims at aligning the source and target data distributions and unifying the feature space by domain-adversarial training. With the assistance of the domain discriminator, the sentence matcher is able to learn domain-consistent knowledge for the final matching prediction. To the best of our knowledge, this work is among the first to investigate the domain adaption problem of sentence matching for community Q&A forums question retrieval. The experiment results suggest that the proposed X-QR model offers better performance than conventional sentence matching methods in accomplishing cross-domain community Q&A tasks.

We propose a weighted outlier mining method called WATCH to identify outliers in high-dimensional categorical datasets. WATCH is composed of two distinctive modules: 1) feature grouping by the virtue of correlation measurement among features and 2) outlier mining by assigning scores to objects in each feature groups. At the heart of WATCH is the feature grouping module, which groups an array of features into multiple groups to discover various aspects of feature patterns in each group. The outlier mining module detects outliers from high-dimensional categorical datasets. Except for the number of outliers specified by users, WATCH is conducive to bypassing the optimization of any user-given parameter. We implement and evaluate WATCH using synthetic and real-world datasets. Our experimental results show that WATCH is a promising and practical algorithm to detect outliers in high-dimensional categorical datasets, because WATCH achieves high performance in terms of precision, efficiency, and interpretability.

Parallel clustering is an important research area of big data analysis. The conventional Hierarchical Agglomerative Clustering (HAC) techniques are inadequate to handle big-scale categorical datasets due to two drawbacks. First, HAC consumes excessive CPU time and memory resources; and second, it is non-trivial to decompose clustering tasks into independent sub-tasks executed in parallel. We solve these two problems by a MapReduce-based hierarchical subspace-clustering algorithm - called PAPU - using LSH-based data partitioning. PAPU is conducive to partitioning a large-scale dataset into multiple independent sub-datasets, into which similar data objects are mapped. Advocating parallel computing, PAPU obtains sub-clusters corresponding to respective attribute subspaces from independent chunks in the local clustering phase. To improve the accuracy of approximated clustering results, PAPU measures various scale clusters by applying the hierarchical clustering scheme to iteratively merge sub-clusters during the global clustering phase. We implement PAPU on a 24-node Hadoop computing platform. The experimental results reveal that hierarchical subspace-clustering coupled with the data-partitioning strategy achieves high clustering efficiency on both synthetic and real-world large-scale datasets. The experiments also demonstrate that PAPU delivers superior performance in terms of extensibility and scalability (e.g., a nearly linear speedup).