Malicious agents pose significant threats to the reliability and decision-making capabilities of Multi-Agent Systems (MAS) powered by Large Language Models (LLMs). Existing defenses often fall short due to reactive designs or centralized architectures which may introduce single points of failure. To address these challenges, we propose SentinelNet, the first decentralized framework for proactively detecting and mitigating malicious behaviors in multi-agent collaboration. SentinelNet equips each agent with a credit-based detector trained via contrastive learning on augmented adversarial debate trajectories, enabling autonomous evaluation of message credibility and dynamic neighbor ranking via bottom-k elimination to suppress malicious communications. To overcome the scarcity of attack data, it generates adversarial trajectories simulating diverse threats, ensuring robust training. Experiments on MAS benchmarks show SentinelNet achieves near-perfect detection of malicious agents, close to 100% within two debate rounds, and recovers 95% of system accuracy from compromised baselines. By exhibiting strong generalizability across domains and attack patterns, SentinelNet establishes a novel paradigm for safeguarding collaborative MAS.

With the rise of generative large language models (LLMs) like LLaMA and ChatGPT, these models have significantly transformed daily life and work by providing advanced insights. However, as jailbreak attacks continue to circumvent built-in safety mechanisms, exploiting carefully crafted scenarios or tokens, the safety risks of LLMs have come into focus. While numerous defense strategies-such as prompt detection, modification, and model fine-tuning-have been proposed to counter these attacks, a critical question arises: do these defenses compromise the utility and usability of LLMs for legitimate users? Existing research predominantly focuses on the effectiveness of defense strategies without thoroughly examining their impact on performance, leaving a gap in understanding the trade-offs between LLM safety and performance.

Recently, how to protect the Intellectual Property (IP) of deep neural networks (DNN) becomes a major concern for the AI industry. To combat potential model piracy, recent works explore various watermarking strategies to embed secret identity messages into the prediction behaviors or the internals (e.g., weights and neuron activation) of the target model. Sacrificing less functionality and involving more knowledge about the target model, the latter branch of watermarking schemes (i.e., white-box model watermarking) is claimed to be accurate, credible and secure against most known watermark removal attacks, with emerging research efforts and applications in the industry.

Graph neural network (GNN) based recommendation models are observed to be more vulnerable against carefully-designed malicious records injected into the system, i.e., shilling attacks, which manipulate the recommendation to common users and therefore impair user trust. In this paper, we for the first time conduct a systematic study on the vulnerability of GNN based recommendation model against the shilling attack. With the aid of theoretical analysis, we attribute the root cause of the vulnerability to its neighborhood aggregation mechanism, which could make the negative impact of attacks propagate rapidly in the system. To restore the robustness of GNN based recommendation model, the key factor lies in detecting malicious records in the system and preventing the propagation of misinformation. To this end, we construct a user-user graph to capture the patterns of malicious behaviors and design a novel GNN based detector to identify fake users. Furthermore, we develop a data augmentation strategy and a joint learning paradigm to train the recommender model and the proposed detector. Extensive experiments on benchmark datasets validate the enhanced robustness of the proposed method in resisting various types of shilling attacks and identifying fake users, e.g., our proposed method fully mitigating the impact of popularity attacks on target items up to , and improving the accuracy of detecting fake users on the Gowalla dataset by .

Open-source knowledge graphs are attracting increasing attention. Nevertheless, the openness also raises the concern of data poisoning attacks, that is, the attacker could submit malicious facts to bias the prediction of knowledge graph embedding (KGE) models. Existing studies on such attacks adopt a clear-box setting and neglect the semantic information of the generated facts, making them fail to attack in real-world scenarios. In this work, we consider a more rigorous setting and propose a model-agnostic, semantic, and stealthy data poisoning attack on KGE models from a practical perspective. The main design of our work is to inject indicative paths to make the infected model predict certain malicious facts. With the aid of the proposed opaque-box path injection theory, we theoretically reveal that the attack success rate under the opaque-box setting is determined by the plausibility of triplets on the indicative path. Based on this, we develop a novel and efficient algorithm to search paths that maximize the attack goal, satisfy certain semantic constraints, and preserve certain stealthiness, i.e., the normal functionality of the target KGE will not be influenced although it predicts wrong facts given certain queries. Through extensive evaluation of benchmark datasets and 6 typical knowledge graph embedding models as the victims, we validate the effectiveness in terms of attack success rate (ASR) under opaque-box setting and stealthiness. For example, on FB15k-237, our attack achieves a ASR on DeepPath, with an average ASR over when attacking various KGE models under the opaque-box setting.

Time series prediction has wide applications in many safety-critical scenarios. According to previous studies, time series of recorded events usually contain a non-trivial proportion of extreme events, featured with extremely large/small values and may have huge societal consequences if overlooked by a predictive model (i.e., predictor). Despite its significance in time series, we however observe the conventional square loss in time series prediction would ignore the modeling of extreme events. Specifically, we prove the square loss as a learning objective of the predictor behaves equivalently as a Gaussian kernel density estimator (KDE) on the recorded events, which is light-tailed itself and unable to model the ground-truth event distribution, usually heavy-tailed due to the existence of extreme events. Considering the benefits of forecasting extreme events, we propose a unified loss form called Generalized Extreme Value Loss (GEVL), which bridges the misalignment between the tail parts of the estimation and the ground-truth via transformations on either the observed events or the estimator. Following the proposed framework, we present three heavy-tailed kernels and derive the corresponding GEVLs which show different levels of trade-off between modeling effectiveness and computational resources.Comprehensive experiments validate our novel loss form substantially enhances representative time series predictors in modeling extreme events.

As a fundamental task in modern data mining, time series classification is powering mission-critical tasks including stock price prediction and network traffic analysis. Due to the non-linear structure of deep neural networks (DNN), deep learning has established as a promising solution to time series classification. However, the excessive learning capacity of DNNs may make them prone to threats of backdoor attacks, where an attacker embeds hidden functionalities (i.e., backdoor) to DNNs and activates the backdoor by specially-designed inputs (i.e., triggers). Despite extensive studies concerning backdoor attacks on image and text domains, there is little known about the vulnerability of DNN based time series classifiers against backdoor attacks. Due to the unique characteristics of time series data, most existing backdoor attack techniques fail to threaten time series classifiers. In this paper, through analyzing the key factors which influence the effectiveness of a backdoor, we systematize a list of practical principles for designing triggers on time series data. In this light, we propose a novel framework called TimeTrojan, which aims to learn to form the trigger pattern through a constrained multi-objective optimization. To solve the hereafter challenging optimization issue, we further design an iterative learning algorithm. Remarkably, the proposed framework is agnostic to a wide range of DNN classifiers. Extensive empirical results on 6 representative DNN classifiers and 6 real-world datasets validate the effectiveness of the proposed attack framework. In most cases, TimeTrojan successfully injects backdoors with 100% attack success rate without affecting the model accuracy on clean samples, which implies the complete control of the behavior of the DNN classifiers by the adversary.

Protecting the intellectual property (IP) of deep neural networks (DNN) becomes an urgent concern for IT corporations. For model piracy forensics, previous model fingerprinting schemes are commonly based on adversarial examples constructed for the owner's model as the fingerprint, and verify whether a suspect model is indeed pirated from the original model by matching the behavioral pattern on the fingerprint examples between one another. However, these methods heavily rely on the characteristics of classification tasks which inhibits their application to more general scenarios. To address this issue, we present MetaV, the first task-agnostic model fingerprinting framework which enables fingerprinting on a much wider range of DNNs independent from the downstream learning task, and exhibits strong robustness against a variety of ownership obfuscation techniques. Specifically, we generalize previous schemes into two critical design components in MetaV: the adaptive fingerprint and the meta-verifier, which are jointly optimized such that the meta-verifier learns to determine whether a suspect model is stolen based on the concatenated outputs of the suspect model on the adaptive fingerprint. As a key of being task-agnostic, the full process makes no assumption on the model internals in the ensemble only if they have the same input and output dimensions. Spanning classification, regression and generative modeling, extensive experimental results validate the substantially improved performance of MetaV over the state-of-the-art fingerprinting schemes and demonstrate the enhanced generality of MetaV for providing task-agnostic fingerprinting. For example, on fingerprinting ResNet-18 trained for skin cancer diagnosis, MetaV achieves simultaneously 100% true positives and 100% true negatives on a diverse test set of 70 suspect models, achieving an about 220% relative improvement in ARUC over the optimal baseline.

Hierarchical community detection, which aims at discovering the hierarchical structure of a graph, attracts increasing attention due to its wide range of applications. However, due to the difficulty of parametrizing the community tree, existing methods mainly rely on heuristic algorithms, which are limited by their low accuracy and inability to handle new observations. As far as we know, how to leverage deep learning techniques to better discover hierarchical communities remains almost blank in the existing literature. In this paper, we present the first deep learning framework called ReinCom for hierarchical community detection. To address the challenge of parametrizing the community tree, we propose a novel growing-up process where, at each step, we first partition nodes into the community tree and then adjust the community tree according to the partition results. To learn an optimal growing-up process, we propose an embedding agent and a community agent to implement the two sub-steps respectively. Furthermore, we also propose an online learning strategy for new observations on the graph. Empirical results show that our proposed model has better modeling effectiveness than the state-of-the-art methods. For example, in terms of modularity, the performance of ReinCom is 33% higher than previous community detection works. Besides, with the aid of the learned node embeddings, we also devise a graph visualization algorithm which can consistently reflect the latent hierarchical structure of a graph.

Location recommendation becomes increasingly important in the mobile era. Particularly, how to exploit personalized geographical preferences determines the quality of recommended results. A number of efforts have been made on this task, however, there exists a common limitation called the out-of-town recommending problem, i.e., those far places can hardly be recommended. In this paper, we first reveal why modeling the geographical patterns is difficult with the help of the extreme value theory. We find that out-of-town distances are heavy-tailed variables with few observations and extreme values, making it difficult to use common distributions to describe them. To address this issue, we propose a new function called volcano function to model out-of-town distances and personalize it for different users. Empirical results show that we can learn effective patterns from limited observations. Finally we extend the volcano function to a ranking-based collaborative filtering framework, naming it as volcano network (VolNet). Experimental results show the superior performance of VolNet, especially the recall is improved from 0.2 to 0.35 in recommending remote venues compared with the state-of-the-art method GeoMF++.

Time series prediction is an intensively studied topic in data mining. In spite of the considerable improvements, recent deep learning-based methods overlook the existence of extreme events, which result in weak performance when applying them to real time series. Extreme events are rare and random, but do play a critical role in many real applications, such as the forecasting of financial crisis and natural disasters. In this paper, we explore the central theme of improving the ability of deep learning on modeling extreme events for time series prediction. Through the lens of formal analysis, we first find that the weakness of deep learning methods roots in the conventional form of quadratic loss. To address this issue, we take inspirations from the Extreme Value Theory, developing a new form of loss called Extreme Value Loss (EVL) for detecting the future occurrence of extreme events. Furthermore, we propose to employ Memory Network in order to memorize extreme events in historical records.By incorporating EVL with an adapted memory network module, we achieve an end-to-end framework for time series prediction with extreme events. Through extensive experiments on synthetic data and two real datasets of stock and climate, we empirically validate the effectiveness of our framework. Besides, we also provide a proper choice for hyper-parameters in our proposed framework by conducting several additional experiments.

Location-based embedding is a fundamental problem to solve in location-based social networks (LBSN). In this paper, we propose a geographical convolutional neural tensor network (GeoCNTN) as a generic embedding model. GeoCNTN first takes the raw location data and extracts from it a more well-conditioned representation by our proposed Geo-CMeans algorithm. We then use a convolutional neural network (CNN) and an embedding structure to extract individual latent structural patterns from the preprocessed data. Finally, we apply a neural tensor network (NTN) to craft the implicitly related features we have obtained into a unified geographical feature. The advantages of our GeoCNTN mainly come from its novel neural network structure, which intrinsically offers a mechanism to extract latent structural features from the geographical data, as well as its wide applicability in various LBSN-related tasks. From two case studies, i.e. link prediction and entity classification in user-group LBSN, we evaluate the embedding efficacy of our model. Results show that GeoCNTN significantly performs better on at least two tasks, with improvement by 9% w.r.t. NDCG and 11% w.r.t. F1 score respectively, using the Meetup-USA dataset.