Vertical Federated Learning (VFL) stands out as a promising approach to safeguard privacy in collaborative machine learning, allowing multiple entities to jointly train models on vertically partitioned datasets without revealing private information. While recent years have seen substantial research on privacy vulnerabilities and defense strategies for VFL, the focus has primarily been on passive scenarios where attackers adhere to the protocol. This perspective undermines the practical threats since the attackers can deviate from the protocol to improve their inference capabilities. To address this gap, our study introduces two innovative data reconstruction attacks designed to compromise data privacy in an active setting. Essentially, both attacks modify the gradients computed during the training phase of VFL to breach privacy. Our first attack uses an Active Inversion Network exploiting a small portion of known data in the training set to coerce the passive participants into training an auto-encoder for the reconstruction of their private data. The second attack, Active Generative Network, utilizes the knowledge of the training data distribution to guide the system into training a conditional generative network (C-GAN) for feature inferences. Our experiments confirm the efficacy of both attacks in inferring private features from real-world datasets.

Recently, there has been significant interest in Temporal Graph Neural Networks (TGNN) because of their capability to learn from time-evolving graph-related data. However, similar to Graph Neural Network (GNN), explaining the predictions of TGNN is non-trivial due to its black-box and complex nature. A major approach for this problem in GNNs is by analyzing the model’s responses to some perturbations of the model’s inputs, called perturbation-based explanation methods. These methods are convenient and flexible as they do not require access to the model’s internals. However, a question arises: Does the lack of internal access limit these methods from uncovering crucial information about the predictions? Motivated by the question, this study explores the limitations of some popular classes of perturbation-based explanation methods. By constructing specific instances of TGNNs, we show (i) Node-perturbation is not reliable for identifying the paths that carry out the prediction, (ii) Edgeperturbation cannot reliably determine all the nodes contributing to the prediction and (iii) perturbing both nodes and edges does not consistently help identify the graph components responsible for the temporal aggregation in TGNNs. Our experimental results further demonstrate situations for failures of explanations can occur frequently in both synthetic and real-world scenarios. Thus, they emphasize the importance of perturbation choices and the internal information of the explained model in determining faithful explanations of the model’s predictions.

Despite recent studies on understanding deep neural networks (DNNs), there exists numerous questions on how DNNs generate their predictions. Especially, given similar predictions on different inputs, are the underlying mechanisms generating those predictions the same? In this work, we propose NeuCEPT, a method to identify critical neurons that are important to the model’s local predictions and learn their underlying mechanisms. We first formulate a critical neurons identification problem as maximizing a sequence of mutual-information objectives and provide a theoretical framework to efficiently solve for critical neurons while keeping the precision under control. NeuCEPT next heuristically learns different model’s mechanisms in an unsupervised manner. Our experiments and case studies show that neurons identified by NeuCEPT not only have strong influence on the model’s predictions but also hold meaningful information about model’s mechanisms.

Knowledge Distillation (KD) has been considered as a key solution in model compression and acceleration in recent years. In KD, a small student model is generally trained from a large teacher model by minimizing the divergence between the probabilistic outputs of the two. However, as demonstrated in our experiments, existing KD methods might not transfer critical explainable knowledge of the teacher to the student, i.e. the explanations of predictions made by the two models are not consistent. In this paper, we propose a novel explainable knowledge distillation model, called XDistillation, through which both the performance the explanations’ information are transferred from the teacher model to the student model. The XDistillation model leverages the idea of convolutional autoencoders to approximate the teacher explanations. Our experiments shows that models trained by XDistillation outperform those trained by conventional KD methods not only in term of predictive accuracy but also faithfulness to the teacher models.

In many image-classification applications, understanding the reasons of model’s prediction can be as critical as the prediction’s accuracy itself. Various feature-based local explainers have been designed to provide explanations on the decision of complex classifiers. Nevertheless, there is no consensus on evaluating the quality of different explanations. In response to this lack of comprehensive evaluation, we introduce the c-Eval metric and its corresponding framework to quantify the feature-based local explanation’s quality. Given a classifier’s prediction and the corresponding explanation on that prediction, c-Eval is the minimum-distortion perturbation that successfully alters the prediction while keeping the explanation’s features unchanged. To show that c-Eval captures the importance of input’s features, we establish a connection between c-Eval and the features returned by explainers in affine and nearly-affine classifiers. We then introduce the c-Eval plot, which not only displays a strong connection between c-Eval and explainers’ quality, but also helps automatically determine explainer’s parameters.