In knowledge distillation, a primary focus has been on transforming and balancing multiple distillation components. In this work, we emphasize the importance of thoroughly examining each distillation component, as we observe that not all elements are equally crucial. From this perspective, we decouple the Kullback-Leibler (KL) divergence into three unique elements: Binary Classification Divergence (BCD), Strong Correlation Divergence (SCD), and Weak Correlation Divergence (WCD). Each of these elements presents varying degrees of influence. Leveraging these insights, we present the Correlation-Aware Knowledge Distillation (CAKD) framework. CAKD is designed to prioritize the facets of the distillation components that have the most substantial influence on predictions, thereby optimizing knowledge transfer from teacher to student models. Our experiments demonstrate that adjusting the effect of each element enhances the effectiveness of knowledge transformation. Furthermore, evidence shows that our novel CAKD framework consistently outperforms the baseline across diverse models and datasets. Our work further highlights the importance and effectiveness of closely examining the impact of different parts of distillation process.

Adversarial generative models, such as Generative Adversarial Networks (GANs), are widely applied for generating various types of data, i.e., images, text, and audio. Accordingly, its promising performance has led to the GAN-based adversarial attack methods in the white-box and black-box attack scenarios. The importance of transferable black-box attacks lies in their ability to be effective across different models and settings, more closely aligning with real-world applications. However, it remains challenging to retain the performance in terms of transferable adversarial examples for such methods. Meanwhile, we observe that some enhanced gradient-based transferable adversarial attack algorithms require prolonged time for adversarial sample generation. Thus, in this work, we propose a novel algorithm named GE-AdvGAN to enhance the transferability of adversarial samples whilst improving the algorithm's efficiency. The main approach is via optimising the training process of the generator parameters. With the functional and characteristic similarity analysis, we introduce a novel gradient editing (GE) mechanism and verify its feasibility in generating transferable samples on various models. Moreover, by exploring the frequency domain information to determine the gradient editing direction, GE-AdvGAN can generate highly transferable adversarial samples while minimizing the execution time in comparison to the state-of-the-art transferable adversarial attack algorithms. The performance of GE-AdvGAN is comprehensively evaluated by large-scale experiments on different datasets, which results demonstrate the superiority of our algorithm. The code for our algorithm is available at: https://github.com/LMBTough/GE-advGAN.

Directional distribution analysis has long served as a fundamental functionality in abstracting dispersion and orientation of spatial datasets. Spatial datasets that describe sensitive information of individuals such as health status and home addresses must be used and shared cautiously to protect individuals' privacy. There is an inherent tension between the need of accurate directional distribution result and the requirement of individuals' location privacy. Plenty of excellent location privacy protection approaches such as geo-indistinguishability can provide strong protection for locations but considerably at the expense of statistical quality of subsequent directional distribution analysis. In this paper, to protect individual location data for directional distribution, we define the geographic feature of community with covariance matrix and then propose ageo-ellipse-indistinguishabilityprivacy notion incorporating this covariance matrix. As an instantiation of metric differential privacy,geo-ellipse-indistinguishabilityguarantees pairwise inputs cannot be distinguishable with the level proportional to privacy budget and Mahalanobis distance between them, given a randomized output. We also present elliptical privacy mechanisms to achieve this privacy definition on the basis of gamma distribution and multivariate normal distribution. We finally evaluate the empirical utility of the proposed mechanism in New York home addresses database. Our experiments demonstrate that under the same privacy level, our proposed elliptical approach can achieve significantly higher directional distribution utility than circular noise function based method.

Recent advances in Deep Neural Networks (DNNs) have dramatically improved the accuracy of DNN inference, but also introduce larger latency. In this paper, we investigate how to utilize early exit, a novel method that allows inference to exit at earlier exit points at the cost of an acceptable amount of accuracy. Scheduling the optimal exit point on a per-instance basis is challenging because the realized performance (i.e., confidence and latency) of each exit point is random and the statistics vary in different scenarios. Moreover, the performance has dependencies among the exit points, further complicating the problem. Therefore, the optimal exit scheduling decision cannot be known in advance but should be learned in an online fashion. To this end, we propose Dynamic Early Exit (DEE), a real-time online learning algorithm based on contextual bandit analysis. DEE observes the performance at each exit point as context and decides whether to exit or keep processing. Unlike standard contextual bandit analyses, the rewards of the decisions in our problem are temporally dependent. Furthermore, the performances of the earlier exit points are inevitably explored more compared to the later ones, which poses an unbalance exploration-exploitation trade-off. DEE addresses the aforementioned challenges, where its regret per inference asymptotically approaches zero. We compare DEE with four benchmark schemes in the real-world experiment. The experiment result shows that DEE can improve the overall performance by up to 98.1% compared to the best benchmark scheme.

Today, Pareto-optimal objects finding has been applied in various fields, such as group decision making and opinion collection. Many of the existing solutions to this problem require explicit attributes for objects. However, these attributes cannot be obtained sometimes. To address this issue, we propose an algorithm, which uses preference relations given by crowdsourcing, to find Pareto-optimal objects with shorter latency and lower monetary costs. It employs two multi-pairwise-comparison question models: BEST-form and BETTER-form questions. Multiple BEST (or BETTER) questions can be sent to crowds concurrently. Extensive experimental results show that the number of questions reduces greatly. In addition, the numerical results show that the latency is significantly shortened at a reasonable monetary cost, compared with the existing methods.