The feature attribution method reveals the contribution of input variables to the decision-making process to provide an attribution map for explanation. Existing methods grounded on the information bottleneck principle compute information in a specific layer to obtain attributions, compressing the features by injecting noise via a parametric damping ratio. However, the attribution obtained in a specific layer neglects evidence of the decision-making process distributed across layers. In this paper, we introduce a comprehensive information bottleneck (CoIBA), which discovers the relevant information in each targeted layer to explain the decision-making process. Our core idea is applying information bottleneck in multiple targeted layers to estimate the comprehensive information by sharing a parametric damping ratio across the layers. Leveraging this shared ratio complements the over-compressed information to discover the omitted clues of the decision by sharing the relevant information across the targeted layers. We suggest the variational approach to fairly reflect the relevant information of each layer by upper bounding layer-wise information. Therefore, CoIBA guarantees that the discarded activation is unnecessary in every targeted layer to make a decision. The extensive experimental results demonstrate the enhancement in faithfulness of the feature attributions provided by CoIBA.

In this paper, we examine a key limitation in query-based detectors for temporal action detection (TAD), which arises from their direct adaptation of originally designed architectures for object detection. Despite the effectiveness of the existing models, they struggle to fully address the unique challenges of TAD, such as the redundancy in multi-scale features and the limited ability to capture sufficient temporal context. To address these issues, we propose a multi-dilated gated encoder and central-adjacent region integrated decoder for temporal action detection transformer (DiGIT). Our approach replaces the existing encoder that consists of multi-scale deformable attention and feedforward network with our multi-dilated gated encoder. Our proposed encoder reduces the redundant information caused by multi-level features while maintaining the ability to capture fine-grained and long-range temporal information. Furthermore, we introduce a central-adjacent region integrated decoder that leverages a more comprehensive sampling strategy for deformable cross-attention to capture the essential information. Extensive experiments demonstrate that DiGIT achieves state-of-the-art performance on THUMOS14, ActivityNet v1.3, and HACS-Segment. Code is available at: https://github.com/Dotori-HJ/DiGIT

In this paper, we investigate that the normalized co-ordinate expression is a key factor as reliance on hand-crafted components in query-based detectors for tempo-ral action detection (TAD). Despite significant advancements towards an end-to-end framework in object detection, query-based detectors have been limited in achieving full end-to-end modeling in TAD. To address this is-sue, we propose TE-TAD, a full end-to-end temporal action detection transformer that integrates time-aligned co-ordinate expression. We reformulate coordinate expression utilizing actual time line values, ensuring length-invariant representations from the extremely diverse video duration environment. Furthermore, our proposed adaptive query selection dynamically adjusts the number of queries based on video length, providing a suitable solution for varying video durations compared to a fixed query set. Our approach not only simplifies the TAD process by eliminating the needfor hand-crafted components but also significantly improves the performance of query-based detectors. Our TE-TAD outperforms the previous query-based detectors and achieves competitive performance compared to state-of-the-art methods on popular benchmark datasets. Code is available at: https://github.com/Dotori-HJ/TE-TAD

Adversarial attacks mislead deep neural network classifiers with slight perturbations, underscoring the necessity for the development of robust defenses to ensure the secure and responsible use of artificial intelligence. Recent research has shown that diffusion-based adversarial purification methods have emerged as a promising defense technique, but often suffer from computational inefficiencies and suboptimal results. To address these issues, we propose RoPAR, an innovative approach that enhances robustness against adversarial attacks by aggregating purified images at intermediate steps of the diffusion process. Our method improves model robustness while reducing the required diffusion steps. We also introduce a technique for reordering Gaussian noise to minimize semantic information loss while removing adversarial perturbations. These enhancements significantly reduce the number of function evaluations from 200 to 6, achieving a robust accuracy of 92.39 % against preprocessor-blind PGD attacks on CIFAR-10, a 2.29 percentage point improvement over state-of-the-art. Moreover, our method demonstrates its effectiveness in real-world scenarios, achieving 87.46 % accuracy on CIFAR-10C.

Adversarial attacks pose a significant threat to security-critical applications by deliberately deceiving model predictions. Numerous works attempt to create robust models by encoding useful information to intermediate representations. However, they still contain too much information about the training data which hinders improving the robustness of the model. To mitigate this issue, we propose a novel approach, CHBaR, that incorporates class-conditioned information into intermediate representations. The class-conditioned information plays the role of weight components which are multiplied with the intermediate representations to produce class-conditioned representations. We utilize an attribution-based explanation method to obtain this class-conditioned information. As a result, the weight components emphasize class-relevant features by highlighting relevant information from the target class. This weighting process easily integrates the target class without complex computations and conceals useless representations, thus enhancing model predictions by masking features unrelated to the class. Extensive experiments demonstrate the effectiveness of our proposed method in enhancing adversarial robustness. Especially, on the SVHN dataset, our proposed method shows an increment of 6.98 % points compared to the baseline model in PGD40adversarial attack with the TRADES training setting.

Revealing the transparency of Deep Neural Networks (DNNs) has been widely studied to describe the decision mechanisms of network inner structures. In this paper, we propose a novel post-hoc framework, Unfold and Conquer Attribution Guidance (UCAG), which enhances the explainability of the network decision by spatially scrutinizing the input features with respect to the model confidence. Addressing the phenomenon of missing detailed descriptions, UCAG sequentially complies with the confidence of slices of the image, leading to providing an abundant and clear interpretation. Therefore, it is possible to enhance the representation ability of explanation by preserving the detailed descriptions of assistant input features, which are commonly overwhelmed by the main meaningful regions. We conduct numerous evaluations to validate the performance in several metrics: i) deletion and insertion, ii) (energy-based) pointing games, and iii) positive and negative density maps. Experimental results, including qualitative comparisons, demonstrate that our method outperforms the existing methods with the nature of clear and detailed explanations and applicability.

Although various face-related tasks have significantly advanced in recent years, occlusion and extreme pose still impede the achievement of higher performance. Existing face rotation or de-occlusion methods only have emphasized the aspect of each problem. In addition, the lack of high-quality paired data remains an obstacle for both methods. In this work, we present a self-supervision strategy called Swap-R&R to overcome the lack of ground-truth in a fully unsupervised manner for joint face rotation and de-occlusion. To generate an input pair for self-supervision, we transfer the occlusion from a face in an image to an estimated 3D face and create a damaged face image, as if rotated from a different pose by rotating twice with the roughly de-occluded face. Furthermore, we propose Complete Face Recovery GAN (CFR-GAN) to restore the collapsed textures and disappeared occlusion areas by leveraging the structural and textural differences between two rendered images. Unlike previous works, which have selected occlusion-free images to obtain ground-truths, our approach does not require human intervention and paired data. We show that our proposed method can generate a de-occluded frontal face image from an occluded profile face image. Moreover, extensive experiments demonstrate that our approach can boost the performance of facial recognition and facial expression recognition. The code is publicly available1