The growing necessity for deep learning applications to adhere to rising data privacy standards has made machine unlearning crucial in removing the impact of specific examples from a given model. Although exact unlearning, which retrains the model from scratch using the remaining dataset, satisfies this objective, it is computationally expensive, leading approximate unlearning to be an active research field. However, we argue that most existing approximate algorithms fail to provide robust privacy guarantees. These methods are often "biased," designed to defend against attacks on a single model property (e.g., loss) while leaving information encoded in other properties (e.g., entropy) exposed. An adaptive attacker can simply exploit this bias, making the unlearning ineffective, as a model's privacy is only as strong as its most vulnerable point. For example, gradient descent using random labels may enhance indistinguishability with respect to loss by directly lowering the probability of the correct label, but it fails to achieve comparable results for entropy, as it does not increase the entropy to levels similar to those of unseen data. To address this problem, we propose a Weight Initialization based on Gradient similarity dubbed WIG, a novel algorithm that provides unbiased unlearning that approximates the retraining process. Instead of targeting a specific property, WIG induces catastrophic forgetting by partially initializing weights based on their gradient similarity between the train set and the data to be forgotten. WIG achieves the best indistinguishability among current state-of-the-art approximate unlearning algorithms across diverse metrics, including various membership inference attacks, Inter-class confusion test, U-LiRA, NeurIPS Machine Unlearning Challenge, and Gaussian Poison, demonstrating its versatility.

Monocular depth estimation has been widely studied, and significant improvements in performance have been recently reported. However, most previous works are evaluated on a few benchmark datasets, such as KITTI datasets, and none of the works provide an in-depth analysis of the generalization performance of monocular depth estimation. In this paper, we deeply investigate the various backbone networks (e.g.CNN and Transformer models) toward the generalization of monocular depth estimation. First, we evaluate state-of-the-art models on both in-distribution and out-of-distribution datasets, which have never been seen during network training. Then, we investigate the internal properties of the representations from the intermediate layers of CNN-/Transformer-based models using synthetic texture-shifted datasets. Through extensive experiments, we observe that the Transformers exhibit a strong shape-bias rather than CNNs, which have a strong texture-bias. We also discover that texture-biased models exhibit worse generalization performance for monocular depth estimation than shape-biased models. We demonstrate that similar aspects are observed in real-world driving datasets captured under diverse environments. Lastly, we conduct a dense ablation study with various backbone networks which are utilized in modern strategies. The experiments demonstrate that the intrinsic locality of the CNNs and the self-attention of the Transformers induce texture-bias and shape-bias, respectively.

Self-supervised monocular depth estimation has been widely studied recently. Most of the work has focused on improving performance on benchmark datasets, such as KITTI, but has offered a few experiments on generalization performance. In this paper, we investigate the backbone networks (e.g., CNNs, Transformers, and CNN-Transformer hybrid models) toward the generalization of monocular depth estimation. We first evaluate state-of-the-art models on diverse public datasets, which have never been seen during the network training. Next, we investigate the effects of texture-biased and shape-biased representations using the various texture-shifted datasets that we generated. We observe that Transformers exhibit a strong shape bias and CNNs do a strong texture-bias. We also find that shape-biased models show better generalization performance for monocular depth estimation compared to texture-biased models. Based on these observations, we newly design a CNN-Transformer hybrid network with a multi-level adaptive feature fusion module, called MonoFormer. The design intuition behind MonoFormer is to increase shape bias by employing Transformers while compensating for the weak locality bias of Transformers by adaptively fusing multi-level representations. Extensive experiments show that the proposed method achieves state-of-the-art performance with various public datasets. Our method also shows the best generalization ability among the competitive methods.