This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images. Conventional deep metric learning methods focus on learning a discriminative embedding to describe the semantic features of images, which ignore the existence of uncertainty in each image resulting from noise or semantic ambiguity. Training without awareness of these uncertainties causes the model to overfit the annotated labels during training and produce overconfident judgments during inference. Motivated by this, we argue that a good similarity model should consider the semantic discrepancies with awareness of the uncertainty to better deal with ambiguous images for more robust training. To achieve this, we propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. We further propose an introspective similarity metric to make similarity judgments between images considering both their semantic differences and ambiguities. The gradient analysis of the proposed metric shows that it enables the model to learn at an adaptive and slower pace to deal with the uncertainty during training. Our framework attains state-of-the-art performance on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets for image retrieval. We further evaluate our framework for image classification on the ImageNet-1 K, CIFAR-10, and CIFAR-100 datasets, which shows that equipping existing data mixing methods with the proposed introspective metric consistently achieves better results (e.g., +0.44% for CutMix on ImageNet-1 K).

Learning a generalizable and comprehensive similarity metric to depict the semantic discrepancies between images is the foundation of many computer vision tasks. While existing methods approach this goal by learning an ensemble of embeddings with diverse objectives, the backbone network still receives a mix of all the training signals. Differently, we propose a deep factorized metric learning (DFML) method to factorize the training signal and employ different samples to train various components of the backbone network. We factorize the network to different sub-blocks and devise a learnable router to adaptively allocate the training samples to each sub-block with the objective to capture the most information. The metric model trained by DFML capture different characteristics with different sub-blocks and constitutes a generalizable metric when using all the sub-blocks. The proposed DFML achieves state-of-the-art performance on all three benchmarks for deep metric learning including CUB-200-20ll, Cars196, and Stanford Online Products. We also generalize DFML to the image classification task on ImageNet-1K and observe consistent improvement in accuracy/computation trade-off. Specifically, we improve the performance of ViT-B on ImageNet (+0.2% accuracy) with less computation load (-24% FLOPs).11Code is available at: https://github.com/wangck20/DFML.

The pretrain-finetune paradigm in modern computer vision facilitates the success of self-supervised learning, which tends to achieve better transferability than supervised learning. However, with the availability of massive labeled data, a natural question emerges: how to train a better model with both self and full supervision signals? In this paper, we propose Omni-suPErvised Representation leArning with hierarchical supervisions (OPERA) as a solution. We provide a unified perspective of supervisions from labeled and unlabeled data and propose a unified framework of fully supervised and self-supervised learning. We extract a set of hierarchical proxy representations for each image and impose self and full supervisions on the corresponding proxy representations. Extensive experiments on both convolutional neural networks and vision transformers demonstrate the superiority of OPERA in image classification, segmentation, and object detection.1

This paper proposes an ASynchronous Autoregressive Prediction (ASAP) method for satellite anomaly detection. We empirically observe that a single classification model can hardly detect unknown anomalous situations and neglect the Markov nature of temporal satellite data. To address this, we adopt an autoregressive model to deal with the prediction of unknown anomaly for satellite data. We further propose a non-uniform temporal encoding method for asynchronous data and a median filtering method for more accurate detection. To reduce the effect of outliers, we employ an adaptive threshold selection method to achieve a more robust classification boundary. Experiments on real satellite data demonstrate that the proposed ASAP method outperforms the baseline classification method by 55.79%.

In this paper, we propose a deep compositional metric learning (DCML) framework for effective and generalizable similarity measurement between images. Conventional deep metric learning methods minimize a discriminative loss to enlarge interclass distances while suppressing intraclass variations, which might lead to inferior generalization performance since samples even from the same class may present diverse characteristics. This motivates the adoption of the ensemble technique to learn a number of sub-embeddings using different and diverse subtasks. However, most subtasks impose weaker or contradictory constraints, which essentially sacrifices the discrimination ability of each sub-embedding to improve the generalization ability of their combination. To achieve a better generalization ability without compromising, we propose to separate the sub-embeddings from direct supervisions from the subtasks and apply the losses on different composites of the sub-embeddings. We employ a set of learnable compositors to combine the sub-embeddings and use a self-reinforced loss to train the compositors, which serve as relays to distribute the diverse training signals to avoid destroying the discrimination ability. Experimental results on the CUB-200-2011, Cars196, and Stanford Online Products datasets demonstrate the superior performance of our framework.1

Aiming at the robot drilling perpendicularity in aircraft assembly, a surface normal measurement method is presented and a drilling end effector is designed. Four laser ranging sensors are uniformly distributed around the drill to measure the surface normal as four non-coplanar points define a unique circumscribed sphere and the sphere center can be calculated. By this principle, the method measures four points coordinates on the curved surface in the drilling area using four laser ranging sensors. If the four laser ranging sensors are close enough to each other, it is reasonable to assume that the drilling point is on the sphere surface. Therefore, the connection line of the sphere center and the drilling point is the surface normal at the drilling point. The angle θ between the normal and the axis of the drill is calculated and if θ is larger than 0.5 degree, the drilling end effector will adjust the attitude of the drill to make sure θ is smaller than 0.5 degree to meet the requirement in aircraft assembly. A novel adjusting mechanism is designed in the end effector which can keep drill vertex immobile when adjusting drill position. Thus, it is not required to remove the drill vertex to the mark point. Simulation results of three kinds curved surfaces show that the surface normal measurement method is accurate and efficient. Experiments on aviation drilling system results also demonstrate the adjusting mechanism is effective with high accuracy.