Although pre-trained large vision foundation models (VFM) yield superior results on various downstream tasks, full fine-tuning is often impractical due to its high computational cost and storage requirements. Recent advancements in parameter-efficient fine-tuning (PEFT) of VFM for image classification show significant promise. However, the application of PEFT techniques to dense prediction tasks remains largely unexplored. Our analysis of existing methods reveals that the underlying premise of utilizing low-rank parameter matrices, despite their efficacy in specific applications, may not be adequately suitable for dense prediction tasks. To this end, we propose a novel PEFT learning approach tailored for dense prediction tasks, namely VFM-Adapter. Specifically, the VFM-Adapter introduces a hybrid operation mapping technique that seamlessly integrates local information with global modeling to the adapter module. It capitalizes on the distinct inductive biases inherent in different operations. Additionally, we dynamically generate parameters for the VFM-Adapter, enabling flexibility of feature extraction given specific inputs. To validate the efficacy of VFM-Adapter, we conduct extensive experiments across object detection, semantic segmentation, and instance segmentation tasks. Results on multiple benchmarks consistently demonstrate the superiority of our method over previous approaches. Notably, with only three percent of the trainable parameters of the SAM-Base backbone, our approach achieves competitive or even superior performance compared to full fine-tuning. The code will be available.

Pretrained language models (LMs) are prone to arithmetic errors.Existing work showed limited success in probing numeric values from models' representations, indicating that these errors can be attributed to the inherent unreliability of distributionally learned embeddings in representing exact quantities.However, we observe that previous probing methods are inadequate for the emergent structure of learned number embeddings with sinusoidal patterns.In response, we propose a novel probing technique that decodes numeric values from input embeddings with near-perfect accuracy across a range of open-source LMs.This proves that after the sole pre-training, LMs represent numbers with remarkable precision.Finally, we find that the embeddings' precision, judged by our probe's accuracy, explains a large portion of LM's errors in elementary arithmetic, and show that aligning the embeddings with the pattern our probes discover can mitigate these errors.

Training high-accuracy 3D detectors necessitates massive labeled 3D annotations with 7 degree-of-freedom, which is laborious and time-consuming. Therefore, the form of point annotations is proposed to offer significant prospects for practical applications in 3D detection, which is not only more accessible and less expensive but also provides strong spatial information for object localization. In this paper, we empirically discover that it is non-trivial to merely adapt Point-DETR to its 3D form, encountering two main bottlenecks: 1) it fails to encode strong 3D prior into the model, and 2) it generates low-quality pseudo labels in distant regions due to the extreme sparsity of LiDAR points. To overcome these challenges, we introduce Point-DETR3D, a teacher-student framework for weakly semi-supervised 3D detection, designed to fully capitalize on point-wise supervision within a constrained instance-wise annotation budget. Different from Point-DETR which encodes 3D positional information solely through a point encoder, we propose an explicit positional query initialization strategy to enhance the positional prior. Considering the low quality of pseudo labels at distant regions produced by the teacher model, we enhance the detector's perception by incorporating dense imagery data through a novel Cross-Modal Deformable RoI Fusion (D-RoI). Moreover, an innovative point-guided self-supervised learning technique is proposed to allow for fully exploiting point priors, even in student models. Extensive experiments on representative nuScenes dataset demonstrate our Point-DETR3D obtains significant improvements compared to previous works. Notably, with only 5% of labeled data, Point-DETR3D achieves over 90% performance of its fully supervised counterpart.

Person re-identification is still a challenging task in actual application due to serious misalignment caused by large scale variation, serious occlusion, or variably truncated body. Conventional holistic methods usually lack the cross-scale aligning ability. Segmentation-based partial methods achieve better aligning performance but generally suffer from the instability of part segmentation. To explicitly address these issues, we define the seriously misaligned Re-ID task and propose a novel framework called adversarial cross-scale alignment pursuit (ACSAP). Instead of dynamically segmenting the feature map for part alignment, the model incorporates the stability of holistic methods and adversarially generates aligned feature maps for similarity metrics. Especially, the spatial reconstruction (SR) module in the generator is proposed for feature filtering and aligning. Then a part visibility calculation (PVC) algorithm is proposed to distinguish the credibility of different generated areas. We propose a novel dataset called Seriously-Misaligned-REID and achieve 10.0% rank-1 outperformance compared to state-of-the-art methods on it. Extensive performance results demonstrate the effectiveness of our framework.

Few-shot learning aims at rapidly adapting to novel categories with only a handful of samples at test time, which has been predominantly tackled with the idea of meta-learning. However, meta-learning approaches essentially learn across a variety of few-shot tasks and thus still require large-scale training data with fine-grained supervision to derive a generalized model, thereby involving prohibitive annotation cost. In this paper, we advance the few-shot classification paradigm towards a more challenging scenario, i.e, cross-granularity few-shot classification, where the model observes only coarse labels during training while is expected to perform fine-grained classification during testing. This task largely relieves the annotation cost since fine-grained labeling usually requires strong domain-specific expertise. To bridge the cross-granularity gap, we approximate the fine-grained data distribution by greedy clustering of each coarse-class into pseudo-fine-classes according to the similarity of image embeddings. We then propose a meta-embedder that jointly optimizes the visual- and semantic-discrimination, in both instance-wise and coarse class-wise, to obtain a good feature space for this coarse-to-fine pseudo-labeling process. Extensive experiments and ablation studies are conducted to demonstrate the effectiveness and robustness of our approach on three representative datasets.

Crowd anomaly detection under surveillance scene is a quite challenging task, which often companies with not rare objects, unexpected bursts in activity and complex dynamic patterns. In this paper, we propose a social multiple-instance learning(MIL) framework with a dual-branch network by considering dynamic interaction among groups, individuals and environment to obtain attentive spatial-temporal feature representation. First, MIL is employed to overcome the challenge of rare training abnormal samples and video-based labels. The social force map is utilized for modeling behavior interaction to supply the prior knowledge. In addition, we introduce the self-attention module, which represents a more discriminative spatial-temporal feature based on C3D network through implementing weight redistribution inside the feature. The results of the experiments conducted on UCF-Crime dataset show that the proposed dual-branch social multiple-instance learning (MIL) anomaly detection framework with the dual-branch network outperforms than existing approaches and obtains the state-of-the-art performance.

In this paper, we propose an online crowd segmentation method based on reinforcement learning. Existing approaches for the task suffers from the fixed parameters applied to different sceneries. We propose to utilize the reinforcement learning to adaptively adjust the critical parameters for better clustering. Specifically, we first propose the velocity-constrained natural nearest neighbor algorithm to adaptively determine the original fixed parameter K in K-nearest-neighbor, thus better reflecting the neighborhood relationship in velocity correlation calculation. Then, we construct an online reinforcement learning module in the grouping process to adaptively decide the segmentation threshold. Furthermore, we combine the motion image semantics with feature point grouping to obtain the pixel-level segmentation. This method enhances the generalization and robustness in different scenarios to increase the accuracy. Comprehensive experimental results conducted on different datasets demonstrate the effectiveness and validity of our method.

Group re-identification (Re-ID) is an important task of computer vision and involves multiple challenges. In this paper, we propose a Hybrid Attention Model (HAM) to address the problem of group Re-ID, solving the spatial variation in the challenging still-image-based group Re-ID task. HAM consists of both the position and channel attention to make the network focus more on the crucial areas and features of the group images. Furthermore, we propose a novel Least Squares Residual Distance (LSRD) based on the least squares algorithm. LSRD can leverage the residual of the fitting function achieved by least squares method, better learning the metric between group image pairs. To evaluate the performance, we propose a new largest group Re-ID dataset. Extensive experimental results conducted on the dataset demonstrate the effectiveness of our approaches.

In this paper, we propose a method to further improve person search by fully utilizing the combination of pedestrian detection and person re-identification tasks. An improved constrain that utilizes comprehensive samples in the dataset is proposed to fully excavate information for recognition. Besides the label constrain for training the model in traditional classification task, unlabeled identities that do not have specific IDs are utilized as well to constitute a tailored triplet loss for more performance improvement. Meanwhile, a novel large-scale challenging dataset, SJTU318, which uses videos acquired through twelve cameras is proposed to demonstrate the effectiveness of our method. It contains 443 identities and 14,610 frames in which pedestrians are annotated with their bounding box positions and identities. Experiments conducted on a public dataset, CUHK-SYSU and our proposed dataset SJTU318 show that our method outperforms existing state-of-the-art approaches.

In this paper, we explore ways to address the challenges such as data bias caused by the lack of data on person re-identification problem. We propose a data generation framework from both intra- and inter-view aspects for data augmentation to advance the performance of the existing person re-identification algorithms. Specifically, for intra-view data generation, the proposed method generates useful predicted sequences within a camera view for certain person data expansion. The generated sequences well preserve the movement information of the camera and objects, which expands the original data with longer sequence length to tackle the problem caused by insufficient data from the root. For more challenging datasets which suffer from background clutters, we propose an inter-view image generation with automatic end-to-end background substitution to eliminate the influence by the background and increase the diversity of the training data as well, which makes the recognition system learn to focus on the regions of objects and image features related to identity. We then propose a flexible data augmentation method based on our data generation approaches to improve the performance of the person re-identification and analyze the advantages and applicability of these approaches respectively. Evaluated on the challenging re-id datasets, our method outperforms existing state-of-the-art approaches without any network structure modification on the baseline neural network. Cross-datasets evaluation results show that our method has favorable generalization ability and is potentially helpful for solving similar recognition tasks due to the common issue of insufficient data.