Due to the irregular and disordered data structure in 3D point clouds, prior works have focused on designing more sophisticated local representation methods to capture these complex local patterns. However, the recognition performance has saturated over the past few years, indicating that increasingly complex and redundant designs no longer make improvements to local learning. This phenomenon prompts us to diverge from the trend in 3D vision and instead pursue an alternative and successful solution: deeper neural networks. In this paper, we propose DeepLA-Net, a series of very deep networks for point cloud analysis. The key insight of our approach is to exploit a small but mighty local learning block, which uses 10× fewer FLOPs, enabling the construction of very deep networks. Furthermore, we design a training supervision strategy to ensure smooth gradient backpropagation and optimization in very deep networks. We construct the DeepLA-Net family with a depth of up to 120 blocks — at least 5× deeper than recent methods — trained on a single RTX 3090. An ensemble of the DeepLA-Net achieves state-of-the-art performance on classification and segmentation tasks of S3DIS Area5 (+2.2% mIoU), ScanNet test set (+1.6% mIoU), ScanObjectNN (+2.1% OA), and ShapeNet-Part (+0.9% cls.mIoU). The code are released at https://github.com/zeng-ziyin/DeepLA-Net.

LiDAR segmentation is crucial for autonomous driving perception. Range view methods have been widely adopted for these applications due to their intuitiveness and ease of implementation. However, the inherent shortcomings of the range view approach (e.g., assuming that point clouds within the same pixel of a range image have the same semantic class) make it difficult to perform accurate fine-grained segmentation tasks, thus limiting its potential in practical applications. To address these issues, we propose RangeFusion, an end-to-end framework that greatly improves the ability to learn and process LiDAR point clouds from range views by employing a multispatial learning model. A novel range-scan space (RSS) is proposed to address the inability of existing range view methods to accurately aggregate features of neighboring points. This space achieves accurate and efficient neighboring point feature aggregation with linear time complexity. In addition, a supervised label smoothing method called multilevel feature selection heads (MFSHs) is designed, which achieves more fine-grained semantic prediction by subdividing the full point cloud into multisemantic hierarchical subclouds and adaptively fusing the features with confidence filtering. The performance of the proposed method was evaluated on several benchmarks, including SemanticKITTI and nuScenes. On these two datasets, mean intersection over union (mIoU) scores of 67.9% and 80.2% were achieved, respectively. This demonstrates that the proposed approach outperforms existing range view- and multiview-based approaches while maintaining efficient performance at 26.5 FPS. In addition, real road data were collected for testing. The code is available athttps://github.com/Wansit99/RangeFusion.

Text-based Person Search (TBPS) aims to retrieve the person images using natural language descriptions. Recently, Contrastive Language Image Pretraining (CLIP), a universal large cross-modal vision-language pre-training model, has remarkably performed over various cross-modal downstream tasks due to its powerful cross-modal semantic learning capacity. TPBS, as a fine-grained cross-modal retrieval task, is also facing the rise of research on the CLIP-based TBPS. In order to explore the potential of the visual-language pre-training model for downstream TBPS tasks, this paper makes the first attempt to conduct a comprehensive empirical study of CLIP for TBPS and thus contribute a straightforward, incremental, yet strong TBPS-CLIP baseline to the TBPS community. We revisit critical design considerations under CLIP, including data augmentation and loss function. The model, with the aforementioned designs and practical training tricks, can attain satisfactory performance without any sophisticated modules. Also, we conduct the probing experiments of TBPS-CLIP in model generalization and model compression, demonstrating the effectiveness of TBPS-CLIP from various aspects. This work is expected to provide empirical insights and highlight future CLIP-based TBPS research.

Few-shot image classification aims to classify images from unseen novel classes with few samples. Recent works demonstrate that deep local descriptors exhibit enhanced representational capabilities compared to image-level features. However, most existing methods solely rely on either employing all local descriptors or directly utilizing partial descriptors, potentially resulting in the loss of crucial information. Moreover, these methods primarily emphasize the selection of query descriptors while overlooking support descriptors. In this paper, we propose a novel Task-Aware Adaptive Local Descriptors Selection Network (TALDS-Net), which exhibits the capacity for adaptive selection of task-aware support descriptors and query descriptors. Specifically, we compare the similarity of each local support descriptor with other local support descriptors to obtain the optimal support descriptor subset and then compare the query descriptors with the optimal support subset to obtain discriminative query descriptors. Extensive experiments demonstrate that our TALDS-Net outperforms state-of-the-art methods on both general and fine-grained datasets1.

Few-shot image classification is a challenging task aim to classific unseen images in scenarios with limited samples. Recent work demonstrate that local discriminative features have better representational capabilities than global features. In this paper, we propose a novel method to extract local discriminative features from images and calculate better class representations through contrastive learning. We divide the training paradigm into two stages, referred to as pre-training and meta-training. In the pre-training stage, we employ a global contrastive loss and introduce an improved Mutual Maximum Local Matching Contrastive Loss (MMCL) to obtain local discriminative features while disregarding irrelevant background features. During the meta-training stage, we introduce a new contrastive learning-based Adaptive Class Representation Computation Network (ACRC-Net) to enhance the generalization capability of class representations. It has the ability to adaptively compute class representations, specifically by comparing the similarity between features of the same and different classes within the same tasks to obtain the optimal subset of class representations. Based on the optimal subset of representations, we obtain the final class representation. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods on general datasets.

Semantic segmentation of large-scale point clouds provides foundational knowledge for various geodetic and cartographic applications, including autonomous driving, smart cities, and indoor navigation. However, point cloud data’s unstructured and inherently disordered characteristics pose challenges in extracting accurate 3-D semantic information. In this study, we introduce a novel semantic segmentation network for large-scale point cloud scenes, referred to as dense dual-branch cross attention network (D2CAN). We propose a local multidimensional feature aggregation (LMFA) module to increase multidimensional feature representation types and preserve rich local details. Based on the augmented local features, an expanded dual-branch cross attention (EDCA) module establishes internal deep connections between multidimensional attributes and semantic features. This assists the network in reducing boundary ambiguities and expanding the receptive field, enabling the parallel capture of long-range contexts specifically adapted for large-scale scene point cloud segmentation. These two modules work collaboratively to constitute a local context deep perception (LCDP) block. To reduce information loss during feature sampling and propagation, we propose a global feature pyramid dense fusion (GFDF) block. This block adaptively integrates features across different scales and effectively captures global context with long-range dependencies. In conclusion, D2CAN combines LCDP and GFDF to aggregate both local and global contexts, resulting in robust feature discrimination for semantic segmentation of large-scale scenes. Our method’s effectiveness and superior generation ability have been validated across three challenging benchmarks and achieve state-of-the-art performance on Toronto-3D, SensatUrban, and Stanford large-scale 3-D indoor spaces (S3DIS) datasets, with mean intersection over union (IoU) values of 83.5%, 61.1%, and 72.3%, respectively.

Given the prominence of 3D sensors in recent years, 3D point clouds are worthy to be further investigated for environment perception and scene understanding. Learning accurate local and global contexts in point clouds is pivotal for semantic segmentation, and neighbor aggregation and Transformers have achieved notable success in local and global perception in point cloud analysis, respectively. Nevertheless, studying each independently is far from the optimal solution for comprehensive feature learning. To address this, we take a novel step towards investigating and integrating the structures of neighbor aggregation and Transformers. In this paper, we introduce Point Neighbor Aggregation with Transformer (PointNAT), a conceptually straightforward and effective approach aiming to enhance the performance of 3D point cloud semantic segmentation. PointNAT consists of a Neighbor Aggregation Block (NAB) for local perception, a Point Transformer Block (PTB) for global modeling, and a Hybrid Block to connect NABs and PTBs. NABs effectively learn complex local features at varying scales through an improved neighbor aggregation operation and a multi-head mechanism. PTBs efficiently perform global attention using a small set of learnable key points. Hybrid Blocks serve as high-and-low frequency signal hybridizers, merging the strengths of these two blocks by adaptively assigning hybrid weights to local and global contexts. We have evaluated the performance of PointNAT with state-of-the-art networks on several benchmarks, including S3DIS, Toronto3D, and SensatUrban. PointNAT achieves mIoU scores of 77.8%, 84.7%, and 65.2% in these three dataset, respectively. Furthermore, it outperforms the baseline approach PointNeXt by 3.0%, 1.3%, and 4.2%, respectively, while utilizing only 59.9% of the parameters and 15.2% of the FLOPs. The results demonstrate PointNAT’s superior ability in accurately segmenting large-scale 3D point cloud scenes, emphasizing its potential to advance environment perception and scene understanding. Our code is available at https://github.com/zeng-ziyin/PointNAT.