Abstract In many computer vision applications, datasets often exhibit an underlying taxonomy within the label space. To adhere to this hierarchical structure, hyperbolic spaces have emerged as an effective manifold for representation learning, thanks to their ability to encode hierarchical relationships, with little distortion, even for low-dimensional embeddings. Hyperbolic prototypical learning, where class labels are represented by prototypes, has recently demonstrated strong potential in this setting. However, existing methods generally assume a uniform distribution of prototypes, overlooking the hierarchical organization of labels that may be available for a given task. To better exploit this prior knowledge, we propose a hierarchically informed method for prototype positioning. Our approach leverages the Gromov-Wasserstein distance to align the hierarchical relationships between labels with the initial uniform spherical distribution of prototypes, leading to more structured and semantically meaningful representations. Additionally, within a deep learning framework, we propose an alternative characterization of decision boundaries using horospheres, which are level sets of the Busemann function. Geometrically, horospheres correspond to spheres tangent to the boundary of hyperbolic space at a virtual point analogous to a prototype, which makes them a compliant tool in the prototypical learning context. Accordingly, we define a new horospherical layer that can be adapted to any neural network backbone. This layer is particularly advantageous when the number of prototypes exceeds the number of classes, offering enhanced flexibility to the classifier. Through our experiments, we demonstrate that the combination of proper initialization and optimized prototype positioning significantly enhances baseline performance for image classification on hierarchical datasets. Additionally, we validate our approach in two semantic segmentation tasks, using both image and point cloud datasets, confirming its effectiveness in leveraging hierarchical label structures for improved classification performance.

Learning models on one labeled dataset that generalize well on another domain is a difficult task, as several shifts might happen between the data domains. This is notably the case for lidar data, for which models can exhibit large performance discrepancies due for instance to different lidar patterns or changes in acquisition conditions. This paper addresses the corresponding Unsupervised Domain Adaptation (UDA) task for semantic segmentation. To mitigate this problem, we introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. This novel strategy differs from classical minimization of statistical divergences or lidar-specific domain adaptation techniques. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.The project repository: github.com/valeoai/SALUDA

We propose a new self-supervised method for pre-training the backbone of deep perception models operating on point clouds. The core idea is to train the model on a pretext task which is the reconstruction of the surface on which the 3D points are sampled, and to use the underlying latent vectors as input to the perception head. The intuition is that if the network is able to reconstruct the scene surface, given only sparse input points, then it probably also captures some fragments of semantic information, that can be used to boost an actual perception task. This principle has a very simple formulation, which makes it both easy to implement and widely applicable to a large range of 3D sensors and deep networks performing semantic segmentation or object detection. In fact, it supports a single-stream pipeline, as opposed to most contrastive learning approaches, allowing training on limited resources. We conducted extensive experiments on various autonomous driving datasets, involving very different kinds of lidars, for both semantic segmentation and object detection. The results show the effectiveness of our method to learn useful representations without any annotation, compared to existing approaches. The code is available at github.com/valeoai/ALSO

While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images, its application to 3D data is still recent and scarce, with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data, that can handle both classification and, for the first time, semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation, we created three benchmarks for evaluating this new ZSL task, using S3DIS, ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines, which we additionally propose for this task.