We introduce HiT, a novel hierarchical neural field representation for 3D shapes that learns general hierarchies in a coarse-to-fine manner across different shape categories in an unsupervised setting. Our key contribution is a hierarchical transformer (HIT), where each level learns parent–child relationships of the tree hierarchy using a compressed codebook. This codebook enables the network to automatically identify common substructures across potentially diverse shape categories. Unlike previous works that constrain the task to a fixed hierarchical structure (e.g., binary), we impose no such restriction, except for limiting the total number of nodes at each tree level. This flexibility allows our method to infer the hierarchical structure directly from data, over multiple shape categories, and representing more general and complex hierarchies than prior approaches. When trained at scale with a reconstruction loss, our model captures meaningful containment relationships between parent and child nodes. We demonstrate its effectiveness through an unsupervised shape segmentation task over all 55 ShapeNet categories, where our method successfully segments shapes into multiple levels of granularity. Project Page: aditya-vora.github.io/HiT/

We introduce ASIA (Adaptive 3D Segmentation using few Image Annotations), a novel framework that enables segmentation of possibly non-semantic and non-text describable "parts" in 3D. Our segmentation is controllable through a few user-annotated in-the-wild images, which are easier to collect than multi-view images, less demanding to annotate than 3D models, and more precise than potentially ambiguous text descriptions. Our method leverages the rich priors of text-to-image diffusion models, such as Stable Diffusion, to transfer segmentations from image space to 3D, even when the annotated and target objects differ significantly in geometry or structure. During training, we optimize a text token for each segment and fine-tune our model with a novel cross-view part correspondence loss. At inference, we segment multi-view renderings of the 3D mesh, fuse the labels in UV-space via voting, refine them with our novel Noise Optimization technique, and finally map the UV-labels back onto the mesh. ASIA provides a practical and generalizable solution for both semantic and non-semantic 3D segmentation tasks, outperforming existing methods by a noticeable margin in both quantitative and qualitative evaluations.

We present a volume rendering-based neural surface reconstruction method that takes as few as three disparate RGB images as input. Our key idea is to regularize the reconstruction, which is severely ill-posed and leaving significant gaps between the sparse views, by learning a set of neural templates that act as surface priors. Our method, coined DiViNet, operates in two stages. The first stage learns the templates, in the form of 3D Gaussian functions, across different scenes, without 3D supervision. In the reconstruction stage, our predicted templates serve as anchors to help “stitch” the surfaces over sparse regions. We demonstrate that our approach is not only able to complete the surface geometry but also reconstructs surface details to a reasonable extent from few disparate input views. On the DTU and BlendedMVS datasets, our approach achieves the best reconstruction quality among existing methods in the presence of such sparse views and performs on par, if not better, with competing methods when dense views are employed as inputs.

The fidelity of a pose transfer system depends on its ability to generate realistic images of a person under novel poses while preserving the desired human attributes (like face, hairstyle, and clothes). However, the visual fidelity is often compromised as the existing methods fail to extract rich appearance and pose features since they propagate the pose and the appearance information through the same pathway. Also, the repeated downsampling in these pathways leads to the loss of finer details, thus producing blurry results. Further, these methods use vanilla convolution that treats all the pixels as important and fail to focus primarily on significant regions needed for the desired transformation. This work proposes an appearance-consistent human pose transfer framework that progressively transforms the person in the source image to the desired target pose using the information from three pathways: an image pathway, a pose pathway, and an appearance pathway. We propose the use of gated convolution to dynamically extract features relevant for generating the transformed image. The appearance pathway generates an appearance code to produce an image consistent in appearance with that of the source image. We establish the efficacy of the proposed framework through an extensive set of experiments on DeepFashion, Market-1501, and the Action Class dataset. We also generate coherent action sequences through a given set of desired poses from the action class dataset that contains humans in three actions: golf, yoga/workouts, and tennis.