Large 3D shape repositories are rapidly expanding, driven by advances in generative modeling, making efficient shape retrieval increasingly important for authoring tools. While text queries capture high-level semantics, they often fail to convey precise geometric details. 3D sketches provide a more expressive means of representing shape geometry, and recent AR/VR developments have made sketch-based retrieval practical. However, existing 3D sketch datasets face three major limitations: (1) reliance on quad meshes or voxel hulls, which often fail on complex or non-manifold shapes; (2) use of fixed-size point clouds that discard stroke connectivity and limit geometric fidelity; and (3) dependence on expensive curve-based or multi-view rendering pipelines, which hinder large-scale data generation. Limited point cloud representations also fail to capture sketch connectivity and topology when used to train retrieval models. To address these challenges, we propose Sketch3R, a scalable framework that converts arbitrary 3D meshes into human-like VR sketches using a graph-based representation that preserves stroke connectivity and adapts to sketch complexity. Leveraging this representation, Sketch3R employs a lightweight graph-attention Siamese network for efficient and accurate sketch-to-shape retrieval. Experiments demonstrate that our method outperforms prior approaches in both accuracy and speed, while robustly handling 3D shapes across diverse topologies.

We present SketchTo3DGen, a novel system for rapid 3D content ideation on a VR headset. SketchTo3DGen combines freehand 3D sketching and audio descriptions to generate photo-realistic 3D assets on-the-fly. Running on a Meta Quest headset with a Unity application, our system leverages remote GPU-accelerated services for AI-driven content creation using intuitive inputs. The user can draw a 3D sketch in mid-air and describe the intended asset verbally; our pipeline transcribes and normalizes the speech into a text prompt, selects informative viewpoints of the 3D VR sketch, generates corresponding images via a state-of-the-art text-to-image model, and finally reconstructs a 3D mesh using an image-to-3D generator. The entire workflow is experienced in VR with minimal interface elements. We describe the design motivations, technical pipeline, and user interaction details of SketchTo3DGen. This VR in-headset pipeline, using intuitive inputs in the form of hand-drawn 3D VR sketch and speech, streamlines 3D modeling, accelerating the generation of articulation ready 3D assets.

Generating realistic one-shot emotional talking head animation on arbitrary faces is a challenging problem, as it requires realistic emotions, head movements, identity preser-vation, and accurate lip sync. Existing emotional talking face generation methods either fail to retain the identity information of arbitrary subjects owing to the limited variability of existing emotional datasets, or they fail to capture emotions accurately even if they preserve identity of arbitrary faces. Moreover, most of the methods rely on additional input videos for driving poses and/or or expressions on the generated video. For practical applications, it is in-feasible to obtain driving videos of the same or different subject with variations in head pose, expressions etc. In this paper, we propose a novel approach for Audio-driven Emotional Talking Head generation from a single image, with emotion-controllable head pose generation. Unlike existing methods, our method does not require a driving video either for pose or emotions, and can generate different emotions and diverse head pose variations from input speech and a single image of an arbitrary subject in neutral emotion. Our method overcomes the limitations of existing emotional audio-visual datasets by learning a disentangled approach for optical flow computation approach for pose and expression. Using our proposed method of independently computing pose-driven and expression-driven optical flow, our image generation network can be pretrained on a large dataset with greater pose variability but lacking emotion annotations. The expression flow generation branch is fine-tuned on a smaller emotional dataset to accurately capture different emotions not present in the original dataset, while retaining the pose variability from the original dataset. We present extensive experiments to demonstrate the superior-ity of our proposed method in generating talking head animation with accurate emotions, diverse head movements, and generalization to arbitrary faces.

We present a particle-based neural garment simulator (dubbed as GarSim) that can simulate template garments on the target arbitrary body poses. Existing learning-based methods majorly work for specific garment type (e.g. top, skirt, etc) or garment topology, and needs retraining for a new type of garment. Similarly, some methods focus on a particular fabric, body shape, and pose. To circumvent these limitations, our method fundamentally learns the physical dynamics of the garment vertices conditioned on underlying body shape, motion, and fabric properties to generalize across garment types, topology, and fabric along with different body shape and pose. In particular, we represent the garment as a graph, where the nodes represent the physical state of the garment vertices, and the edges represent the relation between the two nodes. The nodes and edges of the garment graph encode various properties of garments and the human body to compute the dynamics of the vertices through a learned message-passing. Learning of such dynamics of the garment vertices conditioned on underlying body motion and fabric properties enables our method to be trained simultaneously for multiple types of garments (e.g., tops, skirts, etc) with arbitrary mesh resolutions, varying topologies, and fabric properties. Our experimental results show that GarSim with less amount of training data not only outperforms the SOTA methods on challenging CLOTH3D dataset both qualitatively and quantitatively, but also works reliably well on the unseen poses obtained from YouTube videos, and give satisfactory results on unseen cloth types which were not present during the training.

Deep Neural Networks (DNNs) are often criticized for being susceptible to adversarial attacks. Most successful defense strategies adopt adversarial training or random input transformations that typically require retraining or finetuning the model to achieve reasonable performance. In this work, our investigations of intermediate representations of a pre-trained DNN lead to an interesting discovery pointing to intrinsic robustness to adversarial attacks. We find that we can learn a generative classifier by statistically characterizing the neural response of an intermediate layer to clean training samples. The predictions of multiple such intermediate-layer based classifiers, when aggregated, show unexpected robustness to adversarial attacks. Specifically, we devise an ensemble of these generative classifiers that rank-aggregates their predictions via a Borda count-based consensus. Our proposed approach uses a subset of the clean training data and a pre-trained model, and yet is agnostic to network architectures or the adversarial attack generation method. We show extensive experiments to establish that our defense strategy achieves state-of-the-art performance on the ImageNet validation set.

In this paper, we present an automatic multi-model fitting pipeline that can robustly fit multiple geometric models present in the corrupted and noisy data. Our approach can handle large data corruption and requires no user input, unlike most state-of-the-art approaches. The pipeline can be used as an independent block in many geometric vision applications like 3D reconstruction, motion and planar segmentation. We use residual density as the primary tool to guide hypothesis generation, estimate the fraction of inliers, and perform model selection. We show results for a diverse set of geometric models like planar homographies, fundamental matrices and vanishing points, which often arise in various computer vision applications. Despite being fully automatic, our approach achieves competitive performance compared to state-of-the-art approaches in terms of accuracy and computational time.

We propose a fast and efficient two-stage hypothesis filtering technique that can improve performance of clustering based robust multi-model fitting algorithms. Sampling based hypothesis generation is nondeterministic and permits little control over generating poor model hypotheses, often leading to a significant proportion of bad hypotheses. Our novel filtering approach leverages the asymmetry in the distributions of points around the inlier/outlier boundary via the sample skewness computed in the residual space. The output is a set of promising hypotheses which aid multi-model fitting algorithms in improving accuracy as well as running time. We validate our approach on the AdelaideRMF dataset and show favorable results along with comparisons to state-of-the-art.