Generating 3D images of complex objects conditionally from a few 2D views is a difficult synthesis problem, compounded by issues such as domain gap and geometric misalignment. For instance, a unified framework such as Generative Adversarial Networks cannot achieve this unless they explicitly define both a domain-invariant and geometric-invariant joint latent distribution, whereas Neural Radiance Fields are generally unable to handle both issues as they optimize at the pixel level. By contrast, we propose a simple and novel 2D to 3D synthesis approach based on conditional diffusion with vector-quantized codes. Operating in an information-rich code space enables high-resolution 3D synthesis via full-coverage attention across the views. Specifically, we generate the 3D codes (e.g. for CT images) conditional on previously generated 3D codes and the entire codebook of two 2D views (e.g. 2D X-rays). Qualitative and quantitative results demonstrate state-of-the-art performance over specialized methods across varied evaluation criteria, including fidelity metrics such as density, coverage, and distortion metrics for two complex volumetric imagery datasets from in real-world scenarios.

Deep generative models are a class of techniques that train deep neural networks to model the distribution of training samples. Research has fragmented into various interconnected approaches, each of which make trade-offs including run-time, diversity, and architectural restrictions. In particular, this compendium covers energy-based models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flows, in addition to numerous hybrid approaches. These techniques are compared and contrasted, explaining the premises behind each and how they are interrelated, while reviewing current state-of-the-art advances and implementations.

This paper presents an algorithm, similar to implicit sphere tracing, that ray marches 3D non-convex shapes for efficient collision detection. Instead of finding points on the surface where individual rays strike, an entire shape is marched in unison by a lower bound of the boundary distance, calculated at the closest point between the two surfaces. Advancing one shape towards the other by this new bound allows us to identify a contact in few steps. This method supports arbitrary nonconvex shapes, and can be run in parallel. We apply this to protein-protein docking and show that we can identify around 80 docking poses per second featuring contact but no overlap, irrespective of proteins' specific geometry. This paves the way to future fast docking algorithms, building upon implicit surface representations to quickly find a well-distributed subset of close candidate solutions for further investigation.