We present a stereo-matching method for depth estimation from high-resolution images using visual hulls as priors, and a memory-efficient technique for the correlation computation. Our method uses object masks extracted from supplementary views of the scene to guide the disparity estimation, effectively reducing the search space for matches. This approach is specifically tailored to stereo rigs in volumetric capture systems, where an accurate depth plays a key role in the downstream reconstruction task. To enable training and regression at high resolutions targeted by recent systems, our approach extends a sparse correlation computation into a hybrid sparse-dense scheme suitable for application in leading recurrent network architectures. We evaluate the performance-efficiency tradeoff of our method compared to state-of-the-art approaches and demonstrate the efficacy of the visual hull guidance. In addition, we propose a training scheme for a further reduction of memory requirements during optimization, facilitating training on high-resolution data.

Video frame interpolation has seen important progress in recent years, thanks to developments in several directions. Some works leverage better optical flow methods with improved splatting strategies or additional cues from depth, while others have investigated alternative approaches through direct predictions or transformers. Still, the problem remains unsolved in more challenging conditions such as complex lighting or large motion. In this work, we are bridging the gap towards video production with a novel transformer-based interpolation network architecture capable of estimating the expected error together with the interpolated frame. This offers several advantages that are of key importance for frame interpolation usage: First, we obtained improved visual quality over several datasets. The improvement in terms of quality is also clearly demonstrated through a user study. Second, our method estimates error maps for the interpolated frame, which are essential for real-life applications on longer video sequences where problematic frames need to be flagged. Finally, for rendered content a partial rendering pass of the intermediate frame, guided by the predicted error, can be utilized during the interpolation to generate a new frame of superior quality. Through this error estimation, our method can produce even higher-quality intermediate frames using only a fraction of the time compared to a full rendering.

Research into non-line-of-sight imaging problems has gained momentum in recent years motivated by intriguing prospective applications in e.g. medicine and autonomous driving. While transient image formation is well understood and there exist various reconstruction approaches for non-line-of-sight scenes that combine efficient forward renderers with optimization schemes, those approaches suffer from runtimes in the order of hours even for moderately sized scenes. Furthermore, the ill-posedness of the inverse problem often leads to instabilities in the optimization.Inspired by the latest advances in direct-line-of-sight inverse rendering that have led to stunning results for reconstructing scene geometry and appearance, we present a fast differentiable transient renderer that accelerates the inverse rendering runtime to minutes on consumer hardware, making it possible to apply inverse transient imaging on a wider range of tasks and in more time-critical scenarios. We demonstrate its effectiveness on a series of applications using various datasets and show that it can be used for self-supervised learning.

Abstract 2D Confocal Raman Microscopy (CRM) data consist of high dimensional per‐pixel spectral data of 1000 bands and allows for complex spectral and spatial‐spectral analysis tasks, i.e., in material discrimination, material thickness, and spatial material distributions. Currently, simple integral methods are commonly applied as visual analysis solutions to CRM data which exhibit restricted discrimination power in various regards. In this paper we present a novel approach for the visual analysis of 2D multispectral CRM data using multi‐variate visualization techniques. Due to the large amount of data and the demand of an explorative approach without a‐priori restriction, our system allows for arbitrary interactive (de)selection of varaibles w/o limitation and an unrestricted online definition/construction of new, combined properties. Our approach integrates CRM specific quantitative measures and handles material‐related features for mixed materials in a quantitative manner. Technically, we realize the online definition/construction of new, combined properties as semi‐automatic, cascaded, 1D and 2D multidimensional transfer functions (MD‐TFs). By interactively incorporating new (raw or derived) properties, the dimensionality of the MD‐TF space grows during the exploration procedure and is virtually unlimited. The final visualization is achieved by an enhanced color mixing step which improves saturation and contrast.