Manual creation of character animations using traditional tools is tedious and cumbersome for a novice animator. Video-to-image motion retargeting emerges as an automated approach for animating an input image using an exemplar video. State-of-the-art animation methods often produce artifacts in presence of drastic pose changes within a video. To address this issue, we propose LiveImage, which handles the large pose problem and overcomes various artifacts and shape distortions in the generated video frames. LiveImage uses a single-domain motion model with keypoint regularization first to produce coarse-grained frames that effectively capture motion and structural information. A motion condition network then estimates fine-grained video frames using denoising diffusion model to counteract unnatural artifacts and distortions arising from large pose changes. We perform comprehensive qualitative and quantitative evaluations, demonstrating its efficacy in overcoming challenges associated with motion transfer from video to image.

Abstract Animation of hand‐drawn sketches is an adorable art. It allows the animator to generate animations with expressive freedom and requires significant expertise. In this work, we introduce a novel sketch animation framework designed to address inherent challenges, such as motion extraction, motion transfer, and occlusion. The framework takes an exemplar video input featuring a moving object and utilizes a robust motion transfer technique to animate the input sketch. We show comparative evaluations that demonstrate the superior performance of our method over existing sketch animation techniques. Notably, our approach exhibits a higher level of user accessibility in contrast to conventional sketch‐based animation systems, positioning it as a promising contributor to the field of sketch animation. https://graphics-research-group.github.io/SketchAnim/

Recent research has focused on the effectiveness of Virtual Reality (VR) in games as a more immersive method of interaction. However, there is a lack of robust analysis of the physiological effects between VR and flatscreen (FS) gaming. This paper introduces the first systematic comparison and analysis of emotional and physiological responses to commercially available games in VR and FS environments. To elicit these responses, we first selected four games through a pilot study of 6 participants to cover all four quadrants of the valence-arousal space. Using these games, we recorded the physiological activity, including Blood Volume Pulse and Electrodermal Activity, and self-reported emotions of 33 participants in a user study. Our data analysis revealed that VR gaming elicited more pronounced emotions, higher arousal, increased cognitive load and stress, and lower dominance than FS gaming. The Virtual Reality and Flat Screen (VRFS) dataset, containing over 15 hours of multimodal data comparing FS and VR gaming across different games, is also made publicly available for research purposes. Our analysis provides valuable insights for further investigations into the physiological and emotional effects of VR and FS gaming.

Training systems are used in many industries, ranging from surgery to space missions to rehabilitation. Virtual Reality (VR) is a technology that has been incorporated as an effective tool in such training systems to simulate the environment, especially in situations where the training can’t take place in the actual environment. For a training environment and task to be effective, it must sufficiently challenge the trainee. One parameter that can be used to measure this is cognitive load (CL), which is defined as the amount of working memory used while performing a learning task. This parameter needs to be sufficiently high to maximize learning but not too high as to overload the trainee. However, the challenge is to detect this state using objective physiological measures, which can be collected during the entire task. This paper presents a study to classify CL using a combination of Electroencephalogram (EEG) and Electrodermal Activity (EDA) signals during a procedural VR training task. Thirty participants undertook a study where they built a designated model within a given time over multiple levels that were constructed to induce low to high CL. Features generated from the data were subject to feature selection (FS), which was undertaken using the Mutual Information (MI) technique. Binary classification models were developed using Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (kNN), Extreme Gradient Boosting (Xgboost) and Multi-Layer Perceptrons (MLP). Results illustrated that the Xgboost classifier performed the best with an F1-score of $0.831 \pm 0.030$ and accuracy of $0.805 \pm 0.033.$ SHAP analysis of the features illustrated greater contributions from the frontal and occipital regions of the brain and frequency domain features from tonic skin conductance.

We propose SLI-pSp, a general purpose Image-to-Image (I2I) translation model that encodes spatial layout information as well as style in the generator, using pSp as the base architecture. Previous methods like pSp have shown promising results by leveraging StyleGAN as a generator in various I2I tasks but they seem to miss finer or underrepresented details in facial images like earrings and caps, and break down on complex datasets due to their solely global approach. To address these shortcomings, we propose a technique termed Spatial Layout Injection (SLI-pSp) that encodes spatial layout information in the input image in the StyleGAN generator along with style. We do so without modifying the style vector injection in the generator through pSp’s map2style network, but rather by combining SLI with noise layers in the StyleGAN generator at multiple spatial scales. Such an approach helps preserve global aspects of image generation as well as enhance spatial layout details in the output. We experiment on several challenging datasets and across several I2I tasks that highlight the effectiveness of our approach over previous methods with respect to finer details in the generated image and overall visual quality.

Though Chaos Engineering is a popular method to test reliability and performance assurance, available tools can only inject random or manually curated faults into a target system. Given the vast array of faults that can be injected, it is crucial to a.) intelligently pick the faults that can have tangible effects, b.) increase the test coverage, and c.) reduce the overall time needed to assess the reliability of a system under adverse conditions. To the effect, we are proposing to learn from past major outages and use genetic algorithm-based meta-heuristics to evolve complex fault injections.

Colorization involves the synthesis of colors while preserving structural content as well as the semantics of the target image. This problem has been well studied for 2D photographs with many state-of-the-art solutions. We explore a new challenge in the field of colorization where we aim at colorizing multi-modal 3D medical data using 2D style exemplars. To the best of our knowledge, this work is the first of its kind and poses challenges related to the modality (medical MRI) and dimensionality (3D volumetric images) of the data. Our approach to colorization is motivated by modality conversion that highlights its robustness in handling multi-modal data.

Abstract A good transfer function in volume rendering requires careful consideration of the materials present in a volume. A manual creation is tedious and prone to errors. Furthermore, the user interaction to design a higher dimensional transfer function gets complicated. In this work, we present a graph‐based approach to design a transfer function that takes volumetric structures into account. Our novel contribution is in proposing an algorithm for robust deduction of a material graph from a set of disconnected edges. We incorporate stable graph creation under varying noise levels in the volume. We show that the deduced material graph can be used to automatically create a transfer function using the occlusion spectrum of the input volume. Since we compute material topology of the objects, an enhanced rendering is possible with our method. This also allows us to selectively render objects and depict adjacent materials in a volume. Our method considerably reduces manual effort required in designing a transfer function and provides an easy interface for interaction with the volume.

In this work we present a redirected walking scheme suitable for shared spaces in a virtual reality environment. We show our redirected walking to work for the case of two physical spaces (a host and a guest) being merged into a single virtual host space. The redirection is based on warping the guest space into the host space using a conformal mapping that preserves local shape and features.

Level set method based segmentation provides an efficient tool for topological and geometrical shape handling. Conventional level set surfaces are only C(0) continuous since the level set evolution involves linear interpolation to compute derivatives. Bajaj et al. present a higher order method to evaluate level set surfaces that are C(2) continuous, but are slow due to high computational burden. In this paper, we provide a higher order GPU based solver for fast and efficient segmentation of large volumetric images. We also extend the higher order method to multi-domain segmentation. Our streaming solver is efficient in memory usage.