Homography estimation is crucial for image stitching in autonomous vehicle applications, enabling seamless multi-view transformations. While convolutional neural networks (CNNs) have achieved state-of-the-art performance, they often struggle with capturing global context, limiting their effectiveness. In this paper, we propose a Pyramidal Vision Transformer (PViT) for homography estimation, which integrates a transformer encoder for global feature extraction and convolutional layers for local feature refinement. Our method is evaluated on an image stitching task, demonstrating 75% and 77% error reduction compared to conventional and learning-based homography estimation approaches, respectively. Additionally, our model achieves an efficient performance by reducing computation time, making it suitable for deployment in autonomous vehicle vision systems.

3D face animation has been a critical component of character animation in a wide range of media since the early 90’s. The conventional process for animating a 3D face is usually keyframe-based, which is labor-intensive. Therefore, the film and game industries have started using live-action actors’ performances to animate the faces of 3D characters, the process is also known as performance-driven facial animation. At the core of performance-driven facial animation is facial motion retargeting, which transfers the source facial motions to a target 3D face. However, facial motion retargeting still has many limitations that influence its capability to further assist the facial animation process. Existing motion retargeting frameworks cannot accurately transfer the source motion’s semantic information (i.e., meaning and intensity of the motion), especially when applying the motion to non-human-like or stylized target characters. The retargeting quality relies on the parameterization of the target face, which is time-consuming to build and usually not generalizable across proportionally different faces. In this survey paper, we review the literature relating to 3D facial motion retargeting methods and the relevant topics within this area. We provide a systematic understanding of the essential modules of the retargeting pipeline, a taxonomy of the available approaches under these modules, and a thorough analysis of their advantages and limitations with research directions that could potentially contribute to this area. We also contributed a 3D character categorization matrix, which has been used in this survey and might be useful for future research to evaluate the character compatibility of their retargeting or face parameterization methods.

3D facial motion retargeting has the advantage of capturing and recreating the nuances of human facial motions and speeding up the time-consuming 3D facial animation process. However, the facial motion retargeting pipeline is limited in reflecting the facial motion's semantic information (i.e., meaning and intensity), especially when applied to nonhuman characters. The retargeting quality heavily relies on the target face rig, which requires time-consuming preparation such as 3D scanning of human faces and modeling of blendshapes. In this paper, we propose a facial motion retargeting pipeline aiming to provide fast and semantically accurate retargeting results for diverse characters. The new framework comprises a target face parameterization module based on face anatomy and a compatible source motion interpretation module. From the quantitative and qualitative evaluations, we found that the proposed retargeting pipeline can naturally recreate the expressions performed by a motion capture subject in equivalent meanings and intensities, such semantic accuracy extends to the faces of nonhuman characters without labor-demanding preparations.

This special issue contains 24 full papers selected from the Computer Animation and Social Agents 2021 Conference (CASA2021). This conference was founded by the Computer Graphics Society (CGS) in 1988 in Geneva and is the oldest conference on Computer Animation in the world. It has been held in many countries around the world and in recent years in Beijing, China (2018), Paris, France (2019), Bournemouth, UK (2020), and this year in Ottawa, Canada. The two last conferences have been organized virtually due to the Covid-19 pandemy.

We propose a real-time, data-driven method to synthesize the sound of typical soft bodies (such as cloth and rope) in video games. For a given soft body, we perform a geometric analysis of its shape at each frame. These data are used to calculate a variety of motion events at run time that would produce sounds. We then record a database of sounds, which contains sequences of segmented sound units. Finally, we use a concatenative synthesis method to select and synthesize the actual soft-body sounds according to the extracted motion signals. Our approach is more computationally efficient compared to existing soft-body sound synthesis methods and is compatible with any particle-based soft-body physics in video games. We implement our method in Unreal Engine 4 and demonstrate its efficiency with several examples.

In this paper, we extend the idea of noise shaping for dual-tree complex wavelet transform (DCWT) image coding to a perceptual-based noise shaping. In classical noise shaping, the spatial error information is compensated by adding it back into the whole DCWT domain, which allows the retained to coefficients have better capability to approximate the original image. The proposed perceptual noise shaping introduces a perceptual weight to the spatial errors. The weight involves the structural similarity (SSIM) measurement and other adjustment parameters to shape the spatial errors. Experimental results show that the perceptual noise shaping has better results for visual quality and provides higher SSIM index than classical noise shaping. For example, the proposed perceptual noise shaping achieves an overall SSIM of 0.891 for the 8 bit 512×512 “barbara” compared to 0.878 in classical noise shaping when 5000 coefficients are retained.

Abstract We describe an animation creation system called PASCAL that supports sketch‐based modeling and physics augmented locomotion simultaneously. The system uses sketches and reconfigurable space canvases as basic modeling primitives and uses physics to improve the expressiveness and efficiency of several animation techniques to obtain controllable and plausible locomotion animation. The usability evaluation of the system was conducted both with professional and novice animators. Copyright © 2009 John Wiley & Sons, Ltd.

The diversity of devices in both wired and wireless networks via which multimedia contents are desired to be accessed and interacted with has grown significantly. Applications like video conferencing, surveillance and chatting is challenged by this diversity which requires live adaptation to meet user requirements and device specifications. In this paper, we present an architecture for temporal adaptation of ITU-T H.264 video conforming to ISO/IEC MPEG-21 DIA for live video stream along with the adaptation module implementation detail. Adaptation is performed on demand directly from the live bitstream and its generic bitstream syntax description (gBSD) avoiding conventional approaches seen in traditional transcoders. As a result, any MPEG-21 compliant host can adapt the stream without requiring the video codec. A prototype, based on the proposed architecture, and experimental evaluations of the system and its performance supporting the architecture are also presented

The goal of the research is creation of an adaptation mechanism for the delivery of three-dimensional content. The adaptation of content, for various network and terminal capabilities - as well as for different user preferences, is a key feature that needs to be investigated. Current state-of-the art research of the adaptation shows promising results for specific purpose and limited types of content but still it is not well adaptable for the massive heterogeneous environments. We present a method for transmitting adapted thee-dimensional contents to multiple target devices. We present some theoretical and practical methods for adapting three-dimensional contents, which includes shapes and animation. We also discuss practical details to the integration of our methods into MPEG-21 and MPEG-4 architectures.

To present three-dimensional data both in heavy and light-weight clients, an adaptation scheme is required. Current state-of-the art research shows promising results for specific purposes but it is still not well adoptable for light-weight clients such as mobile devices. In this research, we present a method for transmitting adapted 3D content to multiple target devices. To accomplish this goal, we devised a clustered representation of a multi-resolution model that is flexible, simple, efficient, and works with the MPEG-21 adaptation mechanism.

While level of detail (LoD) methods for the representation of 3D models are efficient and established tools to manage the trade-off between speed and quality of the rendering, LoD for animation has not yet been intensively studied by the community, and especially virtual humans animation has not been focused in the past. Animation, a major step for immersive and credible virtual environments, involves heavy computations and as such, it needs a control on its complexity to be embedded into real-time systems. Today, it becomes even more critical and necessary to provide such a control with the emergence of powerful new mobile devices and their increasing use for cyberworlds. With the help of suitable middleware solutions, executables are becoming more and more multi-platform. However, the adaptation of content, for various network and terminal capabilities - as well as for different user preferences, is still a key feature that needs to be investigated. It would ensure the adoption of the "Multiple Target Devices Single Content" concept for virtual environments, and would in theory provide the possibility of such virtual worlds in any possible condition without the need for multiple content. It is on this issue that we focus, with a particular emphasis on 3D objects and animation. This paper presents some methods for adapting a virtual human's representation and animation stream, both for their skeleton-based body animation and their deformation based facial animation, we also discuss practical details for the integration of our methods into MPEG-21 and MPEG-4 architectures.

Sound rendering requires that many different aspects are considered simultaneously, especially when rendering a real-time virtual environment. In 3D sound rendering, much the same as for graphics, one of the major influencing factors is the number of reflective polygons in a scene and due to the increase in the ability of most common graphics cards this number can now be very high, especially when scene designers produce an optimum scene using other optimizing tools such as Polygon Cruncher or Rational Reducer. In addition, the use of programs such as Lightscape™ [20], which is used to produce realistic lighting, by using per vertex shading, increases the number of polygons in a scene by several factors. Therefore a strong, pre-processing method is proposed that dramatically reduces the number of polygons in the scene to a suitable level for real-time sound rendering. The method can also be combined with other methods (e.g. scene partitioning) for even lower CPU usage.

In this paper we describe a filter that can be used to dramatically reduce the amount of data flowing between clients connected in a networked virtual environment, when using the MPEG4 animation parameters. The approach consists of three parts: real life experimentation and parameter determination, filter construction and filter testing. The real life experiments are used to determine base figures for the filter construction, the filter construction details aspects of design and visual compensation and the final evaluations are used to establish if the filter works correctly, both for data reduction and aesthetically and how well.

We present our system for personalized face and speech communication over the Internet. The overall system consists of three parts: the cloning of real human faces to use as the representative avatars; the Networked Virtual Environment System performing the basic task of network and device management; and the speech system which includes a text-to-speech engine and a real time phoneme extraction engine from natural speech. The combination of these three elements provides a system to allow real humans, represented by their virtual counterparts, to communicate with each other even when they are geographically remote. In addition to this, all elements present use MPEG-4 as a common communication and animation standard and were designed and tested on the Windows operating system (OS). The paper presents the main aim of the work, the methodology and the resulting communication system.

Presents the VPARK (Virtual Park) system, which includes a networked virtual environment (NVE) system called W-VLNET and an "attraction building system" that is able to create and modify the attractions used in the NVE. Both systems have been developed in the Windows NT environment. The paper outlines the techniques for communication, scene management, facial and body animation, and general user interaction modules. The use of VRML97 and MPEG-4 SHNC is overviewed for the purpose of outlining the compatability of the system with other similar virtual reality systems. The software provides realistic virtual actors as well as sets of high-level actions that are applicable to them in real-time. Related issues on obtaining actor models and animating them in real time are presented. The creation process of an attraction incorporates assembling animation units through a timeline. Using this software, the users are able to bring their own scenario-based applications into a shared virtual environment.

In this paper we present our Networked Virtual Environment (NVE) System, called W-VLNET (Windows Virtual Life Network), which has been developed on the Windows NT Operating System (OS). This paper emphasizes the Real-Time aspect of this NVE system, the advanced interactivity that the system provides and its ability to transfer data across the Internet so that geographically distant users can collaborate with each other. Techniques for communication, scene management, facial and body animation, and general user interaction modules are detailed in this paper. The use of VRML97 and MPEG4 SHNC is overviewed to stress the compatibility of the system with other similar Virtual Reality systems. The software provides realistic virtual actors as well as sets of applicable high-level actions in real-time. Related issues on obtaining actor models and animating them in real-time are presented. We also introduce a case study to show an example of how the system can be used.