Cover song generation stands out as a popular way of music making in the music-creative community. In this study, we introduce Piano Cover Generation (PiCoGen), a two-stage approach for automatic cover song generation that transcribes the melody line and chord progression of a song given its audio recording, and then uses the resulting lead sheet as the condition to generate a piano cover in the symbolic domain. This approach is advantageous in that it does not required paired data of covers and their original songs for training. Compared to an existing approach that demands such paired data, our evaluation shows that PiCoGen demonstrates competitive or even superior performance across songs of different musical genres.

It is a necessary but challenging task for creative producers to have an idea of the depth perception of the target audience when watching a stereoscopic film in a cinema during production. This paper proposes a novel metric, geometric perceived depth percentage (GPDP), to numerate and depict the depth perception of a scene before rendering. In addition to the geometric relationship between the object depth and focal distance, GPDP takes the screen width and viewing distance into account. As a result, it provides a more intuitive means for predicting stereoscopy and is universal across different viewing conditions. Based on GPDP, we design a practical tool to visualize the stereoscopic perception without the need for any 3-D device or special environment. The tool utilizes the stereoscopic comfort volume, GPDP-based shading schemes, depth perception markers, and GPDP histograms as visual cues so that animators can set stereoscopic parameters more easily. The tool is easily implemented in any modern rendering pipeline, including interactive Autodesk Maya and offline Pixar's RenderMan renderer. It has been used in several production projects including commercial ones. Finally, two user studies show that GPDP is a proper depth perception indicator and the proposed tool can make the stereoscopic parameter setting process easier and more efficient.

Character speech animation is traditionally considered as important but tedious work, especially when taking lip synchronization (lip-sync) into consideration. Although there are some methods proposed to ease the burden on artists to create facial and speech animation, almost none is fast and efficient. In this paper, we introduce a framework for synthesizing lip-sync character speech animation in real time from a given speech sequence and its corresponding texts, starting from training dominated animeme models (DAMs) for each kind of phoneme by learning the character's animation control signal through an expectation-maximization (EM)-style optimization approach. The DAMs are further decomposed to polynomial-fitted animeme models and corresponding dominance functions while taking coarticulation into account. Finally, given a novel speech sequence and its corresponding texts, the animation control signal of the character can be synthesized in real time with the trained DAMs. The synthesized lip-sync animation can even preserve exaggerated characteristics of the character's facial geometry. Moreover, since our method can perform in real time, it can be used for many applications, such as lip-sync animation prototyping, multilingual animation reproduction, avatar speech, and mass animation production. Furthermore, the synthesized animation control signal can be imported into 3-D packages for further adjustment, so our method can be easily integrated into the existing production pipeline.

The purpose of this paper is to optimize the stereoscopic 3D experience when users watch those contents. We propose two factors in stereoscopic experience: visual fatigue and depth perception. In order to optimize stereoscopic experience, we propose five principles to reduce visual fatigue and enhance depth perception and implement them by cropping and warping. Using our methods, time-consuming problems of existing view interpolation methods such as camera calibration, accurate dense depth map, and inpainting are avoided. We also design a GUI that enables users to efficiently edit the stereoscopic video, optimize the stereoscopic experience, and preview the stereoscopic result. The user study shows that our method is successful in optimizing stereoscopic experience. One example is shown in Figure 1.

Conceptual Farm is a virtual reality platform for generating and observing the behaviors of different autonomous characters. By providing (1) descriptions for characters' behaviors and (2) 3D animations and sound, life-like characters in a realistic habitat can be created, modified, and interact with both users and other characters in real time. The flexible, manageable and scalable nature of Conceptual Farm leads to its desirability in zoological research, general education, game and film production, and even decorative arts.

Digital sculpting is a new trend for creating 3D models, but its application in the manipulation of volumetric data raises several issues that need to be addressed. With the extended marching cubes algorithm (EMC), sharp features of 3D models are well preserved. Additionally, the dynamic nature of modifying models in real time needs to be dealt with in sculpting applications: since the sampling of sharp features is implicit, direct modification on cell data will cause problems. A feature refinement strategy is proposed to preserve the dynamically modified model correct, and efficiently. Overall, the proposed methods provide an adaptive resolution and feature-preserved sculpting system that handles dynamic behavior in real-time performance.