Abstract Sketching is a direct and inexpensive means of visual expression. Though image‐based sketching has been well studied, video‐based sketch animation generation is still very challenging due to the temporal coherence requirement. In this paper, we propose a novel end‐to‐end automatic generation approach for vector sketch animation. To solve the flickering issue, we introduce a Differentiable Motion Trajectory (DMT) representation that describes the frame‐wise movement of stroke control points using differentiable polynomial‐based trajectories. DMT enables global semantic gradient propagation across multiple frames, significantly improving the semantic consistency and temporal coherence, and producing high‐framerate output. DMT employs a Bernstein basis to balance the sensitivity of polynomial parameters, thus achieving more stable optimization. Instead of implicit fields, we introduce sparse track points for explicit spatial modeling, which improves efficiency and supports long‐duration video processing. Evaluations on DAVIS and LVOS datasets demonstrate the superiority of our approach over SOTA methods. Cross‐domain validation on 3D models and text‐to‐video data confirms the robustness and compatibility of our approach.

Human-human interaction generation has garnered significant attention in motion synthesis due to its vital role in understanding humans as social beings. However, existing methods typically rely on transformer-based architectures, which often face challenges related to scalability and efficiency. To address these challenges, we propose InterMamba, a novel and efficient human-human interaction generation method built on the Mamba framework, designed to capture long-sequence dependencies effectively while enabling real-time feedback. Specifically, we introduce an adaptive spatio-temporal Mamba framework that utilizes two parallel SSM branches with an adaptive mechanism to integrate the spatial and temporal features of motion sequences. To further enhance the model's ability to capture dependencies within individual motion sequences and the interactions between different individual sequences, we develop two key modules: the self adaptive spatio-temporal Mamba module and the cross adaptive spatio-temporal Mamba module, enabling efficient feature learning. Extensive experiments demonstrate that our method achieves the state-of-the-art results on both two interaction datasets with remarkable quality and efficiency. Compared to the baseline method InterGen, our approach not only improves accuracy but also reduces the parameter size to just 66 M (36% of InterGen's), while achieving an average inference speed of 0.57 seconds, which is 46% of InterGen's execution time.

Abstract In traditional 2D animation, sketches drawn at distant keyframes are used to design motion, yet it would be far too labor‐intensive to draw all the inbetween frames to fully visualize that motion. We propose a novel efficient interpolation algorithm that generates these intermediate frames in the artist's drawing style. Starting from a set of registered rough vector drawings, we first generate a large number of candidate strokes during a pre‐process, and then, at each intermediate frame, we select the subset of those that appropriately conveys the underlying interpolated motion, interpolates the stroke distributions of the key drawings, and introduces a minimum amount of temporal artifacts. In addition, we propose quantitative error metrics to objectively evaluate different stroke selection strategies. We demonstrate the potential of our method on various animations and drawing styles, and show its superiority over competing raster‐ and vector‐based methods.

Abstract Depth maps are acquirable and irreplaceable geometric information that significantly enhances traditional color images. RGB and Depth (RGBD) images have been widely used in various image analysis applications, but they are still very limited due to challenges from different modalities and misalignment between color and depth. In this paper, a Fully Aligned Fusion Network (FAFNet) for RGBD semantic segmentation is presented. To improve cross‐modality fusion, a new RGBD fusion block is proposed, features from color images and depth maps are first fused by an attention cross fusion module and then aligned by a semantic flow. A multi‐layer structure is also designed to hierarchically utilize the RGBD fusion block, which not only eases issues of low resolution and noises for depth maps but also reduces the loss of semantic features in the upsampling process. Quantitative and qualitative evaluations on both the NYU‐Depth V2 and the SUN RGB‐D dataset demonstrate that the FAFNet model outperforms state‐of‐the‐art RGBD semantic segmentation methods.

Abstract The development of deep learning provides a new way for solving the colorization problem on the grayscale image. Excellent coding‐based methods appear in the automatic image colorization task, avoiding the unsaturated colour effect problem of previous methods based on the L2 loss function. Traditional neural networks come with high computational costs and a large number of parameters. Considering the limitation of memory and computing resources and aiming at lightweight, a novel grey image automatic colorization network is proposed. The basic idea of coding‐based methods is used, regarding the colorization task as a pixel‐level classification problem, meanwhile redesign and improve the colour encoding and decoding process. This network architecture leverages a lightweight convolution to reduce the computation and combines an efficient attention model to form a residual block as the kernel of the backbone network. Furthermore, an efficient image self‐attention mechanism placed at the end of the network is applied to enhance the ultimate colouring results. The method proposed in this paper can maintain the natural colouring effect and significantly reduce the computational amount and network model parameters.

We present a novel 3D instance segmentation framework for Multi-View Stereo (MVS) buildings in urban scenes. Unlike existing works focusing on semantic segmentation of urban scenes, the emphasis of this work lies in detecting and segmenting 3D building instances even if they are attached and embedded in a large and imprecise 3D surface model. Multi-view RGB images are first enhanced to RGBH images by adding a heightmap and are segmented to obtain all roof instances using a fine-tuned 2D instance segmentation neural network. Instance masks from different multi-view images are then clustered into global masks. Our mask clustering accounts for spatial occlusion and overlapping, which can eliminate segmentation ambiguities among multi-view images. Based on these global masks, 3D roof instances are segmented out by mask back-projections and extended to the entire building instances through a Markov random field optimization. A new dataset that contains instance-level annotation for both 3D urban scenes (roofs and buildings) and drone images (roofs) is provided. To the best of our knowledge, it is the first outdoor dataset dedicated for 3D instance segmentation with much more annotations of attached 3D buildings than existing datasets1. Quantitative evaluations and ablation studies have shown the effectiveness of all major steps and the advantages of our multi-view framework over the orthophoto-based method.

Typically, relief generation from an input 3D scene is limited to either bas-relief or high-relief modeling. This paper presents a novel unified scheme for synthesizing reliefs guided by the geometric texture richness of 3D scenes; it can generate both basand high-reliefs. The type of relief and compression coefficient can be specified according to the user’s artistic needs. We use an energy minimization function to obtain the surface reliefs, which contains a geometry preservation term and an edge constraint term. An edge relief measure determined by geometric texture richness and edge z -depth is utilized to achieve a balance between these two terms. During relief generation, the geometry preservation term keeps local surface detail in the original scenes, while the edge constraint term maintains regions of the original models with rich geometric texture. Elsewhere, in highreliefs, the edge constraint term also preserves depth discontinuities in the higher parts of the original scenes. The energy function can be discretized to obtain a sparse linear system. The reliefs are obtained by solving it by an iterative process. Finally, we apply non-linear compression to the relief to meet the user’s artistic needs. Experimental results show the method’s effectiveness for generating both bas- and high-reliefs for complex 3D scenes in a unified manner.

This paper presents SymmSketch—a system for creating symmetric 3D free-form shapes from 2D sketches. The reconstruction task usually separates a 3D symmetric shape into two types of shape components, that is, the self-symmetric shape component and the mutual-symmetric shape components. Each type can be created in an intuitive manner. Using a uniform symmetry plane, the user first draws 2D sketch lines for each shape component on a sketching plane. The z -depth information of the hand-drawn input sketches can be calculated using their property of mirror symmetry to generate 3D construction curves. In order to provide more freedom for controlling the local geometric features of the reconstructed free-form shapes (e.g., non-circular cross-sections), our modeling system creates each shape component from four construction curves. Using one pair of symmetric curves and one pair of general curves, an improved cross-sectional surface blending scheme is applied to generate a parametric surface for each component. The final symmetric free-form shape is progressively created, and is represented by 3D triangular mesh. Experimental results illustrate that our system can generate complex symmetric free-form shapes effectively and conveniently.

Tensor field design plays an essential role in various computer graphics applications. One of the main challenges in field design is how to obtain a smooth field preserving meaningful singularities presented in the original scene. Compared with well-studied point singularities, line singularities, commonly occur on object boundaries and occluding contours, are not exploited enough for field design in previous work. In this paper, we discuss the definition of line singularities, and introduce a line-singularity-based interactive tensor field design method, allowing the user to design feature-preserving tensor fields with less effort and to preserve both input singularities and user-specified stroke directions. To avoid introducing extra interaction burdens to the user, our method automatically locates line singularities using a geodesic-based segmentation. We demonstrate the capabilities of our method on tensor field design with various nonphotorealistic rendering applications and the real-time performance accelerated on GPU.

Abstract We introduce a new approach for defining continuous non‐oriented gradient fields from discrete inputs, a fundamental stage for a variety of computer graphics applications such as surface or curve reconstruction, and image stylization. Our approach builds on a moving least square formalism that computes higher‐order local approximations of non‐oriented input gradients. In particular, we show that our novel isotropic linear approximation outperforms its lower‐order alternative: surface or image structures are much better preserved, and instabilities are significantly reduced. Thanks to its ease of implementation (on both CPU and GPU) and small performance overhead, we believe our approach will find a widespread use in graphics applications, as demonstrated by the variety of our results.

We introduce a non-uniform composition that integrates multiple rendering styles in a picture driven by an importance map. This map, either issued from saliency estimation or designed by a user, is introduced both in the creation of the multiple styles and in the final composition. Our approach accommodates a variety of stylization techniques, such as color desaturation, line drawing, blurring, edge-preserving smoothing and enhancement. We illustrate the versatility of the proposed approach and the variety of rendering styles on different applications such as images, videos, 3D scenes and even mixed reality. We also demonstrate that such an approach may help in directing user attention.

Abstract We introduce a new technique called Implicit Brushes to render animated 3D scenes with stylized lines in realtime with temporal coherence. An Implicit Brush is defined at a given pixel by the convolution of a brush footprint along a feature skeleton; the skeleton itself is obtained by locating surface features in the pixel neighborhood. Features are identified via image‐space fitting techniques that not only extract their location, but also their profile, which permits to distinguish between sharp and smooth features. Profile parameters are then mapped to stylistic parameters such as brush orientation, size or opacity to give rise to a wide range of line‐based styles.

We introduce an on-line framework for the visualizing of underground structures that improves X-Ray vision and Focus and Context Rendering for Augmented Reality. Our approach does not require an accurate reconstruction of the 3D environment and runs on-line on modern hardwares. For these purposes, we extract characteristic features from video frames and create visual cues to reveal occlusion relationships. To enhance the perception of occluding order, the extracted features are either directly rendered, or used to create hybrid blending masks: we thus ensures that the resulting cues are clearly noticeable.