Recent advancements in personalized Text-to-Video (T2V) generation have made significant strides in synthesizing character-specific content. However, these methods face a critical limitation: the inability to perform fine-grained control over motion intensity. This limitation stems from an inherent entanglement of action semantics and their corresponding magnitudes within coarse textual descriptions, hindering the generation of nuanced human videos and limiting their applicability in scenarios demanding high precision, such as animating virtual avatars or synthesizing subtle micro-expressions. Furthermore, existing approaches often struggle to preserve high identity fidelity when other attributes are modified. To address these challenges, we introduce MotionCharacter, a framework for high-fidelity human video generation with precise motion control. At its core, MotionCharacter explicitly decouples motion into two independently controllable components: action type and motion intensity. This is achieved through two key technical contributions: (1) a Motion Control Module that leverages textual phrases to specify the action type and a quantifiable metric derived from optical flow to modulate its intensity, guided by a region-aware loss that localizes motion to relevant subject areas; and (2) an ID Content Insertion Module coupled with an ID-Consistency loss to ensure robust identity preservation during dynamic motions. To facilitate training for such fine-grained control, we also curate Human-Motion, a new large-scale dataset with detailed annotations for both motion and facial features. Extensive experiments demonstrate that MotionCharacter achieves substantial improvements over existing methods. Our framework excels in generating videos that are not only identity-consistent but also precisely adhere to specified motion types and intensities.

In single image super-resolution (SISR), although PSNR is a key metric for signal fidelity, images with high PSNR do not necessarily render high visual quality. As a result, current perception-driven SISR methods employ perceptual metrics close to the human eye to measure the quality of the generated images. Unfortunately, the perceptual loss and adversarial loss, widely used by the perception-driven SISR methods, still underperform on these non-differentiable perceptual metrics. To this end, we propose a generic multi-scale perceptual loss, i.e., the patch loss, which can be easily plugged into off-the-shelf SISR methods to improve a broad range of perceptual metrics. Specifically, the proposed patch loss minimizes the multi-scale similarity of image patches and enhances the restoration of regions with complex textures and sharp edges via parameter-free adaptive patch-wise attention. Our proposed patch loss introduces more realistic details compared to the perceptual loss and fewer artifacts compared to the adversarial loss.

Due to the scarcity of annotated samples, the diversity between support set and query set becomes the main obstacle for few shot semantic segmentation. Most existing prototype-based approaches only exploit the prototype from the support feature and ignore the information from the query sample, failing to remove this obstacle.In this paper, we proposes a dual prototype network (DPNet) to dispose of few shot semantic segmentation from a new perspective. Along with the prototype extracted from the support set, we propose to build the pseudo-prototype based on foreground features in the query image. To achieve this goal, the cycle comparison module is developed to select reliable foreground features and generate the pseudo-prototype with them. Then, a prototype interaction module is utilized to integrate the information of the prototype and the pseudo-prototype based on their underlying correlation. Finally, a multi-scale fusion module is introduced to capture contextual information during the dense comparison between prototype (pseudo-prototype) and query feature. Extensive experiments conducted on two benchmarks demonstrate that our method exceeds previous state-of-the-arts with a sizable margin, verifying the effectiveness of the proposed method.

Few shot segmentation is a newly-developing and challenging computer vision task which is only provided with few labeled samples of the novel class. Some recent works on this problem focus more on how to design an effective comparison module but ignore how to extract the features passed to compare. In this paper we propose a novel model named LTAF-Net for few-shot segmentation. This model aims to adaptively recalibrate the extracted features which could boost the accuracy of dense comparison between support features and query features. Besides an additional prediction refinement module is designed to refine the initial mask. Meanwhile this method can apply to k-shot setting without developing a new specialized architecture and achieve competitive performance. Experiments on PASCAL-5iand FSS-1000 strongly prove the effectiveness of the proposed model. Our model outperforms the second-best method 1.4% in 1-shot and 0.76% in 5-shot respectively in PASCAL-5i.

Dianbo Sui, Yubo Chen, Binjie Mao, Delai Qiu, Kang Liu, Jun Zhao. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.