How much can we infer about an emotional voice solely from an expressive face? This intriguing question holds great potential for applications such as virtual character dubbing and aiding individuals with expressive language disorders. Existing face-to-speech methods offer great promise in capturing identity characteristics but struggle to generate diverse vocal styles with emotional expression. In this paper, we explore a new task, termed emotional face-to-speech, aiming to synthesize emotional speech directly from expressive facial cues. To that end, we introduce DEmoFace, a novel generative framework that leverages a discrete diffusion transformer (DiT) with curriculum learning, built upon a multi-level neural audio codec. Specifically, we propose multimodal DiT blocks to dynamically align text and speech while tailoring vocal styles based on facial emotion and identity. To enhance training efficiency and generation quality, we further introduce a coarse-to-fine curriculum learning algorithm for multi-level token processing. In addition, we develop an enhanced predictor-free guidance to handle diverse conditioning scenarios, enabling multi-conditional generation and disentangling complex attributes effectively. Extensive experimental results demonstrate that DEmoFace generates more natural and consistent speech compared to baselines, even surpassing speech-driven methods. Demos of DEmoFace are shown at our project https://demoface.github.io.

While latent diffusion models (LDMs), such as Stable Diffusion, are designed for high-resolution image generation, they often struggle with significant structural distortions when generating images at resolutions higher than their training one. Instead of relying on extensive retraining, a more resource-efficient approach is to reprogram the pretrained model for high-resolution (HR) image generation; however, existing methods often result in poor image quality and long inference time. We introduce RepLDM, a novel reprogramming framework for pretrained LDMs that enables high-quality, high-efficiency, high-resolution image generation; see Fig. 1. RepLDM consists of two stages: (i) an attention guidance stage, which generates a latent representation of a higher-quality training-resolution image using a novel parameter-free self-attention mechanism to enhance the structural consistency; and (ii) a progressive upsampling stage, which progressively performs upsampling in pixel space to mitigate the severe artifacts caused by latent space upsampling. The effective initialization from the first stage allows for denoising at higher resolutions with significantly fewer steps, improving the efficiency. Extensive experimental results demonstrate that RepLDM significantly outperforms state-of-the-art methods in both quality and efficiency for HR image generation, underscoring its advantages for real-world applications. Codes: https://github.com/kmittle/RepLDM.

Abstract This article investigates a mobile edge computing (MEC) network assisted by multiple unmanned aerial vehicles (UAVs) to address the computational and offloading requirements for mobile intelligent terminals (MITs) within crowded venues. The objective is to tackle intricate task processing and diminish MITs' waiting times. Considering the randomness of task arrival at the MITs and the imbalance between the amount of data and computation for complex tasks, a dual‐queue model with data cache queue and computation queue is proposed, with minimizing the weighted system total energy consumption and average delay as the optimization objectives. Lyapunov optimization theory is employed to convert the stochastic optimization problem into a deterministic one, and the initial deployment quantity and hovering position of the UAVs are determined by the density‐based spatial clustering of applications with noise (DBSCAN) method with noise. Then PPO algorithm for MIT task, resource allocation, and UAV trajectory optimization. Numerical results display the proposed scheme can efficaciously diminish energy consumption and delay by 10% and 33% respectively, compared with the baseline scheme. This paper proposes a practical and feasible solution for stochastic computing offloading in UAV‐assisted MEC, which fills the gap in existing research on regarding the consideration of complex task imbalances.