Video-to-music (V2M) generation aims to create music that aligns with visual content. However, two main challenges persist in existing methods: (1) the lack of explicit rhythm modeling hinders audiovisual temporal alignments; (2) effectively integrating various visual features to condition music generation remains non-trivial. To address these issues, we propose Diff-V2M, a general V2M framework based on a hierarchical conditional diffusion model, comprising two core components: visual feature extraction and conditional music generation. For rhythm modeling, we begin by evaluating several rhythmic representations, including low-resolution mel-spectrograms, tempograms, and onset detection functions (ODF), and devise a rhythmic predictor to infer them directly from videos. To ensure contextual and affective coherence, we also extract semantic and emotional features. All features are incorporated into the generator via a hierarchical cross-attention mechanism, where emotional features shape the affective tone via the first layer, while semantic and rhythmic features are fused in the second cross-attention layer. To enhance feature integration, we introduce timestep-aware fusion strategies, including feature-wise linear modulation (FiLM) and weighted fusion, allowing the model to adaptively balance semantic and rhythmic cues throughout the diffusion process. Extensive experiments identify low-resolution ODF as a more effective signal for modeling musical rhythm and demonstrate that Diff-V2M outperforms existing models on both in-domain and out-of-domain datasets, achieving state-of-the-art performance in terms of objective metrics and subjective comparisons.

In standard reinforcement learning, since the uncertainty of task objectives is not adequately considered in the policy training, the policy achieves poor generalization for the out-of-distribution (OOD) tasks. Although considerable efforts have been made to enhance the generalization for OOD tasks, most of these methods overlook the structural information of task representations in latent space during the generation of extrapolative data, resulting in biased and blurred data embeddings, which then affect the policy generalization. To address this issue, we propose a context-based meta-reinforcement learning (meta-RL) method, namely latent variable distribution enhancement sampler (LVDES), which enhances the policy generalization on OOD tasks by providing efficient task representation space and accurate augmentation policy training data for OOD tasks. Specifically, the proposed LVDES consists of four modules: a task inference module, a task separation module, a latent enhancement module (LEM), and a policy module. The task inference module is used to identify the task. The task separation module (TSM) learns a representation space with highly structured separability. The LEM generates relevant additional task trajectories for augmenting policy training data. The policy module learns a policy to solve tasks. By using efficient task representation space and augmented trajectory data, the exploration efficiency and generalization of the policy for OOD tasks can be enhanced by our LVDES method. Extensive experiments are conducted to demonstrate the effectiveness of our method in comparison with existing methods on the MuJoCo and Meta-World benchmarks. The experimental results show that the task completion accuracy of our LVDES on OOD tasks is increased by 60.20%, with the average exploration time being reduced by 62.99% in comparison with the most effective current method, which demonstrates that our LVDES can achieve great policy generalization on OOD tasks.

In noisy and reverberant environments, the performance of deep learning-based speech separation methods drops dramatically because previous methods are not designed and optimized for such situations. To address this issue, we propose a multi-stage end-to-end learning method that decouples the difficult speech separation problem in noisy and reverberant environments into three sub-problems: speech denoising, separation, and de-reverberation. The probability and speed of searching for the optimal solution of the speech separation model are improved by reducing the solution space. Moreover, since the channel information of the audio sequence in the time domain is crucial for speech separation, we propose a triple-path structure capable of modeling the channel dimension of audio sequences. Experimental results show that the proposed multi-stage triple-path method can improve the performance of speech separation models at the cost of little model parameter increment.