Propelled by the breakthrough in deep generative models, audio-to-image generation has emerged as a pivotal cross-modal task that converts complex auditory signals into rich visual representations. However, previous works only focus on single-source audio inputs for image generation, ignoring the multi-source characteristic in natural auditory scenes, thus limiting the performance in generating comprehensive visual content. To bridge this gap, we propose a method called MACS to conduct multi-source audio-to-image generation. To our best knowledge, this is the first work that explicitly separates multi-source audio to capture the rich audio components before image generation. MACS is a two-stage method. In the first stage, multi-source audio inputs are separated by a weakly supervised method, where the audio and text labels are semantically aligned by casting into a common space using the large pre-trained CLAP model. We introduce a ranking loss to consider the contextual significance of the separated audio signals. In the second stage, effective image generation is achieved by mapping the separated audio signals to the generation condition using only a trainable adapter and a MLP layer. We preprocess the LLP dataset as the first full multi-source audio-to-image generation benchmark. The experiments are conducted on multi-source, mixed-source, and single-source audio-to-image generation tasks. The proposed MACS outperforms the current state-of-the-art methods in 17 out of the 21 evaluation indexes on all tasks and delivers superior visual quality.

Neural Radiance Field (NeRF) models are implicit neural scene representation methods that offer unprecedented capabilities in novel view synthesis. Semantically-aware NeRFs not only capture the shape and radiance of a scene, but also encode semantic information of the scene. The training of semantically-aware NeRFs typically requires pixel-level class labels, which can be prohibitively expensive to collect. In this work, we explore active learning as a potential solution to alleviate the annotation burden. We investigate various design choices for active learning of semantically-aware NeRF, including selection granularity and selection strategies. We further propose a novel active learning strategy that takes into account 3D geometric constraints in sample selection. Our experiments demonstrate that active learning can effectively reduce the annotation cost of training semantically-aware NeRF, achieving more than 2× reduction in annotation cost compared to random sampling.

Traffic flow prediction is the foundation of traffic scheduling and a major component of Intelligent Transportation Systems (ITS). Accurate traffic flow prediction is crucial for numerous real-world traffic applications. However, the complex and dynamic nature of spatiotemporal correlations in traffic flow data poses obstacles to this task. Another important factor hindering model improvement is the scarcity of spatio-temporal data. Meanwhile, most existing studies assume the traffic data collected by sensors in real scenarios is completely correct and trustworthy, thereby neglecting the anomalies and incorrect data. To address these issues, we propose the Contrastive Learning-based Adaptive Graph Fusion Convolution Network with Residual-Enhanced Decomposition Strategy (CDAGF) for traffic flow forecasting, which performs a simple yet effective graph augmentation mechanism imposed only on the learned graphs and reserves the explicit graph to mitigate the data scarcity problem, and designs a negative filter to assign negative pairs based on timestamp information. Besides, CDAGF separates anomalous signals from valid traffic signals through a residual-enhanced decomposition strategy to weaken the impact of anomalies and further improve the prediction accuracy. Moreover, CDAGF generates static graph and dynamic graphs based on traffic signals as well as timestamp information and fuses the learned graphs with the explicit graph to model temporal and spatial characteristics dynamically and adaptively. The experimental results on five popular public datasets demonstrate that CDAGF achieves state-of-the-art performance with 18.29 MAE (Mean Absolute Error) on the PEMS04 dataset.