Audio-to-image generation (AIG) faces challenges in fine-grained semantic alignment, particularly semantic semantic misalignment, and loss of visual detail. To address these issues, we proposed MARS ( M ultimodal- A ssisted R efined S emantic alignment), a novel framework leveraging a Mamba-based audio encoder to manage the complexity of long audio sequences, coupled with a fine-grained multimodal alignment strategy using visual descriptions from multimodal large language models. We enhanced semantic coherence and aesthetic quality by fine-tuning an image generator using an image aesthetic perception generator. Furthermore, we validated MARS on VGGSound and VEGAS benchmarks, comprising 37,250 and 9,500 records, respectively. The results suggest that MARS significantly outperforms existing methods, achieving average improvements of 28.73% in semantic relevance and 127.35% in aesthetic scores compared with the best AIG generation baseline. In addition, cross-domain evaluations on the AudioCaps and Clotho datasets confirmed the robustness and generalization capability of MARS , with an average improvement of 73.9% on the V2A metric. • MARS refines semantic alignment for audio-based image generation. • A Mamba-based encoder processes long audio sequences. • MLLMs supply rich visuals to boost semantics and aesthetics. • Extensive experiments prove effectiveness and robustness.

Scene text recognition is considered as a sequence labeling problem. For the text recognition task, the alignment between the scene text image and the output text is coincident, which means the latter characters corresponding to the image region will also be behind. However, the existing global attention-based method focuses too much irrelevant information which leads to alignment drift. Contrary, local attention selects the subset of feature representation most relevant to the current character. In this paper, we explore the local attention mechanism and attempt to replace the global attention to implement decoding. Therefore, we revise several variants of local attention methods and provide a comprehensive comparison, which is missing in the scene text recognition literature so far. Specially, we introduce two Heuristic approaches for Local Attention (HLA) and prove that monotonic alignment improves performance significantly. Evaluations on the benchmarks show that the local attention method outperforms the existing global attention methods.

Anomaly segmentation is an important task in computer vision. At present, anomaly segmentation has achieved milestone development. Many representative works have been proposed, especially unsupervised learning methods and pre-training methods, but pre-training methods are difficult to solve cross-The gap brought by domains has not been paid much attention to in previous research work on how to mine the hidden information contained in the data itself. Because of the diversity of anomaly detection, it is crucial to make full use of the information of the object itself. This paper proposes a novel dynamic branch fusion structure, through mining the hidden information inside the data, so as to re-model more targeted abnormal segmentation. The method has been tested on two known benchmarks to verify the effectiveness of the proposed method.

We perform fast single image super-resolution with flexible magnification for natural images. A novel coarse-to-fine super-resolution framework is developed for the magnification that is factorized into a maximum integer component and the quotient. Specifically, our framework is embedded with a light-weight upscale network for super-resolution with the integer scale factor, followed by the fine-grained network to guide interpolation on feature maps as well as to generate the super-resolved image. Compared with the previous flexible magnification super-resolution approaches, the proposed framework achieves a tradeoff between computational complexity and performance. We conduct experiments using the coarse-to-fine framework on the standard benchmarks and demonstrate its superiority in terms of effectiveness and efficiency over previous approaches.