Audio-to-image generation (AIG) faces challenges in fine-grained semantic alignment, particularly semantic semantic misalignment, and loss of visual detail. To address these issues, we proposed MARS ( M ultimodal- A ssisted R efined S emantic alignment), a novel framework leveraging a Mamba-based audio encoder to manage the complexity of long audio sequences, coupled with a fine-grained multimodal alignment strategy using visual descriptions from multimodal large language models. We enhanced semantic coherence and aesthetic quality by fine-tuning an image generator using an image aesthetic perception generator. Furthermore, we validated MARS on VGGSound and VEGAS benchmarks, comprising 37,250 and 9,500 records, respectively. The results suggest that MARS significantly outperforms existing methods, achieving average improvements of 28.73% in semantic relevance and 127.35% in aesthetic scores compared with the best AIG generation baseline. In addition, cross-domain evaluations on the AudioCaps and Clotho datasets confirmed the robustness and generalization capability of MARS , with an average improvement of 73.9% on the V2A metric. • MARS refines semantic alignment for audio-based image generation. • A Mamba-based encoder processes long audio sequences. • MLLMs supply rich visuals to boost semantics and aesthetics. • Extensive experiments prove effectiveness and robustness.

In recent years, Text-to-Music (T2M) generation models have rapidly emerged as powerful tools in content creation across fields. While existing models have made notable progress in sound quality, instrument identification, and stylistic alignment, they still exhibit clear limitations in modeling musical structure and musicality-particularly in terms of harmonic coherence and rhythmic alignment. To address these issues, we propose a Temporal-Conditioned Symbolic Alignment for Controllable Text-to-Music Generation(TCSA), which introduces explicit local condition controls to enhance structural fidelity in music generation. Specifically, we design a music theory enrichment strategy based on GPT-2 that transforms input text into detailed descriptions with embedded music theory knowledge, from which accurate chord progressions and rhythmic patterns are extracted as generation conditions. To synchronize these local features effectively, we develop a temporal alignment feature fusion mechanism. Additionally, we propose a layer-skipping fine-tuning strategy to avoid overfitting and enable fine-grained structural modeling. Finally, we introduce a perception-driven loss function based on Mel spectrograms to optimize the harmonic consistency and structural coherence of the generated music. Experimental results demonstrate that TCSA achieves competitive generation quality while offering significantly improved controllability over musical structure, making it well-suited for professional music production and refined content creation.

The Internet of Underwater Things (IoUT) has emerged as a promising paradigm for marine applications, yet faces significant challenges in achieving secure and reliable network communications. In IoUT networks, underwater acoustic communication struggles to maintain both covert operation and efficient data transmission, which are crucial for sensitive monitoring and surveillance applications. Bio-inspired communication schemes that mimic dolphin whistles offer an innovative solution for IoUT networks by embedding information within natural marine sounds, enhancing both communication security and network connectivity. However, in phase-modulated bionic camouflage communication, the multi-path effects of the underwater acoustic channel cause overlapping of multiple whistles at the receiver, severely degrading IoUT network performance and limiting practical applications. To address these challenges in IoUT networks, we propose a novel joint passband-baseband equalization method for bio-inspired phase-modulated communication. The receiver architecture combines an improved learning rate-fast transversal RLS (LR-FTRLS) equalizer for channel shortening in the passband with a decision feedback equalizer (DFE) in the baseband, specifically optimized for underwater IoT scenarios. The effectiveness of the proposed scheme has been verified through both shallow and deep-sea simulations. Sea experiment further demonstrate the system’s practical viability, achieving 300 bps with a bit error rate below 10-3 at 9 km range, demonstrating its effectiveness for high-rate covert underwater communication. This work represents a significant advancement in underwater IoT networks, offering enhanced security, improved reliability, and increased data transmission capabilities for various marine IoT applications including environmental monitoring, underwater surveillance, and marine resource exploration.

To achieve outstanding aspect-level sentiment analysis (ASC), it is crucial to reduce the distance between aspect terms and opinion words. Recently, advanced methods in ASC used graph neural network (GNN)-based methods to leverage the syntactic dependency within the sentence, which can shorten the distance through syntactical dependencies. However, existing approaches that utilize GNNs have difficulty extracting long-distance relations in the dependency tree due to the over-smoothing problem resulting from stacking GNN layers, which limits their ability to detect remote relations. To solve this issue, we propose a Bidirectional Directed Acyclic Graph (BDAG) to reconstruct syntactic dependencies and a Bidirectional Directed Acyclic Graph Neural Network (BDAGNN) to efficiently propagate multi-hop sentiment information. We also enhance the BDAG with affective commonsense knowledge from SenticNet for comprehensive sentiment classification. The BDAGNN we proposed obtains partial state-of-the-art performance on four benchmark datasets, indicating the feasibility of encoding syntactic structures with BDAG.

The field of image generation models has seen substantial progress, characterized by a proliferation of diverse generative models and their associated outputs. However, there currently exists a deficiency in methodologies that can concurrently and effectively evaluate both the intrinsic quality of generated images and the alignment between image features and textual prompts. To address these challenges, we propose a novel Framework for evaluating Visual elegance and Sentiment resonance (FVS). The FVS incorporates a novel image aesthetic assessment model, specifically trained to assess the visual attractiveness of the generated images. Additionally, it evaluates the sentiment and aesthetic consistency between textual prompt and the generated image. Experimental results verify that the evaluations from our framework align more closely with human preferences. Moreover, we apply our framework to filter and construct a higher-quality training set of generated images. This curated dataset is then exploited to adapt the generative model, resulting in enhanced generation quality.

Visual Question Answering (VQA) aims to obtain answers through image-question pairs. Nowadays, the VQA model tends to get answers only through questions, ignoring the information in the images. This phenomenon is caused by bias. As indicated by previous studies, the bias in VQA mainly comes from text modality. Our analysis of bias suggests that the question type is a crucial factor in bias formation. To interrupt the shortcut from question type to answer for de-biasing, we propose a self-supervised method for Shuffling Question Types (SQT) to reduce bias from text modality, which overcomes the prior language problem by mitigating the question-to-answer bias without introducing external annotations. Moreover, we propose a new objective function for negative samples. Experimental results show that our approach can achieve 61.76% accuracy on the VQA-CP v2 dataset, which outperforms the state-of-the-art in both self-supervised and supervised methods.

Document layout analysis (DLA) plays an essential role in information extraction and document understanding. At present, DLA has reached the milestone achievement; however, DLA of non-Manhattan is still challenging because of annotation data limitations. In this paper, we propose an image layer modeling method to mitigate this issue. The image layer modeling method generates document images of non-Manhattan layouts by superimposing images under pre-defined aesthetic rules. Due to the lack of evaluation benchmark for non-Manhattan layout, we have constructed a manually-labeled non-Manhattan layout fine-grained segmentation dataset. To the best of our knowledge, this is the first manually-labeled non-Manhattan layout fine-grained segmentation dataset. Extensive experimental results verify that our proposed image layer modeling method can better deal with the fine-grained segmented document of the non-Manhattan layout.

Few-shot segmentation has attracted growing interest owing to its value in practical applications. The primary challenge of few-shot segmentation lies in semantic information discovery, especially for query images. To tackle this issue, we propose a dual-expert distillation network (DEDN) made up of a scenario-level expert and an object-level expert to obtain semantic information from different perspectives. In DEDN, experts can learn from each other through online knowledge distillation with positive-guided Kullback-Leibler divergence. We innovate the Scenario Normalization and Object Continuity Guidance on dual experts to guarantee the various perspectives respectively. We further propose the Adaptive Weighted Fusion to adapt the trained experts to novel classes and obtain reliable fused predictions. Extensive experiments on Pascal-5i and COCO-20i show that our approach achieves state-of-the-art results.

Scene text erasing aims to wipe text regions in scene images with reasonable background. Most previous approaches employ scene text detectors to assist localization of the text regions. However, detected text boxes contain both text strokes and background clutters, and directly in-painting on the whole boxes may remain text artifacts and make regions unnatural. In this paper, we present an end-to-end network that focuses on modeling text stroke masks that provide more accurate locations to compute erased images. The network consists of two stages, i.e., a basic network with stroke generation and a refinement network with stroke awareness. The basic network predicts the text stroke masks and initial erasing results simultaneously. The refinement network receives the masks as supervision to generate natural erased results. Experiments on both synthetic and real-world scene images demonstrate the effectiveness of our framework in producing high quality erasing results.

Reading scene text in the natural image is of fundamental importance in many real-world problems. Text recognition has a profound effect on information processing by enabling automated extraction and interpretation. Recent scene text recognition methods employ the encoder-decoder framework, which constructs the encoder by obtaining the visual representations based on the last layer of the backbone network and then feeding them into a sequence model. In this article, we propose a novel encoder structure that performs the feature extractor and the sequence modeling within a unified framework. The introduced Aggregated Temporal Convolutional Encoder (ATCE) first incorporates the temporal convolutional layers to consider the long-term temporal relationship in the encoder stage. The aggregation of these temporal convolution modules is designed to utilize visual features from different levels, by augmenting the standard architecture with deeper aggregation to better fuse information across modules. We also study the impact of different attention modules in convolutional blocks for learning accurate text representations. We conduct comparisons on several scene text recognition benchmarks for both Chinese and English; the experiments demonstrate the complementary ability with different decoder variants and the effectiveness of our proposed approach.

Scene text recognition is considered as a sequence labeling problem. For the text recognition task, the alignment between the scene text image and the output text is coincident, which means the latter characters corresponding to the image region will also be behind. However, the existing global attention-based method focuses too much irrelevant information which leads to alignment drift. Contrary, local attention selects the subset of feature representation most relevant to the current character. In this paper, we explore the local attention mechanism and attempt to replace the global attention to implement decoding. Therefore, we revise several variants of local attention methods and provide a comprehensive comparison, which is missing in the scene text recognition literature so far. Specially, we introduce two Heuristic approaches for Local Attention (HLA) and prove that monotonic alignment improves performance significantly. Evaluations on the benchmarks show that the local attention method outperforms the existing global attention methods.

Depth completion is a task that recovers a dense depth map from a sparse depth map with the corresponding color image. Recently, the intensive depth generation guided by image clues in the color map has achieved good results. Color images can provide structural and semantic information as guidance information, but cannot provide the more important information about geometric relationships. In this paper, we propose a novel network to learn latent 3D cues from RGB images and depth images. More specifically, the network contains a 3D clues extractor and a dense depth generator. The extractor is designed to fusion and extract the 3D joint clues from the color image and sparse depth. The generator is trained with the sparse depth map and 3D clues to producing a more accurate dense depth map. Extensive experiments show that our proposed method has a significant improvement over existing image-guided methods.

Document layout analysis plays a vital role in computer vision research. Current document layout analysis methods mostly use pixel-based classification for document layout analysis. However, the method based on pixel classification is insufficient for maintaining the continuity of the classification area. In this paper, we propose a document layout analysis method based on positional encoding and bounding box specification. We maintain the continuity of the analysis area by constructing a document layout analysis framework based on the bounding box. In addition, we also integrate a positional encoding module in the framework to maintain the detailed information in the document layout analysis and modeling process. Experimental results prove that our proposed method has achieved state-of-the-art results.

Anomaly localization is pixel-level regions detection in the image. The challenge is how to generate accurate representations of the novel anomaly types which are multifarious. Besides, the anomaly sample size is often not enough to support model learning to detection because of the limitations of real conditions. In this work, we present a novel few-shot setting for anomaly detection and reorganize the defective datasets. Based on the few-shot learning, we transfer the idea of metric learning and propose the prototype-guided transfer network (PGTNet). Extensive experiment results suggest that PGT-Net outperforms current SOTA methods and provides a novel perspective for the anomaly localization task.

The multi-modal target-oriented sentiment analysis aims at predicting the sentiment polarities for target entities in a sentence by combining vision and language information. However, most existing deep learning approaches fail to extract valuable information from the visual modality and ignore the usability of syntactic dependency information embedded in the text modality. In this paper, we propose a two-stream adaptive multi-feature extraction graph convolutional networks (AME-GCN), which translates the image into a textual caption and dynamically fuses the semantic and syntactic feature from the given sentence and generated caption to model the inter/intra-modality dynamics. Extensive experiments on two multi-modal Twitter datasets show the effectiveness of the proposed model against popular textual and multi-modal approaches, demonstrating that AME-GCN is a best alternative for this task.

Depth completion technology based on multi-level feature fusion (MF) strategy has recently achieved remarkable success. However, the existing MF-based methods treat RGB features and depth map features equally when performing modal fusion but ignore the difference in semantic richness and sparsity between them, which leads to the results generated by these methods overfitting the shape of RGB and harm to the accuracy of depth value. To address this problem, we proposed a novel dual fusion (DF) strategy for MF-based depth completion, which can prevent overfitting by weakening the influence of RGB features on the generated results through two fusion stages. The entire DF framework consists of two multi-level fusion modules. The first fusion module performs a simple fusion of RGB features and depth features, while the second fusion module enriches the sparse image representation with the previously obtained fused features. Besides, we utilize non-local sparse attention to solve the problem that ordinary convolution is not capable of expressing depth map features enough. We test our approach on the outdoor KITTI test set and achieve the state-of-the-art (SOTA) performance in RMSE. Extensive experiments on the indoor NYUv2 dataset and KITTI validation set further demonstrate that our approach outperforms existing MF-based methods.

Aspect-level sentiment classification (ASC), detecting and predicting the sentiment polarity of the given aspecs, has attracted increasing attention in the field of Natural Language Processing (NLP). Recent studies in ASC leveraged the graph based on the dependency tree of the context to incorporate the syntactic information and structure of a sentence for better relation extraction. Some researchers noted that existing methods ignored semantic relations or failed to consider affective dependency information, and then proposed several state-of-art methods tackling the above two limitations. However, these approaches failed to consider both informative relations simultaneously. Therefore, we explore and propose a novel solution based on semantic latent graph and SenticNet to leverage semantic and affective information. Specifically, we build a latent semantic graph based on self-attention networks to parse semantic relations within the contexts. In addition, we utilize affective knowledge from SenticNet to enhance the dependency graphs of sentences. Moreover, we use the gate mechanism to dynamically combine information from both the enhanced dependency graphs and latent semantic graphs. Experimental results on three benchmark datasets illustrate the effectiveness and state-of-the-art performance of our model.

Abstract For the multiprocessor systems modeled by interconnection networks, one of the important properties is the characterization of fault tolerability. Connectivity, as an important parameter to evaluate fault tolerability, has witnessed research achievements. To make the evaluation more practical, conditional connectivity has been promisingly proposed. As one kind of conditional connectivity, $h$-restricted connectivity of a connected graph $G$, denoted by $\kappa ^h (G)$, is defined as the cardinality of the minimum vertex cut set $F$ such that $\delta (G-F)\geq h$. In this paper, we establish a universally $h$-restricted connectivity for a class of hypercube-based compound networks, in which the well-known networks, such as hierarchical cubic network $HCN(n, n)$ and its generalization complete cubic network $CCN(n)$, are involved.

Anomaly segmentation is an important task in computer vision. At present, anomaly segmentation has achieved milestone development. Many representative works have been proposed, especially unsupervised learning methods and pre-training methods, but pre-training methods are difficult to solve cross-The gap brought by domains has not been paid much attention to in previous research work on how to mine the hidden information contained in the data itself. Because of the diversity of anomaly detection, it is crucial to make full use of the information of the object itself. This paper proposes a novel dynamic branch fusion structure, through mining the hidden information inside the data, so as to re-model more targeted abnormal segmentation. The method has been tested on two known benchmarks to verify the effectiveness of the proposed method.

We perform fast single image super-resolution with flexible magnification for natural images. A novel coarse-to-fine super-resolution framework is developed for the magnification that is factorized into a maximum integer component and the quotient. Specifically, our framework is embedded with a light-weight upscale network for super-resolution with the integer scale factor, followed by the fine-grained network to guide interpolation on feature maps as well as to generate the super-resolved image. Compared with the previous flexible magnification super-resolution approaches, the proposed framework achieves a tradeoff between computational complexity and performance. We conduct experiments using the coarse-to-fine framework on the standard benchmarks and demonstrate its superiority in terms of effectiveness and efficiency over previous approaches.

Texts from scene images typically consist of several characters and exhibit a characteristic sequence structure. Existing methods capture the structure with the sequence-to-sequence models by an encoder to have the visual representations and then a decoder to translate the features into the label sequence. In this paper, we study text recognition framework by considering the long-term temporal dependencies in the encoder stage. We demonstrate that the proposed Temporal Convolutional Encoder with increased sequential extents improves the accuracy of text recognition. We also study the impact of different attention modules in convolutional blocks for learning accurate text representations. We conduct comparisons on seven datasets and the experiments demonstrate the effectiveness of our proposed approach.

Segmentation-based scene text detection approaches have been adopted to arbitrary-shaped texts and have achieved a great progress. However, false detection always easily exist when the arbitrary-shaped texts are close to each other. In this paper, we propose the Margin Guidance Network (MGN) that mainly based on the margin constraint residual module (MCRM) to address aforementioned problem. The MCRM considers the margins between multiple text instance masks to guide the training of network and improve the performance on text detection. The MCRM contains two prediction branch, the one can generate the multiple different scale of masks for a text instance and the other branch is used to generate multiple margins between the above masks. Experimental results on three public benchmarks including ICDAR2015, CTW1500 and Total-Text have demonstrated that the proposed MGN achieves the state-of-the-art results.