Compared to single modality, multimodal data can provide more comprehensive information for recognition tasks. As the quality of multimodal data varies in open environments, to make full use of the value of each modality and mitigate the impact of low-quality modality data, uncertainty-aware dynamic multimodal fusion has emerged as a promising learning paradigm. Although such methods are widely used, the robustness of these models is still significantly affected by the lack of samples with dynamically varying multimodal quality in training sets. This lackness limits the models’ ability of uncertainty estimation and dynamic multimodal fusion in open environments. In this study, we propose a data augmentation method towards dynamic multimodal fusion. First, through adversarial learning, we generate model-aware high-uncertainty samples, overcoming the limitations of manually predefined low-quality samples. Second, we mix the generated low-quality modalities with original high-quality modality samples to construct multimodal data with dynamically varying quality, thereby enhancing the scenario diversity of multimodal data. Our method not only integrates easily with existing dynamic fusion methods, but also demonstrates state-of-the-art performance in both efficacy and reliability through comprehensive experiments on various benchmark datasets.

The image enhancement task requires a complex balance between extracting high-level contextual information and optimizing spatial details in the image to improve the visual quality. Most of existing methods have limited capability in capturing contextual features and optimizing spatial details when they only rely on a single modality. To address the above issues, this paper introduces a novel multi-modal image enhancement network based on Transformer, named as TITFormer, which combines textual and simulating infrared modalities firstly for this important task. TITFormer comprises a text channel attention fusion (TCF) network block and an infrared-guided spatial detail optimization (SDO) network block. The TCF extracts contextual features from the high-dimensional features compressed after spatial channel transformation of the textual feature and image feature. The SDO module uses simulating infrared images characterized by pixel intensity to guide the optimization of spatial details with contextual features adaptively. Experimental results demonstrate that TITFormer achieves state-of-the-art performance on two publicly available benchmark datasets.

Researchers have applied 3D Lookup Tables (LUTs) in cameras, offering new possibilities for enhancing image quality and achieving various tonal effects. However, these approaches often overlook the non-uniformity of color distribution in the original images, which limits the performance of learnable LUTs. To address this issue, we introduce a lightweight end-to-end image enhancement method called Simulated Infrared Fusion Guided Image-adaptive 3D Lookup Tables (SIRLUT). SIRLUT enhances the adaptability of 3D LUTs by reorganizing the color distribution of images through the integration of simulated infrared imagery. Specifically, SIRLUT consists of an efficient Simulated Infrared Fusion (SIF) module and a Simulated Infrared Guided (SIG) refinement module. The SIF module leverages a cross-modal channel attention mechanism to perceive global information and generate dynamic 3D LUTs, while the SIG refinement module blends simulated infrared images to match image consistency features from both structural and color aspects, achieving local feature fusion. Experimental results demonstrate that SIRLUT outperforms state-of-the-art methods on different tasks by up to 0.88 ~ 2.25dB while reducing the number of parameters. Code is available at https://github.com/riversky2025/SIRLUT.git .

Fake news is widely spread on social media. Much research works have been done on automatic fake news detection in single domain. However, fake news exists in various domains, so the detection model based on single domain is less effective in multiple domain scenes. To improve the detection ability of multi-domain fake news, we propose a perspective collaboration for multi-domain fake news detection (PCMFND) method to detect fake news across multiple domains by combining the powerful feature extraction ability of expert systems. The method extracts features of different perspectives from news content separately, then interactively combines the features of different perspectives, and ultimately achieves fake news detection by adaptively aggregating features of each perspective through domain knowledge. The effectiveness of the proposed method is demonstrated through comparison experiments with traditional multi-domain detection methods on Chinese and English multi-domain datasets.

To effectively solve the problem of a small proportion of substantia nigra and midbrain regions in Magnetic Resolution Imaging (MRI) images of Parkinson’s disease (PD) patients, unclear boundaries with surrounding tissues, and difficulty in accurately delineating boundaries, an improved U2-Net algorithm for Parkinson’s substantia nigra midbrain image segmentation was proposed. This algorithm first improves the Residual U-blocks (RSU) and RSU-4F modules using the Shuffle Attention (SA) module, enhancing the network’s attention to Parkinson’s substantia nigra and midbrain blurry regions in the encoding and decoding layers. Next, replace some ordinary convolutions in RSU-4F with DynamicConv (DyConv), capture local features through dynamic convolution kernels, and improve the segmentation performance of Parkinson’s images. The experimental data used clinical data, and the improved algorithm achieved Dice, Precision, Recall, and mIou of 80.27%, 89.99%, 88.88%, and 82.38% in substantia nigra segmentation, respectively. The experimental results show that this algorithm can achieve more precise segmentation of Parkinson’s images.

Classification on long-tailed distributed data is a challenging problem, which suffers from serious class-imbalance and accordingly unpromising performance es-pecially on tail classes. Recently, the ensembling based methods achieve the state-of-the-art performance and show great potential. However, there are two limitations for cur-rent methods. First, their predictions are not trustworthy for failure-sensitive applications. This is especially harmful for the tail classes where the wrong predictions is basically fre-quent. Second, they assign unified numbers of experts to all samples, which is redundant for easy samples with excessive computational cost. To address these issues, we propose a Trustworthy Long-tailed Classification (TLC) method to jointly conduct classification and uncertainty estimation to identify hard samples in a multi-expert framework. Our TLC obtains the evidence-based uncertainty (EvU) and ev-idence for each expert, and then combines these uncer-tainties and evidences under the Dempster-Shafer Evidence Theory (DST). Moreover, we propose a dynamic expert en-gagement to reduce the number of engaged experts for easy samples and achieve efficiency while maintaining promising performances. Finally, we conduct comprehensive ex-periments on the tasks of classification, tail detection, OOD detection and failure prediction. The experimental results show that the proposed TLC outperforms existing methods and is trustworthy with reliable uncertainty.

With the rapid development of Internet technology, online learning and online education are becoming more and more popular. Intelligent learning diagnosis has become an effective means to guarantee the quality of online learning, and has become a research hotspot in the direction of education informatization. Concept map is an intuitive visual tool that can discover the concepts poorly mastered by students, and provide useful clues for identifying learning disabilities of students. This paper proposed a learning diagnosis method constructed based on the concept map. First, it groups learners, then uses direct hashing and pruning (DHP) to generate association rules between the different concepts, and finally uses the DHP algorithm to produce and construct the concept map automatically to discover the concepts poorly mastered by students, and realize the automatic diagnosis of learning problems. Case studies that have been done in college mathematics classes in some universities of Lianyungang have fully verified the effectiveness of our method.

Molecular similarity search is a simple but powerful chemoinformatics tool to rapidly find molecules that are structurally similar to a known reference compound from a large molecular database. A variety of indexing structures had been developed to improve the performance of similarity search over the large compound database. However, those algorithms often require a large computational cost to build indices and process queries, especially for a large-scale molecular dataset. We study the problem of accelerating similarity search using high performance computing (HPC) and design general algorithms to speed up existing indexing algorithms. We first propose a parallel algorithm based on data chunking, working for all indexing algorithms for similarity search. We theoretically analyze its computation cost and relationships between the speedup and number of data chunks. We further propose a parallel query algorithm for all graph-based indexing algorithms to accelerate their query processing in HPC. Both of our algorithms consistently offer a greater speedup than the baseline algorithm(s) when evaluated with different datasets and parameter settings.