Multimodal medical image fusion technology provides more comprehensive and accurate image support for clinical diagnosis and treatment by integrating complementary information from different imaging modalities. Aiming at the problem that existing methods are still insufficient in detail feature extraction and inter-modal information fusion, this paper proposes a multimodal medical image fusion method combined with an adaptive attention mechanism. First, we design the Grouped Receptive Field Attentional Convolution (GRFAConv) to solve the problem of insufficient detail feature extraction capability. With the multi-head receptive field adaptive weighting strategy of grouped convolution, the range and weight of the receptive field of the convolution kernel can be adaptively adjusted according to the different demands of local and global features of the image to improve the effect of detail retention. Second, for the problem of information fusion between different modalities, we introduce an improved CBAM attention module in the feature fusion process, which adaptively selects and enhances the features in the key regions through the channel attention and spatial attention mechanisms, which greatly improves the clarity of the fused image details and the accuracy of the information expression in the key regions. Furthermore, experimental results on several medical image datasets show that the algorithm proposed in this paper can generate relatively high-quality fused images. It not only enriches the detailed features of the image, but also achieves significant advantages in several evaluation metrics.

In the fault diagnosis of industrial equipment, transfer learning alleviates the problem of data distribution offset and annotation scarcity through cross domain knowledge migration. However, the existing methods have limitations. Single scale feature alignment ignores the difference between shallow and deep features. The pseudo label strategy with fixed temperature parameters reduces the feature discrimination, and the single distribution alignment is difficult to take into account the global and local structure. Therefore, this paper proposes a multi-scale dynamic confrontation transfer learning framework (MDATL), which includes a hierarchical dynamic confrontation mechanism, and dynamically adjusts the characteristic confrontation intensity of each layer through periodic GRL; the two-stage pseudo label optimization strategy combined with temperature scaling softens the probability distribution of the target domain; a mixed distribution alignment strategy, which combines MMD and CORAL to dynamically balance global statistics and local covariance. Experiments using CWRU and PU data sets of six groups of cross condition task verification show the superior performance of this method in unsupervised fault diagnosis.

The evolution of online social media has reshaped public behaviors, and social data infused with emotions provides crucial decision support for sentiment analysis tasks. Conventional multimodal approaches, influenced by redundant information in feature extraction during sentiment analysis, often focus solely on modality interactions without considering the unique content of each modality. This paper introduces the HGB-HFN model, addressing feature redundancy by optimizing extracted features using metaheuristic algorithm techniques, achieving a higher accuracy optimal feature dataset at lower computational costs. To counteract the neglect of modal-independent information, a hybrid fusion approach is adopted. It prioritizes visual features to extract precise semantic and emotional information from textual content, training multiple base classifiers to learn independent and diverse discriminative information. Comparative experiments on streaming datasets MOSI and MOSEI demonstrate the computational approach’s superiority in both accuracy and efficiency over existing methods.

Masked Language Modeling (MLM) and Image-Text Matching (ITM) are always used in fusion encoder to learn the joint representation of images and text. In existing methods, the masking strategy of MLM leads to the neglect of image details during the modeling process. Meanwhile, the sampling strategy of ITM struggles to consistently select high-difficulty hard negative instances, reducing the effectiveness of constraints. This leads to challenges in aligning fine-grained information in cross-modal retrieval. In response to this challenge, a fine-grained information alignment-based visual language model (FAM) is proposed in this paper. On one hand, the attribute-based masking strategy is employed in MLM, helping the model focus on the details of objects in images during modeling. On the other hand, the robust hard negative sample generation strategy provides challenging negative samples for ITM by altering the relationships between objects. This enables the model to align relationships between objects in different modalities and thus calibrates cross-modal retrieval. Extensive experiments demonstrate the effectiveness of the model in cross-modal retrieval tasks.

In recent years, a single medical image is prone to lose hidden features with low resolution and salient features with high noise but rich information, ignoring the connection of multiple images. Therefore, to remedy for the shortcomings, this paper proposes a multi-modal medical image classification method combined with graph convolutional neural networks(MB-pGCN). First, for the first modality image, we use the ResNet-152 model to extract features, and obtain local disease-related region features with the inter-group comparison method. It utilizes weak attention learning to extract local salient features. For the second modality, we use the patch extraction method to embed. Then it combines the acquired local image features and uses the complementary information carried by the modalities to construct a graph convolutional neural network, designing the convergence layer to achieve graph pooling, which takes advantage of the node features and the local structural relationships between nodes. Ultimately, it splices global features with local features to achieve classification. To validate the proposed classification model framework, we choose ADNI, COVID-19 datasets that are in line with the research, showing that MB-pGCN outperforms other models.

In many underwater application scenarios, recognition tasks need to be executed promptly on computationally limited platforms. However, models designed for this field often exhibit spatial locality, and existing works lack the ability to capture crucial details in images. Therefore, a lightweight and detail-aware vision network (LDVNet) for resource-constrained environments is proposed to overcome the limitations of these approaches. Firstly, in order to enhance the accuracy of target image recognition, we introduce transformer modules to acquire global information, thus addressing the issue of spatial locality inherent in traditional convolutional neural networks (CNNs). Secondly, to maintain the network’s lightweight nature, we integrate the transformer module with convolutional operations, thereby mitigating the substantial parameter and floating point operations (FLOPs) overhead. Thirdly, for the efficient extraction of crucial fine-grained details from feature maps, we have devised a channel and spatial attention module (C&SA). This module aids in recognizing intricate and fine-grained visual tasks and enhances image understanding. It is seamlessly integrated into LDVNet with nearly negligible parameter overhead. The experimental results demonstrate that LDVNet outperforms other lightweight networks and hybrid networks in different recognition tasks, while being suitable for resource-constrained environments.

Image inpainting, which aims to reconstruct reasonably clear and realistic images from known pixel information, is one of the core problems in computer vision. However, due to the complexity and variability of the underwater environment, the inability to extract valid pixel points and insufficient correlation between feature information in existing image inpainting techniques lead to blurring in the generated images. Therefore, a novel gated attention feature fusion image inpainting network based on generative adversarial networks (GAF-GAN) is proposed. The accuracy of feature similarity matching depends heavily on the validity of the information contained in the features. On the one hand, gating values are dynamically generated by gated convolution to reduce the interference of invalid information. On the other hand, semantic information at distant locations in an image is accurately acquired by the attention mechanism. For these reasons, we designed an improved gated attention mechanism. Gated attention mechanism make the network focus on effective information such as high-frequency texture and color fidelity of restored images. In addition, the dense feature fusion module is added to expand the overall receptive field of the network to fully learn the image features. Experimental results show that the proposed method can effectively repair defective images with complex texture structures and improve the reality and integrity of image details and structures.

Underwater images are often affected by problems such as light attenuation, color distortion, noise and scattering, resulting in image defects. A novel image inpainting method is proposed to intelligently predict and fill damaged areas for complete and continuous visualization of the image. First, in order to effectively solve the problem of color distortion caused by light refraction in underwater environments, the improved gated attention mechanism is used. This mechanism improves the local details by learning and weighting the important features of the image. Second, gated convolution automatically determines the degree of restoration for each pixel based on local features of the original image. It eliminates distractions such as low contrast and scattering, retaining more original detailed information. By doing so, image inpainting techniques improve the quality and visualization of underwater images.

In a real scenario, the image is often corrupted by complex degradation, and a lot of useful information is lost, which makes super-resolution (SR) reconstruction seriously ill-posed. To effectively solve such a problem, it is crucial to correctly exploit image prior knowledge. Although existing deep learning-based methods can obtain excellent results, they cannot deal with the complex degradation effectively, which would lead to the loss of texture details and the destruction of edge details. In this paper, an efficient multi-regularization method for SR is proposed, which can simultaneously exploit both internal and external image priors within a unified framework. The hybrid Tikhonov-TV prior and deep denoiser prior are introduced to constrain the reconstruction process. That is, the proposed model combines the superiority of the piecewise-smooth prior and deep prior. Moreover, an adaptive weight parameter is employed to make the hybrid component more detail-preserving. Experimental demonstrate that the proposed method achieves better performance in image detail protection than advanced methods.

In recent years, with the development of deep learning, many lightweight convolution neural networks (CNN) have achieved remarkable results in the field of single image super-resolution (SISR). However, it is difficult to capture long-range dependencies due to the limited perceptual field of lightweight CNN. To solve this problem, we propose the lightweight capture context network (LCCN), which includes the hierarchical residual attention distillation fusion (HRADF) module and lightweight capture context dependency (LCCD) module. In HRADF, we propose a hierarchical feature fusion structure consisting of multiple residual attention distillation blocks to improve the reconstruction effect by fusing multiple layers of feature maps. Moreover, the tpaconv lrelu block (TLB) and mixed spatial channel attention (MSCA) are applied to make the network lightweight and take full advantage of feature information. In LCCD, we introduce asymmetrical double multi-head attention to achieve low resource consumption and improve the long-range context dependency capture capability of the network. Extensive experiments show that LCCN achieves a good balance between performance and model complexity, and obtains satisfactory results on several benchmark datasets.

Learning-based Multi-view Stereo (MVS) and stereo matching methods typically construct 3D cost volumes based on the camera frustum of the reference view. Regularization and regression of the cost volume are performed to obtain a depth map. However, the resolution of the output depth map is limited by the computational cost, and when performing feature extraction, the characteristics of convolution local perception make it impossible to capture global context information. In this paper, we propose CTPMVSNet by using the Global Feature Aware Transformer (GFT) to aggregate global context information within and across images. In order to make better use of GFT, we use Deformable Convolution Module (DCM) to ensure a smooth transition of the extracted feature range. In addition, in the cost volume regularization stage, to improve efficiency and generation accuracy, we design a lightweight regularization network with integrated pseudo-three-dimensional convolution, and our experiments on multiple dataset have achieved promising results.

Among the research methods for HER2 automatic evaluation in recent years, most of the methods using deep learning framework have both segmentation and classification functions. Although these methods provide pathologists with reference lesions, they increase the dependence on dataset and the computational cost. Therefore, we propose a Wide Pyramid Network (WPNet) based on deep learning to solve this problem. Our designed WPNet is different from other neural network models, which is mainly extended in the width of the network, and uses the wide pyramid structure to extract the features of different scales on the image for training. Since HER2 score is determined according to the degree of cell membrane staining and the proportion of cells with different degrees of staining, the WPNet model capable of multi-scale feature extraction can facilitate the determination of HER2 score. Compared with other models, for HER2 score classification based on a small sample set, the proposed model not only accelerates the convergence speed during training, reduces the calculation cost but also improves the classification effect.

In this article, we propose a novel localization and channel estimation integration framework for cell-free massive multiple-input–multiple-output (MIMO) Internet of Things (IoT) systems, in which position information supports accurate channel estimation and accurate channel information can, in turn, improve positioning accuracy. Under this integration framework, we propose a two-phase fingerprint-based localization method consisting of both initial and accurate localization phases and a coarse-location-based (CLB) pilot reassignment scheme. The coarse location information for pilot reassignment is obtained in the initial localization phase of the two-phase localization method, and the fingerprint information used in the accurate localization phase is extracted through channel estimation based on the CLB scheme. Furthermore, for localization, two different fingerprint similarity criteria are proposed to meet the requirements of the different localization phases. Simulation results demonstrate that our proposed two-phase fingerprint-based localization method achieves better positioning performance than existing methods, although there is a slight increase in computational complexity compared to the initial localization. Moreover, our proposed CLB pilot reassignment scheme outperforms the conventional pilot assignment schemes in the comprehensive performance considering both channel estimation performance and complexity.

The binary feature descriptors such as binary robust independent elementary features (BRIEF), oriented rotated binary robust independent elementary features (ORB), and fast retina keypoint (FREAK) usually perform binarisation on the intensity comparisons, thus they lose some useful information. In this study, the authors propose an effective binary image descriptor which is called significant bit‐planes‐based local binary pattern for visual recognition. First, the authors divide an image into several sub regions according to the intensity orders to incorporate the spatial information. Then the authors extract the higher bit planes for all the sub regions and sort the adjacent neighbour bits based on the corresponding intensity orders, which make the descriptor invariant to rotation. In order to further improve the discriminative ability, the authors sample the multi‐scale neighbours and average the adjacent pixels and extract the feature descriptor from the higher bit planes. Since the authors directly perform operation on the significant bit planes without quantisation, the authors decrease the information loss to some extent. The descriptor has demonstrated a better performance over the state‐of‐the‐art binary descriptors as well as scale invariant feature transform on two recognition benchmarks (i.e. Kentucky and ETHZ) and PASCAL 2007 for image classification.

Most of the current plug-in technology researches and designs focus on the implementation of a single plug-in function, but is lack in support from safety technology and metadata. However, as an open, dynamic running system, plug-in framework requires the infrastructure framework to control the legitimacy of plug-ins, and constrain the implementation of its function. This paper presents a plug-in application framework based on .NET reflection technology and safety technology which has not only complemented and improved current plug-in studies, but also enhanced the effectiveness and practicality of the plug-in framework.