In supervised learning, traditional image masking faces two key issues: (i) discarded pixels are underutilized, leading to a loss of valuable contextual information; (ii) masking may remove small or critical features, especially in fine-grained tasks. In contrast, masked image modeling (MIM) has demonstrated that masked regions can be reconstructed from partial input, revealing that even incomplete data can exhibit strong contextual consistency with the original image. This highlights the potential of masked regions as sources of semantic diversity. Motivated by this, we revisit the image masking approach, proposing to treat masked content as auxiliary knowledge rather than ignored. Based on this, we proposed MaskAnyNet, which combines masking with a relearning mechanism to exploit both visible and masked information. It can be easily extended to any model with an additional branch to jointly learn from the recomposed masked region. This approach leverages the semantic diversity of masked regions to enrich features and preserve fine-grained details. Experiments on CNN and Transformer backbones show consistent gains across multiple benchmarks. Further analysis confirms that the proposed method improves semantic diversity through the reuse of masked content.

Electroencephalography (EEG) is vital for brain-computer interfaces (BCIs) due to its non-invasive approach and high temporal resolution data capabilities, amid challenges such as data scarcity and the need for extensive labeling. Significant inter-individual variability in EEG signals further limits model generalization. Concurrently, the use of self-supervised pre-training, particularly through masked modeling, is gaining traction in time series analysis to mitigate labeling costs. Although this method involves reconstructing masked signal from unmasked series, random masking can disrupt critical temporal variations, complicating effective representation learning. We thus introduce SSL-MEMI, a novel self-supervised contrastive learning framework for masked EEG motor imagery modeling, integrating Domain Adaptive Alignment (DAA) and Multi-View Temporal-spatial Attention module (MTSA) to effectively handle EEG variability. This framework utilizes manifold-based masking to reconstruct original sequences from masked series, thereby enhancing classification accuracy. When tested on the BCI Competition IV and High Gamma datasets, SSL-MEMI outperforms existing methods, achieving top accuracies and demonstrating superior domain adaptation through reduced Global ${\mathcal{A}}$-distance scores. This study advances EEG classification and indicates broader applications for self-supervised learning in biomedical signal processing. The source code is available at https://github.com/KunKun-Zhang/SSL-MEMI.git.

Domain generalization (DG) aims to enhance the ability of model learning from source domains to generalize to other unseen domains. Existing gradient-based methods focus on learning better domain-invariant features using gradients from multiple source domains, but do not consider the impact of gradient granularity on model training. In this paper, we rethink how to mitigate the gradient conflicting problem from an optimization perspective. The limitations of existing gradient-based methods are theoretically analyzed in terms of modification ratio and modification frequency, showing that gradient granularity is a key factor in ensuring correct modification of the gradient. To address this issue, a gradient modification method, called CorGrad, is proposed by layering and slicing refinement operations to increase the modification frequency and the modification ratio. It can better reduce domain-specific information so that the model can learn better domain-invariant features. Finally, extensive experiments are conducted to verify the effectiveness of the proposed CorGrad, and the results show that the proposed CorGrad can obtain competitive performance in five DG benchmarks, and an average performance of 60.4% can be obtained on the DomainBed when using ResNet18 as the backbone. The code is publicly available at https://github.com/libzwo/CorGrad.

For the few-shot segmentation (FSS) task, existing methods attempt to capture the diversity of new classes by fully utilizing the limited support images, such as cross-attention and prototype matching. However, they often overlook the fact that there is variability in different regions of the same object, and intra-image similarity is higher than inter-image similarity. To address these limitations, a Self-Guided Transformer (SGT) is proposed by leveraging intra-image similarity to improve intra-object inconsistencies in this paper. The proposed SGT can selectively guide segmentation, emphasizing the regions that are easily distinguishable while adapting to the challenges caused by less discriminative regions within objects. Through a refined feature interaction scheme and the novel SGT module, our method can achieve state-of-the-art performance on various FSS datasets, demonstrating significant advances in few-shot semantic segmentation. The code is publicly available at https://github.com/HuHaigen/SGT.

Effective lesion detection in medical image is not only rely on the features of lesion region, but also deeply relative to the surrounding information. However, most current methods have not fully utilize it. What’s more, multi-scale feature fusion mechanism of most traditional detectors are unable to transmit detail information without loss, which makes it hard to detect small and boundary-ambiguous lesion in early stage disease. To address the above issues, we propose a novel intra- and across-layer feature interaction FCOS model (IAFI-FCOS) with a multi-scale feature fusion mechanism ICAF-FPN, which is a network structure with intra-layer context augmentation (ICA) block and across-layer feature weighting (AFW) block. Therefore, the traditional FCOS detector is optimized by enriching the feature representation from two perspectives. Specifically, the ICA block utilizes dilated attention to augment the context information in order to capture long-range dependencies between the lesion region and the surrounding. The AFW block utilizes dual-axis attention mechanism and weighting operation to obtain the efficient across-layer interaction features, enhancing the representation of detailed features. Our approach has been extensively experimented on both the private pancreatic lesion dataset and the public DeepLesion dataset, with AP50of 62.2% and 60.0%, respectively, and these results are 6.4% and 2.3% higher than the FCOS. Additionally, our model achieves SOTA results on the pancreatic lesion dataset.

Magnetic Resonance Imaging (MRI) with multiple modalities is commonly used for diagnosis, but it is associated with an inherently slow acquisition process. To accelerate multi-modal MRI, recent studies explored the merits of using a fully-sampled reference modality (RM) as a guidance to reconstruct the query modalities (QMs) from their undersampled k-space data via convolutional neural networks (CNNs). However, even aided by the RM, the reconstruction of highly undersampled QM data is still suffering from aliasing artifacts. To enhance reconstruction quality, we suggest to further release the guiding power of the RM data via generating its multiscale variants. To this end, we simultaneously partition the k-space of the RM and QM into several subregions with gradually increasing sizes. We then proposed a k-Space Partition-based Convolutional Network (kSPCN) to fully use the partitioned RM and QM data to perform QM reconstruction subregion by subregion. Extensive experiments on different query modalities and acceleration rates demonstrate that kSPCN consistently outperforms state-of-the-art methods and can preserve anatomical structure faithfully up to 12-fold undersampling.

Deep Cascading Networks (DCNs) are very popular for fast MRI reconstruction. However, DCNs still have limited generalization ability on highly undersampled MRI data. One main reason is that the training data is not well used. A promising solution is to boost the filter diversity of DCNs to well fit the rich features in the training data. This can be achieved by reducing the repetition level of the undersampled input images via using the pixel unshuffle (PU) operator. As different input images and different subnets of DCNs require different PU-scales, we propose a novel PU-Scale Estimation (PUSE) method to automatically infer optimal PU-scales. By incorporating PUSE into DCNs, we construct a new Multi-PU-Scale Diversity based (MSDiv+) architecture for DCNs. To boost training convergence, we further propose to generate mini-batches by mixing data samples with different optimal PU-scales. Experiments on the fastMRI dataset demonstrate the effectiveness of our method.

Recent advancements in medical image segmentation have demonstrated superior performance by combining Transformer and U-Net due to the Transformer’s exceptional ability to capture long-range semantic dependencies. However, existing approaches mostly replace or concatenate the Convolutional Neural Networks (CNNs) and Transformers in series, which limits the potential of their combination. In this paper, we introduce a dual-branch feature encoder, CTI-UNet, that effectively fuses the global representations and local features of the CNN and Transformer branches at different scales through bidirectional feature interaction. Our proposed method outperforms existing approaches on multiple medical datasets, demonstrating state-of-the-art performance. The code for CTI-UNet is publicly available at https://github.com/huhaigen/CTI-UNet.

Recently, many salient object detection (SOD) methods decouple image features into body features and edge features, which imply a new development direction in the field of SOD. Most of them mainly focus on how to decouple features, but the fusion method for the decoupled features can be further improved. In this paper, we propose a network, namely Progressive Coupling and Refining Network (PCRNet), which allows the progressive coupling and refining of the decoupled features to get accurate salient features . Furthermore, a novel loss, namely Non-Binary Intersection over Union (NBIoU), is proposed based on the characteristics of non-binary label images and the principle of Intersection over Union (IoU) loss. Experimental results show that our NBIoU performance surpasses binary cross-entropy (BCE), IoU and Dice on non-binary label images. The results on five popular SOD benchmark datasets show that our PCRNet significantly exceeds the previous state-of-the-art (SOTA) methods on multiple metrics. In addition, although our method is designed for SOD, it is comparable with previous SOTA methods on multiple benchmark datasets for camouflaged object detection without any modification on the network structure, verified the robustness of the proposed method.

Gun violence and misuse pose great threat to the public safety. Real-time monitoring of gun usage and gunshot events are very promising for effective gun control. However, most available monitoring systems are installed in a fixed location instead of the guns, which greatly limits the flexibility and coverage. In this study, we propose a wireless gun monitoring and gunshot recognition system based on a low-cost triaxial acceleration sensor, which can monitor the gun in real time and accurately recognize gunshot events. Addressing the limited resources of the embedded systems, we further propose an efficient gunshot recognition algorithm EfficientNetTime that combines the lightweight neural network and knowledge distillation, so as to enable the deployment on embedded devices. First, a novel lightweight deep learning model is proposed as the basic model, which combines the advantages of 1-D convolution and depthwise separable convolution to effectively characterize the gunshot signal while decreasing the computing cost of convolution. Second, using the knowledge distillation, EfficientNetTime is used as the teacher model to generate a compressed student model that maintains accuracy and greatly reducing model size. Finally, the EfficientNetTime student model can be deployed on resource-limited embedded systems. The proposed method can automatically extract features for end-to-end recognition and is robust to temporal transformations of input signals. Using a publicly available gunshot data set, the proposed EfficientNetTime model is verified and compared against the state-of-the-art models. Experimental results demonstrate that the EfficientNetTime model surpasses other gunshot recognition methods in terms of the accuracy and model size.

To enhance the nonlinearity of neural networks and increase their mapping abilities between the inputs and response variables, activation functions play a crucial role to model more complex relationships and patterns in the data. In this work, a novel methodology is proposed to adaptively customize activation functions only by adding very few parameters to the traditional activation functions such as Sigmoid, Tanh, and rectified linear unit (ReLU). To verify the effectiveness of the proposed methodology, some theoretical and experimental analysis on accelerating the convergence and improving the performance is presented, and a series of experiments are conducted based on various network models (such as AlexNet, VggNet, GoogLeNet, ResNet and DenseNet), and various datasets (such as CIFAR10, CIFAR100, miniImageNet, PASCAL VOC, and COCO). To further verify the validity and suitability in various optimization strategies and usage scenarios, some comparison experiments are also implemented among different optimization strategies (such as SGD, Momentum, AdaGrad, AdaDelta, and ADAM) and different recognition tasks such as classification and detection. The results show that the proposed methodology is very simple but with significant performance in convergence speed, precision, and generalization, and it can surpass other popular methods such as ReLU and adaptive functions such as Swish in almost all experiments in terms of overall performance.

Network acceleration is very important for Magnetic Resonance Imaging (MRI) reconstruction, as the device-specific retraining of a given network is usually necessary to overcome the scanner transfer problem. We propose a novel framework, namely PU-Network-PS (UNS), to accelerate the Deeply Cascaded Convolutional Networks (DCCNs) for MRI reconstruction, where Pixel Unshuffle (PU) and Pixel Shuffle (PS) are imposed at the two ends of DCCNs, respectively. Our UNS accelerates DCCNs by first decomposing the zero-filled MRI images of DCCNs into several sub-images via using PU, which are dubbed PU-channels and will make DCCNs mainly run in a low-resolution space and thus obtain high computation efficiency. However, directly using PU-channels might lead to unstable performance due to the lost spatial correlations between PU-channels. We discovered that the PU-channels can be organized into different groups according to their mutual similarities and presented a novel Multi-Scale Cross Group Convolution to fully fuse their complementary information. We further integrated into UNS the ensemble learning scheme in the PS stage to boost performance. Experiments on the IXI dataset demonstrated that our UNS can enhance both the reconstruction performance and the running efficiency for both simple and complex DCNN models. Specially, the training time for DuDoRNet can be greatly reduced from three days to about 20 hours with small performance enhancements by using two NVIDIA RTX 2080Ti GPUs.

A major challenge in the analysis of tissue imaging data is cell segmentation, the task of identifying precisely the boundary of each cell in a microscopic image. The cell segmentation task is still challenging due to the variable shapes, large size differences, uneven grayscale, and dense distribution of biological cells in microscopic images. In this paper, we propose a joint feature learning method that integrates the density and boundary branch into a multi-scale convolutional U-Net (MC-Unet). To enhance the supervision of cell density and boundary detection, the density and boundary loss is constructed to guide the joint learning of multiple features, where the density loss branch can address the challenges posed by high density, while the boundary loss branch can address the problems of unclear cell boundaries and partial cell occlusion. A series of experiments on different cell datasets show that two auxiliary branches improve the learning of features on cell density and cell boundaries and that the proposed method is effective on different segmentation models. The code is available at: https://github.com/HuHaigen/Joint-Feature-Learning-for-Cell-Segmentation.

Attention is one of the most valuable breakthroughs in the deep learning community, and how to effectively utilize the attention information of channel and spatial is still one of the hot research topics. In this work, we integrate the advantages of channel and spatial mechanism to propose a Channel-Spatial hybrid Attention Module (CSHAM). Specifically, max-average fusion Channel Attention Module and Spatial Attention Neighbor Enhancement Module are firstly proposed, respectively. Then the connection between the two modules is analyzed and designed, and an alternate connection strategy with the transformation of channel weights is proposed. The key idea is to repeatedly use the channel weight information generated by the channel attention module, and to reduce the negative impact of the network complexity caused by the addition of the attention mechanism. Finally, a series of comparison experiments are conducted on CIFAR100 and Caltech-101 based on various backbone models. The results show that the proposed methods can obtain the best Top-1 performance among the existing popular methods, and can rise by nearly 1% in accuracy while basically maintaining the parameters and FLOPs. The code is publicly available at https://github.com/HuHaigen/A-Channel-Spatial-Hybrid-Attention-Mechanism-using-Channel-Weight-Transfer-Strategy. The package includes the proposed CSHAM for reproducibility purposes.

Deep convolutional neural networks (CNNs) have witnessed a great progress in visual recognition over the past years. However, deep CNN models are still suffering from the domain adaptation problem. Most of the existing methods try to resolve this issue by creating more useful samples in the source domain for network training so that the well-trained CNN models can well adapt to more possible variations in the target domain. However, such methods are different with human visual mechanism. Human eyes can effectively recognize images with large variations that were never seen before, as long as human eyes are very familiar with partial contents of the input images. We simulate the visual mechanism of human eyes and make feature responses diverse as far as possible. We proposed a novel angular diversity loss, which contains a pair of angular Spatial Activation Diversity (A-SAD) losses by borrowing the idea of the angular losses. Besides concerning the recognition accuracy, we also focus on understanding deep CNNs. Recent works further pushed the interpretability into the training stage of the CNN models. This helps CNN models learn more meaningful feature representations. Extensive experiment on MNIST dataset and its six variation dataset show the effectiveness of the proposed A-SAD loss.

Liquid crystal display (LCD) defects detection on module level is increasingly important for flat-panel displays (FPD) industry to increase the production capacity via machine vision technology. However, it is an overwhelmingly challenging issue due to various difficulties. This article discloses a practical automatic optical inspection (AOI) system consisting of hardware structure and software algorithm to detect module-level defects. The AOI system is the core component to build a distributed integrated inspection system with the help of the Internet of Things (IoT). Starting from the analysis of the challenges encountered in module-level defects inspection, a delicate photograph scheme is proposed to reveal different kinds of defects. In order to robustly work on the module-level defects detection with complex situations, a novel framework based on YOLOV3 detection unit is proposed in this article, including the preprocessing module, detection module, defects definition module, and interferences elimination module. To the best of our knowledge, this is the first work that designs a practical AOI system for module-level defects detection. In order to demonstrate the effectiveness of the proposed method, extensive experiments have been conducted on the manufacturing lines. The evaluation of the detection performance of the AOI system in comparison with a manual scheme indicates that the proposed system is practical for module-level defects detection. Currently, the proposed system has been deployed in a real-world LCD manufacturing line from a major player in the world.

The identification of reliability-critical primary input leads (RCPIs) plays an important role in the testing and prediction of reliability boundaries of logic circuits. This article presents a gate-sensitive-attributes-based approach to estimate the criticality of the primary input leads in combinational circuits to their reliability. Oriented to the input vector, a subcircuit-based traversal method marks the critical input leads of each gate in a circuit. Gate-sensitive attributes and a reverse recursive algorithm quantify the effect of each RCPI on circuit reliability under the input vector. A parallel calculation method based on subcircuits with only one primary output reduces the computational complexity to accelerate the calculation process. Similarity-based clustering avoids unnecessary calculations, and a self-adaptive strategy is used to check convergence. Experimental results on benchmark circuits show that the average accuracy of this approach is 0.9634 with Monte Carlo (MC) as the reference and it is 3445 times faster than the MC on average while its average memory cost is 1.67 greater than the MC model. Although the fitness of the worst input vector obtained by other reference methods is 1.09 times better than that of this approach on average, this approach is approximately 21 times faster than that reference method on average.

Magnetic Resonance Images (MRIs) of different modalities have different reference values for pathological diagnosis. But it is difficult to obtain multimodality MRIs. So, medical image synthesis has been proposed as an effective solution, with which any missing modalities are synthesized from the existing ones. To train a multimodal MRI synthesizer with limited number of unpaired MRIs, in this paper, we have proposed a novel High-dimensional Knowledge Guided Generative Adversarial Network (HKG-GAN). In our HKG-GAN, a cross-dimensional knowledge transfer network is utilized to extract features from 2D images (slices of MRIs) to measure the perceptual similarity of images of source and synthesized modalities, whose knowledge is transferred from a pre-trained 3D network without accessing its private training dataset. Nevertheless, based on code-splitting and cross-decoding, HKG-GAN is a one-for-all network that encodes MRIs into content codes and style codes, and then cross-decodes the encoding of a random image of different modality to convert MRI to target modality. The effectiveness has been proofed through comparative experiments.

An activation function is a key component of artificial neural networks (ANNs). It has a great impact on the performance and convergence of neural networks. In this work, a self-adapting methodology is proposed to explore the optimal adaptive activation functions for various tasks based on S-shaped or ReLu-shaped activation functions, which are regulated only by introducing several parameters. To verify the effectiveness of the proposed methodology, a series of comparison experiments are performed with MLP, CNN and RNN network structure on the benchmark datasets of image, text and audio. The experimental results are encouraging, and show that the proposed methodology can locate the optimal activation functions for various tasks. Nevertheless, the obtained functions are competitive and the improvements on network performance are significant compared with other popular activation functions, such as ELU, PReLU, ReLU, and Sigmoid.

Due to the poor resolution and low signal-to-noise ratio in PET images, and especially to the wide variation in size, shape, site and SUV value among different patients or even for the same patient, lymphoma segmentation in 3D PET Images is still a challenging task in the field of medical image processing. In this work, a novel non-self background-based data enhancement method is proposed for the deep learning-based lymphoma segmentation problem. Firstly, a lymphoma pool with 1991 lymphoid lesions is created. Then, some lymphomas from the lymphoma pool are randomly selected and integrated into their non-self images of the patients according to their respective coordinates when training networks. Finally, a series of comparison experiments among various network models and methods are conducted to verify the effectiveness of the proposed method. The results indicated that the proposed method was promising, and could obtain better comprehensive performance than other methods without any data enhancements for the lymphoma segmentation problems.

Owing to the high density, low contrast, deformable cell shapes, low inter-cellular shape and appearance variation, and occlusion of the cells by division or fusion especially in phase-contrast microscopy images, it is still a challenging task to segment cells from the complex background. In this work, we proposed a multi-scale convolution Unet (MC-Unet) for bladder cancer cell segmentation in Phase-Contrast microscopy images. More specifically, the second 3x3 convolution of each layer in the standard Unet is replaced with a multi-scale convolution (MC) block with different kernel sizes, such as 1x1, 3x3, and 5x5. To verify the effectiveness of the proposed method, a series of experiments are conducted on the bladder cancer T24 dataset and the MoNuSeg dataset, and the results shows the proposed MC-Unet can obtain better comprehensive performance than the standard Unet.

Nowadays more and more urban residents are aware of the importance of the air quality to their health, especially who are living in the large cities that are seriously threatened by air pollution. Meanwhile, being limited by the spare sense nodes, the air quality information is very coarse in resolution, which brings urgent demands for high-resolution air quality data acquisition. In this paper, we refer the real-time and fine-gained air quality data in city-scale by employing the crowdsource automobiles as well as their built-in sensors, which significantly improves the sensing system's feasibility and practicability. The main idea of this paper is motivated by that the air component concentration within a vehicle is very similar to that of its nearby environment when the vehicle's windows are open, given the fact that the air will exchange between the inside and outside of the vehicle though the opening window. Therefore, this paper first develops an intelligent algorithm to detect vehicular air exchange state, then extracts the concentration of pollutant in the condition that the concentration trend is convergent after opening the windows, finally, the sensed convergent value is denoted as the equivalent air quality level of the surrounding environment. Based on our Internet of Things cloud platform, real-time air quality data streams from all over the city are collected and analyzed in our data center, and then a fine-gained city level air quality map can be exhibited elaborately. In order to demonstrate the effective- ness of the proposed method, experiments crowdsourcing 500 floating vehicles are conducted in Beijing city for three months to ubiquitously sample the air quality data. Evaluations of the algorithm's performance in comparison with the ground truth indicate the proposed system is practical for collecting air quality data in urban environments.

Seismic signal is widely used in ground target classification due to its inherent characteristics. However, its propagation is highly dependent on local underlying geology. It means that nearly every one geographical environment requires a unique classifier. To resolve the problem, this paper presents a robust feature extraction method Log-Sigmoid Frequency Cepstral Coefficients (LSFCC) which evolves from Mel frequency cepstral coefficients (MFCC) for ground target classification by means of geophone. With the LSFCCs, the average classification accuracy of tracked and wheeled vehicle is more than 89% in three different geographical environments by only one classifier which is trained in one of the three environments.