In supervised learning, traditional image masking faces two key issues: (i) discarded pixels are underutilized, leading to a loss of valuable contextual information; (ii) masking may remove small or critical features, especially in fine-grained tasks. In contrast, masked image modeling (MIM) has demonstrated that masked regions can be reconstructed from partial input, revealing that even incomplete data can exhibit strong contextual consistency with the original image. This highlights the potential of masked regions as sources of semantic diversity. Motivated by this, we revisit the image masking approach, proposing to treat masked content as auxiliary knowledge rather than ignored. Based on this, we proposed MaskAnyNet, which combines masking with a relearning mechanism to exploit both visible and masked information. It can be easily extended to any model with an additional branch to jointly learn from the recomposed masked region. This approach leverages the semantic diversity of masked regions to enrich features and preserve fine-grained details. Experiments on CNN and Transformer backbones show consistent gains across multiple benchmarks. Further analysis confirms that the proposed method improves semantic diversity through the reuse of masked content.

Accurate brain tumor segmentation plays an important role in clinical diagnosis, treatment planning, and therapeutic response monitoring. Multi-modal MRI provides complementary structural and functional information, but existing methods remain limited by their inadequate exploitation of cross-modal complementarity and their inability to effectively handle modality-specific disparities and redundant information. To address these challenges, this paper proposes a novel hierarchical teacher-student learning framework with adaptive cross-modal fusion. MRI modalities are grouped into teacher modalities (Flair and T1c) and student modalities (T2 and T1) based on their intrinsic tumor-related characteristics. Central to this framework is the Modality Guidance Module (MGM), which consists of two key components designed to achieve multi-modal feature distillation. Within MGM, the Modality Enhancement Module (MEM) extracts highly discriminative features from teacher modalities. While the Modality Fusion Module (MFM) leverages these features to guide and refine the learning of student modalities. To further capture inter-modal dependencies, a Cross-Modal Fusion Module (CMFM) is introduced to adaptively integrate complementary information across all modalities. Extensive experiments on the BraTS 2018, 2019 and 2020 datasets demonstrate that the proposed method achieves superior performance compared with state-of-the-art approaches. Beyond brain tumor segmentation, the hierarchical teacher-student paradigm and adaptive fusion strategy also hold potential for broader multi-modal image analysis tasks.

Accurate segmentation of brain tumors from multi-modal MRI is crucial for diagnosis and treatment planning. However, challenges such as severe class imbalance, modality-specific feature heterogeneity, and predictive uncertainty hinder reliable performance. In this work, we propose UTriGate-Net, a novel uncertainty-aware multi-modal brain tumor segmentation framework. First, we design a Triaxial Context Encoding (TCE) block that extracts anisotropic spatial features by applying directional convolutions along the axial, coronal, and sagittal planes, thereby enhancing 3D contextual representation. Second, we introduce a Gated Modality Fusion (GMF) module, which adaptively integrates complementary information across modalities through modality-specific gating weights that suppress redundancy while retaining salient features. Finally, to improve segmentation reliability, we develop an Uncertainty-Regularized Weighted Loss (URWL) that combines dynamic class-specific weighting to mitigate class imbalance with an entropy-based uncertainty penalty to encourage well-calibrated predictions. Experiments on the BraTS 2019 and 2020 datasets demonstrate that UTriGate-Net achieves superior segmentation accuracy and robustness, particularly in challenging subregions. Overall, the proposed framework offers a promising solution for reliable and precise brain tumor delineation in clinical practice.

Significant progress has been made in applying deep learning for the automatic diagnosis of skin lesions. However, most models remain unexplainable, which severely hinders their application in clinical settings. Concept-based ante-hoc interpretable models have the potential to clarify the decision-making process of diagnosis by learning high-level, human-understandable concepts, while they can only provide numerical values of conceptual contributions. Pre-trained Vision-Language Models (VLMs) can learn rich vision-language correlations from large-scale image-text pairs. Fine-tuning pre-trained VLMs for specific downstream tasks is an effective way to reduce data requirements. Nevertheless, when there is a substantial disparity between the pre-trained model and the target task, existing tuning methods frequently struggle to generalize, necessitating substantial training data to fully adapt VLMs to specialized medical tasks. In this work, we propose a concept adaptive fine-tuning (CptAFT) method based on the pre-trained VLM, BiomedCLIP, to develop a concept-based multi-modal interpretable skin lesion diagnosis model. By incorporating medical texts, such as reports and conceptual terms, our model can recognize fine-grained features and provide robust, natural language-driven interpretability. Moreover, our concept-adaptive method that reconstructs images using concept logits and imposes a consistency loss with the original image, enabling the VLM to quickly adapt to the task with a small amount of training data. Extensive experimental results demonstrate that our approach outperforms state-of-the-art closed box and interpretable models in both classification performance and medically relevant interpretability. In particular, after fine-tuning with a small amount of data, our model outperforms MONET, a model trained on the large Skin Disease Image-Report dataset, by 8.28% in concept recognition ability, demonstrating the interpretability of our model.

Electroencephalography (EEG) is vital for brain-computer interfaces (BCIs) due to its non-invasive approach and high temporal resolution data capabilities, amid challenges such as data scarcity and the need for extensive labeling. Significant inter-individual variability in EEG signals further limits model generalization. Concurrently, the use of self-supervised pre-training, particularly through masked modeling, is gaining traction in time series analysis to mitigate labeling costs. Although this method involves reconstructing masked signal from unmasked series, random masking can disrupt critical temporal variations, complicating effective representation learning. We thus introduce SSL-MEMI, a novel self-supervised contrastive learning framework for masked EEG motor imagery modeling, integrating Domain Adaptive Alignment (DAA) and Multi-View Temporal-spatial Attention module (MTSA) to effectively handle EEG variability. This framework utilizes manifold-based masking to reconstruct original sequences from masked series, thereby enhancing classification accuracy. When tested on the BCI Competition IV and High Gamma datasets, SSL-MEMI outperforms existing methods, achieving top accuracies and demonstrating superior domain adaptation through reduced Global ${\mathcal{A}}$-distance scores. This study advances EEG classification and indicates broader applications for self-supervised learning in biomedical signal processing. The source code is available at https://github.com/KunKun-Zhang/SSL-MEMI.git.

It is still challenging to accurately recognize some complex sign language gestures and existing techniques based on palm detection have serious shortcomings. This work proposes a novel scheme for ASL recognition by integrating the Mediapipe hand landmark detection into sign language recognition. Specifically, some key points are first detected to construct a model of 3D hand-knuckle coordinates. Then, sign language recognition is achieved by constructing a classification model. Finally, extensive experiments are conducted on the ASL dataset, and the results show that the proposed scheme can obtain a competitive performance. The proposed scheme offers a promising prospect of application, and it provides us with a viable solution to sign language recognition. The source code is available at https://github.com/Keyneswu/asl_ml .

Domain generalization (DG) aims to enhance the ability of model learning from source domains to generalize to other unseen domains. Existing gradient-based methods focus on learning better domain-invariant features using gradients from multiple source domains, but do not consider the impact of gradient granularity on model training. In this paper, we rethink how to mitigate the gradient conflicting problem from an optimization perspective. The limitations of existing gradient-based methods are theoretically analyzed in terms of modification ratio and modification frequency, showing that gradient granularity is a key factor in ensuring correct modification of the gradient. To address this issue, a gradient modification method, called CorGrad, is proposed by layering and slicing refinement operations to increase the modification frequency and the modification ratio. It can better reduce domain-specific information so that the model can learn better domain-invariant features. Finally, extensive experiments are conducted to verify the effectiveness of the proposed CorGrad, and the results show that the proposed CorGrad can obtain competitive performance in five DG benchmarks, and an average performance of 60.4% can be obtained on the DomainBed when using ResNet18 as the backbone. The code is publicly available at https://github.com/libzwo/CorGrad.

For the few-shot segmentation (FSS) task, existing methods attempt to capture the diversity of new classes by fully utilizing the limited support images, such as cross-attention and prototype matching. However, they often overlook the fact that there is variability in different regions of the same object, and intra-image similarity is higher than inter-image similarity. To address these limitations, a Self-Guided Transformer (SGT) is proposed by leveraging intra-image similarity to improve intra-object inconsistencies in this paper. The proposed SGT can selectively guide segmentation, emphasizing the regions that are easily distinguishable while adapting to the challenges caused by less discriminative regions within objects. Through a refined feature interaction scheme and the novel SGT module, our method can achieve state-of-the-art performance on various FSS datasets, demonstrating significant advances in few-shot semantic segmentation. The code is publicly available at https://github.com/HuHaigen/SGT.

Effective lesion detection in medical image is not only rely on the features of lesion region, but also deeply relative to the surrounding information. However, most current methods have not fully utilize it. What’s more, multi-scale feature fusion mechanism of most traditional detectors are unable to transmit detail information without loss, which makes it hard to detect small and boundary-ambiguous lesion in early stage disease. To address the above issues, we propose a novel intra- and across-layer feature interaction FCOS model (IAFI-FCOS) with a multi-scale feature fusion mechanism ICAF-FPN, which is a network structure with intra-layer context augmentation (ICA) block and across-layer feature weighting (AFW) block. Therefore, the traditional FCOS detector is optimized by enriching the feature representation from two perspectives. Specifically, the ICA block utilizes dilated attention to augment the context information in order to capture long-range dependencies between the lesion region and the surrounding. The AFW block utilizes dual-axis attention mechanism and weighting operation to obtain the efficient across-layer interaction features, enhancing the representation of detailed features. Our approach has been extensively experimented on both the private pancreatic lesion dataset and the public DeepLesion dataset, with AP50of 62.2% and 60.0%, respectively, and these results are 6.4% and 2.3% higher than the FCOS. Additionally, our model achieves SOTA results on the pancreatic lesion dataset.

Magnetic Resonance Imaging (MRI) with multiple modalities is commonly used for diagnosis, but it is associated with an inherently slow acquisition process. To accelerate multi-modal MRI, recent studies explored the merits of using a fully-sampled reference modality (RM) as a guidance to reconstruct the query modalities (QMs) from their undersampled k-space data via convolutional neural networks (CNNs). However, even aided by the RM, the reconstruction of highly undersampled QM data is still suffering from aliasing artifacts. To enhance reconstruction quality, we suggest to further release the guiding power of the RM data via generating its multiscale variants. To this end, we simultaneously partition the k-space of the RM and QM into several subregions with gradually increasing sizes. We then proposed a k-Space Partition-based Convolutional Network (kSPCN) to fully use the partitioned RM and QM data to perform QM reconstruction subregion by subregion. Extensive experiments on different query modalities and acceleration rates demonstrate that kSPCN consistently outperforms state-of-the-art methods and can preserve anatomical structure faithfully up to 12-fold undersampling.

Deep Cascading Networks (DCNs) are very popular for fast MRI reconstruction. However, DCNs still have limited generalization ability on highly undersampled MRI data. One main reason is that the training data is not well used. A promising solution is to boost the filter diversity of DCNs to well fit the rich features in the training data. This can be achieved by reducing the repetition level of the undersampled input images via using the pixel unshuffle (PU) operator. As different input images and different subnets of DCNs require different PU-scales, we propose a novel PU-Scale Estimation (PUSE) method to automatically infer optimal PU-scales. By incorporating PUSE into DCNs, we construct a new Multi-PU-Scale Diversity based (MSDiv+) architecture for DCNs. To boost training convergence, we further propose to generate mini-batches by mixing data samples with different optimal PU-scales. Experiments on the fastMRI dataset demonstrate the effectiveness of our method.

Recent advancements in medical image segmentation have demonstrated superior performance by combining Transformer and U-Net due to the Transformer’s exceptional ability to capture long-range semantic dependencies. However, existing approaches mostly replace or concatenate the Convolutional Neural Networks (CNNs) and Transformers in series, which limits the potential of their combination. In this paper, we introduce a dual-branch feature encoder, CTI-UNet, that effectively fuses the global representations and local features of the CNN and Transformer branches at different scales through bidirectional feature interaction. Our proposed method outperforms existing approaches on multiple medical datasets, demonstrating state-of-the-art performance. The code for CTI-UNet is publicly available at https://github.com/huhaigen/CTI-UNet.

To enhance the nonlinearity of neural networks and increase their mapping abilities between the inputs and response variables, activation functions play a crucial role to model more complex relationships and patterns in the data. In this work, a novel methodology is proposed to adaptively customize activation functions only by adding very few parameters to the traditional activation functions such as Sigmoid, Tanh, and rectified linear unit (ReLU). To verify the effectiveness of the proposed methodology, some theoretical and experimental analysis on accelerating the convergence and improving the performance is presented, and a series of experiments are conducted based on various network models (such as AlexNet, VggNet, GoogLeNet, ResNet and DenseNet), and various datasets (such as CIFAR10, CIFAR100, miniImageNet, PASCAL VOC, and COCO). To further verify the validity and suitability in various optimization strategies and usage scenarios, some comparison experiments are also implemented among different optimization strategies (such as SGD, Momentum, AdaGrad, AdaDelta, and ADAM) and different recognition tasks such as classification and detection. The results show that the proposed methodology is very simple but with significant performance in convergence speed, precision, and generalization, and it can surpass other popular methods such as ReLU and adaptive functions such as Swish in almost all experiments in terms of overall performance.

Network acceleration is very important for Magnetic Resonance Imaging (MRI) reconstruction, as the device-specific retraining of a given network is usually necessary to overcome the scanner transfer problem. We propose a novel framework, namely PU-Network-PS (UNS), to accelerate the Deeply Cascaded Convolutional Networks (DCCNs) for MRI reconstruction, where Pixel Unshuffle (PU) and Pixel Shuffle (PS) are imposed at the two ends of DCCNs, respectively. Our UNS accelerates DCCNs by first decomposing the zero-filled MRI images of DCCNs into several sub-images via using PU, which are dubbed PU-channels and will make DCCNs mainly run in a low-resolution space and thus obtain high computation efficiency. However, directly using PU-channels might lead to unstable performance due to the lost spatial correlations between PU-channels. We discovered that the PU-channels can be organized into different groups according to their mutual similarities and presented a novel Multi-Scale Cross Group Convolution to fully fuse their complementary information. We further integrated into UNS the ensemble learning scheme in the PS stage to boost performance. Experiments on the IXI dataset demonstrated that our UNS can enhance both the reconstruction performance and the running efficiency for both simple and complex DCNN models. Specially, the training time for DuDoRNet can be greatly reduced from three days to about 20 hours with small performance enhancements by using two NVIDIA RTX 2080Ti GPUs.

A major challenge in the analysis of tissue imaging data is cell segmentation, the task of identifying precisely the boundary of each cell in a microscopic image. The cell segmentation task is still challenging due to the variable shapes, large size differences, uneven grayscale, and dense distribution of biological cells in microscopic images. In this paper, we propose a joint feature learning method that integrates the density and boundary branch into a multi-scale convolutional U-Net (MC-Unet). To enhance the supervision of cell density and boundary detection, the density and boundary loss is constructed to guide the joint learning of multiple features, where the density loss branch can address the challenges posed by high density, while the boundary loss branch can address the problems of unclear cell boundaries and partial cell occlusion. A series of experiments on different cell datasets show that two auxiliary branches improve the learning of features on cell density and cell boundaries and that the proposed method is effective on different segmentation models. The code is available at: https://github.com/HuHaigen/Joint-Feature-Learning-for-Cell-Segmentation.

Attention is one of the most valuable breakthroughs in the deep learning community, and how to effectively utilize the attention information of channel and spatial is still one of the hot research topics. In this work, we integrate the advantages of channel and spatial mechanism to propose a Channel-Spatial hybrid Attention Module (CSHAM). Specifically, max-average fusion Channel Attention Module and Spatial Attention Neighbor Enhancement Module are firstly proposed, respectively. Then the connection between the two modules is analyzed and designed, and an alternate connection strategy with the transformation of channel weights is proposed. The key idea is to repeatedly use the channel weight information generated by the channel attention module, and to reduce the negative impact of the network complexity caused by the addition of the attention mechanism. Finally, a series of comparison experiments are conducted on CIFAR100 and Caltech-101 based on various backbone models. The results show that the proposed methods can obtain the best Top-1 performance among the existing popular methods, and can rise by nearly 1% in accuracy while basically maintaining the parameters and FLOPs. The code is publicly available at https://github.com/HuHaigen/A-Channel-Spatial-Hybrid-Attention-Mechanism-using-Channel-Weight-Transfer-Strategy. The package includes the proposed CSHAM for reproducibility purposes.

Magnetic Resonance Images (MRIs) of different modalities have different reference values for pathological diagnosis. But it is difficult to obtain multimodality MRIs. So, medical image synthesis has been proposed as an effective solution, with which any missing modalities are synthesized from the existing ones. To train a multimodal MRI synthesizer with limited number of unpaired MRIs, in this paper, we have proposed a novel High-dimensional Knowledge Guided Generative Adversarial Network (HKG-GAN). In our HKG-GAN, a cross-dimensional knowledge transfer network is utilized to extract features from 2D images (slices of MRIs) to measure the perceptual similarity of images of source and synthesized modalities, whose knowledge is transferred from a pre-trained 3D network without accessing its private training dataset. Nevertheless, based on code-splitting and cross-decoding, HKG-GAN is a one-for-all network that encodes MRIs into content codes and style codes, and then cross-decodes the encoding of a random image of different modality to convert MRI to target modality. The effectiveness has been proofed through comparative experiments.

An activation function is a key component of artificial neural networks (ANNs). It has a great impact on the performance and convergence of neural networks. In this work, a self-adapting methodology is proposed to explore the optimal adaptive activation functions for various tasks based on S-shaped or ReLu-shaped activation functions, which are regulated only by introducing several parameters. To verify the effectiveness of the proposed methodology, a series of comparison experiments are performed with MLP, CNN and RNN network structure on the benchmark datasets of image, text and audio. The experimental results are encouraging, and show that the proposed methodology can locate the optimal activation functions for various tasks. Nevertheless, the obtained functions are competitive and the improvements on network performance are significant compared with other popular activation functions, such as ELU, PReLU, ReLU, and Sigmoid.

Cancer cell detection and its stages recognition of life cycle are an important step to analyze cellular dynamics in the automation of cell based-experiments. In this work, a two-stage hierarchical method is proposed to detect and recognize different life stages of bladder cells by using two cascade Convolutional Neural Networks (CNNs). Initially, a hybrid object proposal algorithm (called EdgeSelective) by combining EdgeBoxes and Selective Search is proposed to generate candidate object proposals instead of a single Selective Search method in Region-CNN (R-CNN), and it can exploit the advantages of different mechanisms for generating proposals so that each cell in the image can be fully contained by at least one proposed region during the detection process. Then, the obtained cells from the previous step are used to train and extract features by employing CNNs for the purpose of cell life stage recognition. Finally, a series of comparison experiments are implemented. The results show that the proposed method can obtain better performance than traditional methods either in the stage of cell detection or cell life stage recognition, and it encourages and suggests the application in the development of new anticancer drug and cytopathology analysis of cancer patients in the near future.

Lymphoma detection and segmentation from PET images are critical tasks for cancer staging and treatment monitoring. However, it is still a challenge owing to the complexities of lymphoma PET data themselves, and the huge memory requirements for 3D volume data. In this work, a prior knowledge integrated scheme based on DBSCAN and GAs is proposed to detect and segment lymphomas in 3D PET images. To reduce memory requirements and add more feature information, billions of voxels in 3D volume data are first aggregated into supervoxels. Then, such supervoxels serve as basic data units for further clustering the supervoxels by using DBSCAN algorithm, in which a new similarity measure based on prior knowledge is proposed. Meanwhile, a genetic algorithm is used to search the most appropriate parameters of DBSCAN to obtain the optimal clustering results. Finally, a series of comparison experiments among various feature attributes and various similarity functions are performed, and the results are encouraging, and show that the proposed scheme by intergrating the prior knowledge of organ distributions can achieve better performance than traditional methods.

Due to the poor resolution and low signal-to-noise ratio in PET images, and especially to the wide variation in size, shape, site and SUV value among different patients or even for the same patient, lymphoma segmentation in 3D PET Images is still a challenging task in the field of medical image processing. In this work, a novel non-self background-based data enhancement method is proposed for the deep learning-based lymphoma segmentation problem. Firstly, a lymphoma pool with 1991 lymphoid lesions is created. Then, some lymphomas from the lymphoma pool are randomly selected and integrated into their non-self images of the patients according to their respective coordinates when training networks. Finally, a series of comparison experiments among various network models and methods are conducted to verify the effectiveness of the proposed method. The results indicated that the proposed method was promising, and could obtain better comprehensive performance than other methods without any data enhancements for the lymphoma segmentation problems.

Owing to the high density, low contrast, deformable cell shapes, low inter-cellular shape and appearance variation, and occlusion of the cells by division or fusion especially in phase-contrast microscopy images, it is still a challenging task to segment cells from the complex background. In this work, we proposed a multi-scale convolution Unet (MC-Unet) for bladder cancer cell segmentation in Phase-Contrast microscopy images. More specifically, the second 3x3 convolution of each layer in the standard Unet is replaced with a multi-scale convolution (MC) block with different kernel sizes, such as 1x1, 3x3, and 5x5. To verify the effectiveness of the proposed method, a series of experiments are conducted on the bladder cancer T24 dataset and the MoNuSeg dataset, and the results shows the proposed MC-Unet can obtain better comprehensive performance than the standard Unet.

This paper investigates the issue of PID-controller parameters tuning for a greenhouse climate control system using Evolutionary Algorithms based on multiple performance measures such as good set-point tracking and smooth control signals. A model of nonlinear thermodynamic laws between numerous system variables affecting the greenhouse climate is formulated. The proposed tuning scheme is validated for greenhouse climate control by minimizing the integrated time square error (ITSE) and the control increment or rate in a series of simulations. The results show that the controllers by tuning the gain parameters can achieve good control performance through step responses such as small overshoot, fast settling time, and less rise time and steady state error. Maybe it is quite an effective and promising tuning method using multi-objective algorithms in the complex greenhouse production.

This paper presents a hybrid control strategy, combining RBF network with the conventional PID controller, for the greenhouse climate control. A model of nonlinear thermodynamic laws between numerous system variables affecting the greenhouse climate is formulated. The presented Neuro-PID control scheme is validated through simulations of set-point tracking and disturbance rejection. The results show that the proposed strategy has good adaptability, strong robustness while achieving satisfactory control performance for the complex and nonlinear time-varying greenhouse climate control system, and it may provide a valuable reference to formulate environmental control strategies for actual application in greenhouse production.

Trajectory tracking of the mobile robot is one research hot for the robot. For the control system of the two-wheeled differential drive mobile robot being in nonhonolomic system and the complex relations among the control parameters, it is difficult to solve the problem based on traditional mathematics model. A new control scheme combined with the fuzzy PD (Proportional and Differential) control and the separate integral control is proposed in this paper. The control scheme can not only make full use of the advantage of the fuzzy control, but also have the good steady state tracking ability of the integral control. However, this control scheme introduces so many parameters which are difficult to optimize. In order to realize the online adaptive learning of the control parameters, the modified VFSA (Very Fast Simulated Annealing) is used. The simulation results show that the method is feasible, and can quickly approach the conference trajectory in a short time, and the trajectory tracking error is very small.