Current knowledge distillation (KD) methods for semantic segmentation focus on guiding the student to imitate the teacher's knowledge within homogeneous architectures. However, these methods overlook the diverse knowledge contained in architectures with different inductive biases, which is crucial for enabling the student to acquire a more precise and comprehensive understanding of the data during distillation. To this end, we propose for the first time a generic knowledge distillation method for semantic segmentation from a heterogeneous perspective, named HeteroAKD. Due to the substantial disparities between heterogeneous architectures, such as CNN and Transformer, directly transferring cross-architecture knowledge presents significant challenges. To eliminate the influence of architecture-specific information, the intermediate features of both the teacher and student are skillfully projected into an aligned logits space. Furthermore, to utilize diverse knowledge from heterogeneous architectures and deliver customized knowledge required by the student, a teacher-student knowledge mixing mechanism (KMM) and a teacher-student knowledge evaluation mechanism (KEM) are introduced. These mechanisms are performed by assessing the reliability and its discrepancy between heterogeneous teacher-student knowledge. Extensive experiments conducted on three main-stream benchmarks using various teacher-student pairs demonstrate that our HeteroAKD framework outperforms state-of-the-art KD methods in facilitating distillation between heterogeneous architectures.

In the field of remote sensing, detecting tiny objects remains a significant challenge, essentially due to the insufficient and weak feature representations caused by the limited pixel coverage of these objects, which presents a pressing demand for networks possessing multiscale feature enhancement ability. Furthermore, the unique prior knowledge presented in remote sensing images is not utilized effectively or appropriately, making this information just an obstacle to detecting tiny objects. To address these issues, we propose an effective network for tiny object detection called Multiscale Feature Enhancement and Adaptive Receptive Field YOLO (MA-YOLO). Specifically, the MA-YOLO integrates two innovative plug-and-play modules: the Multiscale Feature Enhancement Module (MFEM) and the Receptive Field Adaptive Multiscale Feature Fusion Module (RFAMFFM). The MFEM is designed to extract multiscale and multi-geometric features through a multi-branch refinement design with multi-size and -shape convolution, thereby enhancing feature representation. Driven by the larger receptive fields of large-kernel convolution and the guiding property of global pooling, the RFAMFFM is employed to dynamically adjust and select the receptive field to leverage prior knowledge effectively. Our proposed network outperforms the existing comparison models on two publicly available datasets, i.e., USOD and VEDAI. Specifically, compared to the state-of-the-art model, MA-YOLO achieves significant improvements of 2.2% on the USOD dataset and of 1.8% on the VEDAI dataset for the mAP50:95 metric.

Non-exemplar class incremental learning (NECIL) aims to continuously assimilate new knowledge while retaining previously acquired knowledge in scenarios where prior examples are unavailable. A prevalent strategy within NECIL mitigates knowledge forgetting by freezing the feature extractor after training on the initial task. However, this freezing mechanism does not provide explicit training to differentiate between new and old classes, resulting in overlapping feature representations. To address this challenge, we propose a **C**onfusion-driven se**L**f-supervised pr**O**gressi**V**ely weighted **E**nsemble lea**R**ning (*CLOVER*) framework for NECIL. Firstly, we introduce a confusion-driven self-supervised learning approach that enhances representation extraction by guiding the model to distinguish between highly confusable classes, thereby reducing class representation overlap. Secondly, we develop a progressively weighted ensemble learning method that gradually adjusts weights to integrate diverse knowledge more effectively, further minimizing representation overlap. Finally, extensive experiments demonstrate that our proposed method achieves state-of-the-art results on the CIFAR100, TinyImageNet, and ImageNet-Subset NECIL benchmarks.

In this work, we present GPDiT, a Generative Pre-trained Autoregressive Diffusion Transformer that unifies the strengths of diffusion and autoregressive modeling for long-range video synthesis, within a continuous latent space. Instead of predicting discrete tokens, GPDiT autoregressively predicts future latent frames using a diffusion loss, enabling natural modeling of motion dynamics and semantic consistency across frames. This continuous autoregressive framework not only enhances generation quality but also endows the model with representation capabilities. Additionally, we introduce a lightweight causal attention variant and a parameter-free rotation-based time-conditioning mechanism, improving both the training and inference efficiency. Extensive experiments demonstrate that GPDiT achieves strong performance in video generation quality, video representation ability, and few-shot learning tasks, highlighting its potential as an effective framework for video modeling in continuous space.

The goal of few-shot class incremental learning (FSCIL) is to learn new concepts from a limited number of novel samples while preserving the knowledge of previously learned classes. The mainstream FSCIL framework begins with training in the base session, after which the feature extractor is frozen to accommodate novel classes. We observed that traditional base-session training approaches often lead to overfitting on challenging samples, which can lead to reduced robustness in the decision boundaries and exacerbate the forgetting phenomenon when introducing incremental data. To address this issue, we proposed the progressive learning strategy (PGLS). First, inspired by curriculum learning, we developed a covariance noise perturbation approach based on the statistical information as a difficulty measure for assessing sample robustness. We then reweighted the samples based on their robustness, initially concentrating on enhancing model stability by prioritizing robust samples and subsequently leveraging weakly robust samples to improve generalization. Second, we predefined forward compatibility for various virtual class augmentation models. Within base class training, we employed a curriculum learning strategy that progressively introduced fewer to more virtual classes in order to mitigate any adverse effects on model performance. This strategy enhances the adaptability of base classes to novel ones and alleviates forgetting problems. Finally, extensive experiments conducted on the CUB200, CIFAR100, and miniImageNet datasets demonstrate the significant advantages of our proposed method over state-of-the-art models.

Self-training has been shown to achieve remarkable gains in semi-supervised semantic segmentation by creating pseudo-labels using unlabeled data. This approach, however, suffers from the quality of the generated pseudo-labels, and generating higher quality pseudo-labels is the main challenge that needs to be addressed. In this paper, we propose a novel method for semi-supervised semantic segmentation based on Multi-perspective pseudo-label Generation and Confidence-weighted Training (MGCT). First, we present a multi-perspective pseudo-label generation strategy that considers both global and local semantic perspectives. This strategy prioritizes pixels in all images by the global and local predictions, and subsequently generates pseudo-labels for different pixels in stages according to the ranking results. Our pseudo-label generation method shows superior suitability for semi-supervised semantic segmentation compared to other approaches. Second, we propose a confidence-weighted training method to alleviate performance degradation caused by unstable pixels. Our training method assigns confident weights to unstable pixels, which reduces the interference of unstable pixels during training and facilitates the efficient training of the model. Finally, we validate our approach on the PASCAL VOC 2012 and Cityscapes datasets, and the results indicate that we achieve new state-of-the-art performance on both datasets in all settings.

In multi-sequence Magnetic Resonance Imaging (MRI), the accurate segmentation of the kidney and tumor based on traditional supervised methods typically necessitates detailed annotation for each sequence, which is both time-consuming and labor-intensive. Unsupervised Domain Adaptation (UDA) methods can effectively mitigate inter-domain differences by aligning cross-modal features, thereby reducing the annotation burden. However, most existing UDA methods are limited to one-to-one domain adaptation, which tends to be inefficient and resource-intensive when faced with multi-target domain transfer tasks. To address this challenge, we propose a novel and efficient One-to-Multiple Progressive Style Transfer Unsupervised Domain-Adaptive (PSTUDA) framework for kidney and tumor segmentation in multi-sequence MRI. Specifically, we develop a multi-level style dictionary to explicitly store the style information of each target domain at various stages, which alleviates the burden of a single generator in a multi-target transfer task and enables effective decoupling of content and style. Concurrently, we employ multiple cascading style fusion modules that utilize point-wise instance normalization to progressively recombine content and style features, which enhances cross-modal alignment and structural consistency. Experiments conducted on the private MSKT and public KiTS19 datasets demonstrate the superiority of the proposed PSTUDA over comparative methods in multi-sequence kidney and tumor segmentation. The average Dice Similarity Coefficients are increased by at least 1.8% and 3.9%, respectively. Impressively, our PSTUDA not only significantly reduces the floating-point computation by approximately 72% but also reduces the number of model parameters by about 50%, bringing higher efficiency and feasibility to practical clinical applications.

Endoscopic video analysis can effectively assist clinicians in disease diagnosis and treatment, and has played an indispensable role in clinical medicine. Unlike regular videos, endoscopic video analysis presents unique challenges, including complex camera movements, uneven distribution of lesions, and concealment, and it typically relies on contrastive learning in self-supervised pretraining as its mainstream technique. However, representations obtained from contrastive learning enhance the discriminability of the model but often lack fine-grained information, which is suboptimal in the pixel-level prediction tasks. In this paper, we develop a Multi-view Masked Contrastive Representation Learning (M$^2$CRL) framework for endoscopic video pre-training. Specifically, we propose a multi-view mask strategy for addressing the challenges of endoscopic videos. We utilize the frame-aggregated attention guided tube mask to capture global-level spatiotemporal sensitive representation from the global views, while the random tube mask is employed to focus on local variations from the local views. Subsequently, we combine multi-view mask modeling with contrastive learning to obtain endoscopic video representations that possess fine-grained perception and holistic discriminative capabilities simultaneously. The proposed M$^2$CRL is pre-trained on 7 publicly available endoscopic video datasets and fine-tuned on 3 endoscopic video datasets for 3 downstream tasks. Notably, our M$^2$CRL significantly outperforms the current state-of-the-art self-supervised endoscopic pre-training methods, e.g., Endo-FM (3.5% F1 for classification, 7.5% Dice for segmentation, and 2.2% F1 for detection) and other self-supervised methods, e.g., VideoMAE V2 (4.6% F1 for classification, 0.4% Dice for segmentation, and 2.1% F1 for detection).

Nasopharyngeal carcinoma (NPC) is a malignant tumor with a high incidence. Accurate segmentation of the tumor region in Computed Tomography (CT) images of NPC is the key to treatment. However, the features of uneven grayscale values and hazy boundaries of NPC regions make accurate NPC segmentation particularly challenging. To address these problems, we propose an accurate and effective NPC segmentation method using Dual Attention and Edge Detection Convolutional Neural Network (DAED-Net). Firstly, we combine a 2.5D convolutional neural network with UNet++ and propose a new backbone called Dual-dimension Dense UNet (DD-UNet), which can extract more beneficial features from 3D images. Secondly, a Dual Attention Module (DAM) is proposed to help the model better segment the target region of NPC by efficiently collecting spatial and channel attention information from feature maps. Moreover, an Edge Detection Module (EDM) is introduced in the network to enhance the segmentation of the target contours. Finally, we evaluate the proposed DAED-Net on the public MICCAI 2019 StructSeg NPC dataset from different perspectives. Numerical and visual results show that the proposed method outperforms nine state-of-the-art segmentation methods and yields more accurate NPC segmentation results.

Glass is ubiquitous in the real world, and its perception has many applications, including robot navigation and drone tracking. However, due to the transparent property of glass, the interior of a glass area can be any surrounding scene or object, which brings challenges for computer vision. Inspired by the human senses, boundary cues are one of the crucial factors for people to judge the location of glass contours. Hence, we propose a boundary cue guidance and contextual feature mining network (BCNet) to accurately and efficiently segment glass. Specifically, we first design a multi-branch boundary extraction module (MBEM) for learning accurate boundary cues combined with multi-level encoded features. Second, we propose a boundary cue guidance module (BCGM), inject the boundary cues into the representation learning, and provide constraints with object structure semantics to guide feature extraction. Besides, we design a contextual feature mining module (CFMM) to dynamically capture the contextual information of different receptive fields for the detection of different sizes and shapes of the glass. Finally, extensive experiments on two benchmark glass datasets, GDD and GSD. The results demonstrate that our BCNet achieves state-of-the-art segmentation performance against existing methods.

Learning how to model global relationships and extract local details is crucial in improving the performance of multi-organ segmentation. Most existing U-shaped structure methods use feature fusion to address these two challenges, but still lack the ability to balance capturing global relationships and local details. To address these issues, we propose a novel multi-organ segmentation framework called TransWnet to mine global relationships and local details from both intra- and inter-scale perspectives. To achieve this, we innovatively design a Row and Column Swin Transformer (RCST) module that can efficiently capture global contextual features and construct local information. Specifically, we design a parallel structure of Row and Column Attention to model the global relationships of multi-scale encoded features, and further mine local information from the global relationships through a local window mechanism. Extensive experiments on the Synapse dataset show that our method outperforms state-of-the-art approaches and achieves accurate segmentation of abdominal multi-organs.

Breast ultrasound (BUS) image segmentation is a critical procedure in the diagnosis and quantitative analysis of breast cancer. Most existing methods for BUS image segmentation do not effectively utilize the prior information extracted from the images. In addition, breast tumors have very blurred boundaries, various sizes and irregular shapes, and the images have a lot of noise. Thus, tumor segmentation remains a challenge. In this article, we propose a BUS image segmentation method using a boundary-guided and region-aware network with global scale-adaptive (BGRA-GSA). Specifically, we first design a global scale-adaptive module (GSAM) to extract features of tumors of different sizes from multiple perspectives. GSAM encodes the features at the top of the network in both channel and spatial dimensions, which can effectively extract multi-scale context and provide global prior information. Moreover, we develop a boundary-guided module (BGM) for fully mining boundary information. BGM guides the decoder to learn the boundary context by explicitly enhancing the extracted boundary features. Simultaneously, we design a region-aware module (RAM) for realizing the cross-fusion of diverse layers of breast tumor diversity features, which can facilitate the network to improve the learning ability of contextual features of tumor regions. These modules enable our BGRA-GSA to capture and integrate rich global multi-scale context, multi-level fine-grained details, and semantic information to facilitate accurate breast tumor segmentation. Finally, the experimental results on three publicly available datasets show that our model achieves highly effective segmentation of breast tumors even with blurred boundaries, various sizes and shapes, and low contrast.

Learning how to fully extract global representations and local features is a key factor in improving the performance of polyp segmentation. In this paper, we explore the potential of combined techniques of Transformers and convolutional neural networks (CNNs) to address the challenges of polyp segmentation. Specifically, we present TransMixer, a hybrid interaction fusion architecture of the Transformer branch and the CNN branch, which is able to enhance the local details of global representations and the global context awareness of local features. To achieve this, we first bridge the semantic gap between the Transformer branch and the CNN branch through the Interaction Fusion Module (IFM), and then make full use of both respective properties to enhance polyp feature representations. After that, we further propose the Hierarchical Attention Module (HAM) to collect polyp semantic information from high-level features to gradually guide the recovery of polyp spatial information in low-level features. Quantitative and qualitative results show that the proposed model is more robust to various complex situations compared to existing methods, and achieves state-of-the-art performance in polyp segmentation.

Camouflaged object segmentation (COS) is a challenging task due to the existence of high intrinsic similarities between the object and background. To overcome this challenge, we pro-pose a new framework, called TransCoop, for COS through the cooperation of Transformers and convolutional neural net-works (CNNs). Specifically, Transformer is used to model global context and structural information for accurately positioning potential target objects. Meanwhile, a well-designed texture feature fusion module (TFFM) is used to fuse the features encoded by CNN with the shallow Transformer to fully mine the low-level features in the scene. Furthermore, we propose a noise removal module (NRM), which can eliminate the background noise of low-level features with the guidance of precise target location. Notably, our TFFM and NRM can effectively realize the interaction between Transformer and CNN features. Extensive experiments on four benchmark datasets demonstrate the superiority of our TransCoop against existing state-of-the-art methods.

Efficient and accurate abdominal multi-organ segmentation is the key to clinical applications such as computer-aided diagnosis and computer-aided surgery, but this task is extremely challenging due to blurred organ boundaries, complex backgrounds, and different organ sizes. Although existing segmentation methods have achieved good segmentation results, we found that the segmentation performance of abdominal small and medium organs is often unsatisfactory, but the accurate location and segmentation of abdominal small and medium organs plays an important role in the diagnosis and screening of clinical diseases. To address this problem, in this paper we propose an intra- and inter-scale collaborative learning network (I2-Net) for the abdominal multi-organ segmentation task. Firstly, we design a Feature Complementary Module (FCM) to adaptively complement the local and global features extracted by CNN and Transformer. Secondly, we propose a Feature Aggregation Module (FAM) to aggregate multi-scale semantic information. Finally, we employ a Focus Module (FM) for collaborative learning of intra- and inter-scale features. Extensive experiments on the Synapse dataset show that our method outperforms the state-of-the-art approaches and achieve accurate segmentation of abdominal multi-organs, especially for small and medium organs.

With the advent of the era of big data, it is troublesome to accurately predict the associations between lncRNAs and diseases based on traditional biological experiments due to its time-consuming and subjective. In this paper, we propose a novel deep learning method for predicting lncRNA-disease associations using multi-feature coding and attention convolutional neural network (MCA-Net). We first calculate six similarity features to extract different types of lncRNA and disease feature information. Second, a multi-feature coding method is proposed to construct the feature vectors of lncRNA-disease association samples by integrating the six similarity features. Furthermore, an attention convolutional neural network is developed to identify lncRNA-disease associations under 10-fold cross-validation. Finally, we evaluate the performance of MCA-Net from different perspectives including the effects of the model parameters, distinct deep learning models, and the necessity of attention mechanism. We also compare MCA-Net with several state-of-the-art methods on three publicly available datasets, i.e., LncRNADisease, Lnc2Cancer, and LncRNADisease2.0. The results show that our MCA-Net outperforms the state-of-the-art methods on all three dataset. Besides, case studies on breast cancer and lung cancer further verify that MCA-Net is effective and accurate for the lncRNA-disease association prediction.

BACKGROUND: A large number of experimental studies show that the mutation and regulation of long non-coding RNAs (lncRNAs) are associated with various human diseases. Accurate prediction of lncRNA-disease associations can provide a new perspective for the diagnosis and treatment of diseases. The main function of many lncRNAs is still unclear and using traditional experiments to detect lncRNA-disease associations is time-consuming. RESULTS: In this paper, we develop a novel and effective method for the prediction of lncRNA-disease associations using network feature similarity and gradient boosting (LDNFSGB). In LDNFSGB, we first construct a comprehensive feature vector to effectively extract the global and local information of lncRNAs and diseases through considering the disease semantic similarity (DISSS), the lncRNA function similarity (LNCFS), the lncRNA Gaussian interaction profile kernel similarity (LNCGS), the disease Gaussian interaction profile kernel similarity (DISGS), and the lncRNA-disease interaction (LNCDIS). Particularly, two methods are used to calculate the DISSS (LNCFS) for considering the local and global information of disease semantics (lncRNA functions) respectively. An autoencoder is then used to reduce the dimensionality of the feature vector to obtain the optimal feature parameter from the original feature set. Furthermore, we employ the gradient boosting algorithm to obtain the lncRNA-disease association prediction. CONCLUSIONS: In this study, hold-out, leave-one-out cross-validation, and ten-fold cross-validation methods are implemented on three publicly available datasets to evaluate the performance of LDNFSGB. Extensive experiments show that LDNFSGB dramatically outperforms other state-of-the-art methods. The case studies on six diseases, including cancers and non-cancers, further demonstrate the effectiveness of our method in real-world applications.