Evolutionary algorithms (EAs) have shown their competitiveness in solving feature selection (FS) problem. However, when facing high-dimensional data with a number of instances, there are two challenges for the existing EAs. (1) The increasing number of features causes the search space of EAs to grow exponentially, which is known as the “curse of dimensionality". (2) The increasing number of instances not only increases the evaluation cost of EAs, but also may degrade the quality of obtained feature subsets. To tackle the two challenges simultaneously, this paper proposes an instance selection (IS) assisted evolutionary FS algorithm, named ISA-EFS. In ISA-EFS, a complementary feature grouping strategy is first suggested, with which the search is performed on the feature group level instead of the single feature level, and the “curse of dimensionality" can be solved effectively. Based on the grouping strategy, two new evolutionary (grouping-oriented crossover and mutation) operators are designed, which achieve the feature subsets with good quality. Then, a novel instance selection algorithm is developed to select a small number of “representative" instances and used for high-dimensional feature selection (HDFS). In ISA-EFS, the suggested IS and FS algorithms are carried on alternately. Meanwhile, since IS is designed to assist FS, the computational resources are gradually removed from IS to FS, with which the quality of feature subsets obtained by ISA-EFS is continuously improved. Experimental results on 12 high-dimensional datasets with a number of instances demonstrate the effectiveness and efficiency of the proposed ISA-EFS, when compared with six state-of-the-art FS algorithms.

Multimodal Federated Learning (MFL) has emerged as a collaborative paradigm for training models across decentralized devices, harnessing various data modalities to facilitate effective learning while respecting data ownership. In this realm, notably, a pivotal shift from homogeneous to heterogeneous MFL has taken place. While the former assumes uniformity in input modalities across clients, the latter accommodates modality-incongruous setups, which is often the case in practical situations. For example, while some advanced medical institutions have the luxury of utilizing both MRI and CT for disease diagnosis, remote hospitals often find themselves constrained to employ CT exclusively due to its cost-effectiveness. Although heterogeneous MFL can apply to a broader scenario, it introduces a new challenge: modality-heterogeneous client drift, arising from diverse modality-coupled local optimization. To address this, we introduce FedMM, a simple yet effective approach. During local optimization, FedMM employs modality dropout, randomly masking available modalities, and promoting weight alignment while preserving model expressivity on its original modality combination. To enhance the modality dropout process, FedMM incorporates a task-specific inter- and intra-modal regularizer, which acts as an additional constraint, forcing that weight distribution remains more consistent across diverse input modalities and therefore eases the optimization process with modality dropout enabled. By combining them, our approach holistically addresses client drift. It fosters convergence among client models while considering each client's unique input modalities, enhancing heterogeneous MFL performance. Comprehensive evaluations in three medical image segmentation datasets demonstrate FedMM's superiority over state-of-the-art heterogeneous MFL methods.

Medical image segmentation is crucial in clinical diagnosis, helping physicians identify and analyze medical conditions. However, this task is often accompanied by challenges like sensitive data, privacy concerns, and expensive annotations. Current research focuses on personalized collaborative training of medical segmentation systems, ignoring that obtaining segmentation annotations is time-consuming and laborious. Achieving a perfect balance between annotation cost and segmentation performance while ensuring local model personalization has become a valuable direction. Therefore, this study introduces a novel Model-Heterogeneous Semi-Supervised Federated (HSSF) Learning framework. It proposes Regularity Condensation and Regularity Fusion to transfer autonomously selective knowledge to ensure the personalization between sites. In addition, to efficiently utilize unlabeled data and reduce the annotation burden, it proposes a Self-Assessment (SA) module and a Reliable Pseudo-Label Generation (RPG) module. The SA module generates self-assessment confidence in real-time based on model performance, and the RPG module generates reliable pseudo-label based on SA confidence. We evaluate our model separately on the Skin Lesion and Polyp Lesion datasets. The results show that our model performs better than other methods characterized by heterogeneity. Moreover, it exhibits highly commendable performance even in homogeneous designs, most notably in region-based metrics. The full range of resources can be readily accessed through the designated repository located at HSSF(github.com) on the platform of GitHub.

Personalized federated learning (PFL) addresses the data heterogeneity challenge faced by general federated learning (GFL). Rather than learning a single global model, with PFL a collection of models are adapted to the unique feature distribution of each site. However, current PFL methods rarely consider self-attention networks which can handle data heterogeneity by long-range dependency modeling and they do not utilize prediction inconsistencies in local models as an indicator of site uniqueness. In this paper, we propose FedDP, a novel fed erated learning scheme with d ual p ersonalization, which improves model personalization from both feature and prediction aspects to boost image segmentation results. We leverage long-range dependencies by designing a local query (LQ) that decouples the query embedding layer out of each local model, whose parameters are trained privately to better adapt to the respective feature distribution of the site. We then propose inconsistency-guided calibration (IGC), which exploits the inter-site prediction inconsistencies to accommodate the model learning concentration. By encouraging a model to penalize pixels with larger inconsistencies, we better tailor prediction-level patterns to each local site. Experimentally, we compare FedDP with the state-of-the-art PFL methods on two popular medical image segmentation tasks with different modalities, where our results consistently outperform others on both tasks. Our code and models are available at https://github.com/jcwang123/PFL-Seg-Trans.

Multi-modal image registration spatially aligns two images with different distributions. One of its major challenges is that images acquired from different imaging machines have different imaging distributions, making it difficult to focus only on the spatial aspect of the images and ignore differences in distributions. In this study, we developed a self-supervised approach, Indescribable Multi-model Spatial Evaluator (IMSE), to address multi-modal image registration. IMSE creates an accurate multi-modal spatial evaluator to measure spatial differences between two images, and then optimizes registration by minimizing the error predicted of the evaluator. To optimize IMSE performance, we also proposed a new style enhancement method called Shuffle Remap which randomizes the image distribution into multiple segments, and then randomly disorders and remaps these segments, so that the distribution of the original image is changed. Shuffle Remap can help IMSE to predict the difference in spatial location from unseen target distributions. Our results show that IMSE outperformed the existing methods for registration using T1-T2 and CT-MRI datasets. IMSE also can be easily integrated into the traditional registration process, and can provide a convenient way to evaluate and visualize registration results. IMSE also has the potential to be used as a new paradigm for image-to-image translation. Our code is available at https://github.com/Kid-Liet/IMSE.

Automated detecting lung infections from computed tomography (CT) data plays an important role for combating coronavirus 2019 (COVID-19). However, there are still some challenges for developing AI system: 1) most current COVID-19 infection segmentation methods mainly relied on 2-D CT images, which lack 3-D sequential constraint; 2) existing 3-D CT segmentation methods focus on single-scale representations, which do not achieve the multiple level receptive field sizes on 3-D volume; and 3) the emergent breaking out of COVID-19 makes it hard to annotate sufficient CT volumes for training deep model. To address these issues, we first build a multiple dimensional-attention convolutional neural network (MDA-CNN) to aggregate multiscale information along different dimension of input feature maps and impose supervision on multiple predictions from different convolutional neural networks (CNNs) layers. Second, we assign this MDA-CNN as a basic network into a novel dual multiscale mean teacher network (DM [Formula: see text]-Net) for semi-supervised COVID-19 lung infection segmentation on CT volumes by leveraging unlabeled data and exploring the multiscale information. Our DM [Formula: see text]-Net encourages multiple predictions at different CNN layers from the student and teacher networks to be consistent for computing a multiscale consistency loss on unlabeled data, which is then added to the supervised loss on the labeled data from multiple predictions of MDA-CNN. Third, we collect two COVID-19 segmentation datasets to evaluate our method. The experimental results show that our network consistently outperforms the compared state-of-the-art methods.

Skin lesion segmentation from dermoscopy images is of great significance in the quantitative analysis of skin cancers, which is yet challenging even for dermatologists due to the inherent issues, i.e., considerable size, shape and color variation, and ambiguous boundaries. Recent vision transformers have shown promising performance in handling the variation through global context modeling. Still, they have not thoroughly solved the problem of ambiguous boundaries as they ignore the complementary usage of the boundary knowledge and global contexts. In this paper, we propose a novel cross-scale boundary-aware transformer, XBound-Former, to simultaneously address the variation and boundary problems of skin lesion segmentation. XBound-Former is a purely attention-based network and catches boundary knowledge via three specially designed learners. First, we propose an implicit boundary learner (im-Bound) to constrain the network attention on the points with noticeable boundary variation, enhancing the local context modeling while maintaining the global context. Second, we propose an explicit boundary learner (ex-Bound) to extract the boundary knowledge at multiple scales and convert it into embeddings explicitly. Third, based on the learned multi-scale boundary embeddings, we propose a cross-scale boundary learner (X-Bound) to simultaneously address the problem of ambiguous and multi-scale boundaries by using learned boundary embedding from one scale to guide the boundary-aware attention on the other scales. We evaluate the model on two skin lesion datasets and one polyp lesion dataset, where our model consistently outperforms other convolution- and transformer-based models, especially on the boundary-wise metrics. All resources could be found in https://github.com/jcwang123/xboundformer.

Automatic delineation of organ-at-risk (OAR) and gross-tumor-volume (GTV) is of great significance for radiotherapy planning. However, it is a challenging task to learn powerful representations for accurate delineation under limited pixel (voxel)-wise annotations. Contrastive learning at pixel-level can alleviate the dependency on annotations by learning dense representations from unlabeled data. Recent studies in this direction design various contrastive losses on the feature maps, to yield discriminative features for each pixel in the map. However, pixels in the same map inevitably share semantics to be closer than they actually are, which may affect the discrimination of pixels in the same map and lead to the unfair comparison to pixels in other maps. To address these issues, we propose a separated region-level contrastive learning scheme, namely SepaReg, the core of which is to separate each image into regions and encode each region separately. Specifically, SepaReg comprises two components: a structure-aware image separation (SIS) module and an intra- and inter-organ distillation (IID) module. The SIS is proposed to operate on the image set to rebuild a region set under the guidance of structural information. The inter-organ representation will be learned from this set via typical contrastive losses cross regions. On the other hand, the IID is proposed to tackle the quantity imbalance in the region set as tiny organs may produce fewer regions, by exploiting intra-organ representations. We conducted extensive experiments to evaluate the proposed model on a public dataset and two private datasets. The experimental results demonstrate the effectiveness of the proposed model, consistently achieving better performance than state-of-the-art approaches. Code is available at https://github.com/jcwang123/Separate_CL.

Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. While numerous methods for detecting, segmenting and tracking of medical instruments based on endoscopic video images have been proposed in the literature, key limitations remain to be addressed: Firstly, robustness, that is, the reliable performance of state-of-the-art methods when run on challenging images (e.g. in the presence of blood, smoke or motion artifacts). Secondly, generalization; algorithms trained for a specific intervention in a specific hospital should generalize to other interventions or institutions. In an effort to promote solutions for these limitations, we organized the Robust Medical Instrument Segmentation (ROBUST-MIS) challenge as an international benchmarking competition with a specific focus on the robustness and generalization capabilities of algorithms. For the first time in the field of endoscopic image processing, our challenge included a task on binary segmentation and also addressed multi-instance detection and segmentation. The challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures from three different types of surgery. The validation of the competing methods for the three tasks (binary segmentation, multi-instance detection and multi-instance segmentation) was performed in three different stages with an increasing domain gap between the training and the test data. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap. While the average detection and segmentation quality of the best-performing algorithms is high, future research should concentrate on detection and segmentation of small, crossing, moving and transparent instrument(s) (parts).