Image decomposition aims to analyze an image into elementary components, which is essential for numerous downstream tasks and also by nature provides certain interpretability to the analysis. Deep learning can be powerful for such tasks, but surprisingly their combination with a focus on interpretability and generalizability is rarely explored. In this work, we introduce a novel framework to decompose an image into the low-rank, sparse, and noise components, combining hierarchical Bayesian modeling and deep learning to create an architecture-modularized and model-generalizable neural network (DNN). The proposed framework includes three steps: (1) hierarchical Bayesian modeling of image decomposition, (2) transforming the inference problem into optimization tasks, and (3) deep inference via a modularized Bayesian DNN under a relaxed amortized formulation. We further analyze the connection between the loss function and the generalization error bound following the PAC-Bayes theory, which then motivates a new test-time adaptation approach for out-of-distribution scenarios. We instantiated the application using two downstream tasks, i.e., image denoising and unsupervised anomaly detection, and the results demonstrated improved generalizability as well as interpretability of our methods. The source code is available at https://github.com/LucyyyyW/InDeed.

In recent years, the deployment of supervised machine learning techniques for segmentation tasks has significantly increased. Nonetheless, the annotation process for extensive datasets remains costly, labor-intensive, and error-prone. While acquiring sufficiently large datasets to train deep learning models is feasible, these datasets often experience a distribution shift relative to the actual test data. This problem is particularly critical in the domain of medical imaging, where it adversely affects the efficacy of automatic segmentation models. In this work, we introduce DiffuSeg, a novel conditional diffusion model developed for medical image data, that exploits any labels to synthesize new images in the target domain. This allows a number of new research directions, including the segmentation task that motivates this work. Our method only requires label maps from any existing datasets and unlabelled images from the target domain for image diffusion. To learn the target domain knowledge, a feature factorization variational autoencoder is proposed to provide conditional information for the diffusion model. Consequently, the segmentation network can be trained with the given labels and the synthetic images, thus avoiding human annotations. Initially, we apply our method to the MNIST dataset and subsequently adapt it for use with medical image segmentation datasets, such as retinal fundus images for vessel segmentation and MRI images for heart segmentation. Our approach exhibits significant improvements over relevant baselines in both image generation and segmentation accuracy, especially in scenarios where annotations for the target dataset are unavailable during training. An open-source implementation of our approach can be released after reviewing..

While multi-task learning (MTL) has been widely developed for natural image analysis, its potential for enhancing performance in medical imaging remains relatively unexplored. Most methods formulate MTL as a multi-objective problem, inherently forcing all tasks to compete with each other during optimization. In this work, we propose a novel approach by formulating MTL as a multi-level optimization problem, in which the features learned from one task are optimized by benefiting from the other tasks. Specifically, we advocate for a cooperative approach where each task considers the features of others, enabling individual performance enhancement without detriment to others. To achieve this objective, we introduce a novel optimization strategy aimed at seeking flat minima for each sub-problem, fostering the learning of robust sub-models resilient to changes in other sub-models. We demonstrate the advantages of our proposed method through comprehensive parameter and comparison studies on the OrganCMNIST dataset. Additionally, we evaluate its efficacy on three eye-related medical image datasets, comparing its performance against other state-of-the-art MTL approaches. The results highlight the superiority of our method over existing approaches, showcasing its potential for training multi-purpose models in medical image analysis.

The success of deep learning on image classification and recognition tasks has led to new applications in diverse contexts, including the field of medical imaging. However, two properties of deep neural networks (DNNs) may limit their future use in medical applications. The first is that DNNs require a large amount of labeled training data, and the second is that the deep learning-based models lack interpretability. In this paper, we propose and investigate a data-efficient framework for the task of general medical image segmentation. We address the two aforementioned challenges by introducing domain knowledge in the form of a strong prior into a deep learning framework. This prior is expressed by a customized dynamical system. We performed experiments on two different datasets, namely JSRT and ISIC2016 (heart and lungs segmentation on chest X-ray images and skin lesion segmentation on dermoscopy images). We have achieved competitive results using the same amount of training data compared to the state-of-the-art methods. More importantly, we demonstrate that our framework is extremely data-efficient, and it can achieve reliable results using extremely limited training data. Furthermore, the proposed method is rotationally invariant and insensitive to initialization.

Unsupervised domain adaptation(UDA) aims to mitigate the performance drop of models tested on the target domain, due to the domain shift from the target to sources. Most UDA segmentation methods focus on the scenario of solely single source domain. However, in practical situations data with gold standard could be available from multiple sources (domains), and the multi-source training data could provide more information for knowledge transfer. How to utilize them to achieve better domain adaptation yet remains to be further explored. This work investigates multi-source UDA and proposes a new framework for medical image segmentation. Firstly, we employ a multi-level adversarial learning scheme to adapt features at different levels between each of the source domains and the target, to improve the segmentation performance. Then, we propose a multi-model consistency loss to transfer the learned multi-source knowledge to the target domain simultaneously. Finally, we validated the proposed framework on two applications, i.e., multi-modality cardiac segmentation and cross-modality liver segmentation. The results showed our method delivered promising performance and compared favorably to state-of-the-art approaches.

Medical images are generally acquired with limited field-of-view (FOV), which could lead to incomplete regions of interest (ROI), and thus impose a great challenge on medical image analysis. This is particularly evident for the learning-based multi-target landmark detection, where algorithms could be misleading to learn primarily the variation of background due to the varying FOV, failing the detection of targets. Based on learning a navigation policy, instead of predicting targets directly, reinforcement learning (RL)-based methods have the potential to tackle this challenge in an efficient manner. Inspired by this, in this work we propose a multi-agent RL framework for simultaneous multi-target landmark detection. This framework is aimed to learn from incomplete or (and) complete images to form an implicit knowledge of global structure, which is consolidated during the training stage for the detection of targets from either complete or incomplete test images. To further explicitly exploit the global structural information from incomplete images, we propose to embed a shape model into the RL process. With this prior knowledge, the proposed RL model can not only localize dozens of targets simultaneously, but also work effectively and robustly in the presence of incomplete images. We validated the applicability and efficacy of the proposed method on various multi-target detection tasks with incomplete images from practical clinics, using body dual-energy X-ray absorptiometry (DXA), cardiac MRI and head CT datasets. Results showed that our method could predict whole set of landmarks with incomplete training images up to 80% missing proportion (average distance error 2.29 cm on body DXA), and could detect unseen landmarks in regions with missing image information outside FOV of target images (average distance error 6.84 mm on 3D half-head CT). Our code will be released via https://zmiclab.github.io/projects.html.

Supervised segmentation can be costly, particularly in applications of biomedical image analysis where large scale manual annotations from experts are generally too expensive to be available. Semi-supervised segmentation, able to learn from both the labeled and unlabeled images, could be an efficient and effective alternative for such scenarios. In this work, we propose a new formulation based on risk minimization, which makes full use of the unlabeled images. Different from most of the existing approaches which solely explicitly guarantee the minimization of prediction risks from the labeled training images, the new formulation also considers the risks on unlabeled images. Particularly, this is achieved via an unbiased estimator, based on which we develop a general framework for semi-supervised image segmentation. We validate this framework on three medical image segmentation tasks, namely cardiac segmentation on ACDC2017, optic cup and disc segmentation on REFUGE dataset and 3D whole heart segmentation on MM-WHS dataset. Results show that the proposed estimator is effective, and the segmentation method achieves superior performance and demonstrates great potential compared to the other state-of-the-art approaches. Our code and data will be released via https://zmiclab.github.io/projects.html, once the manuscript is accepted for publication.

Large training datasets are important for deep learning-based methods. For medical image segmentation, it could be however difficult to obtain large number of labeled training images solely from one center. Distributed learning, such as swarm learning, has the potential to use multi-center data without breaching data privacy. However, data distributions across centers can vary a lot due to the diverse imaging protocols and vendors (known as feature skew). Also, the regions of interest to be segmented could be different, leading to inhomogeneous label distributions (referred to as label skew). With such non-independently and identically distributed (Non-IID) data, the distributed learning could result in degraded models. In this work, we propose a novel swarm learning approach, which assembles local knowledge from each center while at the same time overcomes forgetting of global knowledge during local training. Specifically, the approach first leverages a label skew-awared loss to preserve the global label knowledge, and then aligns local feature distributions to consolidate global knowledge against local feature skew. We validated our method in three Non-IID scenarios using four public datasets, including the Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation (M&Ms) dataset, the Federated Tumor Segmentation (FeTS) dataset, the Multi-Modality Whole Heart Segmentation (MMWHS) dataset and the Multi-Site Prostate T2-weighted MRI segmentation (MSProsMRI) dataset. Results show that our method could achieve superior performance over existing methods. Code will be released via https://zmiclab.github.io/projects.html once the paper gets accepted.

With the rapid development of the Internet of Things (IoT), we have seen a larger number of devices deployed with different wireless communication protocols (i.e., WiFi, ZigBee, Bluetooth). Working in the same place opens a new opportunity for these devices to communicate directly with each other, leveraging on Cross-technology Communication (CTC). However, since these devices operate in the same frequency band which results in the competition against each other for network resources, severe interfere may arise. In this paper, we explore pseudo-random sequence (PR sequence) to design a novel CTC protocol that enables low-cost direct communication between WiFi and ZigBee in noisy indoor environments. Pseudorandom sequence offers a unique statistical feature to accomplish both information transmission and synchronization between heterogeneous devices. We design a dynamic synchronous decoding strategy to handle interference coexisted among different wireless protocols. Our system does not require any modification of communication protocol and underlying hardware and firmware. We implement our system on commercial devices (Intel 5300 WiFi NIC and MicaZ CC2420), and conduct extensive experiments to evaluate the system performance in three typical scenarios. The experimental results show that the synchronization time of our approach is lower than 0.5 ms, and the accuracy is greater than 84% while the channel occupancy is as high as 50%.

Unsupervised domain adaptation is useful in medical image segmentation. Particularly, when ground truths of the target images are not available, domain adaptation can train a target-specific model by utilizing the existing labeled images from other modalities. Most of the reported works mapped images of both the source and target domains into a common latent feature space, and then reduced their discrepancy either implicitly with adversarial training or explicitly by directly minimizing a discrepancy metric. In this work, we propose a new framework, where the latent features of both domains are driven towards a common and parameterized variational form, whose conditional distribution given the image is Gaussian. This is achieved by two networks based on variational auto-encoders (VAEs) and a regularization for this variational approximation. Both of the VAEs, each for one domain, contain a segmentation module, where the source segmentation is trained in a supervised manner, while the target one is trained unsupervisedly. We validated the proposed domain adaptation method using two cardiac segmentation tasks, i.e., the cross-modality (CT and MR) whole heart segmentation and the cross-sequence cardiac MR segmentation. Results show that the proposed method achieved better accuracies compared to two state-of-the-art approaches and demonstrated good potential for cardiac segmentation. Furthermore, the proposed explicit regularization was shown to be effective and efficient in narrowing down the distribution gap between domains, which is useful for unsupervised domain adaptation. The code and data have been released via https://zmiclab.github.io/projects.html.

Late gadolinium enhancement magnetic resonance imaging (LGE MRI) appears to be a promising alternative for scar assessment in patients with atrial fibrillation (AF). Automating the quantification and analysis of atrial scars can be challenging due to the low image quality. In this work, we propose a fully automated method based on the graph-cuts framework, where the potentials of the graph are learned on a surface mesh of the left atrium (LA) using a multi-scale convolutional neural network (MS-CNN). For validation, we have included fifty-eight images with manual delineations. MS-CNN, which can efficiently incorporate both the local and global texture information of the images, has been shown to evidently improve the segmentation accuracy of the proposed graph-cuts based method. The segmentation could be further improved when the contribution between the t-link and n-link weights of the graph is balanced. The proposed method achieves a mean accuracy of 0.856 ± 0.033 and mean Dice score of 0.702 ± 0.071 for LA scar quantification. Compared to the conventional methods, which are based on the manual delineation of LA for initialization, our method is fully automatic and has demonstrated significantly better Dice score and accuracy (p < 0.01). The method is promising and can be potentially useful in diagnosis and prognosis of AF.

Domain adaptation has great values in unpaired cross-modality image segmentation, where the training images with gold standard segmentation are not available from the target image domain. The aim is to reduce the distribution discrepancy between the source and target domains. Hence, an effective measurement for this discrepancy is critical. In this work, we propose a new metric based on characteristic functions of distributions. This metric, referred to as CF distance, enables explicit domain adaptation, in contrast to the implicit manners minimizing domain discrepancy via adversarial training. Based on this CF distance, we propose an unsupervised domain adaptation framework for cross-modality cardiac segmentation, which consists of image reconstruction and prior distribution matching. We validated the method on two tasks, i.e., the CT-MR cross-modality segmentation and the multi-sequence cardiac MR segmentation. Results showed that the proposed explicit metric was effective in domain adaptation, and the segmentation method delivered promising and superior performance, compared to other state-of-the-art techniques. The data and source code of this work has been released via https://zmiclab.github.io/projects.html.