To tackle the problem of limited annotated data, semi-supervised learning is attracting attention as an alternative to fully supervised models. Moreover, optimizing a multiple-task model to learn “multiple contexts” can provide better generalizability compared to single-task models. We propose a novel semi-supervised multiple-task model leveraging self-supervision and adversarial training—namely, self-supervised, semi-supervised, multi-context learning (S4MCL)—and apply it to two crucial medical imaging tasks, classification and segmentation. Our experiments on spine X-rays reveal that the S4MCL model significantly outperforms semi-supervised single-task, semi-supervised multi-context, and fully-supervised single-task models, even with a 50% reduction of classification and segmentation labels.

In this work, we propose to resolve the issue existing in current deep learning based organ segmentation systems that they often produce results that do not capture the overall shape of the target organ and often lack smoothness. Since there is a rigorous mapping between the Signed Distance Map (SDM) calculated from object boundary contours and the binary segmentation map, we exploit the feasibility of learning the SDM directly from medical scans. By converting the segmentation task into predicting an SDM, we show that our proposed method retains superior segmentation performance and has better smoothness and continuity in shape. To leverage the complementary information in traditional segmentation training, we introduce an approximated Heaviside function to train the model by predicting SDMs and segmentation maps simultaneously. We validate our proposed models by conducting extensive experiments on a hippocampus segmentation dataset and the public MICCAI 2015 Head and Neck Auto Segmentation Challenge dataset with multiple organs. While our carefully designed backbone 3D segmentation network improves the Dice coefficient by more than 5% compared to current state-of-the-arts, the proposed model with SDM learning produces smoother segmentation results with smaller Hausdorff distance and average surface distance, thus proving the effectiveness of our method.

Multi-modal imaging plays a critical role in various clinical applications. However, due to the associated high cost and potential risk, the acquisition of multi-modal images could be limited. To address this issue, many cross-modality image synthesis methods have been proposed. The state-of-the-art methods are mainly based on traditional convolutional generative adversarial networks (GANs) for generating target images. In 3D medical image synthesis, an open problem is how to efficiently exploit the spatial correlations of the 3D image sequence to resolve the inter-slice discontinuity and unevenness artifacts. In this paper, we propose a novel Context aware Residual Recurrent Generative Adversarial Network (short for CorGAN) for sequential medical image generation, which jointly exploits the spatial dependencies of the sequences as well as the peer image generation with GANs. Experimental results show the robustness and accuracy of our method, which outperforms the-state-of-the-art methods in synthesizing target 3D images from the corresponding source images.

Scoliosis is a congenital disease in which the spine is deformed from its normal shape. Radiography is the most cost-effective and accessible modality for imaging the spine. Conventional spinal assessment, diagnosis of scoliosis, and treatment planning relies on tedious and time-consuming manual analysis of spine radiographs that is susceptible to observer variation. A reliable, fully-automated method that can accurately identify vertebrae, a crucial step in image-guided scoliosis assessment, is presently unavailable in the literature. Leveraging a novel, deep-learning-based image segmentation model, we develop an end-to-end spine radiograph analysis pipeline that automatically provides an accurate segmentation and identification of the vertebrae, culminating in the reliable estimation of the Cobb angle, the most widely used measurement to quantify the magnitude of scoliosis. Our experimental results with anterior-posterior spine X-ray images indicate that our system is effective in the identification and labeling of vertebrae, and can potentially provide assistance to medical practitioners in the assessment of scoliosis.

Semi-supervised learning has recently been attracting attention as an alternative to fully supervised models that require large pools of labeled data. Moreover, optimizing a model for multiple tasks can provide better generalizability than single-task learning. Leveraging self-supervision and adversarial training, we propose a novel, general purpose semi-supervised, multiple-task model-namely, self-supervised, semi-supervised, multi-task learning (S4MTL)-for accomplishing two important medical image analysis tasks: segmentation and diagnostic classification. Experimental results on chest and spine X-ray datasets confirm that our S4MTL model significantly outperforms semi-supervised single-task, semi/fully-supervised multi-task, and fully-supervised single-task models, even with a 50% reduction in class and segmentation labels.

Existing approaches to image reconstruction in photoacoustic computed tomography (PACT) with acoustically heterogeneous media are limited to weakly varying media, are computationally burdensome, and/or cannot effectively mitigate the effects of measurement data incompleteness and noise. In this work, we develop and investigate a discrete imaging model for PACT that is based on the exact photoacoustic (PA) wave equation and facilitates the circumvention of these limitations. A key contribution of the work is the establishment of a procedure to implement a matched forward and backprojection operator pair associated with the discrete imaging model, which permits application of a wide-range of modern image reconstruction algorithms that can mitigate the effects of data incompleteness and noise. The forward and backprojection operators are based on the k-space pseudospectral method for computing numerical solutions to the PA wave equation in the time domain. The developed reconstruction methodology is investigated by use of both computer-simulated and experimental PACT measurement data.