Recently, the diffusion model has emerged as a superior generative model that can produce high quality and realistic images. However, for medical image translation, the existing diffusion models are deficient in accurately retaining structural information since the structure details of source domain images are lost during the forward diffusion process and cannot be fully recovered through learned reverse diffusion, while the integrity of anatomical structures is extremely important in medical images. For instance, errors in image translation may distort, shift, or even remove structures and tumors, leading to incorrect diagnosis and inadequate treatments. Training and conditioning diffusion models using paired source and target images with matching anatomy can help. However, such paired data are very difficult and costly to obtain, and may also reduce the robustness of the developed model to out-of-distribution testing data. We propose a frequency-guided diffusion model (FGDM) that employs frequency-domain filters to guide the diffusion model for structure-preserving image translation. Based on its design, FGDM allows zero-shot learning, as it can be trained solely on the data from the target domain, and used directly for source-to-target domain translation without any exposure to the source-domain data during training. We evaluated it on three cone-beam CT (CBCT)-to-CT translation tasks for different anatomical sites, and a cross-institutional MR imaging translation task. FGDM outperformed the state-of-the-art methods (GAN-based, VAE-based, and diffusion-based) in metrics of Fréchet Inception Distance (FID), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM), showing its significant advantages in zero-shot medical image translation.

introduction Share on Guest Editorial: ACM JETC Special Issue on Hardware-Aware Learning for Medical Applications Editors: Yiyu Shi University of Notre Dame, Notre Dame, Indiana, USA University of Notre Dame, Notre Dame, Indiana, USAView Profile , Yongpan Liu Tsinghua University, Beijing, China Tsinghua University, Beijing, ChinaView Profile , Jianxu Chen Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V, Dortmund, Germany Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V, Dortmund, GermanyView Profile , Steve Jiang University of Texas Southwestern Medical Center Dallas, Texas, USA University of Texas Southwestern Medical Center Dallas, Texas, USAView Profile Authors Info & Claims ACM Journal on Emerging Technologies in Computing SystemsVolume 18Issue 2April 2022 Article No.: 24pp 1–3https://doi.org/10.1145/3503262Online:31 December 2021Publication History 0citation60DownloadsMetricsTotal Citations0Total Downloads60Last 12 Months60Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

In this Special Section of the IEEE Transactions on Circuits and Systems for Video Technology, it is our honor to present emerging advanced machine learning and data analytics algorithms aiming at catalyzing synergies among image/video processing, text/speech understanding, and multimodal learning in biomedical informatics. Our goals are to 1) introduce novel data-driven models to accelerate knowledge discovery in biomedicine through the seamless integration of medical data collected from imaging systems, laboratory and wearable devices, as well as other related medical devices; 2) promote the development of new multi-modal learning systems to enhance the healthcare quality and patient safety; and 3) promote new applications in biomedical informatics that can leverage or benefits from the integration of multi-modal data and machine learning.

Low-dose computed tomography (LDCT) is desirable for both diagnostic imaging and image-guided interventions. Denoisers are widely used to improve the quality of LDCT. Deep learning (DL)-based denoisers have shown state-of-the-art performance and are becoming mainstream methods. However, there are two challenges to using DL-based denoisers: 1) a trained model typically does not generate different image candidates with different noise-resolution tradeoffs, which are sometimes needed for different clinical tasks; and 2) the model's generalizability might be an issue when the noise level in the testing images differs from that in the training dataset. To address these two challenges, in this work, we introduce a lightweight optimization process that can run on top of any existing DL-based denoiser during the testing phase to generate multiple image candidates with different noise-resolution tradeoffs suitable for different clinical tasks in real time. Consequently, our method allows users to interact with the denoiser to efficiently review various image candidates and quickly pick the desired one; thus, we termed this method deep interactive denoiser (DID). Experimental results demonstrated that DID can deliver multiple image candidates with different noise-resolution tradeoffs and shows great generalizability across various network architectures, as well as training and testing datasets with various noise levels.

OBJECTIVE: accurately classifying the malignancy of lesions detected in a screening scan is critical for reducing false positives. Radiomics holds great potential to differentiate malignant from benign tumors by extracting and analyzing a large number of quantitative image features. Since not all radiomic features contribute to an effective classifying model, selecting an optimal feature subset is critical. METHODS: this work proposes a new multi-objective based feature selection (MO-FS) algorithm that considers sensitivity and specificity simultaneously as the objective functions during feature selection. For MO-FS, we developed a modified entropy-based termination criterion that stops the algorithm automatically rather than relying on a preset number of generations. We also designed a solution selection methodology for multi-objective learning that uses the evidential reasoning approach (SMOLER) to automatically select the optimal solution from the Pareto-optimal set. Furthermore, we developed an adaptive mutation operation to generate the mutation probability in MO-FS automatically. RESULTS: we evaluated the MO-FS for classifying lung nodule malignancy in low-dose CT and breast lesion malignancy in digital breast tomosynthesis. CONCLUSION: the experimental results demonstrated that the feature set selected by MO-FS achieved better classification performance than features selected by other commonly used methods. SIGNIFICANCE: the proposed method is general and more effective radiomic feature selection strategy.

A number of image-processing problems can be formulated as optimization problems. The objective function typically contains several terms specifically designed for different purposes. Parameters in front of these terms are used to control the relative importance among them. It is of critical importance to adjust these parameters, as quality of the solution depends on their values. Tuning parameters are a relatively straight forward task for a human, as one can intuitively determine the direction of parameter adjustment based on the solution quality. Yet manual parameter tuning is not only tedious in many cases, but also becomes impractical when a number of parameters exist in a problem. Aiming at solving this problem, this paper proposes an approach that employs deep reinforcement learning to train a system that can automatically adjust parameters in a human-like manner. We demonstrate our idea in an example problem of optimization-based iterative computed tomography (CT) reconstruction with a pixel-wise total-variation regularization term. We set up a parameter-tuning policy network (PTPN), which maps a CT image patch to an output that specifies the direction and amplitude by which the parameter at the patch center is adjusted. We train the PTPN via an end-to-end reinforcement learning procedure. We demonstrate that under the guidance of the trained PTPN, reconstructed CT images attain quality similar or better than those reconstructed with manually tuned parameters.

Precise patient positioning is fundamental to successful removal of malignant tumors during treatment of head and neck cancers. Errors in patient positioning have been known to damage critical organs and cause complications. To better address issues of patient positioning and motion, we introduce a 3-DOF neuro-adaptive soft-robot, called Soft-NeuroAdapt to correct deviations along 3 axes. The robot consists of inflatable air bladders that adaptively control head deviations from target while ensuring patient safety and comfort. The adaptive-neuro controller combines a state feedback component, a feedforward regulator, and a neural network that ensures correct adaptation. States are measured by a 3D vision system. We validate Soft-NeuroAdapt on a 3D printed head-and-neck dummy, and demonstrate that the controller provides adaptive actuation that compensates for intrafractional deviations in patient positioning.

Despite the rapid developments of X-ray cone-beam CT (CBCT), image noise still remains a major issue for the low dose CBCT. To suppress the noise effectively while retain the structures well for low dose CBCT image, in this paper, a sparse constraint based on the 3-D dictionary is incorporated into a regularized iterative reconstruction framework, defining the 3-D dictionary learning (3-DDL) method. In addition, by analyzing the sparsity level curve associated with different regularization parameters, a new adaptive parameter selection strategy is proposed to facilitate our 3-DDL method. To justify the proposed method, we first analyze the distributions of the representation coefficients associated with the 3-D dictionary and the conventional 2-D dictionary to compare their efficiencies in representing volumetric images. Then, multiple real data experiments are conducted for performance validation. Based on these results, we found: 1) the 3-D dictionary-based sparse coefficients have three orders narrower Laplacian distribution compared with the 2-D dictionary, suggesting the higher representation efficiencies of the 3-D dictionary; 2) the sparsity level curve demonstrates a clear Z-shape, and hence referred to as Z-curve, in this paper; 3) the parameter associated with the maximum curvature point of the Z-curve suggests a nice parameter choice, which could be adaptively located with the proposed Z-index parameterization (ZIP) method; 4) the proposed 3-DDL algorithm equipped with the ZIP method could deliver reconstructions with the lowest root mean squared errors and the highest structural similarity index compared with the competing methods; 5) similar noise performance as the regular dose FDK reconstruction regarding the standard deviation metric could be achieved with the proposed method using (1/2)/(1/4)/(1/8) dose level projections. The contrast-noise ratio is improved by ~2.5/3.5 times with respect to two different cases under the (1/8) dose level compared with the low dose FDK reconstruction. The proposed method is expected to reduce the radiation dose by a factor of 8 for CBCT, considering the voted strongly discriminated low contrast tissues.

Respiration-correlated CBCT, commonly called 4DCBCT, provides respiratory phase-resolved CBCT images. A typical 4DCBCT represents averaged patient images over one breathing cycle and the fourth dimension is actually breathing phase instead of time. In many clinical applications, it is desirable to obtain true 4DCBCT with the fourth dimension being time, i.e., each constituent CBCT image corresponds to an instantaneous projection. Theoretically it is impossible to reconstruct a CBCT image from a single projection. However, if all the constituent CBCT images of a 4DCBCT scan share a lot of redundant information, it might be possible to make a good reconstruction of these images by exploring their sparsity and coherence/redundancy. Though these CBCT images are not completely time resolved, they can exploit both local and global temporal coherence of the patient anatomy automatically and contain much more temporal variation information of the patient geometry than the conventional 4DCBCT. We propose in this work a computational model and algorithms for the reconstruction of this type of semi-time-resolved CBCT, called cine-CBCT, based on low rank approximation that can utilize the underlying temporal coherence both locally and globally, such as slow variation, periodicity or repetition, in those cine-CBCT images.

Under-sampling and insufficient data result in a big challenge in the reconstruction of x-ray computed tomographic (CT) images. In addition, patient's respiratory motion also deteriorates this reconstruction process as it normally leads to blurred outputs. In this work, we propose an iterative method with a combination of total variation (TV) regularization and principle component analysis (PCA) regularization. Partial prior knowledge of the CT images, obtained through PCA analysis of training images is incorporated in the reconstruction process. Numerical experiments are performed in the context of a fan-beam CT reconstruction, which shows advantages of our method over the ones with just TV regularization or just PCA regularization.

Respiratory gated radiotherapy for lung cancer allows for more precise delivery of prescribed radiation dose to the tumor, while minimizing normal tissue complications. Techniques for fluoroscopic gating without implanted fiducial markers have been developed in a classification framework. Due to the high-dimensionality nature of the images, dimensionality reduction techniques such as principal component analysis (PCA) were used to preprocess the data. In this work, we have applied generalized linear discriminant analysis (GLDA) to the respiratory gating problem. The fundamental difference from conventional dimensionality reduction techniques is that GLDA explicitly takes into account the label information available in the training set and therefore is efficient for discrimination among classes. On average, GLDA was demonstrated to perform similarly with PCA trained with SVM at high nominal duty cycles and outperform PCA in terms of classification accuracy (CA) and target coverage (TC) at lower nominal duty cycle (20%). A major advantage of GLDA is its robustness, while CA and TC using PCA can be reduced by up to 10% depending on the data dimensionality. With only 1-dimensional feature vectors, GLDA is much more computationally efficient than PCA. Therefore, GLDA is an effective and efficient method for respiratory gating with markerless fluoroscopic images.

Accurate lung tumor targeting in real time plays a fundamental role in image-guide radiotherapy of lung cancers. Precise tumor targeting is required for both respiratory gating and tracking. Gating is considered as the current state of the art for precise lung cancer radiotherapy, which irradiates the tumor when it moves into a predefined gating window. Tracking seems to be a next-generation technique, and it operates in a more aggressive fashion by following the tumor position with radiation beam in real time. Existing methods for gating and tracking often rely on observed motion patterns of external surrogates or implanted fiducial markers. However, external surrogates suffer from certain degrees of inaccuracy, and implanted fiducial markers are in limited uses due to the risk of pneumothorax. Therefore, direct tumor targeting techniques without implanting fiducial markers are desired. Previous studies in fluoroscopic markerless targeting are mainly based on template matching methods, which may fail when tumor boundary is unclear in fluoroscopic images. In this paper, we propose a novel framework of markerless gating and tracking based on machine learning algorithms. Specifically, gating is treated as a two-class classification problem, which is solved by principal component analysis (PCA) and artificial neural network (ANN). Further, we formulate the tracking problem as a regression task, which employs the correlation between the tumor position and nearby surrogate anatomic features in the image. Four regression methods were tested in this study: 1-degree and 2-degree linear regression, artificial neural network (ANN), and support vector machine (SVM). Finally, we demonstrate the superb performance of the proposed markerless gating and tracking algorithms on 10 fluoroscopic image sequences of 9 patients. For gating, the target coverage (the precision) ranges from 90% to 99%, with mean of 96.5%. For tracking, the mean localization error is about 2.1 pixels and the maximum error at 95% confidence level is about 4.6 pixels (pixel size is about 0.5 mm).

We propose a novel approach for on-line treatment verification using cine EPID (electronic portal imaging device) images for hypofractionated lung radiotherapy based on a machine learning algorithm. Hypofractionated lung radiotherapy has high precision requirement, and it is essential to effectively monitor the target making sure the tumor is within beam aperture. We model the treatment verification problem as a two-class classification problem and apply artificial neural network (ANN) to classify the cine EPID images acquired during the treatment into corresponding classes-tumor inside or outside of the beam aperture. Training samples of ANN are generated using digitally reconstructed radiograph (DRR) with artificially added shifts in tumor location-to simulate cine EPID images with different tumor locations. Principal component analysis (PCA) is used to reduce the dimensionality of the training samples and cine EPID images acquired during the treatment. The proposed treatment verification algorithm has been tested on six hypofrationated lung patients in a retrospective fashion. On average, our proposed algorithm achieved 94.66% classification accuracy, 94.50% recall rate, and 99.79% precision rate.

In an idealized gated radiotherapy treatment, radiation is delivered only when the tumor is at the right position. For gated lung cancer radiotherapy, it is difficult to generate accurate gating signals due to the large uncertainties when using external surrogates and the risk of pneumothorax when using implanted fiducial markers. In this paper, we investigate machine learning algorithms for markerless gated radiotherapy with fluoroscopic images. Previous approach utilizes template matching to localize the tumor position. Here, we investigate two ways to improve the precision of tumor target localization by applying: (1) an ensemble of templates where the representative templates are selected by Gaussian mixture clustering, and (2) a support vector machine (SVM) classifier with radial basis kernels. Template matching only considers images inside the gating window, but images outside the gating window might provide additional information. We take advantage of both states and re-cast the gating problem into a classification problem. Thus, we are able to use the SVM classifier for gated radiotherapy. To verify the effectiveness of the two proposed techniques, we apply them on five sequences of fluoroscopic images from five lung cancer patients against the gating signal of manually contoured tumors as ground truth. Our five-patient case study shows that both ensemble template matching and SVM are reasonable tools for image-guided markerless gated radiotherapy with an average of approximately 95% precision in terms of delivered target dose at approximately 35% duty cycle.

Subsequence matching in time series databases is a useful technique, with applications in pattern matching, prediction, and rule discovery. Internal structure within the time series data can be used to improve these tasks, and provide important insight into the problem domain. This paper introduces our research effort in using the internal structure of a time series directly in the matching process. This idea is applied to the problem domain of respiratory motion data in cancer radiation treatment. We propose a comprehensive solution for analysis, clustering, and online prediction of respiratory motion using subsequence similarity matching. In this system, a motion signal is captured in real time as a data stream, and is analyzed immediately for treatment and also saved in a database for future study. A piecewise linear representation of the signal is generated from a finite state model, and is used as a query for subsequence matching. To ensure that the query subsequence is representative, we introduce the concept of subsequence stability, which can be used to dynamically adjust the query subsequence length. To satisfy the special needs of similarity matching over breathing patterns, a new subsequence similarity measure is introduced. This new measure uses a weighted L1 distance function to capture the relative importance of each source stream, amplitude, frequency, and proximity in time. From the subsequence similarity measure, stream and patient similarity can be defined, which are then used for offline and online applications. The matching results are analyzed and applied for motion prediction and correlation discovery. While our system has been customized for use in radiation therapy, our approach to time series modeling is general enough for application domains with structured time series data.