Objective, task-based measures of image quality (IQ) have been widely advocated for assessing and optimizing medical imaging technologies. Besides signal detection theory-based measures, information-theoretic quantities have been proposed to quantify task-based IQ. For example, task-specific information (TSI), defined as the mutual information between an image and a task variable, represents an optimal measure of how informative an image is for performing a specified task. However, like the ideal observer from signal detection theory, TSI does not quantify the amount of task-relevant information in an image that can be exploited by a sub-ideal observer. A recently proposed relaxation of TSI, termed predictive $\mathcal {V}$ -information ( $\mathcal {V}$ -info), removes this limitation and can quantify the utility of an image with consideration of a specified family of sub-ideal observers. In this study, for the first time, we introduce and investigate $\mathcal {V}$ -info as an objective, task-specific IQ metric. To corroborate its usefulness, a stylized magnetic resonance image restoration problem is considered in which $\mathcal {V}$ -info is employed to quantify signal detection or discrimination performance. The presented experiments show that, for binary classification tasks, $\mathcal {V}$ -info varies consistently with the area under the receiver operating characteristic (ROC) curve in regimes where class separability changes with observer capacity or imaging conditions. However, unlike AUC, $\mathcal {V}$ -info remains sensitive in regimes where discrimination performance approaches saturation. In addition, $\mathcal {V}$ -info is readily applicable to multi-class ( $\gt {2}$ ) tasks where ROC analysis is less natural. These findings suggest that $\mathcal {V}$ -info can serve as a complementary task-based image quality measure alongside traditional signal detection theory-based metrics.

Histopathology whole-slide images (WSI) capture detailed structural and morphological features of tumor tissue, offering rich histological and molecular information. Deep learning (DL) methods have emerged to assist in automatically examining histopathology WSIs and supporting tumor classification. Traditional DL approaches for WSI images face challenges due to the intrinsic complexity of tumor tissue characteristics and the extremely large image size. Multiple instance learning (MIL) methods have been proposed to address these issues by splitting the WSI images into small non-overlapping tiles and aggregating predictions from selected informative tiles for the final classification outcome. However, MIL methods still face challenges such as the need for accurate pseudo-labels, the risk of losing local information, or the failure to learn explicit class-relevant information. To address these limitations, we propose a novel framework that uses a lightweight convolutional neural network (CNN)-based tile encoder (CTE) to extract local tile features and a Transformer-based feature aggregator (TFA) to fuse local features into a representative global feature for WSI classification. Three key contributions of our framework are as follows. Firstly, we design a two-stage training strategy that decouples a lightweight CTE pre-training (using sparsely sampled tiles for efficiency) and TFA fine-tuning (using all tiles for accuracy). It significantly reduces computational costs compared to existing MIL methods while alleviating local information loss. Secondly, dynamic self-attention-based aggregation is designed in TFA, leveraging the Transformer's self-attention to weigh all local tile features without accurate pseudo-labels. It ensures comprehensive integration of local information from both tumor and ambiguous non-tumor regions to enrich global representations from input WSIs, which benefits the final classification performance. Finally, interpretable saliency maps are generated from TFA attention scores, highlighting histopathologically relevant regions to align model decisions with clinical reasoning. Comprehensive experiments on three cancer subtype datasets demonstrate the effectiveness of our proposed method over existing MIL approaches. We also conduct further investigations into the impacts of various factors on model performance, gaining in-depth insights into our method. Our framework achieves higher classification accuracy while maintaining computational efficiency, making it a promising tool for histopathology image analysis.

Diffusion models have emerged as a popular family of deep generative models (DGMs). In the literature, it has been claimed that one class of diffusion models-denoising diffusion probabilistic models (DDPMs)-demonstrate superior image synthesis performance as compared to generative adversarial networks (GANs). To date, these claims have been evaluated using either ensemble-based methods designed for natural images, or conventional measures of image quality such as structural similarity. However, there remains an important need to understand the extent to which DDPMs can reliably learn medical imaging domain-relevant information, which is referred to as 'spatial context' in this work. To address this, a systematic assessment of the ability of DDPMs to learn spatial context relevant to medical imaging applications is reported for the first time. A key aspect of the studies is the use of stochastic context models (SCMs) to produce training data. In this way, the ability of the DDPMs to reliably reproduce spatial context can be quantitatively assessed by use of post-hoc image analyses. Error-rates in DDPM-generated ensembles are reported, and compared to those corresponding to other modern DGMs. The studies reveal new and important insights regarding the capacity of DDPMs to learn spatial context. Notably, the results demonstrate that DDPMs hold significant capacity for generating contextually correct images that are 'interpolated' between training samples, which may benefit data-augmentation tasks in ways that GANs cannot.

The ideal observer (IO) sets an upper performance limit among all observers and has been advocated for assessing and optimizing imaging systems. For general joint detection and estimation (detection-estimation) tasks, estimation ROC (EROC) analysis has been established for evaluating the performance of observers. However, in general, it is difficult to accurately approximate the IO that maximizes the area under the EROC curve. In this study, a hybrid method that employs machine learning is proposed to accomplish this. Specifically, a hybrid approach is developed that combines a multi-task convolutional neural network and a Markov-Chain Monte Carlo (MCMC) method in order to approximate the IO for detection-estimation tasks. Unlike traditional MCMC methods, the hybrid method is not limited to use of specific utility functions. In addition, a purely supervised learning-based sub-ideal observer is proposed. Computer-simulation studies are conducted to validate the proposed method, which include signal-known-statistically/background-known-exactly and signal-known-statistically/background-known-statistically tasks. The EROC curves produced by the proposed method are compared to those produced by the MCMC approach or analytical computation when feasible. The proposed method provides a new approach for approximating the IO and may advance the application of EROC analysis for optimizing imaging systems.

MOTIVATION: Predicting early in treatment whether a tumor is likely to respond to treatment is one of the most difficult yet important tasks in providing personalized cancer care. Most oropharyngeal squamous cell carcinoma (OPSCC) patients receive standard cancer therapy. However, the treatment outcomes vary significantly and are difficult to predict. Multiple studies indicate that microRNAs (miRNAs) are promising cancer biomarkers for the prognosis of oropharyngeal cancer. The reliable and efficient use of miRNAs for patient stratification and treatment outcome prognosis is still a very challenging task, mainly due to the relatively high dimensionality of miRNAs compared to the small number of observation sets; the redundancy, irrelevancy and uncertainty in the large amount of miRNAs; and the imbalanced observation patient samples. RESULTS: In this study, a new machine learning-based prognosis model was proposed to stratify subsets of OPSCC patients with low and high risks for treatment failure. The model cascaded a two-stage prognostic biomarker selection method and an evidential K-nearest neighbors classifier to address the challenges and improve the accuracy of patient stratification. The model has been evaluated on miRNA expression profiling of 150 oropharyngeal tumors by use of overall survival and disease-specific survival as the end points of disease treatment outcomes, respectively. The proposed method showed superior performance compared to other advanced machine-learning methods in terms of common performance quantification metrics. The proposed prognosis model can be employed as a supporting tool to identify patients who are likely to fail standard therapy and potentially benefit from alternative targeted treatments. Availability and implementation: Code is available in https://github.com/shenghh2015/mRMR-BFT-outcome-prediction.

A variety of deep neural network (DNN)-based image denoising methods have been proposed for use with medical images. Traditional measures of image quality (IQ) have been employed to optimize and evaluate these methods. However, the objective evaluation of IQ for the DNN-based denoising methods remains largely lacking. In this work, we evaluate the performance of DNN-based denoising methods by use of task-based IQ measures. Specifically, binary signal detection tasks under signal-known-exactly (SKE) with background-known-statistically (BKS) conditions are considered. The performance of the ideal observer (IO) and common linear numerical observers are quantified and detection efficiencies are computed to assess the impact of the denoising operation on task performance. The numerical results indicate that, in the cases considered, the application of a denoising network can result in a loss of task-relevant information in the image. The impact of the depth of the denoising networks on task performance is also assessed. The presented results highlight the need for the objective evaluation of IQ for DNN-based denoising technologies and may suggest future avenues for improving their effectiveness in medical imaging applications.

Medical imaging systems are commonly assessed and optimized by use of objective measures of image quality (IQ). The Ideal Observer (IO) performance has been advocated to provide a figure-of-merit for use in assessing and optimizing imaging systems because the IO sets an upper performance limit among all observers. When joint signal detection and localization tasks are considered, the IO that employs a modified generalized likelihood ratio test maximizes observer performance as characterized by the localization receiver operating characteristic (LROC) curve. Computations of likelihood ratios are analytically intractable in the majority of cases. Therefore, sampling-based methods that employ Markov-Chain Monte Carlo (MCMC) techniques have been developed to approximate the likelihood ratios. However, the applications of MCMC methods have been limited to relatively simple object models. Supervised learning-based methods that employ convolutional neural networks have been recently developed to approximate the IO for binary signal detection tasks. In this paper, the ability of supervised learning-based methods to approximate the IO for joint signal detection and localization tasks is explored. Both background-known-exactly and background-known-statistically signal detection and localization tasks are considered. The considered object models include a lumpy object model and a clustered lumpy model, and the considered measurement noise models include Laplacian noise, Gaussian noise, and mixed Poisson-Gaussian noise. The LROC curves produced by the supervised learning-based method are compared to those produced by the MCMC approach or analytical computation when feasible. The potential utility of the proposed method for computing objective measures of IQ for optimizing imaging system performance is explored.

Precise delineation of target tumor is a key factor to ensure the effectiveness of radiation therapy. While hybrid positron emission tomography-computed tomography (PET-CT) has become a standard imaging tool in the practice of radiation oncology, many existing automatic/semi-automatic methods still perform tumor segmentation on mono-modal images. In this paper, a co-clustering algorithm is proposed to concurrently segment 3D tumors in PET-CT images, considering that the two complementary imaging modalities can combine functional and anatomical information to improve segmentation performance. The theory of belief functions is adopted in the proposed method to model, fuse, and reason with uncertain and imprecise knowledge from noisy and blurry PET-CT images. To ensure reliable segmentation for each modality, the distance metric for the quantification of clustering distortions and spatial smoothness is iteratively adapted during the clustering procedure. On the other hand, to encourage consistent segmentation between different modalities, a specific context term is proposed in the clustering objective function. Moreover, during the iterative optimization process, clustering results for the two distinct modalities are further adjusted via a belief-functions-based information fusion strategy. The proposed method has been evaluated on a data set consisting of 21 paired PET-CT images for non-small cell lung cancer patients. The quantitative and qualitative evaluations show that our proposed method performs well compared with the state-of-the-art methods.

It is widely accepted that the optimization of medical imaging system performance should be guided by task-based measures of image quality (IQ). Task-based measures of IQ quantify the ability of an observer to perform a specific task, such as detection or estimation of a signal (e.g., a tumor). For binary signal detection tasks, the Bayesian Ideal Observer (IO) sets an upper limit of observer performance and has been advocated for use in optimizing medical imaging systems and data-acquisition designs. Except in special cases, the determination of the IO test statistic is analytically intractable. Markov-chain Monte Carlo (MCMC) techniques can be employed to approximate the IO detection performance, but their reported applications have been limited to relatively simple object models. In cases where the IO test statistic is difficult to compute, the Hotelling Observer (HO) can be employed. To compute the HO test statistic, potentially large covariance matrices must be accurately estimated and subsequently inverted, which can present computational challenges. This paper investigates the supervised learning-based methodologies for approximating the IO and HO test statistics. Convolutional neural networks (CNNs) and single-layer neural networks (SLNNs) are employed to approximate the IO and HO test statistics, respectively. The numerical simulations were conducted for both signal-known-exactly (SKE) and signal-known-statistically (SKS) signal detection tasks. The considered background models include the lumpy object model and the clustered lumpy object model. The measurement noise models considered are Gaussian, Laplacian, and mixed Poisson-Gaussian. The performances of the supervised learning methods are assessed via receiver operating characteristic (ROC) analysis, and the results are compared to those produced by the use of traditional numerical methods or analytical calculations when feasible. The potential advantages of the proposed supervised learning approaches for approximating the IO and HO test statistics are discussed.

Multi-label active learning for image classification has attracted great attention over recent years and a lot of relevant works are published continuously. However, there still remain some problems that need to be solved, such as existing multi-label active learning algorithms do not reflect on the cleanness of sample data and their ways on label correlation mining are defective. For one thing, sample data is usually contaminated in reality, which disturbs the estimation of data distribution and further hinders the model training. For another, previous approaches for label relationship exploration are purely based on the observed label distribution of an incomplete training set, which cannot provide sufficiently efficient information. To address these issues, we propose a novel adaptive low-rank multi-label active learning algorithm, called LRMAL. Specifically, we first use low-rank matrix recovery to learn an effective low-rank feature representation from the noisy data. In a subsequent sampling phase, we make use of its superiorities to evaluate the general informativeness of each unlabeled example-label pair. Based on an intrinsic mapping relation between the example space and the label space of a certain multi-label dataset, we recover the incomplete labels of a training set for a more comprehensive label correlation mining. Furthermore, to reduce the redundancy among the selected example-label pairs, we use a diversity measurement to diversify the sampled data. Finally, an effective sampling strategy is developed by integrating these two aspects of potential information with uncertainty based on an adaptive integration scheme. Experimental results demonstrate the effectiveness of our approach.

Most variational active contour models are designed to find local minima of data-dependent energy functionals with the hope that reasonable initial placement of the active contour will drive it toward a "desirable" local minimum as opposed to an undesirable configuration due to noise or complex image structure. As such, there has been much research into the design of complex region-based energy functionals that are less likely to yield undesirable local minima when compared to simpler edge-based energy functionals whose sensitivity to noise and texture is significantly worse. Unfortunately, most of these more "robust" region-based energy functionals are applicable to a much narrower class of imagery compared to typical edge-based energies due to stronger global assumptions about the underlying image data. Devising new implementation algorithms for active contours that attempt to capture more global minimizers of already proposed image-based energies would allow us to choose an energy that makes sense for a particular class of energy without concern over its sensitivity to local minima. Such implementations have been proposed for capturing global minima. However, sometimes the completely-global minimum is just as undesirable as a minimum that is too local. In this paper, we propose a novel, fast, and flexible dual front implementation of active contours, motivated by minimal path techniques and utilizing fast sweeping algorithms, which is easily manipulated to yield minima with variable "degrees" of localness and globalness. By simply adjusting the size of active regions, the ability to gracefully move from capturing minima that are more local (according to the initial placement of the active contour/surface) to minima that are more global allows this model to more easily obtain "desirable" minimizers (which often are neither the most local nor the most global). Experiments on various 2D and 3D images and comparisons with some active contour models and region-growing methods are also given to illustrate the properties of this model and its performance in a variety of segmentation applications.

In this paper, we propose an innovative approach to the segmentation of tubular structures. This approach combines all of the benefits of minimal path techniques such as global minimizers, fast computation, and powerful incorporation of user input, while also having the capability to represent and detect vessel surfaces directly which so far has been a feature restricted to active contour and surface techniques. The key is to represent the trajectory of a tubular structure not as a 3-D curve but to go up a dimension and represent the entire structure as a 4-D curve. Then we are able to fully exploit minimal path techniques to obtain global minimizing trajectories between two user supplied endpoints in order to reconstruct tubular structures from noisy or low contrast 3-D data without the sensitivity to local minima inherent in most active surface techniques. In contrast to standard purely spatial 3-D minimal path techniques, however, we are able to represent a full tubular surface rather than just a curve which runs through its interior. Our representation also yields a natural notion of a tube's "central curve." We demonstrate and validate the utility of this approach on magnetic resonance (MR) angiography and computed tomography (CT) images of coronary arteries.

In this paper, a hybrid medical image segmentation approach is proposed based on a dual front evolution and fast sweeping evolution. This approach is composed of two stages. In the first stage, a fast sweeping evolution with a stopping criterion based upon gradient information is adopted to give a fast and rough initial boundary estimate close to (or overlapping) the actual boundary. Next, a morphological dilation is used to expand this boundary to a narrow region large enough to contain the actual boundary. In the second stage, a dual front evolution model is used to refine the final segmentation result. In this step, the evolution speeds consider the gradient information together with less local image statistics to improve the veracity and compatibility of the algorithm. The experimental results show that this two-stage algorithm can provide close, smooth and accurate final contours with low computational complexity O(N).

Many practical applications in the field of medical image processing require robust and valid 3D image segmentation results. In this paper, we present a semi-automatic iterative segmentation approach for 3D medical image by combining a 2D boundary tracking algorithm and a boundary mapping process. Upon each of the consecutive slice, the boundary tracking process is accomplished in an alternate procedure of the morphological dilatation and the multi-label front propagation. The multi-label front propagation method is developed based on the minimal path theory and fast sweeping evolution method to ensure the efficiency, and speed of the boundary tracking algorithm. This 3D image segmentation approach can easily extract the close and smooth boundary of the desired object from a 2D medical image series. This approach is efficient and reliable, and requires very limited user intervention. Some experimental results are also presented to demonstrate the efficiency of this approach.