The future of artificial intelligence (AI) safety is expected to include bias mitigation methods from development to application. The complexity and integration of these methods could grow in conjunction with advances in AI and human-AI interactions. Numerous methods are being proposed to mitigate bias, but without a structured way to compare their strengths and weaknesses. In this work, we present two approaches to systematically amplify subgroup performance bias. These approaches allow for the evaluation and comparison of the effectiveness of bias mitigation methods on AI models by varying the degrees of bias, and can be applied to any classification model. We used these approaches to compare four off-the-shelf bias mitigation methods. Both amplification approaches promote the development of learning shortcuts in which the model forms associations between patient attributes and AI output. We demonstrate these approaches in a case study, evaluating bias in the determination of COVID status from chest x-rays. The maximum achieved increase in performance bias, measured as a difference in predicted prevalence, was 72% and 32% for bias between subgroups related to patient sex and race, respectively. These changes in predicted prevalence were not accompanied by substantial changes in the differences in subgroup area under the receiver operating characteristic curves, indicating that the increased bias is due to the formation of learning shortcuts, not a difference in ability to distinguish positive and negative patients between subgroups.

In a well-calibrated risk prediction model, the average predicted probability is close to the true event rate for any given subgroup. Such models are reliable across heterogeneous populations and satisfy strong notions of algorithmic fairness. However, the task of auditing a model for strong calibration is well-known to be difficult—particularly for machine learning (ML) algorithms—due to the sheer number of potential subgroups. As such, common practice is to only assess calibration with respect to a few predefined subgroups. Recent developments in goodness-of-fit testing offer potential solutions but are not designed for settings with weak signal or where the poorly calibrated subgroup is small, as they either overly subdivide the data or fail to divide the data at all. We introduce a new testing procedure based on the following insight: if we can reorder observations by their expected residuals, there should be a change in the association between the predicted and observed residuals along this sequence if a poorly calibrated subgroup exists. This lets us reframe the problem of calibration testing into one of changepoint detection, for which powerful methods already exist. We begin with introducing a sample-splitting procedure where a portion of the data is used to train a suite of candidate models for predicting the residual, and the remaining data are used to perform a score-based cumulative sum (CUSUM) test. To further improve power, we then extend this adaptive CUSUM test to incorporate cross-validation, while maintaining Type I error control under minimal assumptions. Compared to existing methods, the proposed procedure consistently achieved higher power in empirical analyses.

Performance monitoring of machine learning (ML)-based risk prediction models in healthcare is complicated by the issue of performativity: when an algorithm predicts a patient to be at high risk for an adverse event, clinicians are more likely to administer prophylactic treatment and alter the very target that the algorithm aims to predict. A simple approach is to ignore performativity and monitor only the untreated patients, whose outcomes remain unaltered. In general, ignoring performativity may inflate Type I error because (i) untreated patients disproportionally represent those with low predicted risk, and (ii) changes in the clinician’s trust in the ML algorithm and the algorithm itself can induce complex dependencies that violate standard assumptions. Nevertheless, we show that valid inference is still possible when monitoring \textit{conditional} rather than marginal performance measures under either the assumption of conditional exchangeability or time-constant selection bias. Finally, performativity can vary over time and induce nonstationarity in the data, which presents challenges for monitoring. To this end, we introduce a new score-based cumulative sum (CUSUM) monitoring procedure with dynamic control limits. Through extensive simulation studies, we study applications of the score-based CUSUM and how it is affected by various factors, including the efficiency of model updating procedures and the level of clinician trust. Finally, we apply the procedure to detect calibration decay of a risk model during the COVID-19 pandemic.

To generate evidence regarding the safety and efficacy of artificial intelligence (AI) enabled medical devices, AI models need to be evaluated on a diverse population of patient cases, some of which may not be readily available. We propose an evaluation approach for testing medical imaging AI models that relies on in silico imaging pipelines in which stochastic digital models of human anatomy (in object space) with and without pathology are imaged using a digital replica imaging acquisition system to generate realistic synthetic image datasets. Here, we release M-SYNTH, a dataset of cohorts with four breast fibroglandular density distributions imaged at different exposure levels using Monte Carlo x-ray simulations with the publicly available Virtual Imaging Clinical Trial for Regulatory Evaluation (VICTRE) toolkit. We utilize the synthetic dataset to analyze AI model performance and find that model performance decreases with increasing breast density and increases with higher mass density, as expected. As exposure levels decrease, AI model performance drops with the highest performance achieved at exposure levels lower than the nominal recommended dose for the breast type.

After initial release of a machine learning algorithm, the model can be fine-tuned by retraining on subsequently gathered data, adding newly discovered features, or more. Each modification introduces a risk of deteriorating performance and must be validated on a test dataset. It may not always be practical to assemble a new dataset for testing each modification, especially when most modifications are minor or are implemented in rapid succession. Recent work has shown how one can repeatedly test modifications on the same dataset and protect against overfitting by (i) discretizing test results along a grid and (ii) applying a Bonferroni correction to adjust for the total number of modifications considered by an adaptive developer. However, the standard Bonferroni correction is overly conservative when most modifications are beneficial and/or highly correlated. This work investigates more powerful approaches using alpha-recycling and sequentially-rejective graphical procedures (SRGPs). We introduce two novel extensions that account for correlation between adaptively chosen algorithmic modifications: the first leverages the correlation between consecutive modifications using flexible fixed sequence tests, and the second leverages the correlation between the proposed modifications and those generated by a hypothetical prespecified model updating procedure. In empirical analyses, both SRGPs control the error rate of approving deleterious modifications and approve significantly more beneficial modifications than previous approaches.

OBJECTIVE: After deploying a clinical prediction model, subsequently collected data can be used to fine-tune its predictions and adapt to temporal shifts. Because model updating carries risks of over-updating/fitting, we study online methods with performance guarantees. MATERIALS AND METHODS: We introduce 2 procedures for continual recalibration or revision of an underlying prediction model: Bayesian logistic regression (BLR) and a Markov variant that explicitly models distribution shifts (MarBLR). We perform empirical evaluation via simulations and a real-world study predicting Chronic Obstructive Pulmonary Disease (COPD) risk. We derive "Type I and II" regret bounds, which guarantee the procedures are noninferior to a static model and competitive with an oracle logistic reviser in terms of the average loss. RESULTS: Both procedures consistently outperformed the static model and other online logistic revision methods. In simulations, the average estimated calibration index (aECI) of the original model was 0.828 (95%CI, 0.818-0.938). Online recalibration using BLR and MarBLR improved the aECI towards the ideal value of zero, attaining 0.265 (95%CI, 0.230-0.300) and 0.241 (95%CI, 0.216-0.266), respectively. When performing more extensive logistic model revisions, BLR and MarBLR increased the average area under the receiver-operating characteristic curve (aAUC) from 0.767 (95%CI, 0.765-0.769) to 0.800 (95%CI, 0.798-0.802) and 0.799 (95%CI, 0.797-0.801), respectively, in stationary settings and protected against substantial model decay. In the COPD study, BLR and MarBLR dynamically combined the original model with a continually refitted gradient boosted tree to achieve aAUCs of 0.924 (95%CI, 0.913-0.935) and 0.925 (95%CI, 0.914-0.935), compared to the static model's aAUC of 0.904 (95%CI, 0.892-0.916). DISCUSSION: Despite its simplicity, BLR is highly competitive with MarBLR. MarBLR outperforms BLR when its prior better reflects the data. CONCLUSIONS: BLR and MarBLR can improve the transportability of clinical prediction models and maintain their performance over time.

Deep two-dimensional (2-D) convolutional neural networks (CNNs) have been remarkably successful in producing record-breaking results in a variety of computer vision tasks. It is possible to extend CNNs to three dimensions using 3-D kernels to make them suitable for volumetric medical imaging data such as CT or MRI, but this increases the processing time as well as the required number of training samples (due to the higher number of parameters that need to be learned). In this paper, we address both of these issues for a 3-D CNN implementation through the development of a two-stage computer-aided detection system for automatic detection of pulmonary nodules. The first stage consists of a 3-D fully convolutional network for fast screening and generation of candidate suspicious regions. The second stage consists of an ensemble of 3-D CNNs trained using extensive transformations applied to both the positive and negative patches to augment the training set. To enable the second stage classifiers to learn differently, they are trained on false positive patches obtained from the screening model using different thresholds on their associated scores as well as different augmentation types. The networks in the second stage are averaged together to produce the final classification score for each candidate patch. Using this procedure, our overall nodule detection system called DeepMed is fast and can achieve 91% sensitivity at 2 false positives per scan on cases from the LIDC dataset.

The performance of a classifier is largely dependent on the size and representativeness of data used for its training. In circumstances where accumulation and/or labeling of training samples is difficult or expensive, such as medical applications, data augmentation can potentially be used to alleviate the limitations of small datasets. We have previously developed an image blending tool that allows users to modify or supplement an existing CT or mammography dataset by seamlessly inserting a lesion extracted from a source image into a target image. This tool also provides the option to apply various types of transformations to different properties of the lesion prior to its insertion into a new location. In this study, we used this tool to create synthetic samples that appear realistic in chest CT. We then augmented different size training sets with these artificial samples, and investigated the effect of the augmentation on training various classifiers for the detection of lung nodules. Our results indicate that the proposed lesion insertion method can improve classifier performance for small training datasets, and thereby help reduce the need to acquire and label actual patient data.

In this paper, we present development and testing results for a novel colonic polyp classification method for use as part of a computed tomographic colonography (CTC) computer-aided detection (CAD) system. Inspired by the interpretative methodology of radiologists using 3-D fly-through mode in CTC reading, we have developed an algorithm which utilizes sequences of images (referred to here as videos) for classification of CAD marks. For each CAD mark, we created a video composed of a series of intraluminal, volume-rendered images visualizing the detection from multiple viewpoints. We then framed the video classification question as a multiple-instance learning (MIL) problem. Since a positive (negative) bag may contain negative (positive) instances, which in our case depends on the viewing angles and camera distance to the target, we developed a novel MIL paradigm to accommodate this class of problems. We solved the new MIL problem by maximizing a L2-norm soft margin using semidefinite programming, which can optimize relevant parameters automatically. We tested our method by analyzing a CTC data set obtained from 50 patients from three medical centers. Our proposed method showed significantly better performance compared with several traditional MIL methods.

Classification of a given observation to one of three classes is an important task in many decision processes or pattern recognition applications. A general analysis of the performance of three-class classifiers results in a complex 6-D receiver operating characteristic (ROC) space, for which no simple analytical tool exists at present. We investigate the performance of an ideal observer under a specific set of assumptions that reduces the 6-D ROC space to 3-D by constraining the utilities of some of the decisions in the classification task. These assumptions lead to a 3-D ROC space in which the true-positive fraction (TPF) can be expressed in terms of the two types of false-positive fractions (FPFs). We demonstrate that the TPF is uniquely determined by, and therefore is a function of, the two FPFs. The domain of this function is shown to be related to the decision boundaries in the likelihood ratio plane. Based on these properties of the 3-D ROC space, we can define a summary measure, referred to as the normalized volume under the surface (NVUS), that is analogous to the area under the ROC curve (AUC) for a two-class classifier. We further investigate the properties of the 3-D ROC surface and the NVUS for the ideal observer under the condition that the three class distributions are multivariate normal with equal covariance matrices. The probability density functions (pdfs) of the decision variables are shown to follow a bivariate log-normal distribution. By considering these pdfs, we express the TPF in terms of the FPFs, and integrate the TPF over its domain numerically to obtain the NVUS. In addition, we performed a Monte Carlo simulation study, in which the 3-D ROC surface was generated by empirical "optimal" classification of case samples in the multidimensional feature space following the assumed distributions, to obtain an independent estimate of NVUS. The NVUS value obtained by using the analytical pdfs was found to be in good agreemen- t with that obtained from the Monte Carlo simulation study. We also found that, under all conditions studied, the NVUS increased when the difficulty of the classification task was reduced by changing the parameters of the class distributions, thereby exhibiting the properties of a performance metric in analogous to AUC. Our results indicate that, under the conditions that lead to our 3-D ROC analysis, the performance of a three-class classifier may be analyzed by considering the ROC surface, and its accuracy characterized by the NVUS.

In a practical classifier design problem, the sample size is limited, and the available finite sample needs to be used both to design a classifier and to predict the classifier's performance for the true population. Since a larger sample is more representative of the population, it is advantageous to design the classifier with all the available cases, and to use a resampling technique for performance prediction. We conducted a Monte-Carlo simulation study to compare the ability of different resampling techniques in predicting the performance of a neural network classifier designed with the available sample. We investigated a technique based on the cross-validation, the leave-one-out method, and three different types of bootstrapping, namely, the ordinary, .632, and .632+ bootstrap techniques. Our results indicated that, under the study conditions, there can be a large difference in the accuracy of the prediction obtained from different resampling methods, especially when the feature space dimensionality is relatively large and the sample size is small. Under this type of conditions, the .632 and .632+ bootstrap methods were superior to the other techniques studied. Although this investigation is performed under some specific conditions, it reveals important trends for the problem of classifier performance prediction under the constraint of a limited data set.

Mass segmentation is used as the first step in many computer-aided diagnosis (CAD) systems for classification of breast masses as malignant or benign. The goal of this paper was to study the accuracy of an automated mass segmentation method developed in our laboratory, and to investigate the effect of the segmentation stage on the overall classification accuracy. The automated segmentation method was quantitatively compared with manual segmentation by two expert radiologists (R1 and R2) using three similarity or distance measures on a data set of 100 masses. The area overlap measures between R1 and R2, the computer and R1, and the computer and R2 were 0.76 +/- 0.13, 0.74 +/- 0.11, and 0.74 +/- 0.13, respectively. The interobserver difference in these measures between the two radiologists was compared with the corresponding differences between the computer and the radiologists. Using three similarity measures and data from two radiologists, a total of six statistical tests were performed. The difference between the computer and the radiologist segmentation was significantly larger than the interobserver variability in only one test. Two sets of texture, morphological, and spiculation features, one based on the computer segmentation, and the other based on radiologist segmentation, were extracted from a data set of 249 films from 102 patients. A classifier based on stepwise feature selection and linear discriminant analysis was trained and tested using the two feature sets. The leave-one-case-out method was used for data sampling. For case-based classification, the area Az under the receiver operating characteristic (ROC) curve was 0.89 and 0.88 for the feature sets based on the radiologist segmentation and computer segmentation, respectively. The difference between the two ROC curves was not statistically significant.

A new type of classifier combining an unsupervised and a supervised model was designed and applied to classification of malignant and benign masses on mammograms. The unsupervised model was based on an adaptive resonance theory (ART2) network which clustered the masses into a number of separate classes. The classes were divided into two types: one containing only malignant masses and the other containing a mix of malignant and benign masses. The masses from the malignant classes were classified by ART2. The masses from the mixed classes were input to a supervised linear discriminant classifier (LDA). In this way, some malignant masses were separated and classified by ART2 and the less distinguishable benign and malignant masses were classified by LDA. For the evaluation of classifier performance, 348 regions of interest (ROI's) containing biopsy proven masses (169 benign and 179 malignant) were used. Ten different partitions of training and test groups were randomly generated using an average of 73% of ROI's for training and 27% for testing. Classifier design, including feature selection and weight optimization, was performed with the training group. The test group was kept independent of the training group. The performance of the hybrid classifier was compared to that of an LDA classifier alone and a backpropagation neural network (BPN). Receiver operating characteristics (ROC) analysis was used to evaluate the accuracy of the classifiers. The average area under the ROC curve (A(z)) for the hybrid classifier was 0.81 as compared to 0.78 for the LDA and 0.80 for the BPN. The partial areas above a true positive fraction of 0.9 were 0.34, 0.27 and 0.31 for the hybrid, the LDA and the BPN classifier, respectively. These results indicate that the hybrid classifier is a promising approach for improving the accuracy of classification in CAD applications.

Sahiner and Yagle (1992, 1993, 1995) developed a procedure for image reconstruction from projections under wavelet-domain image constraints (WC). We constrain the wavelet transform (WT) of the noisy image to zero in WT regions determined by thresholding the WT of the noisy image. The least-squares image satisfying these constraints shows improved reconstruction not just in the constrained region, but over the entire image. This amounts to image-constrained localized low pass filtering. We now present new results on iterative implementation of this technique. At each iteration we use the image from the previous iteration as the source of WC for the next iteration. This can be viewed as iterative spatially-varying image-constrained Wiener filtering. At the third iteration the RMS error is reduced by more than twice as much as in our previous results, while preserving high-resolution image features.

Presents a novel approach for segmentation of suspicious mass regions in digitized mammograms using a new adaptive density-weighted contrast enhancement (DWCE) filter in conjunction with Laplacian-Gaussian (LG) edge detection. The DWCE enhances structures within the digitized mammogram so that a simple edge detection algorithm can be used to define the boundaries of the objects. Once the object boundaries are known, morphological features are extracted and used by a classification algorithm to differentiate regions within the image. This paper introduces the DWCE algorithm and presents results of a preliminary study based on 25 digitized mammograms with biopsy proven masses. It also compares morphological feature classification based on sequential thresholding, linear discriminant analysis, and neural network classifiers for reduction of false-positive detections.

The authors investigated the classification of regions of interest (ROI's) on mammograms as either mass or normal tissue using a convolution neural network (CNN). A CNN is a backpropagation neural network with two-dimensional (2-D) weight kernels that operate on images. A generalized, fast and stable implementation of the CNN was developed. The input images to the CNN were obtained from the ROI's using two techniques. The first technique employed averaging and subsampling. The second technique employed texture feature extraction methods applied to small subregions inside the ROI. Features computed over different subregions were arranged as texture images, which were subsequently used as CNN inputs. The effects of CNN architecture and texture feature parameters on classification accuracy were studied. Receiver operating characteristic (ROC) methodology was used to evaluate the classification accuracy. A data set consisting of 168 ROIs containing biopsy-proven masses and 504 ROI's containing normal breast tissue was extracted from 168 mammograms by radiologists experienced in mammography. This data set was used for training and testing the CNN. With the best combination of CNN architecture and texture feature parameters, the area under the test ROC curve reached 0.87, which corresponded to a true-positive fraction of 90% at a false positive fraction of 31%. The authors' results demonstrate the feasibility of using a CNN for classification of masses and normal tissue on mammograms.

Combines several ideas, including nonuniform sampling and circular harmonic expansions, into a new procedure for reconstructing a small region of interest (ROI) of an image from a set of its projections that are densely sampled in the ROI and coarsely sampled outside the ROI. Specifically, the radial sampling density of both the projections and the reconstructed image decreases exponentially with increasing distance from the ROI. The problem and data are reminiscent of the local tomography problem; however, the present algorithm reconstructs the ROI of the image itself, not the filtered version of it obtained using local tomography. The new algorithm has the added advantages of speed (it can be implemented entirely using the FFT) and parallelizability (each image harmonic is independent).

A convolution neural network (CNN) was used for classification of masses and normal tissue on mammograms. A generalized CNN was developed that uses multiple images derived from a single region of interest (ROI) as the input. The CNN input images were obtained from the ROIs using (i) averaging and subsampling; and (ii) texture feature extraction methods on smaller sub-regions inside the ROI. In (ii), features computed over different sub-regions were arranged as texture-images, and subsequently used as inputs to the CNN. The results indicate that using texture-images improves the classification accuracy.

The authors combine several ideas, including nonuniform sampling and circular harmonic expansions, into a new procedure for reconstructing a small region of interest (ROI) of an image from a set of its projections that are densely sampled in the ROI and coarsely sampled outside the ROI. Specifically, the radial sampling density of both the projections and the reconstructed image decreases exponentially with increasing distance from the ROI. The problem and data are reminiscent of the recently formulated local tomography problem; however, the authors' algorithm reconstructs the ROI of the image itself, not the filtered version of it obtained using local tomography. The new algorithm has the added advantages of speed (it can be implemented entirely using the FFT) and parallelizability (each image harmonic is computed independently). Numerical examples compare the new algorithm to filtered backprojection.

Low-pass filtering computed tomography (CT) images to reduce noise may smooth or modify image features which are very important to the physician. Image features are often more easily identified and processed in the time-frequency plane. The authors use time-frequency distributions for spatially varying filtering of noisy CT images, constraining time-frequency representation coefficients of the projection data or of the reconstructed image to be zero in certain regions of the time-frequency plane. The authors consider two different applications: 1) filtering the projection data and then performing image reconstruction; and 2) filtering the reconstructed image directly. Criteria minimized, subject to constraints, may be either a deterministic minimum weighted perturbation of the given projection data or a stochastic minimum mean-square error in colored Gaussian noise. Results show improvement over processing the image with a linear spatially invariant filter.

In using filtered backprojection to compute the inverse Radon transform, the ramp filter amplifies noise. Spatially invariant noise filters reduce resolution. It is desirable to filter noise where projections have no local high-frequency components. Using the short-time Fourier transform, the authors apply a time-frequency mask filter that zeroes out projections where local signal energy is below a threshold. Results show improvement over reconstructions using spatially invariant smoothing filters.

In the filtered backprojection procedure for image reconstruction from projections, backprojection dominates the computation time. A simple algorithm that reduces the number of multiplications in linear interpolation and backprojection stage by 50%, with a small increase in the number of additions, is proposed. The algorithm performs the interpolation and backprojection of four views together. Examples of implementation are given and extension to interpolation of more than four views is discussed.

The exponential distribution (ED) developed by H.-I. Choi and W.J. Williams (see IEEE Trans. Acoust. Speech, Sig. Proc., ASSP-37, no.6, p.862-71, 1989) is applied to the problem of simultaneous determination of the velocity and spin density of a moving object in magnetic resonance imaging (MRI). The object velocity is assumed to be an affine function of location (reasonable for selectively imaged thin slice) and is determined from the trajectory of the maxima of the time-frequency distribution. Once the velocity is computed, its effects can be deconvolved from the MRI signal. This improves on previous results which require prior knowledge of either the velocity or discontinuities in the spin density. Numerical examples are presented.>