Masked diffusion language models (MDMs) have recently gained traction as a viable generative framework for natural language. This can be attributed to its scalability and ease of training compared to other diffusion model paradigms for discrete data, establishing itself as the state-of-the-art non-autoregressive generator for discrete data. Diffusion models, in general, have shown excellent ability to improve the generation quality by leveraging inference-time scaling either by increasing the number of denoising steps or by using external verifiers on top of the outputs of each step to guide the generation. In this work, we propose a verifier-based inference-time scaling method that aids in finding a better candidate generation during the denoising process of the MDM. Our experiments demonstrate the application of MDMs for standard text-style transfer tasks and establish MDMs as a better alternative to autoregressive language models. Additionally, we show that a simple soft-value-based verifier setup for MDMs using off-the-shelf pre-trained embedding models leads to significant gains in generation quality even when used on top of typical classifier-free guidance setups in the existing literature.

For object segmentation in medical images, deep neural networks (DNNs) typically perform poorly on out-of-distribution (OOD) images stemming from the large variability in image-acquisition equipment and protocols across sites. However, we observe that the variability in the underlying object-segmentation maps is far lower. Thus, we propose a novel DNN framework to model this variability in segmentation maps, and leverage it to revive poor segmentations produced by existing DNNs on OOD images. Our DNN framework (i) learns the principal modes of variation in a class of segmentation maps, (ii) models each segmentation map using a low-dimensional mixture-of-modes latent representation on a simplex, (iii) enables sampling-free variational learning and uncertainty estimation, and (iv) trains using small in-distribution image sets. In special cases when OOD-image segmentations are extremely poor, we propose a human-in-the-loop method needing minuscule human intervention. Results using 6 publicly-available datasets and 8 existing DNN segmenters show the benefits of our framework in OOD-image object segmentation.

Biological image segmentation often involves an object of interest that naturally exhibits a specific property about its geometry or shape, e.g., convexity. While typical deep-neural-networks (DNNs) for object segmentation ignore object properties relating to geometry/shape, the DNNs that employ shape information fail to enforce hard constraints on geometry/shape. We design a brand-new DNN framework that guarantees convexity of the output object-segment by leveraging fundamental geometrical insights into the boundaries of convex-shaped objects. Moreover, we design our framework to build on typical existing DNNs for per-pixel segmentation, while adding little overhead during training. Empirical evaluation on two publicly available datasets demonstrates that our framework provides significant improvements in the robust segmentation of convex objects in out-of-distribution images.

For image-quality enhancement, typical deep neural networks (DNNs) use large training sets and full supervision, but they generalize poorly to out-of-distribution (OOD) images exhibiting degradations absent during training. Also, having pairs of corresponding images at the desired quality and low quality becomes infeasible in many scenarios in medical image analysis. We propose a novel adversarial-learning framework for DNN-based image-quality enhancement which also incorporates variational modeling in latent space using expectation maximization (EM). Our EM framework extends to partially supervised learning that relaxes the quality requirement for reference images-used for DNN-loss computation during training-to a range in between the input/low quality and the desired/high quality. Results on two public datasets of positron-emission tomography show our framework’s benefits in generalizing to OOD images and visualizing DNN-output uncertainty while learning without full supervision.

Some key applications in medical image segmentation involve object complexes having specific topologies, but typical deep neural networks (DNNs) ignore such topologies. We propose a novel DNN framework to model topology-constrained object boundaries, incorporating both individual-object and multi-object topology constraints. Unlike typical DNNs, our topology-constrained DNN makes the learning significantly more robust to out-of-distribution images. Moreover, our DNN combines variational modeling in latent-space with uncertainty modeling of boundary points along with inter-point correlations. Results on publicly available datasets show our framework to outperform existing methods.

Abnormality detection is a one-class classification (OCC) problem where the methods learn either a generative model of the inlier class (e.g., in the variants of kernel principal component analysis) or a decision boundary to encapsulate the inlier class (e.g., in the one-class variants of the support vector machine). Learning schemes for OCC typically train on data solely from the inlier class, but some recent OCC methods have proposed semi-supervised extensions that also leverage a small amount of training data from outlier classes. Other recent methods extend existing principles to employ deep neural network (DNN) models for learning (for the inlier class) either latent-space distributions or autoencoders, but not both. We propose a semi-supervised variational formulation, leveraging generalized-Gaussian (GG) models leading to data-adaptive, robust, and uncertainty-aware distribution modeling in both latent space and image space. We propose a reparameterization for sampling from the latent-space GG to enable backpropagation-based optimization. Results on many publicly available real-world image sets and a synthetic image set show the benefits of our method over existing methods.

Deep neural networks (DNNs) for nonlinear generative mixture modeling typically rely on unsupervised learning that employs hard clustering schemes, or variational learning with loose / approximate bounds, or under-regularized modeling. We propose a novel statistical framework for a DNN mixture model using a single generative adversarial network. Our learning formulation proposes a novel data-likelihood term relying on a well-regularized / constrained Gaussian mixture model in the latent space along with a prior term on the DNN weights. Our min-max learning increases the data likelihood using a tight variational lower bound using expectation maximization (EM). We leverage our min-max EM learning scheme for semi-supervised learning. Results on three real-world image datasets demonstrate the benefits of our compact modeling and learning formulation over the state of the art for nonlinear generative image (mixture) modeling and image clustering.

Current approaches for semantic image inpainting rely on deep neural networks (DNNs) that learn under full supervision, i.e., using a training set comprising pairs of (i) corrupted images with holes and (ii) corresponding uncorrupted images. However, for several real-world applications, obtaining large sets of uncorrupted images is challenging or infeasible. Current methods also rely on adversarial training involving min-max optimization that is prone to instability during learning. We propose a novel self-supervised image-inpainting DNN framework that can learn in both completely unsupervised and semi-supervised modes. Moreover, our DNN learning formulation bypasses adversarial training and, thereby, lends itself to more stable training. Results on the publicly available CelebA dataset show that our method, even when learning unsupervisedly, outperforms the state of the art that learns with full supervision.

Abnormality detection in medical images is a one-class classification problem for which existing methods typically involve variants of kernel principal component analysis or one-class support vector machines. However, existing methods rely on highly-curated training sets with full supervision, often using heuristics for model fitting or ignore the variances of the data within principal subspaces. In contrast, we propose novel methods that can work with imperfectly curated datasets using robust statistical learning, by extending the multivariate generalized-Gaussian distribution to a reproducing kernel Hilbert space (RKHS) and employing it within a mixture model. We propose a novel semi-supervised extension of our learning scheme, showing that a small amount of expert feedback through high-quality labeled data of the outlier class can boost performance. We propose expectation maximization for our semi-supervised robust mixture-model learning in RKHS, using solely the Gram matrix and without the explicit lifting map. Our methods incorporate optimal component means, principal directions, and variances for abnormality detection. Results on four large public datasets on retinopathy and cancer, compared against a variety of contemporary methods, show that our method gives benefits over the state of the art in one-class classification for abnormality detection.

High-angular-resolution diffusion imaging (HARDI) relies on multicoil acquisitions for clinical applications. HARDI scan time can be reduced by undersampling the set of gradient directions. Typical methods for undersampled HARDI reconstruction use two-stage schemes that first take scanner-reconstructed magnitude images for the acquired directions, and then fit a model to the reconstructed under-sampled diffusion signals, where they assume a Gaussian noise model that behaves poorly for high b values or high noise. In contrast, we propose a novel Bayesian framework for undersampled-HARDI reconstruction that directly fits to multicoil data. We use magnitude images per coil and a Rician noise model to bypass complicated phase-related artifacts and accurately reconstruct at large b values. We use a sparse dictionary prior on the diffusion signal across directions, and a multiscale wavelet regularity on each diffusion weighted image. Results on simulated and clinical HARDI data show that our method improves over the state of the art.

In high-angular-resolution diffusion imaging (HARDI), simultaneous multislice (SMS) acquisition incorporated in multi-coil parallel imaging offers speedups in addition to the speedup obtained from undersampling gradient directions. We propose a novel learning-based method for reconstructing direction-undersampled SMS HARDI data. Our method relies on random-forest regression that also informs on the uncertainty in the reconstructions stemming from noise and artifacts. Results on a large clinical HARDI dataset show that our method significantly improves over the state of the art on SMS HARDI reconstruction qualitatively and quantitatively.

Abnormality detection in medical images is a one-class classification problem for which typical methods use variants of kernel principal component analysis or one-class support vector machines. However, in practical deployment scenarios, many such methods are sensitive to the outliers present in the imperfectly-curated training sets. Current robust methods use heuristics for model fitting or lack formulations to leverage even a small amount of high-quality expert feedback. In contrast, we propose a novel method combining (i) robust statistical modeling, extending the multivariate generalized-Gaussian to a reproducing kernel Hilbert space, with (ii) semi-supervised learning to leverage a small expert-labeled outlier set. Results on simulated and real-world data, including endoscopy data, show that our method outperforms the state of the art in accurately detecting abnormalities.

In this paper, we propose a new method to perform Sparse Kernel Principal Component Analysis (SKPCA) and also mathematically analyze the validity of SKPCA. We formulate SKPCA as a constrained optimization problem with elastic net regularization in kernel feature space and solve it. We consider outlier detection (where KPCA is employed) as an application for SKPCA, using the RBF kernel. We test it on 5 real world datasets and show that by using just 4% (or even less) of the principal components (PCs), where each PC has on average less than 12% non-zero elements in the worst case among all 5 datasets, we are able to nearly match and in 3 datasets even outperform KPCA. We also compare the performance of our method with a recently proposed method for SKPCA and show that our method performs better in terms of both accuracy and sparsity. We also provide a novel probabilistic proof to justify the existence of sparse solutions for KPCA using the RBF kernel. To the best of our knowledge, this is the first attempt at theoretically analyzing the validity of SKPCA.

Typical methods for abnormality detection in medical images, which is a one-class classification problem, rely on kernel principal component analysis (KPCA) and its robust invariants. However, typical methods for robust KPCA appear heuristical in nature and often ignore the variances of the data along the principal modes of variation. In this paper, we propose a novel method for robust statistical learning in a reproducing kernel Hilbert space (RKHS) that relies on our extension of the multivariate generalized Gaussian distribution to RKHS. We propose novel algorithms to fit our kernel generalized Gaussian (KGG) in RKHS, using solely the Gram matrix and without the explicit lifting map. We exploit the KGG model, including mean, principal directions, and variances, for abnormality detection in medical images. The results on two large publicly available retinopathy datasets show that our method outperforms the state of the art.

Identifying plant species based on photographs of their leaves is an important problem in computer vision and biology. Previous approaches for leaf image classification typically rely on hand-crafted shape features or texture features. In contrast, we propose a dual-path deep convolutional neural network (CNN) to (i) learn joint feature representations for leaf images, exploiting their shape and texture characteristics, and (ii) optimize these features for the classification task. We compare our CNN approach against (i) vanilla CNN classifiers and (ii) popular hand-crafted shape features, including a novel shape-context based feature that is extremely computationally efficient, which we call the marginalized shape context. Our results on three large public datasets demonstrate that our dual-path CNN leads to higher accuracy and consistency than the state of the art.

Kernel principal component analysis (kPCA) learns nonlinear modes of variation in the data by nonlinearly mapping the data to kernel feature space and performing (linear) PCA in the associated reproducing kernel Hilbert space (RKHS). However, several widely-used Mercer kernels map data to a Hilbert sphere in RKHS. For such directional data in RKHS, linear analyses can be unnatural or suboptimal. Hence, we propose an alternative to kPCA by extending principal nested spheres (PNS) to RKHS without needing the explicit lifting map underlying the kernel, but solely relying on the kernel trick. It generalizes the model for the residual errors by penalizing the Lpnorm / quasi-norm to enable robust learning from corrupted training data. Our method, termed robust kernel PNS (rkPNS), relies on the Riemannian geometry of the Hilbert sphere in RKHS. Relying on rkPNS, we propose novel algorithms for dimensionality reduction and classification (with and without outliers in the training data). Evaluation on real-world datasets shows that rkPNS compares favorably to the state of the art.

Statistical shape analysis has relied on various models, each with its strengths and limitations. For multigroup analyses, while typical methods pool data to fit a single statistical model, partial pooling through hierarchical modeling can be superior. For pointset shape representations, we propose a novel hierarchical model in Riemannian shape space . The inference treats individual shapes and group-mean shapes as latent variables, and uses expectation maximization that relies on sampling shapes. Our generative model, including shape-smoothness priors, can be robust to segmentation errors, producing more compact per-group models and realistic shape samples. We propose a method for efficient sampling in Riemannian shape space. The results show the benefits of our hierarchical Riemannian generative model for hypothesis testing, over the state of the art.

This paper proposes a novel theoretical framework to model and analyze the statistical characteristics of a wide range of segmentation methods that incorporate a database of label maps or atlases; such methods are termed as label fusion or multiatlas segmentation. We model these multiatlas segmentation problems as nonparametric regression problems in the high-dimensional space of image patches. We analyze the nonparametric estimator's convergence behavior that characterizes expected segmentation error as a function of the size of the multiatlas database. We show that this error has an analytic form involving several parameters that are fundamental to the specific segmentation problem (determined by the chosen anatomical structure, imaging modality, registration algorithm, and label-fusion algorithm). We describe how to estimate these parameters and show that several human anatomical structures exhibit the trends modeled analytically. We use these parameter estimates to optimize the regression estimator. We show that the expected error for large database sizes is well predicted by models learned on small databases. Thus, a few expert segmentations can help predict the database sizes required to keep the expected error below a specified tolerance level. Such cost-benefit analysis is crucial for deploying clinical multiatlas segmentation systems.

An effective approach to structure learning and parameter estimation for Gaussian graphical models is to impose a sparsity prior, such as a Laplace prior, on the entries of the precision matrix. Such an approach involves a hyperparameter that must be tuned to control the amount of sparsity. In this paper, we introduce a parameter-free method for estimating a precision matrix with sparsity that adapts to the data automatically. We achieve this by formulating a hierarchical Bayesian model of the precision matrix with a non-informative Jeffreys’ hyperprior. We also naturally enforce the symmetry and positive-definiteness constraints on the precision matrix by parameterizing it with the Cholesky decomposition. Experiments on simulated and real (cell signaling) data demonstrate that the proposed approach not only automatically adapts the sparsity of the model, but it also results in improved estimates of the precision matrix compared to the Laplace prior model with sparsity parameter chosen by cross-validation.

We introduce a labeled point set registration algorithm based on a family of novel information-theoretic measures derived as a generalization of the well-known Shannon entropy. This generalization, known as the Havrda-Charvat-Tsallis entropy, permits a fine-tuning between solution types of varying degrees of robustness of the divergence measure between multiple point sets. A variant of the traditional free-form deformation approach, known as directly manipulated free-form deformation, is used to model the transformation of the registration solution. We provide an overview of its open source implementation based on the Insight Toolkit of the National Institutes of Health. Characterization of the proposed framework includes comparison with other state of the art kernel-based methods and demonstration of its utility for lung registration via labeled point set representation of lung anatomy.

This paper presents a novel method for Bayesian denoising of magnetic resonance (MR) images that bootstraps itself by inferring the prior, i.e., the uncorrupted-image statistics, from the corrupted input data and the knowledge of the Rician noise model. The proposed method relies on principles from empirical Bayes (EB) estimation. It models the prior in a nonparametric Markov random field (MRF) framework and estimates this prior by optimizing an information-theoretic metric using the expectation-maximization algorithm. The generality and power of nonparametric modeling, coupled with the EB approach for prior estimation, avoids imposing ill-fitting prior models for denoising. The results demonstrate that, unlike typical denoising methods, the proposed method preserves most of the important features in brain MR images. Furthermore, this paper presents a novel Bayesian-inference algorithm on MRFs, namely iterated conditional entropy reduction (ICER). This paper also extends the application of the proposed method for denoising diffusion-weighted MR images. Validation results and quantitative comparisons with the state of the art in MR-image denoising clearly depict the advantages of the proposed method.

This paper presents a novel fuzzy-segmentation method for diffusion tensor (DT) and magnetic resonance (MR) images. Typical fuzzy-segmentation schemes, e.g., those based on fuzzy C means (FCM), incorporate Gaussian class models that are inherently biased towards ellipsoidal clusters characterized by a mean element and a covariance matrix. Tensors in fiber bundles, however, inherently lie on specific manifolds in Riemannian spaces. Unlike FCM-based schemes, the proposed method represents these manifolds using nonparametric data-driven statistical models. The paper describes a statistically-sound (consistent) technique for nonparametric modeling in Riemannian DT spaces. The proposed method produces an optimal fuzzy segmentation by maximizing a novel information-theoretic energy in a Markov-random-field framework. Results on synthetic and real, DT and MR images, show that the proposed method provides information about the uncertainties in the segmentation decisions, which stem from imaging artifacts including noise, partial voluming, and inhomogeneity. By enhancing the nonparametric model to capture the spatial continuity and structure of the fiber bundle, we exploit the framework to extract the cingulum fiber bundle. Typical tractography methods for tract delineation, incorporating thresholds on fractional anisotropy and fiber curvature to terminate tracking, can face serious problems arising from partial voluming and noise. For these reasons, tractography often fails to extract thin tracts with sharp changes in orientation, such as the cingulum. The results demonstrate that the proposed method extracts this structure significantly more accurately as compared to tractography.

Image restoration is an important and widely studied problem in computer vision and image processing. Various image filtering strategies have been effective, but invariably make strong assumptions about the properties of the signal and/or degradation. Hence, these methods lack the generality to be easily applied to new applications or diverse image collections. This paper describes a novel unsupervised, information-theoretic, adaptive filter (UINTA) that improves the predictability of pixel intensities from their neighborhoods by decreasing their joint entropy. In this way, UINTA automatically discovers the statistical properties of the signal and can thereby restore a wide spectrum of images. The paper describes the formulation to minimize the joint entropy measure and presents several important practical considerations in estimating neighborhood statistics. It presents a series of results on both real and synthetic data along with comparisons with current state-of-the-art techniques, including novel applications to medical image processing.

The restoration of images is an important and widely studied problem in computer vision and image processing. Various image filtering strategies have been effective, but invariably make strong assumptions about the properties of the signal and/or degradation. Therefore, these methods typically lack the generality to be easily applied to new applications or diverse image collections. This paper describes a novel unsupervised, information-theoretic, adaptive filter (UINTA) that improves the predictability of pixel intensities from their neighborhoods by decreasing the joint entropy between them. Thus UINTA automatically discovers the statistical properties of the signal and can thereby restore a wide spectrum of images and applications. This paper describes the formulation required to minimize the joint entropy measure, presents several important practical considerations in estimating image-region statistics, and then presents results on both real and synthetic data.