We propose a multi-scale deep energy model that is strongly convex in the local neighbourhood around the data manifold to represent its probability density, with application in inverse problems. In particular, we represent the negative log-prior as a multi-scale energy model parameterized by a Convolutional Neural Network (CNN). We restrict the gradient of the CNN to be locally monotone, which constrains the model as a Locally Convex Multi-Scale Energy (LC-MuSE). We use the learned energy model in image-based inverse problems, where the formulation offers several desirable properties: i) uniqueness of the solution, ii) convergence guarantees to a minimum of the inverse problem, and iii) robustness to input perturbations. In the context of parallel Magnetic Resonance (MR) image reconstruction, we show that the proposed method performs better than the state-of-the-art convex regularizers, while the performance is comparable to plug-and-play regularizers and end-to-end trained methods.

Image reconstruction using deep learning algorithms offers improved reconstruction quality and lower reconstruction time than classical compressed sensing and model-based algorithms. Unfortunately, clean and fully sampled ground-truth data to train the deep networks is often unavailable in several applications, restricting the applicability of the above methods. We introduce a novel metric termed the ENsemble Stein's Unbiased Risk Estimate (ENSURE) framework, which can be used to train deep image reconstruction algorithms without fully sampled and noise-free images. The proposed framework is the generalization of the classical SURE and GSURE formulation to the setting where the images are sampled by different measurement operators, chosen randomly from a set. We evaluate the expectation of the GSURE loss functions over the sampling patterns to obtain the ENSURE loss function. We show that this loss is an unbiased estimate for the true mean-square error, which offers a better alternative to GSURE, which only offers an unbiased estimate for the projected error. Our experiments show that the networks trained with this loss function can offer reconstructions comparable to the supervised setting. While we demonstrate this framework in the context of MR image recovery, the ENSURE framework is generally applicable to arbitrary inverse problems.

Bilinear models such as low-rank and dictionary methods, which decompose dynamic data to spatial and temporal factor matrices are powerful and memory-efficient tools for the recovery of dynamic MRI data. Current bilinear methods rely on sparsity and energy compaction priors on the factor matrices to regularize the recovery. Motivated by deep image prior, we introduce a novel bilinear model, whose factor matrices are generated using convolutional neural networks (CNNs). The CNN parameters, and equivalently the factors, are learned from the undersampled data of the specific subject. Unlike current unrolled deep learning methods that require the storage of all the time frames in the dataset, the proposed approach only requires the storage of the factors or compressed representation; this approach allows the direct use of this scheme to large-scale dynamic applications, including free breathing cardiac MRI considered in this work. To reduce the run time and to improve performance, we initialize the CNN parameters using existing factor methods. We use sparsity regularization of the network parameters to minimize the overfitting of the network to measurement noise. Our experiments on free-breathing and ungated cardiac cine data acquired using a navigated golden-angle gradient-echo radial sequence show the ability of our method to provide reduced spatial blurring as compared to classical bilinear methods as well as a recent unsupervised deep-learning approach.

Current deep learning-based manifold learning algorithms such as the variational autoencoder (VAE) require fully sampled data to learn the probability density of real-world datasets. However, fully sampled data is often unavailable in a variety of problems, including the recovery of dynamic and high-resolution magnetic resonance imaging (MRI). We introduce a novel variational approach to learn a manifold from undersampled data. The VAE uses a decoder fed by latent vectors, drawn from a conditional density estimated from the fully sampled images using an encoder. Since fully sampled images are not available in our setting, we approximate the conditional density of the latent vectors by a parametric model whose parameters are estimated from the undersampled measurements using back-propagation. We use the framework for the joint alignment and recovery of multi-slice free breathing and ungated cardiac MRI data from highly undersampled measurements. Experimental results demonstrate the utility of the proposed scheme in dynamic imaging alignment and reconstructions.

Deep learning algorithms are emerging as powerful alternatives to compressed sensing methods, offering improved image quality and computational efficiency. Unfortunately, fully sampled training images may not be available or are difficult to acquire in several applications, including high-resolution and dynamic imaging. Previous studies in image reconstruction have utilized Stein's Unbiased Risk Estimator (SURE) as a mean square error (MSE) estimate for the image denoising step in an unrolled network. Unfortunately, the end-to-end training of a network using SURE remains challenging since the projected SURE loss is a poor approximation to the MSE, especially in the heavily undersampled setting. We propose an ENsemble SURE (ENSURE) approach to train a deep network only from undersampled measurements. In particular, we show that training a network using an ensemble of images, each acquired with a different sampling pattern, can closely approximate the MSE. Our preliminary experimental results show that the proposed ENSURE approach gives comparable reconstruction quality to supervised learning and a recent unsupervised learning method.

Several imaging algorithms including patch-based image denoising, image time series recovery, and convolutional neural networks can be thought of as methods that exploit the manifold structure of signals. While the empirical performance of these algorithms is impressive, the understanding of recovery of the signals and functions that live on manifold is less understood. In this paper, we focus on the recovery of signals that live on a union of surfaces. In particular, we consider signals living on a union of smooth band-limited surfaces in high dimensions. We show that an exponential mapping transforms the data to a union of low-dimensional subspaces. Using this relation, we introduce a sampling theoretical framework for the recovery of smooth surfaces from few samples and the learning of functions living on smooth surfaces. The low-rank property of the features is used to determine the number of measurements needed to recover the surface. Moreover, the low-rank property of the features also provides an efficient approach, which resembles a neural network, for the local representation of multidimensional functions on the surface. The direct representation of such a function in high dimensions often suffers from the curse of dimensionality; the large number of parameters would translate to the need for extensive training data. The low-rank property of the features can significantly reduce the number of parameters, which makes the computational structure attractive for learning and inference from limited labeled training data.

We introduce a generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The model assumes that the images in the dataset are non-linear mappings of low-dimensional latent vectors. We use the deep convolutional neural network (CNN) to represent the non-linear transformation. The parameters of the generator as well as the low-dimensional latent vectors are jointly estimated only from the undersampled measurements. This approach is different from traditional CNN approaches that require extensive fully sampled training data. We penalize the norm of the gradients of the non-linear mapping to constrain the manifold to be smooth, while temporal gradients of the latent vectors are penalized to obtain a smoothly varying time-series. The proposed scheme brings in the spatial regularization provided by the convolutional network. The main benefit of the proposed scheme is the improvement in image quality and the orders-of-magnitude reduction in memory demand compared to traditional manifold models. To minimize the computational complexity of the algorithm, we introduce an efficient progressive training-in-time approach and an approximate cost function. These approaches speed up the image reconstructions and offers better reconstruction performance.

Multichannel imaging techniques are widely used in MRI to reduce the scan time. These schemes typically perform undersampled acquisition and utilize compressed-sensing based regularized reconstruction algorithms. Model-based deep learning (MoDL) frameworks are now emerging as powerful alternatives to compressed sensing, with significantly improved image quality. In this work, we investigate the impact of sampling patterns on the quality of the image recovered using the MoDL algorithm. We introduce a scheme to jointly optimize the sampling pattern and the reconstruction network parameters in MoDL for parallel MRI. The improved decoupling of the network parameters from the sampling patterns offered by the MoDL scheme translates to improved optimization and thus improved performance. Preliminary experimental results demonstrate that the proposed joint opti-mization framework significantly improves the image quality.

The efficient representation of data in high-dimensional spaces is a key problem in several machine learning tasks. To capture the non-linear structure of the data, we model the data as points living on a smooth surface. We model the surface as the zero level-set of a bandlimited function. We show that this representation allows a non-linear lifting of the surface model, which will map the points to a low-dimensional subspace. This mapping between surfaces and the well-understood subspace model allows us to introduce novel algorithms (a) to recover the surface from few of its samples and (b) to learn a multidimensional bandlimited function from training data. The utility of these algorithms is introduced in practical applications including image denoising.

Hyperspectral image (HSI) restoration is an important preprocessing step in HSI data analysis to improve the image quality for subsequent applications of HSI. In this article, we introduce a spatial-spectral patch-based nonconvex sparsity and low-rank regularization method for HSI restoration. In contrast to traditional approaches based on convex penalties or nonconvex spectral penalty alone, we consider the sparsity of HSI in the spatial-spectral domain and combine the nonconvex low-rank penalty and the nonconvex 3-D total variation (TV)-like sparsity regularization to fully exploit the correlations in both spatial-spectral dimensions of the HSI data set. In addition, we propose a fast iterative variable splitting-based algorithm to effectively solve the corresponding optimization problem. Numerical experiments on both simulated and real HSI data sets demonstrate that the proposed nonconvex low-rank and TV (NonLRTV) method significantly improves the recovered image quality compared with the state-of-the-art algorithms.

We introduce a model-based deep learning architecture termed MoDL-MUSSELS for the correction of phase errors in multishot diffusion-weighted echo-planar MR images. The proposed algorithm is a generalization of the existing MUSSELS algorithm with similar performance but significantly reduced computational complexity. In this work, we show that an iterative re-weighted least-squares implementation of MUSSELS alternates between a multichannel filter bank and the enforcement of data consistency. The multichannel filter bank projects the data to the signal subspace, thus exploiting the annihilation relations between shots. Due to the high computational complexity of the self-learned filter bank, we propose replacing it with a convolutional neural network (CNN) whose parameters are learned from exemplary data. The proposed CNN is a hybrid model involving a multichannel CNN in the k-space and another CNN in the image space. The k-space CNN exploits the annihilation relations between the shot images, while the image domain network is used to project the data to an image manifold. The experiments show that the proposed scheme can yield reconstructions that are comparable to state-of-the-art methods while offering several orders of magnitude reduction in run-time.

We introduce a kernel low-rank algorithm to recover free-breathing and ungated dynamic MRI from spiral acquisitions without explicit k-space navigators. It is often challenging for low-rank methods to recover free-breathing and ungated images from undersampled measurements; extensive cardiac and respiratory motion often results in the Casorati matrix not being sufficiently low-rank. Therefore, we exploit the non-linear structure of the dynamic data, which gives the low-rank kernel matrix. Unlike prior work that rely on navigators to estimate the manifold structure, we propose a kernel low-rank matrix completion method to directly fill in the missing k-space data from variable density spiral acquisitions. We validate the proposed scheme using simulated data and in-vivo data. Our results show that the proposed scheme provides improved reconstructions compared to the classical methods such as low-rank and XD-GRASP. The comparison with breath-held cine data shows that the quantitative metrics agree, whereas the image quality is marginally lower.

Structured low-rank (SLR) algorithms, which exploit annihilation relations between the Fourier samples of a signal resulting from different properties, is a powerful image reconstruction framework in several applications. This scheme relies on low-rank matrix completion to estimate the annihilation relations from the measurements. The main challenge with this strategy is the high computational complexity of matrix completion. We introduce a deep learning (DL) approach to significantly reduce the computational complexity. Specifically, we use a convolutional neural network (CNN)-based filterbank that is trained to estimate the annihilation relations from imperfect (under-sampled and noisy) k-space measurements of Magnetic Resonance Imaging (MRI). The main reason for the computational efficiency is the pre-learning of the parameters of the non-linear CNN from exemplar data, compared to SLR schemes that learn the linear filterbank parameters from the dataset itself. Experimental comparisons show that the proposed scheme can enable calibration-less parallel MRI; it can offer performance similar to SLR schemes while reducing the runtime by around three orders of magnitude. Unlike pre-calibrated and self-calibrated approaches, the proposed uncalibrated approach is insensitive to motion errors and affords higher acceleration. The proposed scheme also incorporates image domain priors that are complementary, thus significantly improving the performance over that of SLR schemes.

We introduce a model-based image reconstruction framework with a convolution neural network (CNN)-based regularization prior. The proposed formulation provides a systematic approach for deriving deep architectures for inverse problems with the arbitrary structure. Since the forward model is explicitly accounted for, a smaller network with fewer parameters is sufficient to capture the image information compared to direct inversion approaches. Thus, reducing the demand for training data and training time. Since we rely on end-to-end training with weight sharing across iterations, the CNN weights are customized to the forward model, thus offering improved performance over approaches that rely on pre-trained denoisers. Our experiments show that the decoupling of the number of iterations from the network complexity offered by this approach provides benefits, including lower demand for training data, reduced risk of overfitting, and implementations with significantly reduced memory footprint. We propose to enforce data-consistency by using numerical optimization blocks, such as conjugate gradients algorithm within the network. This approach offers faster convergence per iteration, compared to methods that rely on proximal gradients steps to enforce data consistency. Our experiments show that the faster convergence translates to improved performance, primarily when the available GPU memory restricts the number of iterations.

We introduce a structured low rank algorithm for the calibration-free compensation of field inhomogeneity artifacts in echo planar imaging (EPI) MRI data. We acquire the data using two EPI readouts that differ in echo-time. Using time segmentation, we reformulate the field inhomogeneity compensation problem as the recovery of an image time series from highly undersampled Fourier measurements. The temporal profile at each pixel is modeled as a single exponential, which is exploited to fill in the missing entries. We show that the exponential behavior at each pixel, along with the spatial smoothness of the exponential parameters, can be exploited to derive a 3-D annihilation relation in the Fourier domain. This relation translates to a low rank property on a structured multi-fold Toeplitz matrix, whose entries correspond to the measured k-space samples. We introduce a fast two-step algorithm for the completion of the Toeplitz matrix from the available samples. In the first step, we estimate the null space vectors of the Toeplitz matrix using only its fully sampled rows. The null space is then used to estimate the signal subspace, which facilitates the efficient recovery of the time series of images. We finally demonstrate the proposed approach on spherical MR phantom data and human data and show that the artifacts are significantly reduced.

Recent theory of mapping an image into a structured low-rank Toeplitz or Hankel matrix has become an effective method to restore images. In this paper, we introduce a generalized structured low-rank algorithm to recover images from their undersampled Fourier coefficients using infimal convolution regularizations. The image is modeled as the superposition of a piecewise constant component and a piecewise linear component. The Fourier coefficients of each component satisfy an annihilation relation, which results in a structured Toeplitz matrix. We exploit the low-rank property of the matrices to formulate a combined regularized optimization problem. In order to solve the problem efficiently and to avoid the high-memory demand resulting from the large-scale Toeplitz matrices, we introduce a fast and a memory-efficient algorithm based on the half-circulant approximation of the Toeplitz matrix. We demonstrate our algorithm in the context of single and multi-channel MR images recovery. Numerical experiments indicate that the proposed algorithm provides improved recovery performance over the state-of-the-art approaches.

We introduce a local manifold regularization approach to recover dynamic MRI data from highly undersampled measurements. The proposed scheme relies on the manifold structure of local image patches at the same spatial location in a free-breathing cardiac MRI dataset; this approach is a generalization of the SmooThness Regularization on Manifolds (SToRM) scheme that exploits the global manifold structure of images in the dataset. Since the manifold structure of the patches varies depending on the spatial location and is often considerably simpler than the global one, this approach significantly reduces the data demand, facilitating the recovery from shorter scans. Since the navigator-based estimation of manifold structure pursued in SToRM is not feasible in this setting, a reformulation of SToRM is introduced. Specifically, the regularization term of the cost function involves the sum of robust distances between images sub-patches in the dataset. The optimization algorithm alternates between updating the images and estimating the manifold structure of the image patches. The utility of the proposed scheme is demonstrated in the context of in-vivo prospective free-breathing cardiac CINE MRI imaging with multichannel acquisitions and simulated phantoms. The new framework facilitates a reduction in scan time, as compared to the SToRM strategy.

We introduce a model-based reconstruction framework with deep learned (DL) and smoothness regularization on manifolds (STORM) priors to recover free breathing and ungated (FBU) cardiac MRI from highly undersampled measurements. The DL priors enable us to exploit the local correlations, while the STORM prior enables us to make use of the extensive non-local similarities that are subject dependent. We introduce a novel model-based formulation that allows the seamless integration of deep learning methods with available prior information, which current deep learning algorithms are not capable of. The experimental results demonstrate the preliminary potential of this work in accelerating FBU cardiac MRI.

The analysis of large datasets is often complicated by the presence of missing entries, mainly because most of the current machine learning algorithms are designed to work with full data. The main focus of this work is to introduce a clustering algorithm, that will provide good clustering even in the presence of missing data. The proposed technique solves an lofusion penalty based optimization problem to recover the clusters. We theoretically analyze the conditions needed for the successful recovery of the clusters. We also propose an algorithm to solve a relaxation of this problem using saturating non-convex fusion penalties. The method is demonstrated on simulated and real datasets, and is observed to perform well in the presence of large fractions of missing entries.

We introduce a continuous domain framework for the recovery of points on a surface in high dimensional space, represented as the zero-level set of a bandlimited function. We show that the exponential maps of the points on the surface satisfy annihilation relations, implying that they lie in a finite dimensional subspace. The subspace properties are used to derive sampling conditions, which will guarantee the perfect recovery of the surface from finite number of points. We rely on nuclear norm minimization to exploit the low-rank structure of the maps to recover the points from noisy measurements. Since the direct estimation of the surface is computationally prohibitive in very high dimensions, we propose an iterative reweighted algorithm using the "kernel trick". The iterative algorithm reveals deep links to Laplacian based algorithms widely used in graph signal processing; the theory and the sampling conditions can serve as a basis for discrete-continuous domain processing of signals on a graph.

We consider the problem of recovering the superposition of R distinct complex exponential functions from compressed non-uniform time-domain samples. Total Variation (TV) minimization or atomic norm minimization was proposed in the literature to recover the R frequencies or the missing data. However, in order for TV minimization and atomic norm minimization to recover the missing data or the frequencies, the underlying R frequencies are required to be well-separated, even when the measurements are noiseless. This paper shows that the Hankel matrix recovery approach can super-resolve the R complex exponentials and their frequencies from compressed nonuniform measurements, regardless of how close their frequencies are to each other. We propose a new concept of orthonormal atomic norm minimization (OANM), and demonstrate that the success of Hankel matrix recovery in separation-free super-resolution comes from the fact that the nuclear norm of a Hankel matrix is an orthonormal atomic norm. More specifically, we show that, in traditional atomic norm minimization, the underlying parameter values must be well separated to achieve successful signal recovery, if the atoms are changing continuously with respect to the continuously-valued parameter. In contrast, for the OANM, it is possible the OANM is successful even though the original atoms can be arbitrarily close.

We introduce a structured low rank matrix completion algorithm to recover a series of images from their under-sampled measurements, where the signal along the parameter dimension at every pixel is described by a linear combination of exponentials. We exploit the exponential behavior of the signal at every pixel, along with the spatial smoothness of the exponential parameters to derive an annihilation relation in the Fourier domain. This relation translates to a low-rank property on a structured matrix constructed from the Fourier samples. We enforce the low-rank property of the structured matrix as a regularization prior to recover the images. Since the direct use of current low rank matrix recovery schemes to this problem is associated with high computational complexity and memory demand, we adopt an iterative re-weighted least squares algorithm, which facilitates the exploitation of the convolutional structure of the matrix. Novel approximations involving 2-D fast Fourier transforms are introduced to drastically reduce the memory demand and computational complexity, which facilitates the extension of structured low-rank methods to large scale 3-D problems. We demonstrate our algorithm in the MR parameter mapping setting and show improvement over the state-of-the-art methods.

We introduce a fast structured low-rank matrix completion algorithm with low memory & computational demand to recover the dynamic MRI data from undersampled measurements. The 3-D dataset is modeled as a piecewise smooth signal, whose discontinuities are localized to the zero sets of a bandlimited function. We show that a structured matrix corresponding to convolution with the Fourier coefficients of the signal derivatives is highly low-rank. This property enables us to recover the signal from undersampled measurements. The application of this scheme in dynamic MRI shows significant improvement over state of the art methods.

We introduce a novel image regularization termed as multiple degree total variation (MDTV). This type of regularization combines the first and second degree directional derivatives, thus providing a good balance between preservation of edges and region smoothness. In order to solve the resulting optimization problem, we proposed a fast majorize minimize algorithm. We demonstrate the utility of the MDTV regularization in the context of image denoising and compressed sensing. We compare the proposed method with standard TV, and the state of the art higher degree methods, including higher degree total variation (HDTV) and total generalized variation (TGV) based schemes. Numerical results indicate that MDTV penalty provides improved image recovery performance.

We derive theoretical guarantees for the exact recovery of piecewise constant two-dimensional images from a minimal number of non-uniform Fourier samples using a convex matrix completion algorithm. We assume the discontinuities of the image are localized to the zero level-set of a bandlimited function, which induces certain linear dependencies in Fourier domain, such that a multifold Toeplitz matrix built from the Fourier data is known to be low-rank. The recovery algorithm arranges the known Fourier samples into the structured matrix then attempts recovery of the missing Fourier data by minimizing the nuclear norm subject to structure and data constraints. This work adapts results by Chen and Chi on the recovery of isolated Diracs via nuclear norm minimization of a similar multifold Hankel structure. We show that exact recovery is possible with high probability when the bandlimited function describing the edge set satisfies an incoherency property. Finally, we demonstrate the algorithm on the recovery of undersampled MRI data.

We propose a convex clustering and reconstruction algorithm for data with missing entries. The algorithm uses a similarity measure between every pair of points to cluster and recover the data. The cluster centres can be recovered reliably when the ground-truth similarity matrix is available. Moreover, the similarity matrix can also be reliably estimated from the partially observed data, when the clusters are well-separated and the coherence of the difference between points from different clusters is low. The algorithm performs well using the estimated similarity matrix on a simulated dataset. The method is also successful in reconstructing images from under-sampled Fourier data.

We introduce a method to recover a continuous domain representation of a piecewise constant two-dimensional image from few low-pass Fourier samples. Assuming the edge set of the image is localized to the zero set of a trigonometric polynomial, we show the Fourier coefficients of the partial derivatives of the image satisfy a linear annihilation relation. We present necessary and sufficient conditions for unique recovery of the image from finite low-pass Fourier samples using the annihilation relation. We also propose a practical two-stage recovery algorithm which is robust to model-mismatch and noise. In the first stage we estimate a continuous domain representation of the edge set of the image. In the second stage we perform an extrapolation in Fourier domain by a least squares two-dimensional linear prediction, which recovers the exact Fourier coefficients of the underlying image. We demonstrate our algorithm on the super-resolution recovery of MRI phantoms and real MRI data from low-pass Fourier samples, which shows benefits over standard approaches for single-image super-resolution MRI.

We introduce a novel algorithm to recover real time dynamic MR images from highly under-sampled k- t space measurements. The proposed scheme models the images in the dynamic dataset as points on a smooth, low dimensional manifold in high dimensional space. We propose to exploit the non-linear and non-local redundancies in the dataset by posing its recovery as a manifold smoothness regularized optimization problem. A navigator acquisition scheme is used to determine the structure of the manifold, or equivalently the associated graph Laplacian matrix. The estimated Laplacian matrix is used to recover the dataset from undersampled measurements. The utility of the proposed scheme is demonstrated by comparisons with state of the art methods in multi-slice real-time cardiac and speech imaging applications.

We provide a two-step approach to recover a jointly k-sparse matrix X, (at most k rows of X are nonzero), with rank r <; <; k from its under sampled measurements. Unlike the classical recovery algorithms that use the same measurement matrix for every column of X, the proposed algorithm comprises two stages, in each of which the measurement is taken by a different measurement matrix. The first stage uses a standard algorithm, [4] to recover any r columns (e.g. the first r) of X. The second uses a new set of measurements and the subspace estimate provided by these columns to recover the rest. We derive conditions on the second measurement matrix to guarantee perfect subspace aware recovery for two cases: First a worst-case setting that applies to all matrices. The second a generic case that works for almost all matrices. We demonstrate both theoretically and through simulations that when r <; <; k our approach needs far fewer measurements. It compares favorably with recent results using dense linear combinations, that do not use column-wise measurements.

We introduce a family of novel regularization penalties to enable the recovery of discrete discontinuous piecewise polynomial signals from undersampled or degraded linear measurements. The penalties promote the group sparsity of the signal analyzed under a nth order derivative. We introduce an efficient alternating minimization algorithm to solve linear inverse problems regularized with the proposed penalties. Our experiments show that promoting group sparsity of derivatives enhances the compressed sensing recovery of discontinuous piecewise linear signals compared with an unstructured sparse prior. We also propose an extension to 2-D, which can be viewed as a group sparse version of higher degree total variation, and illustrate its effectiveness in denoising experiments.

We propose a novel deformation corrected compressed sensing (DC-CS) framework to recover contrast enhanced dynamic magnetic resonance images from undersampled measurements. We introduce a formulation that is capable of handling a wide class of sparsity/compactness priors on the deformation corrected dynamic signal. In this work, we consider example compactness priors such as sparsity in temporal Fourier domain, sparsity in temporal finite difference domain, and nuclear norm penalty to exploit low rank structure. Using variable splitting, we decouple the complex optimization problem to simpler and well understood sub problems; the resulting algorithm alternates between simple steps of shrinkage-based denoising, deformable registration, and a quadratic optimization step. Additionally, we employ efficient continuation strategies to reduce the risk of convergence to local minima. The decoupling enabled by the proposed scheme enables us to apply this scheme to contrast enhanced MRI applications. Through experiments on numerical phantom and in vivo myocardial perfusion MRI datasets, we observe superior image quality of the proposed DC-CS scheme in comparison to the classical k-t FOCUSS with motion estimation/correction scheme, and demonstrate reduced motion artifacts over classical compressed sensing schemes that utilize the compact priors on the original deformation uncorrected signal.

We introduce a fast iterative shrinkage algorithm for patch-smoothness regularization of inverse problems in medical imaging. This approach is enabled by the reformulation of current non-local regularization schemes as an alternating algorithm to minimize a global criterion. The proposed algorithm alternates between evaluating the denoised inter-patch differences by shrinkage and computing an image that is consistent with the denoised inter-patch differences and measured data. We derive analytical shrinkage rules for several penalties that are relevant in non-local regularization. The redundancy in patch comparisons used to evaluate the shrinkage steps are exploited using convolution operations. The resulting algorithm is observed to be considerably faster than current alternating non-local algorithms. The proposed scheme is applicable to a large class of inverse problems including deblurring, denoising, and Fourier inversion. The comparisons of the proposed scheme with state-of-the-art regularization schemes in the context of recovering images from undersampled Fourier measurements demonstrate a considerable reduction in alias artifacts and preservation of edges.

We introduce novel algorithms for the joint recovery of an ensemble of signals that live on a smooth manifold from their under sampled measurements. Unlike current methods that are designed to recover a single signal assuming perfect knowledge of the manifold model, the proposed algorithms exploit similarity between the signals without prior knowledge of the underlying manifold structure. Our first algorithm is a two-step scheme, where the Laplacian of the graph associated with the manifold is estimated from similar under sampled measurements made on the signals; this Laplacian is used to formulate the problem as a penalized optimization scheme, where smoothness of the signals on the manifold is chosen as the penalty. The second algorithm is an iterative scheme that alternates between computation of the Laplacian and the signals. Validation of the proposed algorithms using simulations and experimental MRI data demonstrate their utility in accelerating free breathing cardiac MRI.

We introduce a family of novel image regularization penalties called generalized higher degree total variation (HDTV). These penalties further extend our previously introduced HDTV penalties, which generalize the popular total variation (TV) penalty to incorporate higher degree image derivatives. We show that many of the proposed second degree extensions of TV are special cases or are closely approximated by a generalized HDTV penalty. Additionally, we propose a novel fast alternating minimization algorithm for solving image recovery problems with HDTV and generalized HDTV regularization. The new algorithm enjoys a tenfold speed up compared with the iteratively reweighted majorize minimize algorithm proposed in a previous paper. Numerical experiments on 3D magnetic resonance images and 3D microscopy images show that HDTV and generalized HDTV improve the image quality significantly compared with TV.

Validation is arguably the bottleneck in the diffusion magnetic resonance imaging (MRI) community. This paper evaluates and compares 20 algorithms for recovering the local intra-voxel fiber structure from diffusion MRI data and is based on the results of the "HARDI reconstruction challenge" organized in the context of the "ISBI 2012" conference. Evaluated methods encompass a mixture of classical techniques well known in the literature such as diffusion tensor, Q-Ball and diffusion spectrum imaging, algorithms inspired by the recent theory of compressed sensing and also brand new approaches proposed for the first time at this contest. To quantitatively compare the methods under controlled conditions, two datasets with known ground-truth were synthetically generated and two main criteria were used to evaluate the quality of the reconstructions in every voxel: correct assessment of the number of fiber populations and angular accuracy in their orientation. This comparative study investigates the behavior of every algorithm with varying experimental conditions and highlights strengths and weaknesses of each approach. This information can be useful not only for enhancing current algorithms and develop the next generation of reconstruction methods, but also to assist physicians in the choice of the most adequate technique for their studies.

We introduce a unifying energy minimization framework for nonlocal regularization of inverse problems. In contrast to the weighted sum of square differences between image pixels used by current schemes, the proposed functional is an unweighted sum of inter-patch distances. We use robust distance metrics that promote the averaging of similar patches, while discouraging the averaging of dissimilar patches. We show that the first iteration of a majorize-minimize algorithm to minimize the proposed cost function is similar to current nonlocal methods. The reformulation thus provides a theoretical justification for the heuristic approach of iterating nonlocal schemes, which re-estimate the weights from the current image estimate. Thanks to the reformulation, we now understand that the widely reported alias amplification associated with iterative nonlocal methods are caused by the convergence to local minimum of the nonconvex penalty. We introduce an efficient continuation strategy to overcome this problem. The similarity of the proposed criterion to widely used nonquadratic penalties (e.g., total variation and lp semi-norms) opens the door to the adaptation of fast algorithms developed in the context of compressive sensing; we introduce several novel algorithms to solve the proposed nonlocal optimization problem. Thanks to the unifying framework, these fast algorithms are readily applicable for a large class of distance metrics.

We propose a novel blind compressive sensing (BCS) frame work to recover dynamic magnetic resonance images from undersampled measurements. This scheme models the dynamic signal as a sparse linear combination of temporal basis functions, chosen from a large dictionary. In contrast to classical compressed sensing, the BCS scheme simultaneously estimates the dictionary and the sparse coefficients from the undersampled measurements. Apart from the sparsity of the coefficients, the key difference of the BCS scheme with current low rank methods is the nonorthogonal nature of the dictionary basis functions. Since the number of degrees-of-freedom of the BCS model is smaller than that of the low-rank methods, it provides improved reconstructions at high acceleration rates. We formulate the reconstruction as a constrained optimization problem; the objective function is the linear combination of a data consistency term and sparsity promoting l1 prior of the coefficients. The Frobenius norm dictionary constraint is used to avoid scale ambiguity. We introduce a simple and efficient majorize-minimize algorithm, which decouples the original criterion into three simpler subproblems. An alternating minimization strategy is used, where we cycle through the minimization of three simpler problems. This algorithm is seen to be considerably faster than approaches that alternates between sparse coding and dictionary estimation, as well as the extension of K-SVD dictionary learning scheme. The use of the l1 penalty and Frobenius norm dictionary constraint enables the attenuation of insignificant basis functions compared to the l0 norm and column norm constraint assumed in most dictionary learning algorithms; this is especially important since the number of basis functions that can be reliably estimated is restricted by the available measurements. We also observe that the proposed scheme is more robust to local minima compared to K-SVD method, which relies on greedy sparse coding. Our phase transition experiments demonstrate that the BCS scheme provides much better recovery rates than classical Fourier-based CS schemes, while being only marginally worse than the dictionary aware setting. Since the overhead in additionally estimating the dictionary is low, this method can be very useful in dynamic magnetic resonance imaging applications, where the signal is not sparse in known dictionaries. We demonstrate the utility of the BCS scheme in accelerating contrast enhanced dynamic data. We observe superior reconstruction performance with the BCS scheme in comparison to existing low rank and compressed sensing schemes.

We introduce novel image regularization penalties to overcome the practical problems associated with the classical total variation (TV) scheme. Motivated by novel reinterpretations of the classical TV regularizer, we derive two families of functionals involving higher degree partial image derivatives; we term these families as isotropic and anisotropic higher degree TV (HDTV) penalties, respectively. The isotropic penalty is the L(1) - L(2) mixed norm of the directional image derivatives, while the anisotropic penalty is the separable L(1) norm of directional derivatives. These functionals inherit the desirable properties of standard TV schemes such as invariance to rotations and translations, preservation of discontinuities, and convexity. The use of mixed norms in isotropic penalties encourages the joint sparsity of the directional derivatives at each pixel, thus encouraging isotropic smoothing. In contrast, the fully separable norm in the anisotropic penalty ensures the preservation of discontinuities, while continuing to smooth along the linelike features; this scheme thus enhances the linelike image characteristics analogous to standard TV. We also introduce efficient majorize-minimize algorithms to solve the resulting optimization problems. The numerical comparison of the proposed scheme with classical TV penalty, current second-degree methods, and wavelet algorithms clearly demonstrate the performance improvement. Specifically, the proposed algorithms minimize the staircase and ringing artifacts that are common with TV and wavelet schemes, while better preserving the singularities. We also observe that anisotropic HDTV penalty provides consistently improved reconstructions compared with the isotropic HDTV penalty.

We introduce a novel algorithm to recover sparse and low-rank matrices from noisy and undersampled measurements. We pose the reconstruction as an optimization problem, where we minimize a linear combination of data consistency error, nonconvex spectral penalty, and nonconvex sparsity penalty. We majorize the nondifferentiable spectral and sparsity penalties in the criterion by quadratic expressions to realize an iterative three-step alternating minimization scheme. Since each of these steps can be evaluated either analytically or using fast schemes, we obtain a computationally efficient algorithm. We demonstrate the utility of the algorithm in the context of dynamic magnetic resonance imaging (MRI) reconstruction from sub-Nyquist sampled measurements. The results show a significant improvement in signal-to-noise ratio and image quality compared with classical dynamic imaging algorithms. We expect the proposed scheme to be useful in a range of applications including video restoration and multidimensional MRI.

We introduce a novel algorithm to reconstruct dynamic magnetic resonance imaging (MRI) data from under-sampled k-t space data. In contrast to classical model based cine MRI schemes that rely on the sparsity or banded structure in Fourier space, we use the compact representation of the data in the Karhunen Louve transform (KLT) domain to exploit the correlations in the dataset. The use of the data-dependent KL transform makes our approach ideally suited to a range of dynamic imaging problems, even when the motion is not periodic. In comparison to current KLT-based methods that rely on a two-step approach to first estimate the basis functions and then use it for reconstruction, we pose the problem as a spectrally regularized matrix recovery problem. By simultaneously determining the temporal basis functions and its spatial weights from the entire measured data, the proposed scheme is capable of providing high quality reconstructions at a range of accelerations. In addition to using the compact representation in the KLT domain, we also exploit the sparsity of the data to further improve the recovery rate. Validations using numerical phantoms and in vivo cardiac perfusion MRI data demonstrate the significant improvement in performance offered by the proposed scheme over existing methods.

We introduce a novel algorithm to address the challenges in magnetic resonance (MR) spectroscopic imaging. In contrast to classical sequential data processing schemes, the proposed method combines the reconstruction and postprocessing steps into a unified algorithm. This integrated approach enables us to inject a range of prior information into the data processing scheme, thus constraining the reconstructions. We use high resolution, 3-D estimate of the magnetic field inhomogeneity map to generate an accurate forward model, while a high resolution estimate of the fat/water boundary is used to minimize spectral leakage artifacts. We parameterize the spectrum at each voxel as a sparse linear combination of spikes and polynomials to capture the metabolite and baseline components, respectively. The constrained model makes the problem better conditioned in regions with significant field inhomogeneity, thus enabling the recovery even in regions with high field map variations. To exploit the high resolution MR information, we formulate the problem as an anatomically constrained total variation optimization scheme on a grid with the same spacing as the magnetic resonance imaging data. We analyze the performance of the proposed scheme using phantom and human subjects. Quantitative and qualitative comparisons indicate a significant improvement in spectral quality and lower leakage artifacts.

In this paper we reconstruct the 3-D MR spectroscopic data acquired using an echo-planar spectroscopic imaging (EPSI) sequence. We propose to compensate for B0inhomogeneity artifact by using the B0field map and T2* decay estimated from a higher resolution, multi-slice proton imaging data scanned with similar shimming to the EPSI one. We employ total variation (TV) regularization to obtain more uniform spectral peaks and lineshapes, and exploit the sparsity of the spectral data by using the lscr1norm in the reconstruction.

The main focus of this paper is the efficient approximation of the non-uniform Fourier transform (NUFFT). We reformulate the standard NUFFT approximation as a projection of the exact discrete Fourier transform onto a shift-invariant space. This reformulation enables the use of sophisticated tools, developed in the context of shift-invariant representations, to analyze the performance of the approximation. Using these techniques, we derive the optimal scale factors for a specified interpolator. Assuming these scale factors, we develop a worst-case error criterion that is only dependent on the interpolating function. We propose an iterative re-weighted optimization algorithm to derive the optimized least square (OLS) interpolator. This interpolator significantly reduces the approximation error in comparison to the standard methods. The improved performance of this scheme, for low oversampling factors, could lead to a memory efficient algorithm for non-Cartesian Fourier inversion.

The decomposition of magnetic resonance imaging (MRI) data to generate water and fat images has several applications in medical imaging, including fat suppression and quantification of visceral fat. We introduce a novel algorithm to overcome some of the problems associated with current analytical and iterative decomposition schemes. In contrast to traditional analytical schemes, our approach is general enough to accommodate any uniform echo-shift pattern, any number of metabolites and signal samples. In contrast to region-growing method that use a smooth field-map initialization to resolve the ambiguities with the IDEAL algorithm, we propose to use an explicit smoothness constraint on the final field-map estimate. Towards this end, we estimate the number of feasible solutions at all the voxels, prior to the evaluation of the roots. This approach enables the algorithm to evaluate all the feasible roots, thus avoiding the convergence to the wrong solution. The estimation procedure is based on a modification of the harmonic retrieval (HR) framework to account for the chemical shift dependence in the frequencies. In contrast to the standard linear HR framework, we obtain the frequency shift as the common root of a set of quadratic equations. On most of the pixels with multiple feasible solutions, the correct solution can be identified by a simple sorting of the solutions. We use a region-merging algorithm to resolve the remaining ambiguity and phase-wrapping. Experimental results indicate that the proposed algebraic scheme eliminates most of the difficulties with the current schemes, without compromising the noise performance. Moreover, the proposed algorithm is also computationally more efficient.

Model-based techniques have the potential to reduce the artifacts and improve resolution in magnetic resonance spectroscopic imaging, without sacrificing the signal-to-noise ratio. However, the current approaches have a few drawbacks that limit their performance in practical applications. Specifically, the classical schemes use less flexible image models that lead to model misfit, thus resulting in artifacts. Moreover, the performance of the current approaches is negatively affected by the magnetic field inhomogeneity and spatial mismatch between the anatomical references and spectroscopic imaging data. In this paper, we propose efficient solutions to overcome these problems. We introduce a more flexible image model that represents the signal as a linear combination of compartmental and local basis functions. The former set represents the signal variations within the compartments, while the latter captures the local perturbations resulting from lesions or segmentation errors. Since the combined set is redundant, we obtain the reconstructions using sparsity penalized optimization. To compensate for the artifacts resulting from field inhomogeneity, we estimate the field map using alternate scans and use it in the reconstruction. We model the spatial mismatch as an affine transformation, whose parameters are estimated from the spectroscopy data.

Magnetic resonance spectroscopy imaging (MRSI) is an attractive tool for medical imaging. However, its practical use is often limited by the intrinsic low spatial resolution and long acquisition time. Spectral localization by imaging (SLIM) has been proposed as a non-Fourier reconstruction algorithm that incorporates spatial a priori information about spectroscopically uniform compartments. Unfortunately, the influence of the magnetic field inhomogeneity--in particular, the susceptibility effects at tissues' boundaries--undermines the validity of the compartmental model. Therefore, we propose BSLIM as an extension of SLIM with field inhomogeneity compensation. A B0-field inhomogeneity map, which can be acquired rapidly and at high resolution, is used by the new algorithm as additional a priori information. We show that the proposed method is distinct from the generalized SLIM (GSLIM) framework. Experimental results of a two-compartment phantom demonstrate the feasibility of the method and the importance of inhomogeneity compensation.

We introduce a three-dimensional (3-D) parametric active contour algorithm for the shape estimation of DNA molecules from stereo cryo-electron micrographs. We estimate the shape by matching the projections of a 3-D global shape model with the micrographs; we choose the global model as a 3-D filament with a B-spline skeleton and a specified radial profile. The active contour algorithm iteratively updates the B-spline coefficients, which requires us to evaluate the projections and match them with the micrographs at every iteration. Since the evaluation of the projections of the global model is computationally expensive, we propose a fast algorithm based on locally approximating it by elongated blob-like templates. We introduce the concept of projection-steerability and derive a projection-steerable elongated template. Since the two-dimensional projections of such a blob at any 3-D orientation can be expressed as a linear combination of a few basis functions, matching the projections of such a 3-D template involves evaluating a weighted sum of inner products between the basis functions and the micrographs. The weights are simple functions of the 3-D orientation and the inner-products are evaluated efficiently by separable filtering. We choose an internal energy term that penalizes the average curvature magnitude. Since the exact length of the DNA molecule is known a priori, we introduce a constraint energy term that forces the curve to have this specified length. The sum of these energies along with the image energy derived from the matching process is minimized using the conjugate gradients algorithm. We validate the algorithm using real, as well as simulated, data and show that it performs well.

We present a framework for feature detection in 3-D using steerable filters. These filters can be designed to optimally respond to a particular type of feature by maximizing several Canny-like criteria. The detection process involves the analytical computation of the orientation and corresponding response of the template. A post-processing step consisting of the suppression of non-maximal values followed by thresholding to eliminate insignificant features concludes the detection procedure. We illustrate the approach with the design of feature templates for the detection of surfaces and curves, and demonstrate their efficiency with practical applications.

We propose a general approach for the design of 2D feature detectors from a class of steerable functions based on the optimization of a Canny-like criterion. In contrast with previous computational designs, our approach is truly 2D and provides filters that have closed-form expressions. It also yields operators that have a better orientation selectivity than the classical gradient or Hessian-based detectors. We illustrate the method with the design of operators for edge and ridge detection. We present some experimental results that demonstrate the performance improvement of these new feature detectors. We propose computationally efficient local optimization algorithms for the estimation of feature orientation. We also introduce the notion of shape-adaptable feature detection and use it for the detection of image corners.

Parametric active contour models are one of the preferred approaches for image segmentation because of their computational efficiency and simplicity. However, they have a few drawbacks which limit their performance. In this paper, we identify some of these problems and propose efficient solutions to get around them. The widely-used gradient magnitude-based energy is parameter dependent; its use will negatively affect the parametrization of the curve and, consequently, its stiffness. Hence, we introduce a new edge-based energy that is independent of the parameterization. It is also more robust since it takes into account the gradient direction as well. We express this energy term as a surface integral, thus unifying it naturally with the region-based schemes. The unified framework enables the user to tune the image energy to the application at hand. We show that parametric snakes can guarantee low curvature curves, but only if they are described in the curvilinear abscissa. Since normal curve evolution do not ensure constant arc-length, we propose a new internal energy term that will force this configuration. The curve evolution can sometimes give rise to closed loops in the contour, which will adversely interfere with the optimization algorithm. We propose a curve evolution scheme that prevents this condition.

We present a new approach for the design of optimal steerable 2-D templates for feature detection. As opposed to classical schemes where the optimal 1-D template is derived and extended to 2-D, we directly obtain the 2-D template. We choose the template from a class of steerable functions based on the analytic optimization of a Canny-like criterion. Our approach gives more orientation selective templates that have simple closed form expression. We illustrate the method with the design of operators for edge and ridge detection and demonstrate their performance improvement in practical applications.

We present a new family of 2D orthogonal wavelets which use quincunx sampling. The orthogonal refinement filters have a simple analytical expression in the Fourier domain as a function of the order /spl alpha/, which may be non-integer. The wavelets have good isotropy properties. We can also prove that they yield wavelet bases of L/sub 2/(R/sup 2/) for any /spl alpha/>0. The wavelets are fractional in the sense that the approximation error at a given scale /spl alpha/ decays like O(a/sup /spl alpha//); they also essentially behave like fractional derivative operators. To make our construction practical, we propose an FFT-based implementation that turns out to be surprisingly fast. In fact, our method is almost as efficient as the standard Mallat algorithm for separable wavelets.

We present a method for the exact computation of the moments of a region bounded by a curve represented by a scaling function or wavelet basis. Using Green's theorem, we show that the computation of the area moments is equivalent to applying a suitable multidimensional filter on the coefficients of the curve and thereafter computing a scalar product. The multidimensional filter coefficients are precomputed exactly as the solution of a two-scale relation. To demonstrate the performance improvement of the new method, we compare it with existing methods such as pixel-based approaches and approximation of the region by a polygon. We also propose an alternate scheme when the scaling function is sinc(x).

We present an exact algorithm for the computation of the moments of a region bounded by a curve represented in a scaling function or wavelet basis. Using Green's theorem, we show that the computation of the area moments is equivalent to applying a suitable multidimensional filter on the coefficients of the curve and thereafter computing a scalar product. We compare this algorithm with existing methods such as pixel-based approached and approximation of the region by a polygon.