The interest in leveraging physics-based inductive bias in deep learning has resulted in recent development of _hybrid deep generative models (hybrid-DGMs)_ that integrates known physics-based mathematical expressions in neural generative models. To identify these hybrid-DGMs requires inferring parameters of the physics-based component along with their neural component. The identifiability of these hybrid-DGMs, however, has not yet been theoretically probed or established. How does the existing theory of the un-identifiability of general DGMs apply to hybrid-DGMs? What may be an effective approach to consutrct a hybrid-DGM with theoretically-proven identifiability? This paper provides the first theoretical probe into the identifiability of hybrid-DGMs, and present meta-learning as a novel solution to construct identifiable hybrid-DGMs. On synthetic and real-data benchmarks, we provide strong empirical evidence for the un-identifiability of existing hybrid-DGMs using unconditional priors, and strong identifiability results of the presented meta-formulations of hybrid-DGMs.

To address the lack of high-quality training labels in positron emission tomography (PET) imaging, weakly-supervised reconstruction methods that generate network-based mappings between prior images and noisy targets have been developed. However, the learned model has an intrinsic variance proportional to the average variance of the target image. To suppress noise and improve the accuracy and generalizability of the learned model, we propose a conditional weakly-supervised multi-task learning (MTL) strategy, in which an auxiliary task is introduced serving as an anatomical regularizer for the PET reconstruction main task. In the proposed MTL approach, we devise a novel multi-channel self-attention (MCSA) module that helps learn an optimal combination of shared and task-specific features by capturing both local and global channel-spatial dependencies. The proposed reconstruction method was evaluated on NEMA phantom PET datasets acquired at different positions in a PET/CT scanner and 26 clinical whole-body PET datasets. The phantom results demonstrate that our method outperforms state-of-the-art learning-free and weakly-supervised approaches obtaining the best noise/contrast tradeoff with a significant noise reduction of approximately 50.0% relative to the maximum likelihood (ML) reconstruction. The patient study results demonstrate that our method achieves the largest noise reductions of 67.3% and 35.5% in the liver and lung, respectively, as well as consistently small biases in 8 tumors with various volumes and intensities. In addition, network visualization reveals that adding the auxiliary task introduces more anatomical information into PET reconstruction than adding only the anatomical loss, and the developed MCSA can abstract features and retain PET image details.

The integration of Time-of-Flight (TOF) information in the reconstruction process of Positron Emission Tomography (PET) improves image qualities. However, implementing the cutting-edge model-based deep learning methods for TOF-PET reconstruction is challenging due to the substantial memory requirements. In this study, we presented a novel model-based deep learning approach, LMPDNet, for TOF-PET reconstruction from list-mode data. We addressed the issue of real-time parallel computation of the projection matrix for list-mode data, and proposed an iterative model-based module that utilized a dedicated network model for list-mode data. Our experimental results indicated that the proposed LMPDNet outperformed traditional iteration-based TOF-PET list-mode reconstruction algorithms. Additionally, we compared the spatial and temporal consumption of list-mode data and sinogram data in model-based deep learning methods, demonstrating the superiority of list-mode data in model-based TOF-PET reconstruction.

The reconstruction of epicardial and endocardial extracellular potentials (EEP) by noninvasive methods has become a significant topic in cardiac electrophysiology over recent years. It is of great importance for the diagnosis of arrhythmia and for guidance of radiofrequency ablation, based on the difference in potentials between different locations on the heart's surface. In this study, we propose a non-local regularization of total variation (TV) in a low-rank (LR) and sparse decomposition framework, suitable for the rank-deficient problem of EEP reconstruction. LR and sparse decomposition can be utilized to extract the spatial-temporal information resulting from the sparse properties of EEP data, and the non-local similarities in the LR part can be a constraint for a non-local total variation regularization. The proposed method is implemented in simulated myocardial infarction (MI), interventional, and clinical premature ventricular contraction (PVC) experiments to verify its feasibility and reliability. Compared with the existing LR and TV methods, the proposed method performs better at potential reconstruction as well as PVC localization, particularly in the boundary of the MI region, while the results of this method are also consistent with those of invasive measurements using an EnSite 3000 system in the clinical experiment.

Many efforts have been made for decades in order to improve the accuracy of radioactivity map in positron emission tomography (PET) images, which has important clinical implications for better diagnosis and understanding of diseases. However, there is still a challenging problem for reconstructing high resolution image with the limited acquired photon counts. In this paper, we present a nonlocal self-similar constraint for the purpose of exploiting structured sparsity within the PET reconstructed images. It is based on image patches and approached by low-rank approximation. Moreover, we adopt total variation regulation into our method to further denoise and compensate the demerits inherited in patch-based methods. These two regulation terms are firstly employed in the Poisson model, and are jointly solved in a distributed optimization framework. Experiments have presented that our proposed PNLTV method substantially outperforms existing state-of-the-art methods in PET reconstruction.

The problem of using surface data to reconstruct transmural electrophysiological (EP) signals is intrinsically ill-posed without a unique solution in its unconstrained form. Incorporating physiological spatiotemporal priors through probabilistic integration of dynamic EP models, we have previously developed a Bayesian approach to transmural electrophysiological imaging (TEPI) using body-surface electrocardiograms. In this study, we generalize TEPI to using electrical signals collected from heart surfaces, and we test its feasibility on two pre-clinical swine models provided through the STACOM 2011 EP simulation Challenge. Since this new application of TEPI does not require whole-body imaging, there may be more immediate potential in EP laboratories where it could utilize catheter mapping data and produce transmural information for therapy guidance. Another focus of this study is to investigate the consistency among three modalities in delineating scar after myocardial infarction: TEPI, electroanatomical voltage mapping (EAVM), and magnetic resonance imaging (MRI). Our preliminary data demonstrate that, compared to the low-voltage scar area in EAVM, the 3-D electrical scar volume detected by TEPI is more consistent with anatomical scar volume delineated in MRI. Furthermore, TEPI could complement anatomical imaging by providing EP functional features related to both scar and healthy tissue.

Model-based reconstruction algorithms have shown potentials over conventional strain-based methods in static elas-tographic image by using ”accurate” finite element(FE) or bio-mechanical models. Strictly speaking, however, the measurement noise are always exists and thus do not meet basic assumptions of these algorithms. In addition, the difficulty in determining the proper system response model also greatly affects the quality of the reconstructed images. In this paper, we explore the usage of state space principles for the estimation of materials properties in elastographic imaging. The model-data discrepancy is modeled as uncertainties, i.e. Gaussian white noise, and the measurement noise is treated as another independent Gaussian white noise in the stochastic state space space, and an optimal estimation is computed of full displacement field and Young's modulus simultaneously using an extended Kalman filter (EKF). The performance of the proposed framework is evaluated using phantom data and real data with favorable results.

In this paper, we explore the usage of graphics processing units (GPU)-accelerated particle filter strategies for the estimation of activity map in tomographic PET imaging. The proposed framework formulates the physiological model of the imaging tissues through state space evolution equations and the photon counting statistics through observation equations, and then reconstruction is performed using particle filter estimation. For fast reconstruction, the calculations are implemented using highly-parallel GPU. Experiments show that the image quality is improved through particle filter. Furthermore, thanks to the computing power of graphics hardware, reconstruction times are practical for clinical applications.

The recent advances in meaningful constraining models have resulted in increasingly useful quantitative information recovered from cardiac images. Nevertheless, as most frameworks utilize either functional or structural images, the analyses cannot benefit from the complementary information provided by the other image sources. To better characterize subject-specific cardiac physiology and pathology, data fusion of multiple image sources is essential. Traditional image fusion strategies are performed by fusing information of commensurate images through various mathematical operators. Nevertheless, when image data are dissimilar in physical nature and spatiotemporal quantity, such approaches may not provide meaningful connections between different data. In fact, as different image sources provide partial measurements of the same cardiac system dynamics, it is more natural and suitable to utilize cardiac physiological models for the fusions. Therefore, we propose to use the cardiac physiome model as the central link to fuse functional and structural images for more subject-specific cardiac deformation recovery through state-space filtering. Experiments were performed on synthetic and real data for the characteristics and potential clinical applicability of our framework, and the results show an increase of the overall subject specificity of the recovered deformations.

A large number of real images possess the property of intensity non-homogeneity, which hinders them from being segmented by many image segmentation approaches. Recently, region-based active contour models utilizing local information have been introduced to segment images with intensity non-homogeneity. However, all these models are not convex, thus a good initial guess is required, which limits their practical application. In this paper, we propose a convex neighbor-constrained active contour model to segment images with intensity non-homogeneity. With different shapes and sizes of the neighborhood for each point, our model can accurately capture the region information of a given image. Our model is convex, and therefore it is independent of the initial condition and allows for automatic segmentation. To minimize energy functional of the model, we choose the efficient and fast Split Bregman method. Experimental results on synthetic and real images demonstrate the superior performance of our model.

Volumetric details of cardiac electrophysiology, such as transmembrane potential dynamics and tissue excitability of the myocardium, are of fundamental importance for understanding normal and pathological cardiac mechanisms, and for aiding the diagnosis and treatment of cardiac arrhythmia. Noninvasive observations, however, are made on body surface as an integration-projection of the volumetric phenomena inside patient's heart. We present a physiological-model-constrained statistical framework where prior knowledge of general myocardial electrical activity is used to guide the reconstruction of patient-specific volumetric cardiac electrophysiological details from body surface potential data. Sequential data assimilation with proper computational reduction is developed to estimate transmembrane potential and myocardial excitability inside the heart, which are then utilized to depict arrhythmogenic substrates. Effectiveness and validity of the framework is demonstrated through its application to evaluate the location and extent of myocardial infract using real patient data.

Quantitative estimation of nonrigid motion from image sequences has important technical and practical significance. State-space analysis provides powerful and convenient ways to construct and incorporate the physically meaningful system dynamics of an object, the image-derived observations, and the process and measurement noise disturbances. In this paper, we present a biomechanical-model constrained state-space analysis framework for the multiframe estimation of the periodic cardiac motion and deformation. The physical constraints take the roles as spatial regulator of the myocardial behavior and spatial filter/interpolator of the data measurements, while techniques from statistical filtering theory impose spatiotemporal constraints to facilitate the incorporation of multiframe information to generate optimal estimates of the heart kinematics. Physiologically meaningful results have been achieved from estimated displacement fields and strain maps using in vivo left ventricular magnetic resonance tagging and phase contrast image sequences, which provide the tag-tag and tag-boundary displacement inputs, and the mid-wall instantaneous velocity information and boundary displacement measures, respectively.

We have developed a geometric deformable model that employs neighborhood influence to achieve robust segmentation for noisy and broken edges. The fundamental power of this strategy rests with the explicitly combination of regional inter-point constraints, image forces, and a priori boundary information for each geometric contour point within its adaptively determined local influence domain. This formulation thus naturally unifies the essences of the geometric and parametric snakes through automatic local scale selection, and exhibits their respective fundamental strengths of allowing stable boundary detection when the edge information is weak and possibly discontinuous, while maintaining the abilities to handle topological changes during front evolution. In particular, this paper presents an implementation of the method through local integration of the level set function and the image/prior-driven evolution forces, where the resulting partial differential equation is solved numerically using standard finite difference method. Experimental results on synthetic and real images demonstrate its superior performance.

The primary goal of dynamic positron emission tomography (PET) is to quantify the physiological and biological processes through tracer kinetics analysis. However, the process is difficult and complicated because of the compromising imaging data quality, i.e. either longer scans with good counting statistics but poor temporal resolution, or noisy shorter scans with good temporal resolution. In this paper, we explore the usage of state space principles for physiological parameter estimation in dynamic PET imaging. The system equation is constructed from particular tracer kinetic models, with the number and relationship between tissue compartments dictated by the physiological and biochemical properties of the process under study. And the observation equation on measurement data is formed based on the specific types of imaging or image-derived data. Once the Poisson distributed PET data are converted to Gaussian ones through the Anscombe transformation, an extended Kalman filter is adopted to estimate the tracer kinetics parameters from the system and observations. More appropriate estimation strategies which better take care of the PET statistics are also under development. The framework is tested on simulated digital phantom data, and the results are of sufficient accuracy and robustness.

We present a novel level set representation and front propagation scheme for active contours where the analysis/evolution domain is sampled by unstructured point cloud. These sampling points are adaptively distributed according to both local data and level set geometry, hence allow extremely convenient enhancement/reduction of local front precision by simply putting more/fewer points on the computation domain without grid refinement (as the cases in finite difference schemes) or remeshing (typical infinite element methods). The front evolution process is then conducted on the point-sampled domain, without the use of computational grid or mesh, through the precise but relatively expensive moving least squares (MLS) approximation of the continuous domain, or the faster yet coarser generalized finite difference (GFD) representation and calculations. Because of the adaptive nature of the sampling point density, our strategy performs fast marching and level set local refinement concurrently. We have evaluated the performance of the method in image segmentation and shape recovery applications using real and synthetic data.

In positron emission tomography (PET), an optimal estimate of the radio activity concentration is obtained from the measured emission data under some criteria. So far, all the well-known reconstruction algorithms require exact known system probability matrix a priori, where the quality of such system model largely determines the quality of the reconstructed images, especially for the least-squares strategies. In this paper, we propose an algorithm for PET reconstruction for the real world case where the PET system model is subject to uncertainties. The method is based on the formulation of PET reconstruction as a regularization problem and the image estimation is achieved with the aid of an uncertainty-weighted least squares framework. The performance of our work is evaluated using the Shepp-Logan simulated phantom data, where it yields significant improvement in image quality over the conventional least-squares reconstruction efforts.

Proper spatial and temporal constraints are essential for image-based motion recovery of deforming objects. Since biological organs, such as the heart, are typically composed of fibrous tissues of anisotropic nature, one must adopt realistic spatial models, in addition to those important considerations for temporal modeling, in order to properly regularize the object behavior for kinematics recovery. We present a biomechanically constrained state space analysis framework for the multiframe estimation of the heart motion and deformation. While the anisotropic physical constraints enforce spatial regulations on the myocardial behavior and spatial filtering of the image data measurements, statistical filtering techniques impose temporal constraints to incorporate multiframe information. Implemented within a mesh-free particle representation and computation framework, excellent experimental results are achieved for both synthetic data with known ground truth and canine magnetic resonance image sequences with known clinical gold standard.

In addition to its technical merits as a challenging nonrigid motion and structural integrity analysis problem, quantitative estimation of cardiac regional functions and material characteristics has significant physiological and clinical values. We earlier developed a stochastic finite element framework for the simultaneous estimation of myocardial motion and material parameters from medical image sequences with an extended Kalman filter approach. In this paper, we present a computational strategy for the framework based upon the maximum a posteriori estimation principles, realized through the extended Kalman smoother, that produce a sequence of kinematics state and material parameter estimates from the entire sequence of observations. The system dynamics equations of the heart is constructed using a biomechanical model with stochastic parameters, and the tissue material and deformation parameters are jointly estimated from the periodic imaging data. Experiments with canine magnetic resonance images have been conducted with very promising results, as validated through comparison to the histological staining of post mortem myocardium.

A biomechanical model constrained stochastic finite element framework has been developed to estimate jointly myocardium kinematics and material parameters from medical image sequences. In an extended Kalman filter formulation, we have observed that the augmented state error covariance matrix must be carefully chosen in order to avoid divergence (Shi, P. and Liu, H., MICCAI, pp.634-41, 2002). We incorporate confidence measures of the input imaging and imaging-derived data into the initialization of the state error covariance matrix. These confidence measures come from the shape-matching process of boundary points and from the local phase coherence of the magnetic resonance velocity images. Experiments with two types of imaging inputs have shown vastly improved filtering efficiency and physiologically meaningful results.

Many of the computer vision algorithms have been posed in various forms of differential equations, derived from minimization of specific energy functionals, and the finite element representation and computation have become the de facto numerical strategies for solving these problems. However, for cases where domain mappings between numerical iterations or image frames involve large geometrical shape changes, such as deformable models for object segmentation and nonrigid motion tracking, these strategies may exhibit considerable loss of accuracy when the mesh elements become extremely skewed or compressed. We present a new computational paradigm, the meshfree particle method, where the object representation and the numerical calculation are purely based on the nodal points and do not require the meshing of the analysis domain. This meshfree strategy can naturally handle large deformation and domain discontinuity issues and achieve desired numerical accuracy through adaptive node and polynomial shape function refinement. We discuss in detail the element-free Galerkin method, including the shape function construction using the moving least square approximation and the Galerkin weak form formulation, and we demonstrate its applications to deformable model based segmentation and mechanically motivated left ventricular motion analysis.

A robust H/sub /spl infin// algorithm for object kinematics estimation from image sequences is presented. The framework relies on both the physical modeling of the object structure and behavior, and the minimization of the worst-case error filtering criterion. By employing the finite element method, the system dynamics of the object is constructed as a set of physically meaningful partial differential equations, which are then converted into continuous- and discrete-time state space representations. In contrast to the popular Kalman filtering strategy which produces the minimum-mean-square-error estimates, the mini-max H/sub /spl infin// filter is adopted which assumes no prior statistics knowledge on the external disturbances. A series of experiments are performed using synthetic data of various noise types and levels to assess the accuracy and robustness of the H/sub /spl infin// filtering framework, and to make comparisons to the Kalman filtering results. Practical applications to magnetic resonance image sequences of the heart are also presented.

We present a velocity-constrained front propagation approach for myocardium segmentation from magnetic resonance intensity image (MRI) and its matching phase contrast velocity (PCV) images. Our curve evolution criterion is dependent on the prior probability distribution of the myocardial boundary and the conditional boundary probability distribution, which is constructed from the MRI intensity gradient, the PCV magnitude, and the local phase coherence of the PCV direction. A two-step boundary finding strategy is employed to facilitate the computation. For the first image frame, a gradient-only fast marching/level set step is used to approach the boundary, and a narrowband is formed around the curve. The initial boundary is then refined using the full information from priors and all three image sources. For the other frames, the resulting contours from the previous frames are used as the initialization contours, and only refinement step is needed. Experiment results from canine MRI sequence are presented, and are compared to results from gradient-only segmentation.