In image-guided robotic surgery, segmenting the endoscopic video stream into meaningful parts provides important contextual information that surgeons can exploit to enhance their perception of the surgical scene. This information provides surgeons with real-time decision-making guidance before initiating critical tasks such as tissue cutting. Segmenting endoscopic video is a challenging problem due to a variety of complications including significant noise attributed to bleeding and smoke from cutting, poor appearance contrast between different tissue types, occluding surgical tools, and limited visibility of the objects' geometries on the projected camera views. In this paper, we propose a multi-modal approach to segmentation where preoperative 3D computed tomography scans and intraoperative stereo-endoscopic video data are jointly analyzed. The idea is to segment multiple poorly visible structures in the stereo/multichannel endoscopic videos by fusing reliable prior knowledge captured from the preoperative 3D scans. More specifically, we estimate and track the pose of the preoperative models in 3D and consider the models' non-rigid deformations to match with corresponding visual cues in multi-channel endoscopic video and segment the objects of interest. Further, contrary to most augmented reality frameworks in endoscopic surgery that assume known camera parameters, an assumption that is often violated during surgery due to non-optimal camera calibration and changes in camera focus/zoom, our method embeds these parameters into the optimization hence correcting the calibration parameters within the segmentation process. We evaluate our technique on synthetic data, ex vivo lamb kidney datasets, and in vivo clinical partial nephrectomy surgery with results demonstrating high accuracy and robustness.

A fundamental means for understanding the brain's organizational structure is to group its spatially disparate regions into functional subnetworks based on their interactions. Most community detection techniques are designed for generating partitions, but certain brain regions are known to interact with multiple subnetworks. Thus, the brain's underlying subnetworks necessarily overlap. In this paper, we propose a technique for identifying overlapping subnetworks from weighted graphs with statistical control over false node inclusion. Our technique improves upon the replicator dynamics formulation by incorporating a graph augmentation strategy to enable subnetwork overlaps, and a graph incrementation scheme for merging subnetworks that might be falsely split by replicator dynamics due to its stringent mutual similarity criterion in defining subnetworks. To statistically control for inclusion of false nodes into the detected subnetworks, we further present a procedure for integrating stability selection into our subnetwork identification technique. We refer to the resulting technique as stable overlapping replicator dynamics (SORD). Our experiments on synthetic data show significantly higher accuracy in subnetwork identification with SORD than several state-of-the-art techniques. We also demonstrate higher test-retest reliability in multiple network measures on the Human Connectome Project data. Further, we illustrate that SORD enables identification of neuroanatomically-meaningful subnetworks and network hubs.

Noise confounds present serious complications to functional magnetic resonance imaging (fMRI) analysis. The amount of discernible signals within a single dataset of a subject is often inadequate to obtain satisfactory intra-subject activation detection. To remedy this limitation, we propose a novel group Markov random field (GMRF) that extends each subject's neighborhood system to other subjects to enable information coalescing. A distinct advantage of GMRF over standard fMRI group analysis is that no stringent one-to-one voxel correspondence is required. Instead, intra- and inter-subject neighboring voxels are jointly regularized to encourage spatially proximal voxels to be assigned similar labels across subjects. Our proposed group-extended graph structure thus provides an effective means for handling inter-subject variability. Also, adopting a group-wise approach by integrating group information into intra-subject activation, as opposed to estimating a single average group map, permits inter-subject differences to be characterized and studied. GMRF can be elegantly implemented as a single MRF, thus enabling all subjects' activation maps to be simultaneously and collaboratively segmented with global optimality guaranteed in the case of binary labeling. We validate our technique on synthetic and real fMRI data and demonstrate GMRF's superior performance over standard fMRI analysis.

Functional magnetic resonance imaging (fMRI) is increasingly used for studying functional integration of the brain. However, large inter-subject variability in functional connectivity, particularly in disease populations, renders detection of representative group networks challenging. In this paper, we propose a novel technique, "group replicator dynamics" (GRD), for detecting sparse functional brain networks that are common across a group of subjects. We extend the replicator dynamics (RD) approach, which we show to be a solution of the nonnegative sparse principal component analysis problem, by integrating group information into each subject's RD process. Our proposed strategy effectively coaxes all subjects' networks to evolve towards the common network of the group. This results in sparse networks comprising the same brain regions across subjects yet with subject-specific weightings of the identified brain regions. Thus, in contrast to traditional averaging approaches, GRD enables inter-subject variability to be modeled, which facilitates statistical group inference. Quantitative validation of GRD on synthetic data demonstrated superior network detection performance over standard methods. When applied to real fMRI data, GRD detected task-specific networks that conform well to prior neuroscience knowledge.

The perplexing effects of noise and high feature dimensionality greatly complicate functional magnetic resonance imaging (fMRI) classification. In this paper, we present a novel formulation for constructing “Generalized Group Sparse Classifiers” (GSSC) to alleviate these problems. In particular, we propose an extension of group LASSO that permits associations between features within (predefined) groups to be modeled. Integrating this new penalty into classifier learning enables incorporation of additional prior information beyond group structure. In the context of fMRI, GGSC provides a flexible means for modeling how the brain is functionally organized into specialized modules (i.e. groups of voxels) with spatially proximal voxels often displaying similar level of brain activity (i.e. feature associations). Applying GSSC to real fMRI data improved predictive performance over standard classifiers, while providing more neurologically interpretable classifier weight patterns. Our results thus demonstrate the importance of incorporating prior knowledge into classification problems.

Noise confounds present serious complications to accurate data analysis in functional magnetic resonance imaging (fMRI). Simply relying on contextual image information often results in unsatisfactory segmentation of active brain regions. To remedy this, we propose a novel Group Markov Random Field (Group MRF) that extends the neighborhood system to other subjects to incorporate group information in modeling each subject's brain activation. Our approach has the distinct advantage of being able to regularize the states of both intra- and inter-subject neighbors without having to create a stringent one-to-one voxel correspondence as in standard fMRI group analysis. Also, our method can be efficiently implemented as a single MRF, hence enabling activation maps of a group of subjects to be simultaneously and collaboratively segmented. We validate on both synthetic and real fMRI data and demonstrate superior performance over standard analysis techniques.

We present a novel 3-D region-based hidden Markov model (rbHMM) for efficient unsupervised 3-D image segmentation. Our contribution is twofold. First, rbHMM employs a more efficient representation of the image data than current state-of-the-art HMM-based approaches that are based on either voxels or rectangular lattices/grids, thus resulting in a faster optimization process. Second, our proposed novel tree-structured parameter estimation algorithm for the rbHMM provides a locally optimal data labeling that is invariant to object rotation, which is a highly valuable property in segmentation tasks, especially in medical imaging where the segmentation results need to be independent of patient positioning in scanners in order to minimize methodological variability in data analysis. We demonstrate the advantages of our proposed technique over grid-based HMMs by validating on synthetic images of geometric shapes as well as both simulated and clinical brain MRI scans. For the geometric shapes data, our method produced consistently accurate segmentation results that were also invariant to object rotation. For the brain MRI data, our white matter and gray matter segmentation resulted in substantially higher robustness and accuracy levels with improved Dice similarity indices of 4.60% (p=0.0022) and 7.71% , respectively.

We present a novel 3-D scalable compression method for medical images with optimized volume of interest (VOI) coding. The method is presented within the framework of interactive telemedicine applications, where different remote clients may access the compressed 3-D medical imaging data stored on a central server and request the transmission of different VOIs from an initial lossy to a final lossless representation. The method employs the 3-D integer wavelet transform and a modified EBCOT with 3-D contexts to create a scalable bit-stream. Optimized VOI coding is attained by an optimization technique that reorders the output bit-stream after encoding, so that those bits belonging to a VOI are decoded at the highest quality possible at any bit-rate, while allowing for the decoding of background information with peripherally increasing quality around the VOI. The bit-stream reordering procedure is based on a weighting model that incorporates the position of the VOI and the mean energy of the wavelet coefficients. The background information with peripherally increasing quality around the VOI allows for placement of the VOI into the context of the 3-D image. Performance evaluations based on real 3-D medical imaging data showed that the proposed method achieves a higher reconstruction quality, in terms of the peak signal-to-noise ratio, than that achieved by 3D-JPEG2000 with VOI coding, when using the MAXSHIFT and general scaling-based methods.

We recently proposed a symmetry-based scalable lossless compression method for 3D medical images using the 2D integer wavelet transform and the embedded block coder with optimized truncation (EBCOT). In this paper, we present two major contributions that enhance our early work: 1) a new block-based intra-band prediction method that exploits the global and local symmetries of the wavelet-transform sub-bands based on the main axis of symmetry as detected using the analytical Fourier-Mellin transform; and 2) a new inter-slice DPCM prediction method that exploits the correlation between slices. Performance evaluations on real 3D medical images show an average improvement of up to 17% in lossless compression ratios when compared to the state-of-the-art compression methods including 3D-JPEG2000, JPEG2000 and H.264 intra-coding.

We recently proposed a method for lossless compression of 4-D medical images based on the advanced video coding standard (H.264/AVC). In this paper, we present two major contributions that enhance our previous work for compression of functional MRI (fMRI) data: 1) a new multiframe motion compensation process that employs 4-D search, variable-size block matching, and bidirectional prediction; and 2) a new context-based adaptive binary arithmetic coder designed for lossless compression of the residual and motion vector data. We validate our method on real fMRI sequences of various resolutions and compare the performance to two state-of-the-art methods: 4D-JPEG2000 and H.264/AVC. Quantitative results demonstrate that our proposed technique significantly outperforms current state of the art with an average compression ratio improvement of 13%.

A novel approach is proposed for quantitatively characterizing the spatial patterns of activation statistics in functional magnetic resonance imaging (fMRI) activation maps. Specifically, we propose using 3-D invariant moment descriptors, as opposed to the traditionally-employed magnitude-based features such as mean voxel statistics or percentage of activated voxels, to characterize the task-specific spatial distribution of activation statistics within a given region of interest (ROI). The proposed method is applied to real fMRI data collected from 21 healthy subjects performing previously-learned right-handed finger tapping sequences that are either externally guided (EG) by a cue or internally guided (IG)--tasks expected to incur subtle differences in motor-related cortical and subcortical ROIs. Voxel-based activation statistics contrasting EG versus rest and IG versus rest are examined in multiple manually-drawn ROIs on unwarped brain images. Analyzing the activation statistics within each ROI using the proposed 3-D invariant moment descriptors detected significant group differences between the two tasks, thus quantitatively demonstrating that the spatial distribution of activation statistics within an ROI represent an important task-related attribute of brain activation. In contrast, conventional methods that solely rely on activation statistic magnitudes and disregard spatial information showed reduced discriminability. Normally, incorporating spatial information would merely increase inter-subject variability partly due to differences in brain size and subject's orientation in the scanner. Yet, our results suggest that the proposed spatial features, which are invariant to similarity transformations, can effectively account for such inter-subject variability, while enhancing the sensitivity in detecting task-specific activation. Thus, we argue that this novel quantitative description of the "3-D texture" of activation maps provides new directions to explore for ROI-based fMRI analysis.

We propose a novel symmetry-based technique for scalable lossless compression of 3D medical image data. The proposed method employs the 2D integer wavelet transform to decorrelate the data and an intraband prediction method to reduce the energy of the sub-bands by exploiting the anatomical symmetries typically present in structural medical images. A modified version of the embedded block coder with optimized truncation (EBCOT), tailored according to the characteristics of the data, encodes the residual data generated after prediction to provide resolution and quality scalability. Performance evaluations on a wide range of real 3D medical images show an average improvement of 15% in lossless compression ratios when compared to other state-of-the art lossless compression methods that also provide resolution and quality scalability including 3D-JPEG2000, JPEG2000, and H.264/AVC intra-coding.

Dynamic volumetric (four dimensional- 4D) medical images are typically huge in file size and require a vast amount of resources for storage and transmission purposes. In this paper, we propose an efficient lossless compression method for 4D medical images that is based on a multi-frame motion compensation process employing a 4D search, variable block- sizes and bi-directional prediction. Data redundancies are reduced by recursively applying multi-frame motion compensation in the spatial and temporal dimensions. The proposed method also uses a novel differential coding algorithm to reduce redundancies in motion vectors and a new context-based adaptive binary arithmetic coder (CABAC) for compression of the residual data. Performance evaluations on real medical images of varying modality resulted in lossless compression ratios of up to 16:1.

This paper presents an efficient lossless compression method for 4-D medical images based on the advanced video coding scheme (H.264/AVC). The proposed method efficiently reduces data redundancies in all four dimensions by recursively applying multiframe motion compensation. Performance evaluations on real 4-D medical images of varying modalities including functional magnetic resonance show an improvement in compression efficiency of up to three times that of other state-of-the-art compression methods such as 3D-JPEG2000.

Four dimensional (4D) medical data are sequences of volumetric images captured in time. These data sets are typically very large in size and demand a great amount of resources for storage and transmission. In this paper, we present a lossless compression technique for 4D medical images which is based on the H.264/AVC video coding standard. Our lossless compression technique efficiently exploits spatial and temporal redundancies between 2D image slices and 3D images in 4D medical images and eliminates any concerns regarding the effects of compression on image quality for diagnostic purposes. Performance evaluations have shown that the proposed compression technique outperforms current 4D compression methods by 70%