Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.

In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task.Our method is trained and evaluated on an in-house collected data-set of 34 volunteers and when compared to pure RL and supervised learning (SL) techniques, it performs substantially better, which highlights the suitability of RL navigation for US-guided procedures. When testing our proposed model, we obtained a 82.91% chance of navigating correctly to the sacrum from 165 different starting positions on 5 different unseen simulated environments.

Automated measurement of the intensity of spontaneous facial Action Units (AU) defined by the Facial Action Coding System (FACS) in video sequences is a challenging problem. This paper proposes a person-adaptive methodology for the intensity estimation of spontaneous AUs. We formulate this problem as a source separation problem where we consider the observed AUs as the source signals to be separated from each other and other information given by a sequence of facial images. We first compute an initial estimation of the sources, called observations, using sparse linear regression functions. We then develop and apply a Bayesian source separation method that recruits the prior information of the sources to iteratively improve the initial estimations/observations in an adaptive fashion. Furthermore, our approach adaptively uses some testing information (but not the ground-truth labels) to improve the performance of the approach (i.e., Person-Adaptive model). Our experimental results on DISFA, UNBC-McMaster and FERA2015 databases show that this approach is very promising for automated measurement of the intensity of spontaneous facial AUs.

Association mapping of genetic diseases has attracted extensive research interest during the recent years. However, most of the methodologies introduced so far suffer from spurious inference of the associated sites due to population inhomogeneities. In this paper, we introduce a statistical framework to compensate for this shortcoming by equipping the current methodologies with a state-of-the-art clustering algorithm being widely used in population genetics applications. The proposed framework jointly infers the disease-associated factors and the hidden population structures. In this regard, a Markov Chain-Monte Carlo (MCMC) procedure has been employed to assess the posterior probability distribution of the model parameters. We have implemented our proposed framework on a software package whose performance is extensively evaluated on a number of synthetic datasets, and compared to some of the well-known existing methods such as STRUCTURE. It has been shown that in extreme scenarios, up to $10-15$10-15 percent of improvement in the inference accuracy is achieved with a moderate increase in computational complexity.

Similarity measure is a main core of image registration algorithms. Spatially varying intensity distortion is an important challenge, which affects the performance of similarity measures. Correlation among the pixels is the main characteristic of this distortion. Similarity measures such as sum-of-squared-differences (SSD) and mutual information ignore this correlation; hence, perfect registration cannot be achieved in the presence of this distortion. In this paper, we model this correlation with the aid of the low rank matrix theory. Based on this model, we compensate this distortion analytically and introduce rank-regularized SSD (RRSSD). This new similarity measure is a modified SSD based on singular values of difference image in mono-modal imaging. In fact, image registration and distortion correction are performed simultaneously in the proposed model. Based on our experiments, the RRSSD similarity measure achieves clinically acceptable registration results, and outperforms other state-of-the-art similarity measures, such as the well-known method of residual complexity.

Nonlinear dimensionality reduction methods have demonstrated top-notch performance in many pattern recognition and image classification tasks. Despite their popularity, they suffer from highly expensive time and memory requirements, which render them inapplicable to large-scale datasets. To leverage such cases we propose a new method called "Path-Based Isomap". Similar to Isomap, we exploit geodesic paths to find the low-dimensional embedding. However, instead of preserving pairwise geodesic distances, the low-dimensional embedding is computed via a path-mapping algorithm. Due to the much fewer number of paths compared to number of data points, a significant improvement in time and memory complexity with a comparable performance is achieved. The method demonstrates state-of-the-art performance on well-known synthetic and real-world datasets, as well as in the presence of noise.

Automatic measurement of spontaneous facial action units (AUs) defined by the facial action coding system (FACS) is a challenging problem. The recent FACS user manual defines 33 AUs to describe different facial activities and expressions. In spontaneous facial expressions, a subset of AUs are often occurred or activated at a time. Given this fact that AUs occurred sparsely over time, we propose a novel method to detect the absence and presence of AUs and estimate their intensity levels via sparse representation (SR). We use the robust principal component analysis to decompose expression from facial identity and then estimate the intensity of multiple AUs jointly using a regression model formulated based on dictionary learning and SR. Our experiments on Denver intensity of spontaneous facial action and UNBC-McMaster shoulder pain expression archive databases show that our method is a promising approach for measurement of spontaneous facial AUs.

Similarity measure is an important block in image registration. Most traditional intensity-based similarity measures (e.g., sum-of-squared-difference, correlation coefficient, and mutual information) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation between pixel intensities; hence, perfect image registration cannot be achieved, especially in the presence of spatially varying intensity distortions. Here, we assume that spatially varying intensity distortion (such as bias field) is a low-rank matrix. Based on this assumption, we formulate the image registration problem as a nonlinear and low-rank matrix decomposition (NLLRMD). Therefore, image registration and correction of spatially varying intensity distortion are simultaneously achieved. We illustrate the uniqueness of NLLRMD, and therefore, we propose the rank of difference image as a robust similarity in the presence of spatially varying intensity distortion. Finally, by incorporating the Gaussian noise, we introduce rank-induced similarity measure based on the singular values of the difference image. This measure produces clinically acceptable registration results on both simulated and real-world problems examined in this paper, and outperforms other state-of-the-art measures such as the residual complexity approach.

Automatic recognition of facial expression and facial identity from visual data are two challenging problems that are tied together. In the past decade, researchers have mostly tried to solve these two problems separately to come up with face identification systems that are expression-independent and facial expressions recognition systems that are person-independent. This paper presents a new framework using sparse representation for simultaneous recognition of facial expression and identity. Our framework is based on the assumption that any facial appearance is a sparse combination of identities and expressions (i.e., one identity and one expression). Our experimental results using the CK+ and MMI face datasets show that the proposed approach outperforms methods that conduct face identification and face recognition individually.

Similarity measure is an important key in image registration. Most traditional intensity‐based similarity measures (e.g. sum‐of‐squared‐differences, correlation coefficient, mutual information and correlation ratio) assume a stationary image and pixel‐by‐pixel independence. These similarity measures ignore the correlation among pixel intensities; hence, a perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions and outlier objects that appear in one image but not in the other. It is supposed here that non‐stationary intensity distortion (such as bias field) has a sparse representation in the transformation domain. Based on this assumption, a novel similarity measure is proposed here based on sparse representation in a mono‐modal setting. The zero norm (ℓ 0 ) in the transform domain is introduced as a new similarity measure in the presence of non‐stationary intensity distortion. The present study defines a sparsity similarity measure that indicates the complexity of the residual image between two registered images in the transform domain such as discrete cosine transform or wavelet. It is attempted here to analytically and statistically illustrate that the proposed similarity measure has important properties such as metric properties in vector space, from correntropy perspective. This measure produces accurate registration results on both artificial and real‐world problems which are examined in the present study.

Independent Component Analysis (ICA) can be used to decompose functional Magnetic Resonance Imaging (fMRI) data into a set of statistically independent images which are likely to be the sources of fMRI data. After applying ICA, a set of independent components are produced, and then, a “meaningful” subset from these components must be identified, because a large majority of components are non-interesting. So, interpreting the components is an important and also difficult task. In this paper, we propose a criterion based on the entropy of time courses to automatically select the components of interest. This method does not require to know the stimulus pattern of the experiment.

Clustering approaches are going to efficiently define the activated regions of the brain in fMRI studies. However, choosing appropriate clustering algorithms and defining optimal number of clusters are still key problems of these methods. In this paper, we apply an improved version of Growing Neural Gas algorithm, which automatically operates on the optimal number of clusters. The decision criterion for creating new clusters at the heart of this online clustering is taken from MB cluster validity index. Comparison with other so-called clustering methods for fMRI data analysis shows that the proposed algorithm outperforms them in both artificial and real datasets.

In this study, we have proposed a novel approach to investigate the features of four subbands of 2-D wavelet transform in magnetic resonance images (MRIs) for normal and abnormal brains which defected by multiple sclerosis (MS). Concurrently, another method extracts different kinds of features in spatial domain. Totally, 116 features have been extracted. Before applying the algorithm, we have to use a registration method because of variety in size of brain images. All extracted features have been passed over the principal component analysis (PCA) and have been pushed to an artificial neural network (ANN) that is a feed-forward type. According to changing in position of defected parts of brain, we have analyzed four different MRI datasets in different stages of MS progression, including 101 MRIs of normal and abnormal brain images. In all cases, certain diagnosis is gained. Meantime, 40 percent of the datasets have been reserved as the "test data ".

A new method for automatic landmark extraction from MR brain images is presented. In this method, landmark extraction is accomplished by modifying growing neural gas (GNG), which is a neural-network-based cluster-seeking algorithm. Using modified GNG (MGNG) corresponding dominant points of contours extracted from two corresponding images are found. These contours are borders of segmented anatomical regions from brain images. The presented method is compared to: 1) the node splitting-merging Kohonen model and 2) the Teh-Chin algorithm (a well-known approach for dominant points extraction of ordered curves). It is shown that the proposed algorithm has lower distortion error, ability of extracting landmarks from two corresponding curves simultaneously, and also generates the best match according to five medical experts.