In this paper we present an introduction to Variational Bayesian (VB) methods in the context of probabilistic graphical models, and discuss their application in multimedia related problems. VB is a family of deterministic probability distribution approximation procedures that offer distinct advantages over alternative approaches based on stochastic sampling and those providing only point estimates. VB inference is flexible to be applied in different practical problems, yet is broad enough to subsume as its special cases several alternative inference approaches including Maximum A Posteriori (MAP) and the Expectation-Maximization (EM) algorithm. In this paper we also show the connections between VB and other posterior approximation methods such as the marginalization-based Loopy Belief Propagation (LBP) and the Expectation Propagation (EP) algorithms. Specifically, both VB and EP are variational methods that minimize functionals based on the Kullback-Leibler (KL) divergence. LBP, traditionally developed using graphical models, can also be viewed as a VB inference procedure. We present several multimedia related applications illustrating the use and effectiveness of the VB algorithms discussed herein. We hope that by reading this tutorial the readers will obtain a general understanding of Bayesian methods and establish connections among popular algorithms used in practice.

When a probabilistic model and its prior are given, Bayesian learning offers inference with automatic parameter tuning. However, Bayesian learning is often obstructed by computational difficulty: the rigorous Bayesian learning is intractable in many models, and its variational Bayesian (VB) approximation is prone to suffer from local minima. In this paper, we overcome this difficulty for low-rank subspace clustering (LRSC) by providing an exact global solver and its efficient approximation. LRSC extracts a low-dimensional structure of data by embedding samples into the union of low-dimensional subspaces, and its variational Bayesian variant has shown good performance. We first prove a key property that the VB-LRSC model is highly redundant. Thanks to this property, the optimization problem of VB-LRSC can be separated into small subproblems, each of which has only a small number of unknown variables. Our exact global solver relies on another key property that the stationary condition of each subproblem is written as a set of polynomial equations, which is solvable with the homotopy method. For further computational efficiency, we also propose an efficient approximate variant, of which the stationary condition can be written as a polynomial equation with a single variable. Experimental results show the usefulness of our approach.

In this paper, we consider the problem of clustering data points into low-dimensional subspaces in the presence of outliers. We pose the problem using a density estimation formulation with an associated generative model. Based on this probability model, we first develop an iterative expectation-maximization (EM) algorithm and then derive its global solution. In addition, we develop two Bayesian methods based on variational Bayesian (VB) approximation, which are capable of automatic dimensionality selection. While the first method is based on an alternating optimization scheme for all unknowns, the second method makes use of recent results in VB matrix factorization leading to fast and effective estimation. Both methods are extended to handle sparse outliers for robustness and can handle missing values. Experimental results suggest that proposed methods are very effective in clustering and identifying outliers.

The variational Bayesian (VB) approach is one of the best tractable approximations to the Bayesian estimation, and it was demonstrated to perform well in many applications. However, its good performance was not fully understood theoretically. For example, VB sometimes produces a sparse solution, which is regarded as a practical advantage of VB, but such sparsity is hardly observed in the rigorous Bayesian estimation. In this paper, we focus on probabilistic PCA and give more theoretical insight into the empirical success of VB. More specifically, for the situation where the noise variance is unknown, we derive a sufficient condition for perfect recovery of the true PCA dimensionality in the large-scale limit when the size of an observed matrix goes to infinity. In our analysis, we obtain bounds for a noise variance estimator and simple closed-form solutions for other parameters, which themselves are actually very useful for better implementation of VB-PCA.

We propose a novel design for light field image acquisition based on compressive sensing principles. By placing a randomly coded mask at the aperture of a camera, incoherent measurements of the light passing through different parts of the lens are encoded in the captured images. Each captured image is a random linear combination of different angular views of a scene. The encoded images are then used to recover the original light field image via a novel Bayesian reconstruction algorithm. Using the principles of compressive sensing, we show that light field images with a large number of angular views can be recovered from only a few acquisitions. Moreover, the proposed acquisition and recovery method provides light field images with high spatial resolution and signal-to-noise-ratio, and therefore is not affected by limitations common to existing light field camera designs. We present a prototype camera design based on the proposed framework by modifying a regular digital camera. Finally, we demonstrate the effectiveness of the proposed system using experimental results with both synthetic and real images.

There has been a significant interest in the recovery of low-rank matrices from an incomplete of measurements, due to both theoretical and practical developments demonstrating the wide applicability of the problem. A number of methods have been developed for this recovery problem, however, a principled method for choosing the unknown target rank is generally missing. In this paper, we present a recovery algorithm based on sparse Bayesian learning (SBL) and automatic relevance determination principles. Starting from a matrix factorization formulation and enforcing the low-rank constraint in the estimates as a sparsity constraint, we develop an approach that is very effective in determining the correct rank while providing high recovery performance. We provide empirical results and comparisons with current state-of-the-art methods that illustrate the potential of this approach.

In this paper, we present a novel passive millimeter-wave (PMMW) imaging system designed using compressive sensing principles. We employ randomly encoded masks at the focal plane of the PMMW imager to acquire incoherent measurements of the imaged scene. We develop a Bayesian reconstruction algorithm to estimate the original image from these measurements, where the sparsity inherent to typical PMMW images is efficiently exploited. Comparisons with other existing reconstruction methods show that the proposed reconstruction algorithm provides higher quality image estimates. Finally, we demonstrate with simulations using real PMMW images that the imaging duration can be dramatically reduced by acquiring only a few measurements compared to the size of the image.

Every day, a huge amount of video data is generated for different purposes and applications. Fast and accurate algorithms for efficient video search and retrieval are therefore essential. The interesting properties of sparse representation and the new sampling theory named Compressive Sensing (CS) constitute the core of the new approach to video representation and retrieval we are presenting in this paper. Once the representation (where sparsity is expected) has been chosen and the observations have been taken, the proposed approach utilizes Bayesian modeling and inference to tackle the retrieval problem. In order to speed up the inference process the use of Principal Components Analysis (PCA) to provide an alternative representation of the frames is analyzed. Experimental results validate the proposed approach whose robustness against noise is also examined.

Variational Bayesian matrix factorization (VBMF) efficiently approximates the posterior distribution of factorized matrices by assuming matrix-wise independence of the two factors. A recent study on fully-observed VBMF showed that, under a stronger assumption that the two factorized matrices are column-wise independent, the global optimal solution can be analytically computed. However, it was not clear how restrictive the column-wise independence assumption is. In this paper, we prove that the global solution under matrix-wise independence is actually column-wise independent, implying that the column-wise independence assumption is harmless. A practical consequence of our theoretical finding is that the global solution under matrix-wise independence (which is a standard setup) can be obtained analytically in a computationally very efficient way without any iterative algorithms. We experimentally illustrate advantages of using our analytic solution in probabilistic principal component analysis.

In this paper, we address the super resolution (SR) problem from a set of degraded low resolution (LR) images to obtain a high resolution (HR) image. Accurate estimation of the sub-pixel motion between the LR images significantly affects the performance of the reconstructed HR image. In this paper, we propose novel super resolution methods where the HR image and the motion parameters are estimated simultaneously. Utilizing a bayesian formulation, we model the unknown HR image, the acquisition process, the motion parameters and the unknown model parameters in a stochastic sense. Employing a variational bayesian analysis, we develop two novel algorithms which jointly estimate the distributions of all unknowns. The proposed framework has the following advantages: 1) Through the incorporation of uncertainty of the estimates, the algorithms prevent the propagation of errors between the estimates of the various unknowns; 2) the algorithms are robust to errors in the estimation of the motion parameters; and 3) using a fully bayesian formulation, the developed algorithms simultaneously estimate all algorithmic parameters along with the HR image and motion parameters, and therefore they are fully-automated and do not require parameter tuning. We also show that the proposed motion estimation method is a stochastic generalization of the classical Lucas-Kanade registration algorithm. Experimental results demonstrate that the proposed approaches are very effective and compare favorably to state-of-the-art SR algorithms.

In this paper we propose two fast Total Variation (TV) based algorithms for image restoration by utilizing variational posterior distribution approximation. The unknown image and the hyperparameters for the image and observation models are formulated and estimated simultaneously within a hierachical Bayesian framework, rendering the algorithms fully-automated without any free parameters. Experimental results demonstrate that the proposed algorithms provide restoration results competitive to existing methods in terms of image quality while achieving superior computational efficiency.

In this paper, we propose a symmetrical EEG/fMRI fusion algorithm which combines EEG and fMRI by means of a common generative model. The use of a total variation (TV) prior as well as spatially adaptive temporal priors enables adaptation to the local characteristics of the estimated responses. We utilize an approximate variational Bayesian framework and obtain a fully automatic fusion algorithm. Simulation results demonstrate that the proposed algorithm outperforms existing EEG/fMRI fusion methods.

In this paper we propose a novel Bayesian algorithm for image restoration and parameter estimation. We utilize an image prior where Gaussian distributions are placed per pixel in the high-pass filter outputs of the image. By following the hierarchical Bayesian framework, we simultaneously estimate the unknown image and hyperparameters for both the image prior and the image degradation noise. We show that the proposed formulation is a special case of the popular lp-norm based formulations with p = 0, and therefore enforces sparsity to an high extent in the filtered image coefficients. Moreover, the proposed formulation results in a convex optimization problem, and therefore does not suffer from the robustness issues common with non-convex image priors. Experimental results demonstrate that the proposed algorithm provides superior performance compared to state-of-the-art restoration algorithms although no user-supervision is required.

This paper is devoted to the combination of image priors in Super Resolution (SR) image reconstruction. Taking into account that each combination of a given observation model and a prior model produces a different posterior distribution of the underlying High Resolution (HR) image, the use of variational posterior distribution approximation on each posterior will produce as many posterior approximations as priors we want to combine. A unique approximation is obtained here by finding the distribution on the HR image given the observations that minimizes a linear convex combination of the Kullback-Leibler divergences associated with each posterior distribution. We find this distribution in closed form and also relate the proposed approach to other prior combination methods in the literature. The estimated HR images are compared with images provided by other SR reconstruction methods.

In this paper, we model the components of the compressive sensing (CS) problem, i.e., the signal acquisition process, the unknown signal coefficients and the model parameters for the signal and noise using the Bayesian framework. We utilize a hierarchical form of the Laplace prior to model the sparsity of the unknown signal. We describe the relationship among a number of sparsity priors proposed in the literature, and show the advantages of the proposed model including its high degree of sparsity. Moreover, we show that some of the existing models are special cases of the proposed model. Using our model, we develop a constructive (greedy) algorithm designed for fast reconstruction useful in practical settings. Unlike most existing CS reconstruction methods, the proposed algorithm is fully automated, i.e., the unknown signal coefficients and all necessary parameters are estimated solely from the observation, and, therefore, no user-intervention is needed. Additionally, the proposed algorithm provides estimates of the uncertainty of the reconstructions. We provide experimental results with synthetic 1-D signals and images, and compare with the state-of-the-art CS reconstruction algorithms demonstrating the superior performance of the proposed approach.

Photographs acquired under low-lighting conditions require long exposure times and therefore exhibit significant blurring due to the shaking of the camera. Using shorter exposure times results in sharper images but with a very high level of noise. In this paper we address the problem of utilizing two such images in order to obtain an estimate of the original scene and present a novel blind deconvolution algorithm for solving it. We formulate the problem in a hierarchical Bayesian framework by utilizing prior knowledge on the unknown image and blur, and also on the dependency between the two observed images. By incorporating a fully Bayesian analysis, the developed algorithm estimates all necessary model parameters along with the unknown image and blur, such that no user-intervention is needed. Moreover, we employ a variational Bayesian inference procedure, which allows for the statistical compensation of errors occurring at different stages of the restoration, and also provides uncertainties of the estimates. Experimental results with synthetic and real images demonstrate that the proposed method provides very high quality restoration results and compares favorably to existing methods even though no user supervision is needed.

In this paper we model the components of the compressive sensing (CS) problem using the Bayesian framework by utilizing a hierarchical form of the Laplace prior to model sparsity of the unknown signal. This signal prior includes some of the existing models as special cases and achieves a high degree of sparsity. We develop a constructive (greedy) algorithm resulting from this formulation where necessary parameters are estimated solely from the observation and therefore no user-intervention is needed. We provide experimental results with synthetic 1D signals and images, and compare with the state-of-the-art CS reconstruction algorithms demonstrating the superior performance of the proposed approach.

We propose a novel camera design for light field image acquisition using compressive sensing. By utilizing a randomly coded non-refractive mask in front of the aperture, incoherent measurements of the light passing through different regions are encoded in the captured images. A novel reconstruction algorithm is proposed to recover the original light field image from these acquisitions. Using the principles of compressive sensing, we demonstrate that light field images with high angular dimension can be captured with only a few acquisitions. Moreover, the proposed design provides images with high spatial resolution and signal-to-noise-ratio (SNR), and therefore does not suffer from limitations common to existing light-field camera designs. Experimental results demonstrate the efficiency of the proposed system.

Photographs acquired under low-light conditions require long exposure times and therefore exhibit significant blurring due to the shaking of the camera. Using shorter exposure times results in sharper images but with a very high level of noise. In this paper we address this problem and present a novel blind deconvolution algorithm for a pair of differently exposed images. We formulate the problem in a hierarchical Bayesian framework by utilizing prior knowledge on the unknown image and blur, and also on the dependency between two observed images. By incorporating a fully Bayesian analysis, the developed algorithm estimates all necessary algorithm parameters along with the unknowns, such that no user-intervention is needed. Moreover, we employ a variational Bayesian inference procedure, which allows for the statistical compensation of errors occurring at different stages of the restoration, and also provides uncertainties of the estimates. Experimental results demonstrate the high restoration performance of the proposed algorithm.

Super-resolution (SR) algorithms for compressed video aim at recovering high-frequency information and estimating a high-resolution (HR) image or a set of HR images from a sequence of low-resolution (LR) video frames. In this paper we present a novel SR algorithm for compressed video based on the maximum a posteriori (MAP) framework. We utilize a new multichannel image prior model, along with the state-of-the art image prior and observation models. Moreover, relationship between model parameters and the decoded bitstream are established. Numerical experiments demonstrate the improved performance of the proposed method compared to existing algorithms for different compression ratios.

We present a new image restoration method based on modelling the coefficients of an overcomplete wavelet response to natural images with a mixture of two Gaussian distributions, having non-zero and zero mean respectively, and reflecting the assumption that this response is close to be sparse. Including the observation model, the resulting procedure iterates between image reconstruction from the hard-thresholding of the response to the current estimate and a fast blur compensation step. Results indicate that our method compares favorably with current wavelet-based restoration methods.

In this paper, we present novel algorithms for total variation (TV) based blind deconvolution and parameter estimation utilizing a variational framework. Using a hierarchical Bayesian model, the unknown image, blur, and hyperparameters for the image, blur, and noise priors are estimated simultaneously. A variational inference approach is utilized so that approximations of the posterior distributions of the unknowns are obtained, thus providing a measure of the uncertainty of the estimates. Experimental results demonstrate that the proposed approaches provide higher restoration performance than non-TV-based methods without any assumptions about the unknown hyperparameters.

In this paper we propose novel algorithms for image restoration and parameter estimation with a Generalized Gaussian Markov Random Field (GGMRF) prior utilizing variational distribution approximation. The restored image and the unknown hyperparameters for both the image prior and the image degradation noise are simultaneously estimated within a hierarchical Bayesian framework. We develop two algorithms resulting from this formulation which provide approximations to the posterior distributions of the latent variables. Experimental results are provided to demonstrate the performance of the algorithms.

In this paper we propose a novel algorithm for super resolution based on total variation prior and variational distribution approximations. We formulate the problem using a hierarchical Bayesian model where the reconstructed high resolution image and the model parameters are estimated simultaneously from the low resolution observations. The algorithm resulting from this formulation utilizes variational inference and provides approximations to the posterior distributions of the latent variables. Due to the simultaneous parameter estimation, the algorithm is fully automated so parameter tuning is not required. Experimental results show that the proposed approach outperforms some of the state-of-the-art super resolution algorithms.

For most MR imaging applications multiple surface coils are used to obtain images with high signal-to-noise ratios (SNR). However, signal intensity strongly diminishes with distance. Although there are a number of approaches to combine the surface coil images to obtain a high SNR and bias-free image, most of them are developed in an ad hoc manner and lack a systematic treatment. In this work we propose a new approach, an iterative weighted constrained least squares (WCLS) restoration method, for combining surface coil images. The algorithm is fully automated and outperforms approaches which appeared in the literature.

In this paper, we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. Within the hierarchical Bayesian formulation, the reconstructed image and the unknown hyper parameters for the image prior and the noise are simultaneously estimated. The proposed algorithms provide approximations to the posterior distributions of the latent variables using variational methods. We show that some of the current approaches to TV-based image restoration are special cases of our framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyper parameters and clearly outperform existing methods when additional information is included.

Background modeling and subtraction is a fundamental task in many computer vision and video processing applications. We present a novel probabilistic background modeling and subtraction method that exploits spatial and temporal dependencies between pixels. By using an initial clustering of the background scene, we model each pixel by a mixture of spatiotemporal Gaussian distributions, where each distribution represents locally a region in the neighborhood of the pixel. By extracting the local properties around each pixel, the proposed method obtains accurate models of dynamic backgrounds that are highly effective in detecting foreground objects. Experimental results for indoor and outdoor surveillance videos in comparison with other multimodal methods demonstrate the performance advantages of the proposed method.

In this paper we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. By following the hierarchical Bayesian framework, we simultaneously estimate the reconstructed image and the unknown hyper parameters for both the image prior and the image degradation noise. Our algorithms provide an approximation to the posterior distributions of the unknowns so that both the uncertainty of the estimates can be measured and different values from these distributions can be used for the estimates. We also show that some of the current approaches to TV-based image restoration are special cases of our variational framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyper parameters and clearly outperform existing methods when additional information is included.

In this paper, a new algorithm for lossless compression of hyperspectral images is proposed. The spectral redundancy in hyperspectral images is exploited using a context-match method driven by the correlation between adjacent bands. This method is suitable for hyperspectral images in the band-sequential format. Moreover, this method compares favorably with the recent proposed lossless compression algorithms in terms of compression, with a lower complexity.

In this paper, we propose a compression algorithm focused on the peculiarities of hyperspectral images. The spectral redundancy in hyperspectral images is exploited by using a context matching method driven by the correlation between adjacent bands of hyperspectral spectral images. The method compares favorably with recent proposed lossless compression algorithms in terms of compression, with significantly lower complexity.

This paper describes the predictive image compression using conditional averages. compare the performance of the proposed predictor with the GAP predictor used in CALIC and the MED predictor used in JPEG-LS and LOCO-I.