Multiple Instance Learning (MIL) is a powerful framework for weakly supervised learning, particularly useful when fine-grained annotations are unavailable. Despite growing interest in deep MIL methods, the field lacks standardized tools for model development, evaluation, and comparison, which hinders reproducibility and accessibility. To address this, we present torchmil , an open-source Python library built on PyTorch. torchmil offers a unified, modular, and extensible framework, featuring basic building blocks for MIL models, a standardized data format, and a curated collection of benchmark datasets and models. The library includes comprehensive documentation and tutorials to support both practitioners and researchers. torchmil aims to accelerate progress in MIL and lower the entry barrier for new users. Available at https://torchmil.readthedocs.io .

The Multiple Instance Learning (MIL) paradigm is attracting plenty of attention in medical imaging classification, where labeled data is scarce. MIL methods cast medical images as bags of instances (e.g. patches in whole slide images, or slices in CT scans), and only bag labels are required for training. Deep MIL approaches have obtained promising results by aggregating instance-level representations via an attention mechanism to compute the bag-level prediction. These methods typically capture both local interactions among adjacent instances and global, long-range dependencies through various mechanisms. However, they treat attention values deterministically, potentially overlooking uncertainty in the contribution of individual instances. In this work we propose a novel probabilistic framework that estimates a probability distribution over the attention values, and accounts for both global and local interactions. In a comprehensive evaluation involving eleven state-of-the-art baselines and three medical datasets, we show that our approach achieves top predictive performance in different metrics. Moreover, the probabilistic treatment of the attention provides uncertainty maps that are interpretable in terms of illness localization.

Deep learning (DL) networks have recently achieved excellent performance on image compressed sensing. However, most existing methods rely on burdened and complex network structures, resulting in significant computational and storage requirements that defeat the purpose of compressed sensing. This severely hinders their applicability in real-world resource-limited devices. In this paper, a lightweight cycle network driven by physical and deep priors for image compressed sensing is proposed which integrates the learning of the sensing matrix and compressive image reconstruction. Specifically, the regularization terms and a likelihood term derived from the physical observation model are learned in an end-to-end cycle network, simultaneously estimating the reconstructed image and sensing matrix in the image and feature domains. Moreover, a dual-domain fusion reconstruction module is proposed. It creates simulated measurement residuals for enhancing reconstruction in the compressed domain, which leads to high reconstruction performance and reduces computational load by bonding together the compressed image domains in the cyclic network. Extensive experiments demonstrate that our model delivers superior performance and alleviates model complexity, which is of great importance in low-budget applications.

In this paper we propose the use of the Hyperbolic Secant (HS) distribution as a prior for the Blind Image Deconvolution (BID) problem. It is well-known that when high-pass filters are applied to natural images, the resulting coefficients are sparse. We leverage this property using the HS distribution, a seldom explored Super Gaussian distribution with suitable properties for this problem. Using the Pólya-Gamma distribution, we derive an explicit Gaussian Scale Mixture representation. This representation is then used to propose a novel variational Bayesian algorithm that outperforms state-of-the-art BID methods.

Multiple Instance Learning (MIL) is widely used in medical imaging classification to reduce the labeling effort. While only bag labels are available for training, one typically seeks predictions at both bag and instance levels (classification and localization tasks, respectively). Early MIL methods treated the instances in a bag independently. Recent methods account for global and local dependencies among instances. Although they have yielded excellent results in classification, their performance in terms of localization is comparatively limited. We argue that these models have been designed to target the classification task, while implications at the instance level have not been deeply investigated. Motivated by a simple observation -- that neighboring instances are likely to have the same label -- we propose a novel, principled, and flexible mechanism to model local dependencies. It can be used alone or combined with any mechanism to model global dependencies (e.g., transformers). A thorough empirical validation shows that our module leads to state-of-the-art performance in localization while being competitive or superior in classification. Our code is at https://github.com/Franblueee/SmMIL.

Multiple Instance Learning (MIL) is a weakly supervised paradigm that has been successfully applied to many different scientific areas and is particularly well suited to medical imaging. Probabilistic MIL methods, and more specifically Gaussian Processes (GPs), have achieved excellent results due to their high expressiveness and uncertainty quantification capabilities. One of the most successful GP-based MIL methods, VGPMIL, resorts to a variational bound to handle the intractability of the logistic function. Here, we formulate VGPMIL using Pólya-Gamma random variables. This approach yields the same variational posterior approximations as the original VGPMIL, which is a consequence of the two representations that the Hyperbolic Secant distribution admits. This leads us to propose a general GP-based MIL method that takes different forms by simply leveraging distributions other than the Hyperbolic Secant one. Using the Gamma distribution we arrive at a new approach that obtains competitive or superior predictive performance and efficiency. This is validated in a comprehensive experimental study including one synthetic MIL dataset, two well-known MIL benchmarks, and a real-world medical problem. We expect that this work provides useful ideas beyond MIL that can foster further research in the field.

Hematoxylin and Eosin (H&E) color variation among histological images from different laboratories can significantly degrade the performance of Computer-Aided Diagnosis systems. The staining procedure is the primary factor responsible for color variation, and consequently, the methods designed to reduce such variations are designed in concordance with this procedure. In particular, Blind Color Deconvolution (BCD) methods aim to identify the true underlying colors in the image and to separate the tissue structure from the color information. Unfortunately, BCD methods often assume that images are stained solely with pure staining colors (e.g., blue and pink for H&E). This assumption does not hold true when common artifacts such as blood are present, requiring an additional color component to represent them. This is a challenge for color standardization algorithms, which are unable to correctly identify the stains in the image, leading to unexpected results. In this work, we propose a Blood-Robust Bayesian K-Singular Value Decomposition model designed to simultaneously detect blood and extract color from histological images while preserving structural details. We evaluate our method using both synthetic and real images, which contain varying amounts of blood pixels. • Blind Color Deconvolution separates color from structure in histological images. • Few BCD methods consider the effects of noise and artifacts. • We propose a blood robust approach that combines blood detection with BCD. • The proposed approach significantly improves the quality of stain separation in the presence of blood.

Real-time video services are gaining popularity in our daily life, yet limited network bandwidth can constrain the delivered video quality. Video Super Resolution (VSR) technology emerges as a key solution to enhance user experience by reconstructing high-resolution (HR) videos. The existing real-time VSR frameworks have primarily emphasized spatial quality metrics like PSNR and SSIM, which often lack consideration of temporal coherence, a critical factor for accurately reflecting the overall quality of super-resolved videos. Inspired by Video Quality Assessment (VQA) strategies, we propose a dual-frame training framework and a lightweight multi-branch network to address VSR processing in real time. Such designs thoroughly leverage the spatio-temporal correlations between consecutive frames so as to ensure efficient video restoration. Furthermore, we incorporate ST-RRED, a powerful VQA approach that separately measures spatial and temporal consistency aligning with human perception principles, into our loss functions. This guides us to synthesize quality-aware perceptual features across both space and time for realistic reconstruction. Our model demonstrates remarkable efficiency, achieving near real-time processing of 4K videos. Compared to the state-of-the-art lightweight model MRVSR, ours is more compact and faster, 60% smaller in size (0.483M vs. 1.21M parameters), and 106% quicker (96.44fps vs. 46.7fps on 1080p frames), with significantly improved perceptual quality.

Multiple instance learning (MIL) is a weakly supervised learning paradigm that is becoming increasingly popular because it requires less labeling effort than fully supervised methods. This is especially interesting for areas where the creation of large annotated datasets remains challenging, as in medicine. Although recent deep learning MIL approaches have obtained state-of-the-art results, they are fully deterministic and do not provide uncertainty estimations for the predictions. In this work, we introduce the attention Gaussian process (AGP) model, a novel probabilistic attention mechanism based on Gaussian processes (GPs) for deep MIL. AGP provides accurate bag-level predictions as well as instance-level explainability and can be trained end-to-end. Moreover, its probabilistic nature guarantees robustness to overfit on small datasets and uncertainty estimations for the predictions. The latter is especially important in medical applications, where decisions have a direct impact on the patient's health. The proposed model is validated experimentally as follows. First, its behavior is illustrated in two synthetic MIL experiments based on the well-known MNIST and CIFAR-10 datasets, respectively. Then, it is evaluated in three different real-world cancer detection experiments. AGP outperforms state-of-the-art MIL approaches, including deterministic deep learning ones. It shows a strong performance even on a small dataset with less than 100 labels and generalizes better than competing methods on an external test set. Moreover, we experimentally show that predictive uncertainty correlates with the risk of wrong predictions, and therefore it is a good indicator of reliability in practice. Our code is publicly available.

Medical image segmentation is a challenging task, particularly due to inter- and intra-observer variability, even between medical experts. In this paper, we propose a novel model, called Probabilistic Inter-Observer and iNtra-Observer variation NetwOrk (Pionono). It captures the labeling behavior of each rater with a multidimensional probability distribution and integrates this information with the feature maps of the image to produce probabilistic segmentation predictions. The model is optimized by variational inference and can be trained end-to-end. It outperforms state-of-the-art models such as STAPLE, Probabilistic U-Net, and models based on confusion matrices. Additionally, Pionono predicts multiple coherent segmentation maps that mimic the rater’s expert opinion, which provides additional valuable information for the diagnostic process. Experiments on real-world cancer segmentation datasets demonstrate the high accuracy and efficiency of Pionono, making it a powerful tool for medical image analysis.

This paper proposes a deep learning model for robust image restoration when the degradation is not precisely known. We show how the Moore-Penrose pseudo-inverse of a blur convolution operator can be approximated by a Wiener filter’s impulse response. The image restoration problem is then cast as the learning of a residual on the frequencies where the blurring filter is zero which, when added to the Wiener restoration, will satisfy the image formation model. A Dynamic Filter Network removes artifacts introduced by inaccurate blur estimations and other image formation model inconsistencies. The experiments conducted on synthetic and real image datasets assert the performance and robustness of the proposed method and show its superiority to existing ones.

Histological images are often tainted with two or more stains to reveal their underlying structures and conditions. Blind Color Deconvolution (BCD) techniques separate colors (stains) and structural information (concentrations), which is useful for the processing, data augmentation, and classification of such images. Classical BCD methods rely on a complicated optimization procedure that has to be carried out on each image independently, i.e., they are not amortized methods. In contrast, once they have been trained, deep neural networks can be used in a fast, amortized manner on unseen inputs. Unfortunately, the lack of large databases of ground truth color and concentrations has limited the development of deep models for BCD. In this work, we propose a deep variational Bayesian BCD neural network (BCD-Net) for stain separation and concentration estimation. BCD-Net is trained by maximizing the evidence lower bound of the observed images, which does not require the use of ground truth examples of stains and concentrations. Results obtained using two multicenter databases (Camelyon-17 and a stain separation benchmark) demonstrate the effectiveness of BCD-Net in the stain separation tasks, while drastically reducing the computation time compared to classical non-amortized methods.

Network anomaly detection is a very relevant research area nowadays, especially due to its multiple applications in the field of network security. The boost of new models based on variational autoencoders and generative adversarial networks has motivated a reevaluation of traditional techniques for anomaly detection. It is, however, essential to be able to understand these new models from the perspective of the experience attained from years of evaluating network security data for anomaly detection. In this paper, we revisit anomaly detection techniques based on PCA from a probabilistic generative model point of view, and contribute a mathematical model that relates them. Specifically, we start with the probabilistic PCA model and explain its connection to the Multivariate Statistical Network Monitoring (MSNM) framework. MSNM was recently successfully proposed as a means of incorporating industrial process anomaly detection experience into the field of networking. We have evaluated the mathematical model using two different datasets. The first, a synthetic dataset created to better understand the analysis proposed, and the second, UGR’16, is a specifically designed real-traffic dataset for network security anomaly detection. We have drawn conclusions that we consider to be useful when applying generative models to network security detection.

The automatic classification of volcano-seismic events is a key problem in volcanology. Due to its complexity, deep learning (DL) techniques have become the tool of choice for this problem, outperforming classical classifiers. The main drawback of this approach, when applied to the classification of volcano-seismic events, is its tendency to overfit because of the small-size available databases. In this work, we propose and analyze the use of the Gaussian processes (GPs) and Deep GPs (DGPs), and their hierarchical extension, for volcano-seismic event classification. We empirically prove the adequacy of the proposed modeling with an insightful and exhaustive comparison with state-of-the-art DL-based methods on a seismic database recorded at “Volcán de Fuego,” Colima, Mexico. The hierarchical structure of DGPs and the reduced number of parameters to be automatically estimated become essential to achieve excellent performance even on small databases, capturing well the complex patterns of seismic signals for all classes and, in particular, for those that have been hardly observed.

Color deconvolution aims at separating multi-stained images into single stained ones. In digital histopathological images, true stain color vectors vary between images and need to be estimated to obtain stain concentrations and separate stain bands. These band images can be used for image analysis purposes and, once normalized, utilized with other multi-stained images (from different laboratories and obtained using different scanners) for classification purposes. In this paper we propose the use of Super Gaussian (SG) priors for each stain concentration together with the similarity to a given reference matrix for the color vectors. Variational inference and an evidence lower bound are utilized to automatically estimate all the latent variables. The proposed methodology is tested on real images and compared to classical and state-of-the-art methods for histopathological blind image color deconvolution.

Most whole-slide histological images are stained with two or more chemical dyes. Slide stain separation or color deconvolution is a crucial step within the digital pathology workflow. In this paper, the blind color deconvolution problem is formulated within the Bayesian framework. Starting from a multi-stained histological image, our model takes into account both spatial relations among the concentration image pixels and similarity between a given reference color-vector matrix and the estimated one. Using Variational Bayes inference, three efficient new blind color deconvolution methods are proposed which provide automated procedures to estimate all the model parameters in the problem. A comparison with classical and current state-of-the-art color deconvolution algorithms using real images has been carried out demonstrating the superiority of the proposed approach.

Due to their limited depth of field, conventional brightfield microscopes cannot image thick specimens entirely in focus. A common way to obtain an all-in-focus image is to acquire a z-stack of images by optically sectioning the specimen and then apply a multi-focus fusion method. Unfortunately, for undersampled image stacks, fusion methods cannot remove the blur in regions where the in-focus position is between two optical sections. In this work, we propose a parameter-free Gaussian PSF model in which the all-in-focus image together with both the depth map and sampling distances in image plane are estimated from the image sequence automatically, without knowledge on the z-stack acquisition. In a maximum a posteriori framework, an iteratively reweighted least squares method is used to estimate the image and an adaptive scaled gradient descent method is utilized to estimate the depth map and sampling distances efficiently. Experiments on synthetic and real data demonstrate that the proposed method outperforms the current state-of-the-art, mitigating fusion artifacts and recovering sharper edges.

Deep Learning techniques and more specifically Generative Adversarial Networks (GANs) have recently been used for solving the video super-resolution (VSR) problem. In some of the published works, feature-based perceptual losses have also been used, resulting in promising results. While there has been work in the literature incorporating temporal information into the loss function, studies which make use of the spatial activity to improve GAN models are still lacking. Towards this end, this paper aims to train a GAN guided by a spatially adaptive loss function. Experimental results demonstrate that the learned model achieves improved results with sharper images, fewer artifacts and less noise.

While high and ultra high definition displays are becoming popular, most of the available content has been acquired at much lower resolutions. In this work we propose to pseudo-invert with regularization the image formation model using GANs and perceptual losses. Our model, which does not require the use of motion compensation, utilizes explicitly the low resolution image formation model and additionally introduces two feature losses which are used to obtain perceptually improved high resolution images. The experimental validation shows that our approach outperforms current video super resolution learning based models.

While Deep Neural Networks trained for solving inverse imaging problems (such as super-resolution, denoising, or inpainting tasks) regularly achieve new state-of-the-art restoration performance, this increase in performance is often accompanied with undesired artifacts generated in their solution. These artifacts are usually specific to the type of neural network architecture, training, or test input image used for the inverse imaging problem at hand. In this paper, we propose a fast, efficient post-processing method for reducing these artifacts. Given a test input image and its known image formation model, we fine-tune the parameters of the trained network and iteratively update them using a data consistency loss. We show that in addition to being efficient and applicable to large variety of problems, our post-processing through fine-tuning approach enhances the solution originally provided by the neural network by maintaining its restoration quality while reducing the observed artifacts, as measured qualitatively and quantitatively.

Passive millimeter wave images (PMMWIs) can be used to detect and localize objects concealed under clothing. Unfortunately, the quality of the acquired images and the unknown position, shape, and size of the hidden objects render these tasks challenging. In this paper, we discuss a deep learning approach to this detection/localization problem. The effect of the nonstationary acquisition noise on different architectures is analyzed and discussed. A comparison with shallow architectures is also presented. The achieved detection accuracy defines a new state of the art in object detection on PMMWIs. The low computational training and testing costs of the solution allow its use in real-time applications.

Video super-resolution (VSR) has become one of the most critical problems in video processing. In the deep learning literature, recent works have shown the benefits of using adversarial-based and perceptual losses to improve the performance on various image restoration tasks; however, these have yet to be applied for video super-resolution. In this paper, we propose a generative adversarial network (GAN)-based formulation for VSR. We introduce a new generator network optimized for the VSR problem, named VSRResNet, along with new discriminator architecture to properly guide VSRResNet during the GAN training. We further enhance our VSR GAN formulation with two regularizers, a distance loss in feature-space and pixel-space, to obtain our final VSRResFeatGAN model. We show that pre-training our generator with the mean-squared-error loss only quantitatively surpasses the current state-of-the-art VSR models. Finally, we employ the PercepDist metric to compare the state-of-the-art VSR models. We show that this metric more accurately evaluates the perceptual quality of SR solutions obtained from neural networks, compared with the commonly used PSNR/SSIM metrics. Finally, we show that our proposed model, the VSRResFeatGAN model, outperforms the current state-of-the-art SR models, both quantitatively and qualitatively.

Whole-slide histological images are routinely used by medical doctors in diagnosis. Most of these images are stained with the very common and inexpensive hematoxylin and eosin dyes. Slide stain separation and color normalization are crucial steps within the digital pathology workflow which require a previous color deconvolution step. This image processing task is not easy, especially when working with images taken from different microscopes and slides stained in different laboratories. In this paper, based on Variational Bayes inference, an efficient new blind color deconvolution method is proposed. The new model takes into account both spatial relations among image pixels and similarity to a given reference color-vector matrix. A comparison with classical and current state-of-the-art color deconvolution algorithms, using real images with known ground truth hematoxylin and eosin values, has been carried out. This comparison has demonstrated the superiority of the proposed approach.

Recent research on image super-resolution (SR) has shown that the use of perceptual losses such as feature-space loss functions and adversarial training can greatly improve the perceptual quality of the resulting SR output. In this paper, we extend the use of these perceptual-focused approaches for image SR to that of video SR. We design a 15-block residual neural network, VSRResNet, which is pre-trained on a the traditional mean -squared -error (MSE) loss and later fine-tuned with a feature-space loss function in an adversarial setting. We show that our proposed system, VSRRes-FeatGAN, produces super-resolved frames of much higher perceptual quality than those provided by the MSE-based model.

This paper introduces deep Gaussian processes (DGPs) for geo-physical parameter retrieval. Unlike the standard full GP model, the DGP accounts for complicated (modular, hierarchical) processes, provides a efficient solution that scales well to large datasets, and improves prediction accuracy over standard full and sparse GP models. We give empirical evidence of performance for estimation of surface dew point temperature from infrared sounding data.

Current remote sensing image classification problems have to deal with an unprecedented amount of heterogeneous and complex data sources. Upcoming missions will soon provide large data streams that will make land cover/use classification difficult. Machine-learning classifiers can help at this, and many methods are currently available. A popular kernel classifier is the Gaussian process classifier (GPC), since it approaches the classification problem with a solid probabilistic treatment, thus yielding confidence intervals for the predictions as well as very competitive results to the state-of-the-art neural networks and support vector machines. However, its computational cost is prohibitive for large-scale applications, and constitutes the main obstacle precluding wide adoption. This paper tackles this problem by introducing two novel efficient methodologies for GP classification. We first include the standard random Fourier features approximation into GPC, which largely decreases its computational cost and permits large-scale remote sensing image classification. In addition, we propose a model which avoids randomly sampling a number of Fourier frequencies and alternatively learns the optimal ones within a variational Bayes approach. The performance of the proposed methods is illustrated in complex problems of cloud detection from multispectral imagery and infrared sounding data. Excellent empirical results support the proposal in both computational cost and accuracy.

Passive Millimeter Wave Images (PMMWIs) are being increasingly used to identify and localize objects concealed under clothing. Taking into account the quality of these images and the unknown position, shape, and size of the hidden objects, large data sets are required to build successful classification/detection systems. Kernel methods, in particular Gaussian Processes (GPs), are sound, flexible, and popular techniques to address supervised learning problems. Unfortunately, their computational cost is known to be prohibitive for large scale applications. In this work, we present a novel approach to PMMWI classification based on the use of Gaussian Processes for large data sets. The proposed methodology relies on linear approximations to kernel functions through random Fourier features. Model hyperparameters are learned within a variational Bayes inference scheme. Our proposal is well suited for real-time applications, since its computational cost at training and test times is much lower than the original GP formulation. The proposed approach is tested on a unique, large, and real PMMWI database containing a broad variety of sizes, types, and locations of hidden objects.

In this work, we propose a new variational blind deconvolution method for spike and slab prior models. Soft-sparse or shrinkage priors such as the Laplace and other related Gaussian Scale Mixture priors may not be ideal sparsity promoting priors. They assign zero probability mass to events we may be interested in assigning a probability greater than zero. The truly sparse nature of the spike and slab priors allows us to discard irrelevant information in the blur estimation process, resulting in improved performance. We present an efficient inference algorithm to estimate the unknown blur kernel in the filter space, from which we estimate the final deblurred image. The VB approach we propose in this paper handles the inference in a much more efficient way than MCMC, and is more accurate than the standard mean field variational approximation. We prove the efficacy of our method by means of a series of experiments on both synthetically generated and real images.

This work presents a greedy Bayesian dictionary learning (DL) algorithm where not only the signals but also the dictionary representation matrix accept a sparse representation. This double-sparsity (DS) model has been shown to be superior to the standard sparse approach in some image processing tasks, where sparsity is only imposed on the signal coefficients. We present a new Bayesian approach which addresses typical shortcomings of regularization-based DS algorithms: the prior knowledge of the true noise level and the need of parameter tuning. Our model estimates the noise and sparsity levels as well as the model parameters from the observations and frequently outperforms state-of-the-art dictionary based techniques by taking into account the uncertainty of the estimates. Additionally, we introduce a versatile notation which generalizes denoising, inpainting and compressive sensing problem formulations. Finally, theoretical results are validated with denoising experiments on a set of images.

This paper presents an efficient methodology for approximating kernel functions in Gaussian process classification (GPC). Two models are introduced. We first include the standard random Fourier features (RFF) approximation into GPC, which largely improves the computational efficiency and permits large scale remote sensing data classification. In addition, we develop a novel approach which avoids randomly sampling a number of Fourier frequencies, and alternatively learns the optimal ones using a variational Bayes approach. The performance of the proposed methods is illustrated in complex problems of cloud detection from multispectral imagery.

In this paper, we propose an iterative method to address the face identification problem with block occlusions. Our approach utilizes a robust representation based on two characteristics in order to model contiguous errors (e.g., block occlusion) effectively. The first fits to the errors a distribution described by a tailored loss function. The second describes the error image as having a specific structure (resulting in low-rank in comparison with image size). We will show that this joint characterization is effective for describing errors with spatial continuity. Our approach is computationally efficient due to the utilization of the alternating direction method of multipliers. A special case of our fast iterative algorithm leads to the robust representation method, which is normally used to handle non-contiguous errors (e.g., pixel corruption). Extensive results on representative face databases (in constrained and unconstrained environments) document the effectiveness of our method over existing robust representation methods with respect to both identification rates and computational time.

Recent work in signal processing in general and image processing in particular deals with sparse representation related problems. Two such problems are of paramount importance: an overriding need for designing a well-suited overcomplete dictionary containing a redundant set of atoms-i.e., basis signals-and how to find a sparse representation of a given signal with respect to the chosen dictionary. Dictionary learning techniques, among which we find the popular K-singular value decomposition algorithm, tackle these problems by adapting a dictionary to a set of training data. A common drawback of such techniques is the need for parameter-tuning. In order to overcome this limitation, we propose a fully-automated Bayesian method that considers the uncertainty of the estimates and produces a sparse representation of the data without prior information on the number of non-zeros in each representation vector. We follow a Bayesian approach that uses a three-tiered hierarchical prior to enforce sparsity on the representations and develop an efficient variational inference framework that reduces computational complexity. Furthermore, we describe a greedy approach that speeds up the whole process. Finally, we present experimental results that show superior performance on two different applications with real images: denoising and inpainting.

Passive Millimeter Wave Images currently used to detect hidden threats suffer from low resolution, blur, and a very low signal-to-noise-ratio. These shortcomings render threat detection, both visual and automatic, very challenging. Furthermore, due to the presence of very severe noise, most of the blind image restoration methods fail to recover the system blurring kernel from a single image. In this paper we propose a robust Bayesian multiframe blind image deconvolution method that approximates the posterior distribution of the blur by a Dirichlet distribution. We show that this approach naturally incorporates the non-negativity and normalization constraints for the blur and cope well with the image noise. The performance of the proposed method is tested on both synthetic and real images.

In this work we address the multispectral image classification problem from a Bayesian perspective. We develop an algorithm which utilizes the logistic regression function as the observation model in a probabilistic framework, Super-Gaussian (SG) priors which promote sparsity on the adaptive coefficients, and Variational inference to obtain estimates of all the model unknowns. The proposed algorithm is validated on both synthetic and real experiments and compared with other state-of-the-art methods, such as Support Vector Machine and Gaussian Processes, demonstrating its improved performance.

Sequence labeling aims at assigning a label to every sample of a signal (or pixel of an image) while considering the sequentiality (or vicinity) of the samples. To perform this task, many works in the literature first filter and then label the data. Unfortunately, the filtering, which is performed independently from the labeling, is far from optimal and frequently makes the latter task harder. In this paper, a novel approach that trains a Gaussian process classifier and estimates the coefficients of an optimal filter jointly is presented. The new approach, based on Bayesian modeling and alternating direction method of multipliers (ADMMs) optimization, performs both tasks simultaneously. All unknowns are treated as stochastic variables, which are estimated using variational inference and filtering and labeling are linked with the use of ADMM. In the experimental section, synthetic and real experiments are presented to compare the proposed method with other existing approaches.

In this work we propose a novel framework to obtain High Resolution (HR) images from Compressed Sensing (CS) imaging systems capturing multiple Low Resolution (LR) images of the same scene. The proposed CS Super Resolution (SR) approach combines existing CS reconstruction algorithms with an LR to HR approach based on the use of a Super Gaussian (SG) regularization term. The reconstruction is formulated as a constrained optimization problem which is solved using the Alternate Direction Methods of Multipliers (ADMM). The image estimation subproblem is solved using Majorization-Minimization (MM) while the CS reconstruction becomes an l1-minimization subject to a quadratic constraint. The performed experiments show that the proposed method compares favorably to classical SR methods at compression ratio 1, obtaining excellent SR reconstructions at ratios below one.

In this paper, we review recent results on audiovisual (AV) fusion. We also discuss some of the challenges and report on approaches to address them. One important issue in AV fusion is how the modalities interact and influence each other. This review will address this question in the context of AV speech processing, and especially speech recognition, where one of the issues is that the modalities both interact but also sometimes appear to desynchronize from each other. An additional issue that sometimes arises is that one of the modalities may be missing at test time, although it is available at training time; for example, it may be possible to collect AV training data while only having access to audio at test time. We will review approaches to address this issue from the area of multiview learning, where the goal is to learn a model or representation for each of the modalities separately while taking advantage of the rich multimodal training data. In addition to multiview learning, we also discuss the recent application of deep learning (DL) toward AV fusion. We finally draw conclusions and offer our assessment of the future in the area of AV fusion.

Blind image deconvolution involves two key objectives: 1) latent image and 2) blur estimation. For latent image estimation, we propose a fast deconvolution algorithm, which uses an image prior of nondimensional Gaussianity measure to enforce sparsity and an undetermined boundary condition methodology to reduce boundary artifacts. For blur estimation, a linear inverse problem with normalization and nonnegative constraints must be solved. However, the normalization constraint is ignored in many blind image deblurring methods, mainly because it makes the problem less tractable. In this paper, we show that the normalization constraint can be very naturally incorporated into the estimation process by using a Dirichlet distribution to approximate the posterior distribution of the blur. Making use of variational Dirichlet approximation, we provide a blur posterior approximation that considers the uncertainty of the estimate and removes noise in the estimated kernel. Experiments with synthetic and real data demonstrate that the proposed method is very competitive to the state-of-the-art blind image restoration methods.

Many real classification tasks are oriented to sequence (neighbor) labeling, that is, assigning a label to every sample of a signal while taking into account the sequentiality (or neighborhood) of the samples. This is normally approached by first filtering the data and then performing classification. In consequence, both processes are optimized separately, with no guarantee of global optimality. In this work we utilize Bayesian modeling and inference to jointly learn a classifier and estimate an optimal filterbank. Variational Bayesian inference is used to approximate the posterior distributions of all unknowns, resulting in an iterative procedure to estimate the classifier parameters and the filterbank coefficients. In the experimental section we show, using synthetic and real data, that the proposed method compares favorably with other classification/filtering approaches, without the need of parameter tuning.

Iteratively reweighted least squares (IRLS) is one of the most effective methods to minimize the lpregularized linear inverse problem. Unfortunately, the regularizer is nonsmooth and nonconvex when 0 <; p <; 1. In spite of its properties and mainly due to its high computation cost, IRLS is not widely used in image deconvolution and reconstruction. In this paper, we first derive the IRLS method from the perspective of majorization minimization and then propose an Alternating Direction Method of Multipliers (ADMM) to solve the reweighted linear equations. Interestingly, the resulting algorithm has a shrinkage operator that pushes each component to zero in a multiplicative fashion. Experimental results on both image deconvolution and reconstruction demonstrate that the proposed method outperforms state-of-the-art algorithms in terms of speed and recovery quality.

We propose a Bayesian based algorithm to recover sparse signals from compressed noisy measurements in the presence of a smooth background component. This problem is closely related to robust principal component analysis and compressive sensing, and is found in a number of practical areas. The proposed algorithm adopts a hierarchical Bayesian framework for modeling, and employs approximate inference to estimate the unknowns. Numerical examples demonstrate the effectiveness of the proposed algorithm and its advantage over the current state-of-the-art solutions.

In recent years, kernel methods, in particular support vector machines (SVMs), have been successfully introduced to remote sensing image classification. Their properties make them appropriate for dealing with a high number of image features and a low number of available labeled spectra. The introduction of alternative approaches based on (parametric) Bayesian inference has been quite scarce in the more recent years. Assuming a particular prior data distribution may lead to poor results in remote sensing problems because of the specificities and complexity of the data. In this context, the emerging field of nonparametric Bayesian methods constitutes a proper theoretical framework to tackle the remote sensing image classification problem. This paper exploits the Bayesian modeling and inference paradigm to tackle the problem of kernel-based remote sensing image classification. This Bayesian methodology is appropriate for both finite- and infinite-dimensional feature spaces. The particular problem of active learning is addressed by proposing an incremental/active learning approach based on three different approaches: 1) the maximum differential of entropies; 2) the minimum distance to decision boundary; and 3) the minimum normalized distance. Parameters are estimated by using the evidence Bayesian approach, the kernel trick, and the marginal distribution of the observations instead of the posterior distribution of the adaptive parameters. This approach allows us to deal with infinite-dimensional feature spaces. The proposed approach is tested on the challenging problem of urban monitoring from multispectral and synthetic aperture radar data and in multiclass land cover classification of hyperspectral images, in both purely supervised and active learning settings. Similar results are obtained when compared to SVMs in the supervised mode, with the advantage of providing posterior estimates for classification and automatic parameter learning. Comparison with random sampling as well as standard active learning methods such as margin sampling and entropy-query-by-bagging reveals a systematic overall accuracy gain and faster convergence with the number of queries.

Our work addresses the problem of analyzing and understanding dynamic video scenes. A two-level motion pattern mining approach is proposed. At the first level, activities are modeled as distributions over patch-based features, including spatial location, moving direction, and speed. At the second level, traffic states are modeled as distributions over activities. Both patterns are shared among video clips. Compared to other works, one advantage of our method is that moving speed is considered to describe visual word. The other advantage is that traffic states are detected and assigned to every video frame. These enable finer semantic interpretation, more precise video segmentation, and anomaly detection. Specifically, every video frame is labeled by a certain traffic state, and the video is segmented frame by frame accordingly. Moving pixels in each frame, which do not belong to any activity or cannot exist in the corresponding traffic state, are detected as anomalies. We have successfully tested our approach on some challenging traffic surveillance sequences containing both pedestrian and vehicle motions.

In this paper we present an introduction to Variational Bayesian (VB) methods in the context of probabilistic graphical models, and discuss their application in multimedia related problems. VB is a family of deterministic probability distribution approximation procedures that offer distinct advantages over alternative approaches based on stochastic sampling and those providing only point estimates. VB inference is flexible to be applied in different practical problems, yet is broad enough to subsume as its special cases several alternative inference approaches including Maximum A Posteriori (MAP) and the Expectation-Maximization (EM) algorithm. In this paper we also show the connections between VB and other posterior approximation methods such as the marginalization-based Loopy Belief Propagation (LBP) and the Expectation Propagation (EP) algorithms. Specifically, both VB and EP are variational methods that minimize functionals based on the Kullback-Leibler (KL) divergence. LBP, traditionally developed using graphical models, can also be viewed as a VB inference procedure. We present several multimedia related applications illustrating the use and effectiveness of the VB algorithms discussed herein. We hope that by reading this tutorial the readers will obtain a general understanding of Bayesian methods and establish connections among popular algorithms used in practice.

We propose a novel blind image deconvolution (BID) regularization framework for compressive sensing (CS) based imaging systems capturing blurred images. The proposed framework relies on a constrained optimization technique, which is solved by a sequence of unconstrained sub-problems, and allows the incorporation of existing CS reconstruction algorithms in compressive BID problems. As an example, a non-convex lp quasi-norm with is employed as a regularization term for the image, while a simultaneous auto-regressive regularization term is selected for the blur. Nevertheless, the proposed approach is very general and it can be easily adapted to other state-of-the-art BID schemes that utilize different, application specific, image/blur regularization terms. Experimental results, obtained with simulations using blurred synthetic images and real passive millimeter-wave images, show the feasibility of the proposed method and its advantages over existing approaches.

We propose a novel blind image deconvolution (BID) regularization framework for compressive passive millimeter-wave (PMMW) imaging systems. The proposed framework is based on the variable-splitting optimization technique, which allows us to utilize existing compressive sensing reconstruction algorithms in compressive BID problems. In addition, a non-convex lpquasi-norm with 0 <; p <; 1 is employed as a regularization term for the image, while a simultaneous auto-regressive (SAR) regularization term is utilized for the blur. Furthermore, the proposed framework is very general and it can be easily adapted to other state-of-the-art BID approaches that utilize different image/blur regularization terms. Experimental results, obtained with simulations using a synthetic image and real PMMW images, show the advantage of the proposed approach compared to existing ones.

We propose a novel design for light field image acquisition based on compressive sensing principles. By placing a randomly coded mask at the aperture of a camera, incoherent measurements of the light passing through different parts of the lens are encoded in the captured images. Each captured image is a random linear combination of different angular views of a scene. The encoded images are then used to recover the original light field image via a novel Bayesian reconstruction algorithm. Using the principles of compressive sensing, we show that light field images with a large number of angular views can be recovered from only a few acquisitions. Moreover, the proposed acquisition and recovery method provides light field images with high spatial resolution and signal-to-noise-ratio, and therefore is not affected by limitations common to existing light field camera designs. We present a prototype camera design based on the proposed framework by modifying a regular digital camera. Finally, we demonstrate the effectiveness of the proposed system using experimental results with both synthetic and real images.

There has been a significant interest in the recovery of low-rank matrices from an incomplete of measurements, due to both theoretical and practical developments demonstrating the wide applicability of the problem. A number of methods have been developed for this recovery problem, however, a principled method for choosing the unknown target rank is generally missing. In this paper, we present a recovery algorithm based on sparse Bayesian learning (SBL) and automatic relevance determination principles. Starting from a matrix factorization formulation and enforcing the low-rank constraint in the estimates as a sparsity constraint, we develop an approach that is very effective in determining the correct rank while providing high recovery performance. We provide empirical results and comparisons with current state-of-the-art methods that illustrate the potential of this approach.

Synthetic aperture radar (SAR) imagery suffers from the speckle phenomenon. Speckle gives rise to the presence of multiplicative noise which severely degrades the observed images. It is known that logarithmically transformed speckle can be well approximated by a Gaussian distribution. In this paper we propose an algorithm for despeckling images, within the log-transformed spatial domain, using a TV prior whose model parameter is automatically determined using the Evidence Analysis within the Hierarchical Bayesian Paradigm. The effectiveness of the proposed algorithm, over both synthetically speckled and real SAR images, is studied.

In this paper, we propose a novel algorithm for image restoration based on combining nonstationary edge-preserving priors. We develop a Bayesian modeling followed by an evidence analysis inference approach for deriving the foundations of the proposed iterative restoration algorithm. Simulation results over a variety of blurred and noisy standard test images indicate that the presented method outperforms current state-of-the-art image restoration algorithms. We finally present experimental results by digitally refocusing images captured with controlled defocus, successfully confirming the ability of the proposed restoration algorithm in recovering extra features and details, while still preserving edges.

Every day, a huge amount of video data is generated for different purposes and applications. Fast and accurate algorithms for efficient video search and retrieval are therefore essential. The interesting properties of sparse representation and the new sampling theory named Compressive Sensing (CS) constitute the core of the new approach to video representation and retrieval we are presenting in this paper. Once the representation (where sparsity is expected) has been chosen and the observations have been taken, the proposed approach utilizes Bayesian modeling and inference to tackle the retrieval problem. In order to speed up the inference process the use of Principal Components Analysis (PCA) to provide an alternative representation of the frames is analyzed. Experimental results validate the proposed approach whose robustness against noise is also examined.

In this paper, we address the super resolution (SR) problem from a set of degraded low resolution (LR) images to obtain a high resolution (HR) image. Accurate estimation of the sub-pixel motion between the LR images significantly affects the performance of the reconstructed HR image. In this paper, we propose novel super resolution methods where the HR image and the motion parameters are estimated simultaneously. Utilizing a bayesian formulation, we model the unknown HR image, the acquisition process, the motion parameters and the unknown model parameters in a stochastic sense. Employing a variational bayesian analysis, we develop two novel algorithms which jointly estimate the distributions of all unknowns. The proposed framework has the following advantages: 1) Through the incorporation of uncertainty of the estimates, the algorithms prevent the propagation of errors between the estimates of the various unknowns; 2) the algorithms are robust to errors in the estimation of the motion parameters; and 3) using a fully bayesian formulation, the developed algorithms simultaneously estimate all algorithmic parameters along with the HR image and motion parameters, and therefore they are fully-automated and do not require parameter tuning. We also show that the proposed motion estimation method is a stochastic generalization of the classical Lucas-Kanade registration algorithm. Experimental results demonstrate that the proposed approaches are very effective and compare favorably to state-of-the-art SR algorithms.

In this paper we propose two fast Total Variation (TV) based algorithms for image restoration by utilizing variational posterior distribution approximation. The unknown image and the hyperparameters for the image and observation models are formulated and estimated simultaneously within a hierachical Bayesian framework, rendering the algorithms fully-automated without any free parameters. Experimental results demonstrate that the proposed algorithms provide restoration results competitive to existing methods in terms of image quality while achieving superior computational efficiency.

In this paper, we propose a symmetrical EEG/fMRI fusion algorithm which combines EEG and fMRI by means of a common generative model. The use of a total variation (TV) prior as well as spatially adaptive temporal priors enables adaptation to the local characteristics of the estimated responses. We utilize an approximate variational Bayesian framework and obtain a fully automatic fusion algorithm. Simulation results demonstrate that the proposed algorithm outperforms existing EEG/fMRI fusion methods.

In this paper we propose a novel Bayesian algorithm for image restoration and parameter estimation. We utilize an image prior where Gaussian distributions are placed per pixel in the high-pass filter outputs of the image. By following the hierarchical Bayesian framework, we simultaneously estimate the unknown image and hyperparameters for both the image prior and the image degradation noise. We show that the proposed formulation is a special case of the popular lp-norm based formulations with p = 0, and therefore enforces sparsity to an high extent in the filtered image coefficients. Moreover, the proposed formulation results in a convex optimization problem, and therefore does not suffer from the robustness issues common with non-convex image priors. Experimental results demonstrate that the proposed algorithm provides superior performance compared to state-of-the-art restoration algorithms although no user-supervision is required.

This paper is devoted to the combination of image priors in Super Resolution (SR) image reconstruction. Taking into account that each combination of a given observation model and a prior model produces a different posterior distribution of the underlying High Resolution (HR) image, the use of variational posterior distribution approximation on each posterior will produce as many posterior approximations as priors we want to combine. A unique approximation is obtained here by finding the distribution on the HR image given the observations that minimizes a linear convex combination of the Kullback-Leibler divergences associated with each posterior distribution. We find this distribution in closed form and also relate the proposed approach to other prior combination methods in the literature. The estimated HR images are compared with images provided by other SR reconstruction methods.

In this paper, we model the components of the compressive sensing (CS) problem, i.e., the signal acquisition process, the unknown signal coefficients and the model parameters for the signal and noise using the Bayesian framework. We utilize a hierarchical form of the Laplace prior to model the sparsity of the unknown signal. We describe the relationship among a number of sparsity priors proposed in the literature, and show the advantages of the proposed model including its high degree of sparsity. Moreover, we show that some of the existing models are special cases of the proposed model. Using our model, we develop a constructive (greedy) algorithm designed for fast reconstruction useful in practical settings. Unlike most existing CS reconstruction methods, the proposed algorithm is fully automated, i.e., the unknown signal coefficients and all necessary parameters are estimated solely from the observation, and, therefore, no user-intervention is needed. Additionally, the proposed algorithm provides estimates of the uncertainty of the reconstructions. We provide experimental results with synthetic 1-D signals and images, and compare with the state-of-the-art CS reconstruction algorithms demonstrating the superior performance of the proposed approach.

Photographs acquired under low-lighting conditions require long exposure times and therefore exhibit significant blurring due to the shaking of the camera. Using shorter exposure times results in sharper images but with a very high level of noise. In this paper we address the problem of utilizing two such images in order to obtain an estimate of the original scene and present a novel blind deconvolution algorithm for solving it. We formulate the problem in a hierarchical Bayesian framework by utilizing prior knowledge on the unknown image and blur, and also on the dependency between the two observed images. By incorporating a fully Bayesian analysis, the developed algorithm estimates all necessary model parameters along with the unknown image and blur, such that no user-intervention is needed. Moreover, we employ a variational Bayesian inference procedure, which allows for the statistical compensation of errors occurring at different stages of the restoration, and also provides uncertainties of the estimates. Experimental results with synthetic and real images demonstrate that the proposed method provides very high quality restoration results and compares favorably to existing methods even though no user supervision is needed.

In this paper, a new image prior is introduced and used in image restoration. This prior is based on products of spatially weighted total variations (TV). These spatial weights provide this prior with the flexibility to better capture local image features than previous TV based priors. Bayesian inference is used for image restoration with this prior via the variational approximation. The proposed restoration algorithm is fully automatic in the sense that all necessary parameters are estimated from the data and is faster than previous similar algorithms. Numerical experiments are shown which demonstrate that image restoration based on this prior compares favorably with previous state-of-the-art restoration algorithms.

In this paper we model the components of the compressive sensing (CS) problem using the Bayesian framework by utilizing a hierarchical form of the Laplace prior to model sparsity of the unknown signal. This signal prior includes some of the existing models as special cases and achieves a high degree of sparsity. We develop a constructive (greedy) algorithm resulting from this formulation where necessary parameters are estimated solely from the observation and therefore no user-intervention is needed. We provide experimental results with synthetic 1D signals and images, and compare with the state-of-the-art CS reconstruction algorithms demonstrating the superior performance of the proposed approach.

We propose a novel camera design for light field image acquisition using compressive sensing. By utilizing a randomly coded non-refractive mask in front of the aperture, incoherent measurements of the light passing through different regions are encoded in the captured images. A novel reconstruction algorithm is proposed to recover the original light field image from these acquisitions. Using the principles of compressive sensing, we demonstrate that light field images with high angular dimension can be captured with only a few acquisitions. Moreover, the proposed design provides images with high spatial resolution and signal-to-noise-ratio (SNR), and therefore does not suffer from limitations common to existing light-field camera designs. Experimental results demonstrate the efficiency of the proposed system.

Photographs acquired under low-light conditions require long exposure times and therefore exhibit significant blurring due to the shaking of the camera. Using shorter exposure times results in sharper images but with a very high level of noise. In this paper we address this problem and present a novel blind deconvolution algorithm for a pair of differently exposed images. We formulate the problem in a hierarchical Bayesian framework by utilizing prior knowledge on the unknown image and blur, and also on the dependency between two observed images. By incorporating a fully Bayesian analysis, the developed algorithm estimates all necessary algorithm parameters along with the unknowns, such that no user-intervention is needed. Moreover, we employ a variational Bayesian inference procedure, which allows for the statistical compensation of errors occurring at different stages of the restoration, and also provides uncertainties of the estimates. Experimental results demonstrate the high restoration performance of the proposed algorithm.

We present a new image restoration method based on modelling the coefficients of an overcomplete wavelet response to natural images with a mixture of two Gaussian distributions, having non-zero and zero mean respectively, and reflecting the assumption that this response is close to be sparse. Including the observation model, the resulting procedure iterates between image reconstruction from the hard-thresholding of the response to the current estimate and a fast blur compensation step. Results indicate that our method compares favorably with current wavelet-based restoration methods.

In this paper we present a new Bayesian methodology for the restoration of blurred and noisy images. Bayesian methods rely on image priors that encapsulate prior image knowledge and avoid the ill-posedness of image restoration problems. We use a spatially varying image prior utilizing a gamma-normal hyperprior distribution on the local precision parameters. This kind of hyperprior distribution, which to our knowledge has not been used before in image restoration, allows for the incorporation of information on local as well as global image variability, models correlation of the local precision parameters and is a conjugate hyperprior to the image model used in the paper. The proposed restoration technique is compared with other image restoration approaches, demonstrating its improved performance.

Journal Article Guest Editorial Get access Aggelos K. Katsaggelos, Aggelos K. Katsaggelos 1Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL, USA Search for other works by this author on: Oxford Academic Google Scholar Rafael Molina Rafael Molina * 2Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain *Corresponding author:[email protected] Search for other works by this author on: Oxford Academic Google Scholar The Computer Journal, Volume 52, Issue 4, July 2009, Pages 395–396, https://doi.org/10.1093/comjnl/bxp029 Published: 20 April 2009

In this paper we propose and analyze a globally and locally adaptive super-resolution Bayesian methodology for pansharpening of multispectral images. The methodology incorporates prior knowledge on the expected characteristics of the multispectral images uses the sensor characteristics to model the observation process of both panchromatic and multispectral images and includes information on the unknown parameters in the model in the form of hyperprior distributions. Using real and synthetic data, the pansharpened multispectral images are compared with the images obtained by other pansharpening methods and their quality is assessed both qualitatively and quantitatively.

In this paper, we present novel algorithms for total variation (TV) based blind deconvolution and parameter estimation utilizing a variational framework. Using a hierarchical Bayesian model, the unknown image, blur, and hyperparameters for the image, blur, and noise priors are estimated simultaneously. A variational inference approach is utilized so that approximations of the posterior distributions of the unknowns are obtained, thus providing a measure of the uncertainty of the estimates. Experimental results demonstrate that the proposed approaches provide higher restoration performance than non-TV-based methods without any assumptions about the unknown hyperparameters.

In this paper we propose novel algorithms for image restoration and parameter estimation with a Generalized Gaussian Markov Random Field (GGMRF) prior utilizing variational distribution approximation. The restored image and the unknown hyperparameters for both the image prior and the image degradation noise are simultaneously estimated within a hierarchical Bayesian framework. We develop two algorithms resulting from this formulation which provide approximations to the posterior distributions of the latent variables. Experimental results are provided to demonstrate the performance of the algorithms.

In this paper we propose a novel algorithm for super resolution based on total variation prior and variational distribution approximations. We formulate the problem using a hierarchical Bayesian model where the reconstructed high resolution image and the model parameters are estimated simultaneously from the low resolution observations. The algorithm resulting from this formulation utilizes variational inference and provides approximations to the posterior distributions of the latent variables. Due to the simultaneous parameter estimation, the algorithm is fully automated so parameter tuning is not required. Experimental results show that the proposed approach outperforms some of the state-of-the-art super resolution algorithms.

We propose a new image and blur prior model, based on non-stationary autoregressive (AR) models, and use these to blindly deconvolve blurred photographic images, using the Gibbs sampler. As far as we are aware, this is the first attempt to tackle a real-world blind image deconvolution (BID) problem using Markov chain Monte Carlo (MCMC) methods. We give examples with simulated and real out-of-focus images, which show the state-of-the-art results that the proposed approach provides.

In this paper, we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. Within the hierarchical Bayesian formulation, the reconstructed image and the unknown hyper parameters for the image prior and the noise are simultaneously estimated. The proposed algorithms provide approximations to the posterior distributions of the latent variables using variational methods. We show that some of the current approaches to TV-based image restoration are special cases of our framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyper parameters and clearly outperform existing methods when additional information is included.

In this paper we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. By following the hierarchical Bayesian framework, we simultaneously estimate the reconstructed image and the unknown hyper parameters for both the image prior and the image degradation noise. Our algorithms provide an approximation to the posterior distributions of the unknowns so that both the uncertainty of the estimates can be measured and different values from these distributions can be used for the estimates. We also show that some of the current approaches to TV-based image restoration are special cases of our variational framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyper parameters and clearly outperform existing methods when additional information is included.

The variational Bayesian approach has recently been proposed to tackle the blind image restoration (BIR) problem. We consider extending the procedures to include realistic boundary modelling and non-stationary image restoration. Correctly modelling the boundaries is essential for achieving accurate blind restorations of photographic images, whilst nonstationary models allow for better adaptation to local image features, and therefore improvements in quality.

In this paper we present a new Bayesian methodology for the restoration of blurred and noisy images. Bayesian methods rely on image priors that encapsulate prior image knowledge and avoid the ill-posedness of the image restoration problems. Some of these priors depend on global variance parameters, unable to account for local characteristics. Here we first use variational methods to approximate probability posterior distributions for the global model to later use those distributions to define local and more realistic image models which lead to better restored images as it is shown in the experimental section.

In this paper we present a new super resolution Bayesian method for pansharpening of multispectral images which: a) incorporates prior knowledge on the expected characteristics of the multispectral images, b) uses the sensor characteristics to model the observation process of both panchromatic and multispectral images, and c) performs the estimation of all the unknown parameters in the model. Using real data, the pansharpened multispectral images are compared with the images obtained by other pansharpening methods and their quality is assessed both qualitatively and quantitatively.

Following the hierarchical Bayesian framework for blind deconvolution problems, in this paper, we propose the use of simultaneous autoregressions as prior distributions for both the image and blur, and gamma distributions for the unknown parameters (hyperparameters) of the priors and the image formation noise. We show how the gamma distributions on the unknown hyperparameters can be used to prevent the proposed blind deconvolution method from converging to undesirable image and blur estimates and also how these distributions can be inferred in realistic situations. We apply variational methods to approximate the posterior probability of the unknown image, blur, and hyperparameters and propose two different approximations of the posterior distribution. One of these approximations coincides with a classical blind deconvolution method. The proposed algorithms are tested experimentally and compared with existing blind deconvolution methods.

In this paper the blind deconvolution problem is formulated using the variational framework. With its use approximations of the involved probability distributions are developed resulting in two algorithms for the estimation of the posterior distributions of the hyperparameters, the blur, and the original image. The performance of the two proposed restoration algorithms is demonstrated experimentally.

Super-resolution algorithms recover high-frequency information from a sequence of low-resolution observations. In this paper, we consider the impact of video compression on the super-resolution task. Hybrid motion-compensation and transform coding schemes are the focus, as these methods provide observations of the underlying displacement values as well as a variable noise process. We utilize the Bayesian framework to incorporate this information and fuse the super-resolution and post-processing problems. A tractable solution is defined, and relationships between algorithm parameters and information in the compressed bitstream are established. The association between resolution recovery and compression ratio is also explored. Simulations illustrate the performance of the procedure with both synthetic and nonsynthetic sequences.

We consider the estimation of the unknown hyperparameters for the problem of reconstructing a high-resolution image from multiple undersampled, shifted, degraded frames with subpixel displacement errors. We derive mathematical expressions for the iterative calculation of the maximum likelihood estimate (MLE) of the unknown hyperparameters given the low resolution observed images. Experimental results are presented for evaluating the accuracy of the proposed method.

We consider the problem of reconstructing a high-resolution image from an incomplete set of undersampled, shifted, degraded frames with subpixel displacement errors. We derive mathematical expressions for the calculation of the maximum a posteriori (MAP) estimate of the high resolution image given the low resolution observed images. We also examine the role played by the prior model when an incomplete set of low resolution images is used. Finally, the proposed method is tested on real and synthetic images.

In this paper we consider the estimation of the unknown hyperparameters for the problem of reconstructing a high-resolution image from multiple undersampled, shifted, blurred and degraded frames with subpixel displacement errors. We derive mathematical expressions for the iterative calculation of the maximum likelihood estimate (mle) of the unknown hyperparameters given the low resolution observed images. Finally, the proposed method is tested on a synthetic image.

In this paper, we develop a multichannel image restoration algorithm using compound Gauss-Markov random fields (CGMRF) models. The line process in the CGMRF allows the channels to share important information regarding the objects present in the scene. In order to estimate the underlying multichannel image, two new iterative algorithms are presented and their convergence is established. They can be considered as extensions of the classical simulated annealing and iterative conditional methods. Experimental results with color images demonstrate the effectiveness of the proposed approaches.

In this paper, we consider the estimation of the unknown parameters for the problem of reconstructing a high-resolution image from multiple undersampled, shifted, degraded frames with subpixel displacement errors. We derive mathematical expressions for the iterative calculation of the maximum likelihood estimate of the unknown parameters given the low resolution observed images. These iterative procedures require the manipulation of block-semi circulant (BSC) matrices, that is, block matrices with circulant blocks. We show how these BSC matrices can be easily manipulated in order to calculate the unknown parameters. Finally the proposed method is tested on real and synthetic images.

A framework for recovering high-resolution information from a sequence of sub-sampled and compressed observations is presented. Compression schemes that describe a video sequence through a combination of motion vectors and transform coefficients are the focus (e.g. the MPEG and ITU family of standards), and we consider the influence of both the motion vectors and transform coefficients within the reconstruction algorithm. A Bayesian approach is utilized to incorporate the information, and results show a discemable improvement in resolution, as compared to standard interpolation methods.

We propose a new iterative method for Maximum a Posteriori (MAP) reconstruction of SPECT (Single Photon Emission Computed Tomography) images. The method uses Compound Gauss Markov Random Fields (CGMRF) as prior model and is stochastic for the line process and deterministic for the reconstruction. Synthetic and real images are used to compare the new method with existing ones.

SPECT (single photon emission computed tomography) is used in nuclear medicine to determine the distribution of a radioactive isotope within a patient from tomographic views or projection data. These images are severely degraded due to the presence of noise and several physical factors like attenuation and scattering. We use, within the Bayesian framework, a compound Gauss Markov random field (CGMRF) as prior model to reconstruct such images. In order to find the maximum a posteriori (MAP) estimate we propose a new iterative method, which is stochastic for the line process and deterministic for the reconstruction. The proposed method is tested and compared with other reconstruction methods on both synthetic and real SPECT images.

A method for simultaneously estimating the high-resolution frames and the corresponding motion field from a compressed low-resolution video sequence is presented. The algorithm incorporates knowledge of the spatio-temporal correlation between low and high-resolution images to estimate the original high-resolution sequence from the degraded low-resolution observation. Information from the encoder is also exploited, including the transmitted motion vectors, quantization tables, coding modes and quantizer scale factors. Simulations illustrate an improvement in the peak signal-to-noise ratio when compared with traditional interpolation techniques and are corroborated with visual results.

We propose an iterative algorithm for the estimation of high resolution frames from a low resolution compressed video sequence. The algorithm exploits the existing correlation between the high and low resolution frames and the information provided by the encoder to obtain a high resolution frame. The performance of the algorithm is demonstrated experimentally.

A solution to the multichannel image restoration problem is provided using compound Gauss-Markov random fields. For the single channel deblurring problem the convergence of the simulated annealing (SA) and iterative conditional mode (ICM) algorithms has not been established. We propose two new iterative multichannel restoration algorithms which can be considered as extensions of the classical SA and ICM approaches and whose convergence is established. Experimental results with color images demonstrate the effectiveness of the proposed algorithms.

We propose an iterative algorithm for simultaneously estimating the motion field and high resolution frames from a compressed low resolution video sequence. The algorithm exploits the existing correlation between high and low resolution frames and information provided by the encoder, such as coding modes and motion vectors (when available), to obtain a higher resolution frame. The performance of the algorithm is demonstrated experimentally.

In this paper, we examine the restoration problem when the point-spread function (PSF) of the degradation system is partially known. For this problem, the PSF is assumed to be the sum of a known deterministic and an unknown random component. This problem has been examined before; however, in most previous works the problem of estimating the parameters that define the restoration filters was not addressed. In this paper, two iterative algorithms that simultaneously restore the image and estimate the parameters of the restoration filter are proposed using evidence analysis (EA) within the hierarchical Bayesian framework. We show that the restoration step of the first of these algorithms is in effect almost identical to the regularized constrained total least-squares (RCTLS) filter, while the restoration step of the second is identical to the linear minimum mean square-error (LMMSE) filter for this problem. Therefore, in this paper we provide a solution to the parameter estimation problem of the RCTLS filter. We further provide an alternative approach to the expectation-maximization (EM) framework to derive a parameter estimation algorithm for the LMMSE filter. These iterative algorithms are derived in the discrete Fourier transform (DFT) domain; therefore, they are computationally efficient even for large images. Numerical experiments are presented that test and compare the proposed algorithms.

With block-based compression approaches for both still images and sequences of images annoying blocking artifacts are exhibited, primarily at high compression ratios. They are due to the independent processing (quantization) of the block transformed values of the intensity or the displaced frame difference. We propose the application of the hierarchical Bayesian paradigm to the reconstruction of block discrete cosine transform (BDCT) compressed images and the estimation of the required parameters. We derive expressions for the iterative evaluation of these parameters applying the evidence analysis within the hierarchical Bayesian paradigm. The proposed method allows for the combination of parameters estimated at the coder and decoder. The performance of the proposed algorithms is demonstrated experimentally.

The subband decomposition of a single channel image restoration problem is examined. The decomposition is carried out in the image model (prior model) in order to take into account the frequency activity of each band of the original image. The hyperparameters associated with each band together with the original image are rigorously estimated within the Bayesian framework. Finally, the proposed method is tested and compared with other methods on real images.

Although many statistical methods have been proposed for the restoration of tomographic images, their use in medical environments has been limited due to two important factors. These factors are the need for greater computational time than deterministic methods and the selection of the hyperparameters in the image models. Consequently, deterministic methods, like the classical filtered back-projection (FBP) and algebraic reconstruction (AR), are commonly used. In this work, we propose a method to estimate, from observed image data in emission tomography, the hyperparameters in a Generalized Gaussian Markov Random Field (GGMRF). We use the hierarchical Bayesian approach and evidence analysis to reconstruct the image and estimate the unknown hyperparameters. The method is tested on synthetic images.

In this paper, we propose the application of the hierarchical Bayesian paradigm to the image restoration problem. We derive expressions for the iterative evaluation of the two hyperparameters applying the evidence and maximum a posteriori (MAP) analysis within the hierarchical Bayesian paradigm. We show analytically that the analysis provided by the evidence approach is more realistic and appropriate than the MAP approach for the image restoration problem. We furthermore study the relationship between the evidence and an iterative approach resulting from the set theoretic regularization approach for estimating the two hyperparameters, or their ratio, defined as the regularization parameter. Finally the proposed algorithms are tested experimentally.

A number of restoration filters have been proposed for the restoration problem from partially-known blurs. Previously we proposed the regularized constrained least-squares filter (RCTLS) and showed that it has a number of advantages over previous ones (Mesarovic et al. 1995). However, the problem of estimating the parameters that define the RCTLS filter has not yet been addressed. In this paper we propose a two-step algorithm based on the hierarchical Bayesian approach to simultaneously restore the image and estimate the parameters of the RCTLS restoration filter. The algorithm is derived in the DFT domain; thus, it is very efficient even for very large images.

We use the information in the chrominance bands to reconstruct color block transformed compressed images. For the luminance and the two chrominance channels, we define a reconstruction problem and show how to estimate the unknown hyperparameters and reconstruct each band automatically. The method is tested on real images.

This paper deals with the simultaneous identification of the blur and the restoration of a noisy and blurred image. We propose the use of Dirichlet distributions to model our prior knowledge about the blurring function together with smoothness constraints on the restored image to solve the blind deconvolution problem. We show that the use of Dirichlet distributions offers a lot of flexibility in incorporating vague or very precise knowledge about the blurring process into the blind deconvolution process. The proposed MAP estimator offers additional flexibility in modeling the original image. Experimental results demonstrate the performance of the proposed algorithm.

We examine the use of compound Gauss Markov random fields (CGMRF) to restore severely blurred high range images. For this deblurring problem, the convergence of the simulated annealing (SA) and iterative conditional mode (ICM) algorithms has not been established. We propose two new iterative restoration algorithms which extend the classical SA and ICM approaches. Their convergence is established and they are tested on real and synthetic images.

In an image restoration problem one usually has two different kinds of information. In the first stage, one has knowledge about the structural form of the noise and local characteristics of the restoration. These noise and image models normally depend on unknown hyperparameters. The hierarchical Bayesian approach adds a second stage by putting a hyperprior on the hyperparameters, where information about those hyperparameters is included. In this work the author applies the hierarchical Bayesian approach to image restoration problems and compares it with other approaches in handling the estimation of the hyperparameters.>

Considers Bayesian methods and spatial stochastic processes applied to the deconvolution of images of planets. Under simple but realistic prior assumptions about the true underlying image of a planet the Bayesian framework is put to work. The method has been tested on CCD images of Jupiter.>