Abstract This paper proposes a novel knowledge-guided learning model for low-light image enhancement. The developed pipeline improves image visibility by addressing noise and contrast issues, detail preservation, and color balancing. This is achieved by integrating denoising and Retinex techniques with vision transformers. Given as little as a single low-light input image, a set of intermediate exposures are generated by means of gamma transform, which serve as inputs to an ensemble of ten transformer models to produce enhanced outputs. An adaptive exposure selection process is then applied based on a composite image quality score. Finally, the selected outputs are fused in a multi-scale manner using weight maps based on contrast, saturation, and well-exposedness features. Extensive experiments on benchmark datasets, LOL-v1, LOL-v2-Real, LOL-v2-Synthetic, and a unified dataset, demonstrate that the proposed method is competitive with state-of-the-art techniques and shows a significant advantage when processing images captured in extremely low-light conditions. In addition, the developed method is successfully applied to the image dehazing problem without any further optimization.

High dynamic range (HDR) imaging enables to immortalize natural scenes similar to the way that they are perceived by human observers. With regular low dynamic range (LDR) capture/display devices, significant details may not be preserved in images due to the huge dynamic range of natural scenes. To minimize the information loss and produce high quality HDR-like images for LDR screens, this study proposes an efficient multi-exposure fusion (MEF) approach with a simple yet effective weight extraction method relying on principal component analysis, adaptive well-exposedness and saliency maps. These weight maps are later refined through a guided filter and the fusion is carried out by employing a pyramidal decomposition. Experimental comparisons with existing techniques demonstrate that the proposed method produces very strong statistical and visual results.

The restrictions of accessing high-end microscopes, microscale cameras and high-tech imaging lenses result in a high demand on low-cost microscopes. However, low-cost microscopes are facing with many image capture and quality limitations due to incompatible equipped instrumentation. This study aims at overcoming illumination and contrast problems, color aberration issues, and blur and noise corruption in low-cost microscopes at high image magnification rates. The three color channels of the input image are enhanced via principal component analysis and well-exposedness feature maps by means of cross-channel histogram matching, Laplacian and non-local means filtering. The proposed approach produces sharper, and better color and illumination fixed outputs when compared to existing methods in literature.

Tone-mapping is one of the prevailing methods to overcome high dynamic range imaging limitations over low dynamic range display devices, but the tone-mapped output image may suffer from saturated regions with texture and color information loss. In this paper, a novel approach is proposed to solve the so-called clipping problem in tone-mapped high dynamic range images. A successful saturation correction framework, which relies on linear embeddings, difference of pixel intensities and gradient-guided block-search, is developed as a post-processing technique to tone-mapping. Experimental results demonstrate that the proposed method successfully recovers clipped regions for the saturation problem in tone-mapped output images while avoiding artifacts.

This paper proposes an effective technique for multi-exposure image fusion and visible-infrared image fusion problems. Multi-exposure fusion algorithms generally extract faulty weight maps when the input stack contains multiple and/or severely over-exposed images. To overcome this issue, an alternative method is developed for weight map characterization and refinement in addition to the perspectives of linear embeddings of images and adaptive morphological masking. This framework has then been extended to the visible and infrared image fusion problem. The comprehensive experimental comparisons demonstrate that the proposed algorithm significantly enhances the fused image quality both statistically and visually.

Dictionary learning for sparse representations is generative in nature, hence discriminative modifications are commonly observed for classification problems. Classical dictionary learning bears a fundamental problem of not being capable of distinguishing two different classes lying on the same subspace, that cannot be resolved by any discriminative modification. This paper proposes an evolutionary simplicial learning method as a generative and compact sparse framework that solves the aforementioned problem for classification. Simplicial learning is an adaptation of conventional dictionary learning, in which subspaces designated by dictionary elements take the form of simplices through additional constraints on sparse codes. On top, an evolutionary approach is developed to determine the dimensionality and the number of simplices composing the simplicial. The proposed evolutionary learning is considered within multi-class classification tasks through synthetic and handwritten digit datasets and the superiority of it even as a generative-only approach is demonstrated. Simplicial learning loses its superiority over discriminative methods in high-dimensional real-world cases but can further be modified with discriminative elements to achieve state-of-the-art for classification.

Thanks to the increasing number of images stored in the cloud, external image redundancies can be leveraged to efficiently compress images by exploiting inter-images correlations. In this paper, we propose a novel cloud-based image coding scheme. Unlike current state-of-the-art systems, our method relies on a data dimensionality reduction technique. A global compensation is associated to a locally-weighted template matching compensation method to predict a reference frame, to be then differential-coded with classic video coding tools. Experimental results demonstrate that the proposed approach yields significant rate-distortion performance improvements compared to current image coding solutions.

We describe a novel epitomic image representation scheme that factors a given image content into a condensed epitome and a low-resolution image to reduce the memory space for images. Given an input image, we construct a condensed epitome such that all image patches can successfully be reconstructed from the factored representation by means of an optimized neighbor-embedding strategy. Under this new scope of epitomic image representations aligned with the manifold sampling assumption, we end up a more generic epitome learning scheme with increased optimality, compactness, and reconstruction stability. We present the performance of the proposed method for image and video up-scaling (super-resolution) while extensions to other image and video processing are straightforward.

We propose an exemplar-based super-resolution algorithm based on sparsity constrained neighbor-embeddings of local image patches. We extract exemplar patch pairs from as little as the given low-resolution image, and we rely on local geometric similarities of low-and high-resolution patch spaces. While sparsely coding the local geometry with a greedy patch selection method, we refine our solution by iteratively updating the obtained high-resolution image. We finally apply an adaptive back-projection to ensure the global consistency. Our experimental results indicate promising performance on synthesizing natural looking textures and sharp edges when compared to other super-resolution methods from the literature.

The paper first describes an examplar-based image inpainting algorithm using a locally linear neighbor embedding technique with low-dimensional neighborhood representation (LLE-LDNR). The inpainting algorithm first searches the K nearest neighbors ( ) of the input patch to be filled-in and linearly combine them with LLE-LDNR to synthesize the missing pixels. Linear regression is then introduced for improving the K-NN search. The performance of the LLE-LDNR with the enhanced K-NN search method is assessed for two applications: loss concealment and object removal.

We describe a self-content single-image super-resolution algorithm based on multi-scale neighbor embeddings of small image patches. Given an input low-resolution patch, we gradually expand its size by relying on local geometric similarities of low- and high-resolution patch spaces under small scaling factors. We characterize the local geometry with K-similar patches taken from an exemplar set and we collect exemplar patch pairs from the input image and its appropriately rescaled versions. While ensuring local images compatibility with an optimization on K, we satisfy image smoothness by patch overlapping. We further enforce global consistency through an adaptive back-projection. Our experimental results show better performance on synthesizing natural looking textures and sharp edges with less artifacts when compared to other methods.

The paper studies several non-negative matrix factorization methods with nearest neighbors constrained dictionaries for image prediction. The methods considered include the multiplicative update algorithm, the projected gradient algorithm, as well as the graph-regularized NMF solution which aims at taking into account the geometrical structure of the input data. The Intra prediction problem based on these NMF solutions amounts to a neighbor embedding problem. Both prediction and rate-distortion performances are then given in comparison with other neighbor embedding methods like locally linear embedding (LLE) and locally linear embedding with low dimensional neigborhood representation (LLE-LDNR).

The template matching algorithm is a simple extension to exemplar-based texture synthesis. Average of template matching predictors or non-local means based approaches can be seen as heuristic extensions to template matching. These methods which linearly combine several texture patches have been shown to be more robust in synthesis and to give better results when compared to simple template matching. However, they do not search to minimize an approximation error on the known pixel values in the template. They are rather heuristic methods for calculating the linear weighting coefficients. This paper proposes a neighbor embedding based texture synthesis method by formulating the problem as a least-squares optimization using locally linear embedding. By this means, one calculates the linear weighting coefficients by solving a constrained optimization for approximating the template. The proposed texture synthesis framework has first been applied to the image prediction (predictive coding) problem. It has then been extended to a loss concealment application for transmission errors. Experimental results demonstrate the effectiveness of the proposed method for both image compression and error concealment.

This paper describes two new intraimage prediction methods based on two data dimensionality reduction methods: nonnegative matrix factorization (NMF) and locally linear embedding. These two methods aim at approximating a block to be predicted in the image as a linear combination of k-nearest neighbors determined on the known pixels in a causal neighborhood of the input block. Variable k can be seen as a parameter controlling some sort of sparsity constraints of the approximation vector. The impact of this parameter as well as of the nonnegativity and sum-to-one constraints for the addressed prediction problem has been analyzed. The prediction and RD performances of these two new image prediction methods have then been evaluated in a complete image coding-and-decoding algorithm. Simulation results show gains up to 2 dB in terms of the PSNR of the reconstructed signal after coding and decoding of the prediction residue when compared with H.264/AVC intraprediction modes, up to 3 dB when compared with template matching, and up to 1 dB when compared with a sparse prediction method.

This paper presents a novel spatial texture prediction method based on non-negative matrix factorization. As an extension of template matching, approximation based iterative texture prediction methods have recently been considered for image prediction. These approaches rely on the assumption that the given basis functions (atoms) span the signal residue space at each iteration of the algorithm. However, in the case of signal prediction with a sup port region approximation, the atoms may not approximate residue signals very well even though the dictionary has been well adapted in the spatial domain. The underlying main idea is to consider a factorization based algorithm in which the given atoms approximate the signal without going further into signal residue space. The proposed spatial prediction method has first been assessed against the prediction methods based on template matching and sparse approximations. It has then been assessed in a compression scheme where the prediction residue is transform encoded. Experimental results obtained show that the proposed method outperforms the template matching and sparse approximations based techniques in terms of encoding efficiency.

This paper presents a novel dictionary learning method which, because of its simplicity and the limited number of training samples it requires, can be used for online learning of dictionaries for spatial texture prediction. The proposed learning method has first been described to address the problem of intra image prediction based on signal expansion on overcomplete dictionaries. It has then been evaluated in a complete image codec. The experimental results obtained show a significant improvement in terms of the quality of the predicted image compared to H.264/AVC intra prediction. Significant rate-distortion gains have also been achieved on the reconstructed image, after coding and decoding the prediction residue, compared with the H.264/AVC and a sparse spatial prediction method which will be referred to as the generalized template matching approach.

The paper compares a sparse approximation based spatial texture prediction method with the template matching based prediction. Template matching algorithms have been widely considered for image prediction. These approaches rely on the assumption that the predicted texture contains a similar textural structure with the template in the sense of a simple distance metric between template and candidate. However, in real images, there are more complex textured areas where template matching fails. The basic idea instead is to consider sparse approximation algorithms. The proposed sparse spatial prediction is assessed against the prediction method based on template matching with a static and optimized dynamic templates. The spatial prediction method is then assessed in a coding scheme where the prediction residue is encoded with a coding approach similar to JPEG. Experimental observations show that the proposed method outperforms the conventional template matching based prediction.

In this paper, we consider the problem of spatial prediction based on sparse representations. Several algorithms dealing with this problem can be found in the literature. We propose a novel method involving a mixture of sparse representations. We first place this approach into a probabilistic framework and then derive a practical procedure to solve it. Comparisons of the rate-distortion performance show the superiority of the proposed algorithm with regard to other state-of-the-art algorithms.

The paper presents a dictionary construction method for spatial texture prediction based on sparse approximations. Sparse approximations have been recently considered for image prediction using static dictionaries such as a DCT or DFT dictionary. These approaches rely on the assumption that the texture is periodic, hence the use of a static dictionary formed by pre-defined waveforms. However, in real images, there are more complex and non-periodic textures. The main idea underlying the proposed spatial prediction technique is instead to consider a locally adaptive dictionary, A, formed by atoms derived from texture patches present in a causal neighborhood of the block to be predicted. The sparse spatial prediction method is assessed against the sparse prediction method based on a static DCT dictionary. The spatial prediction method is then assessed in a complete image coding scheme where the prediction residue is encoded using a coding approach similar to JPEG.