Raw data acquired by imaging devices are converted into digital images by post-processing algorithms. However, these algorithms are significantly affected by numerous illumination conditions. Particularly, in the case in which illumination conditions depend on canonical light sources, unwanted light sources that locally illuminate the scene or mixed light from several light sources are recognized as the spatially varying illumination conditions. These complex illumination conditions cause several artifacts by affecting the digital image acquisition process. For example, optical aberrations are generated by refraction of complex light, or illumination estimation is significantly affected by the different color temperatures of multiple light sources. To overcome these problems, this study proposes an algorithm that decomposes the complex illumination from several light sources. First, the spatially varying illumination condition is discussed, and the artifacts generated by the conditions, such as false colors and aberrations, are analyzed. Second, mixed light from several canonical light sources is decomposed based on the imaging devices and spectral information of the canonical light sources. The proposed method has low complexity and increased applicability. Furthermore, the improvement scheme based on the proposed method has a parallelized structure and can be easily applied to various types of algorithms, such as color constancy, deconvolution, denoising, and contrast enhancement. The algorithms employing the improvement scheme show the potential of the proposed method for solving the problems associated with multiple light sources.

Conventional contrast enhancement methods, including global and local enhancements, produce enhanced images with some limitations. Global contrast enhancement does not take the local characteristics into consideration, and therefore, the enhancement performance could be limited. On the other hand, a local contrast enhancement method achieves a noticeable improvement, but it generates unnatural improvement results compared with the input image. Due to the complementary characteristics of these two methods, it is hard to achieve remarkable contrast enhancement without visual artifacts. To overcome the limitations, we propose a new tone mapping function for contrast enhancement using an optimization approach that is subject to constraints such as the output image needing to be enhanced naturally and noticeably. Since contrast enhancement without artificiality is possible when the enhancement process mimics the human eye, we model the human visual perception system, and then, the model is incorporated into the proposed tone mapping function. Consequently, the contrast of the image is adaptively enhanced according to a region that is more attractive to a person. The experimental results demonstrate that the proposed algorithm outperforms other contrast enhancement methods in terms of both objective and subjective criteria.

Image deconvolution is an ill-posed problem that usually requires prior knowledge for regularizing the feasible solutions. In literature, iterative methods estimate an intrinsic image, minimizing a cost function regularized by specific prior information. However, it is difficult to directly minimize the constrained cost function, if a nondifferentiable regularization (e.g., the sparsity constraint) is employed. In this paper, we propose a nonderivative image deconvolution algorithm that solves the under-constrained problem (i.e., a non-blind image deconvolution) by successively solving the permuted subproblems. The subproblems, arranged in permuted sequences, directly minimize the nondifferentiable cost functions. Various Lp-regularized (0 < p ≤ 1, p = 2) objective functions are utilized to demonstrate the pixel-wise optimization, in which the projection operator generates simplified, low-dimensional subproblems for estimating each pixel. The subproblems, after projection, are dealt with in the corresponding hyperplanes containing the adjacent pixels of each image coordinate. Furthermore, successively solving the subproblems can accelerate the deconvolution process with a linear speed-up, by parallelizing the subproblem sequences. The image deconvolution results with various regularization functionals are presented and the linear speed-up is also demonstrated with a parallelized version of the proposed algorithm. Experimental results demonstrate that the proposed method outperforms the conventional methods in terms of the improved-signal-to-noise ratio and structural similarity index measure.

In this paper, we propose a chromatic aberration (CA) correction algorithm based on a false color filtering technique. In general, CA produces color distortions called color fringes near the contrasting edges of captured images, and these distortions cause false color artifacts. In the proposed method, a false color filtering technique is used to filter out the false color components from the chroma-signals of the input image. The filtering process is performed with the adaptive weights obtained from both the gradient and color differences, and the weights are designed to reduce the various types of color fringes regardless of the colors of the artifacts. Moreover, as preprocessors of the filtering process, a transient improvement (TI) technique is applied to enhance the slow transitions of the red and blue channels that are blurred by the CA. The TI process improves the filtering performance by narrowing the false color regions before the filtering process when severe color fringes (typically purple fringes) occur widely. Last, the CA-corrected chroma-signal is combined with the TI chroma-signal to avoid incorrect color adjustment. The experimental results show that the proposed method substantially reduces the CA artifacts and provides natural-looking replacement colors, while it avoids incorrect color adjustment.

In this paper, we formulate the colorization-based coding problem into an optimization problem, i.e., an L1 minimization problem. In colorization-based coding, the encoder chooses a few representative pixels (RP) for which the chrominance values and the positions are sent to the decoder, whereas in the decoder, the chrominance values for all the pixels are reconstructed by colorization methods. The main issue in colorization-based coding is how to extract the RP well therefore the compression rate and the quality of the reconstructed color image becomes good. By formulating the colorization-based coding into an L1 minimization problem, it is guaranteed that, given the colorization matrix, the chosen set of RP becomes the optimal set in the sense that it minimizes the error between the original and the reconstructed color image. In other words, for a fixed error value and a given colorization matrix, the chosen set of RP is the smallest set possible. We also propose a method to construct the colorization matrix that colorizes the image in a multiscale manner. This, combined with the proposed RP extraction method, allows us to choose a very small set of RP. It is shown experimentally that the proposed method outperforms conventional colorization-based coding methods as well as the JPEG standard and is comparable with the JPEG2000 compression standard, both in terms of the compression rate and the quality of the reconstructed color image.

In this letter, we propose a new detection method for video surveillance which provides for a robust and real-time working object detection under various global illumination conditions. The proposed scheme needs no manual parameter settings for different illumination conditions, which makes the algorithm applicable to automatic surveillance systems. Two special filters are designed to eliminate the spurious object regions that occur due to the charge coupled device (CCD) noise, making the scheme stable even in very low illumination conditions. We demonstrate the effectiveness of the proposed algorithm experimentally with different illumination conditions, changes in contrast, and noise level.

In this paper, we propose a level set based energy functional, the minimization of which results in simultaneous background modeling, foreground segmentation, and contour smoothing. The simultaneous dealing of background modeling and foreground segmentation has the effect that the two processes constrain each other positively, such that a good estimate of the background can be obtained with a small number of frames, and a temporal change in the scene is reflected quickly in the construction of the background image. Furthermore, the simultaneous level set based contour smoothing eliminates spurious regions, and smoothes the contour that encompasses the object, so that a good representation for the boundary of the object is obtained. The level set based approach makes it possible to derive a level set based Euler-Lagrangian equation, which can be directly implemented and works in real-time.

For the last two decades, two related approaches have been studied independently in conjunction with limitations of image sensors. The one is to reconstruct a high-resolution (HR) image from multiple low-resolution (LR) observations suffering from various degradations such as blur, geometric deformation, aliasing, noise, spatial sampling and so on. The other one is to reconstruct a high dynamic range (HDR) image from differently exposed multiple low dynamic range (LDR) images. LDR is due to the limitation of the capacitance of analogue-to-digital converter and the nonlinearity of the imaging system's response function. In practical situations, since observations suffer from limitations of both spatial resolution and dynamic range, it is reasonable to address them in a unified context. Most super-resolution (SR) image reconstruction methods that enhance the spatial resolution assume that the dynamic ranges of observations are the same or the imaging system's response function is already known. In this paper, the conventional approaches are overviewed and the SR image reconstruction, which simultaneously enhances spatial resolution and dynamic range, is proposed. The image degradation process including limited spatial resolution and limited dynamic range is modelled. With the observation model, the maximum a posteriori estimates of the response function of the imaging system as well as the single HR image and HDR image are obtained. Experimental results indicate that the proposed algorithm outperforms the conventional approaches that perform the HR and HDR reconstructions sequentially with respect to both objective and subjective criteria.

De-interlacing based on motion compensation (MC) is one of the best ways of improving the resolution of a progressive video converted from an interlaced source. However, the converted frames often suffer from serious defects like feathering artifacts in regions with inaccurate motion vectors (MVs). In such regions, an intra-field method that is robust to MV errors can be used to correct motion compensation artifacts (MCAs). In this letter, we propose an adaptive arbitration method to combine intra-field and MC methods adequately. The proposed method considers the reliability of MC results along with the MV reliability measured by the spatio-temporal consistency of MVs and displaced pixel differences. The MC reliability is determined by detecting MCAs in MC results, and then the MV reliability is adjusted according to the MC reliability. Also, adaptive-weight MC and pseudo MC methods are proposed to provide more reliable MC results and to improve the accuracy of MCA detection, respectively. Experimental results show that the proposed method provides high-quality video sequences while reducing many visible artifacts.

Recently, kernel-based tracking algorithms such as the mean shift tracking algorithm has been proposed, which use the information of color histogram together with some spatial information provided by the kernel. However, in spite of the fast speed, there exists an inherent instability problem which is due to the use of an isotropic kernel for spatiality and the use of the Bhattacharyya coefficient as the similarity function. In this paper, we will analyze how the use of the kernel and the Bhattacharyya coefficient can arouse the instability problem. Based on the analysis, we propose a tracking scheme that uses a new representation of the location of the target which is constrained by the color, the area, and the spatiality information of the target in a more stable way than the mean shift algorithm. With this representation, the target localization in the next frame can be achieved by a direct one step computation, and the tracking becomes stable, even in difficult situations such as low-rate-frame environment, and partial occlusion.

In this letter, we analyze the relationship between the change in the intensity value and the scale of an image feature, when a generalized function is used as the fidelity term in the total variation-based noise removal scheme. Based on the analysis, we propose a designing method of the fidelity function that results in any desired monotonic relationship between the intensity change and the scale. As an example, we designed a fidelity function that results in a larger contrast between the intensity change of a small scaled feature and that of a large scaled one than the original total variation-based noise removal scheme that uses the norm as the fidelity function.

In this paper, we propose a high-resolution image reconstruction algorithm to reduce the distortion in the reconstructed high-resolution image due to the inaccuracy of motion estimation. For this purpose, we analyze the noise caused by the inaccurate motion information. Based on this analysis, we propose a new regularization functional. The proposed algorithm requires no prior information about the original image or the inaccurate motion information. Experimental results indicate that the proposed algorithm outperforms conventional approaches with respect to both objective and subjective criteria.

In this paper, a new EDI-based deinterlacing algorithm is proposed. Generally, EDI algorithms perform visually better than any other intrafield deinterlacing algorithm. However, they produce unpleasant results due to their failure in estimating edge direction. To estimate the edge direction precisely, not only simple difference between adjacent two lines but also edge patterns are used. Here, we analyze properties of edge patterns and model them as weight functions. The weight functions help the proposed method to estimate the edge direction precisely.

The problem of recovering full-color images from a noisy and color-sampled observation is considered in this paper. The noise in imaging sensors corrupts the color filter array (CFA) and introduces artifacts during the color interpolation step. Instead of filtering the noise before or after the color interpolation step, we remove the noise during the color interpolation step. This allows us to obtain a color interpolated image without noise and artifacts especially along the edges and in the detail of the image. Color interpolation is carried out on the weighted color difference domain to consider cross-channel correlation. In order to avoid artifacts in high frequency regions and improve the performance, an edge indicator function is used and directions of edges are considered in the proposed interpolation method. Interpolation artifacts are removed by the modified filtering on color difference domain. Experimental results illustrate the effectiveness of the proposed method.

We propose a robust de-interlacing algorithm which combines edge dependent interpolation (EDI) and global motion compensation (GMC). Generally, the EDI algorithm shows a visually better performance than any other de-interlacing algorithm using one field. However, due to the restricted information in one field, a high quality progressive image from interlaced sources cannot be acquired by intrafield methods. Hence, the proposed algorithm makes use of a process of mixing EDI and GMC. In order to obtain the best result, an adaptive thresholding algorithm for detecting the failure of GMC is proposed. Experimental results indicate that the proposed algorithm outperforms conventional approaches with respect to both objective and subjective criteria.

Wavelet is a well-known noise reduction tool for its several nice properties. Since the wavelet domain is a kind of transform domain, thresholding in it can cause problems at the neighborhood of discontinuous points, seen as a ringing artifact in a 2D image. In a low-bit rate network and a heavy error-prone environment, it is a serious problem for an end-user. We propose an efficient ringing reduction algorithm based on the standard wavelet decomposition. Since its structure is the reverse of the wavelet denoising method, our algorithm has advantages regarding computational load and memory efficiency over other ringing reduction filters. Experimental results show that the proposed algorithm dramatically reduces ringing along edges.

The problem of recovering a high-resolution image from a sequence of low-resolution DCT-based compressed observations is considered in this paper. The introduction of compression complicates the recovery problem. We analyze the DCT quantization noise and propose to model it in the spatial domain as a colored Gaussian process. This allows us to estimate the quantization noise at low bit-rates without explicit knowledge of the original image frame, and we propose a method that simultaneously estimates the quantization noise along with the high-resolution data. We also incorporate a nonstationary image prior model to address blocking and ringing artifacts while still preserving edges. To facilitate the simultaneous estimate, we employ a regularization functional to determine the regularization parameter without any prior knowledge of the reconstruction procedure. The smoothing functional to be minimized is then formulated to have a global minimizer in spite of its nonlinearity by enforcing convergence and convexity requirements. Experiments illustrate the benefit of the proposed method when compared to traditional high-resolution image reconstruction methods. Quantitative and qualitative comparisons are provided.

Among many image compression approaches, the discrete cosine transform (DCT) is widely used. However, dividing an image into small blocks prior to coding causes "blocking artifacts". A deblocking algorithm is proposed for DCT-based compressed images using anisotropic diffusion; it is derived from the ALM (Alvarez, Lions, Morel) diffusion model (Alvarez, L. et al., SIAM J. Numer. Anal., vol.29, no.3, p.845-66, 1992). It can control the diffusion rate in the normal direction of the edges using a "rate control parameter". It functions not only as an isotropic diffusion at block boundaries of smooth regions, but also as an anisotropic diffusion that diffuses the image only in the normal direction of the edges at edges or block boundaries of texture regions. The rate control parameters at block boundaries are chosen carefully to reduce discontinuities. To avoid oversmoothing of the texture region, a "speed parameter" is employed. The speed parameter makes the diffusion process slow at the texture region, while making it fast at the smooth region.

In this paper, we propose a high-resolution image reconstruction algorithm considering inaccurate subpixel registration. A regularized iterative reconstruction algorithm is adopted to overcome the ill-posedness problem resulting from inaccurate subpixel registration. In particular, we use multichannel image reconstruction algorithms suitable for applications with multiframe environments. Since the registration error in each low-resolution image has a different pattern, the regularization parameters are determined adaptively for each channel. We propose two methods for estimating the regularization parameter automatically. The proposed algorithms are robust against registration error noise, and they do not require any prior information about the original image or the registration error process. Information needed to determine the regularization parameter and to reconstruct the image is updated at each iteration step based on the available partially reconstructed image. Experimental results indicate that the proposed algorithms outperform conventional approaches in terms of both objective measurements and visual evaluation.

The problem of recovering a high-resolution image from a sequence of low-resolution DCT-based compressed images is considered in this paper. The presence of the compression system complicates the recovery problem, as the operation reduces the amount of frequency aliasing in the low-resolution frames and introduces a non-linear quantization process. The effect of the quantization error and resulting inaccurate sub-pixel motion information is modeled as a zero-mean additive correlated Gaussian noise. A regularization functional is introduced not only to reflect the relative amount of registration error in each low-resolution image but also to determine the regularization parameter without any prior knowledge in the reconstruction procedure. The effectiveness of the proposed algorithm is demonstrated experimentally.

The problem of recovering a high-resolution image from a sequence of low-resolution DCT-based compressed images is considered. The presence of the compression system complicates the recovery problem, as the operation reduces the amount of frequency aliasing in the low-resolution frames and introduces a non-linear quantization process, The effect of the quantization error and resulting inaccurate sub-pixel motion information is modeled as a zero-mean additive correlated Gaussian noise. A regularization functional is introduced, not only to reflect the relative amount of registration error in each low-resolution image, but also to determine the regularization parameter without any prior knowledge in the reconstruction procedure. The effectiveness of the proposed algorithm is demonstrated experimentally.

The efficiency of shape coding is an important problem concerning content-based image manipulations and object-based coding of the video sequences. In order to encode the shape information of an object, the boundary is approximated by a polygon which can be encoded with the smallest number of bits for maximum allowable distortion. The conventional boundary coding schemes, however, does not successfully remove the temporal redundancy of the video sequences. This paper proposes a new boundary encoding scheme by which the temporal redundancy between two successive frames is efficiently removed, resulting in lower bit-rate than the conventional algorithms. The interframe vertex selection problem is solved by finding the path with the minimum cost in the directed acyclic graph (DAG) and its fast version using a simplified graph is introduced to reduce the computational load. The vertices were selected from both the current frame to be encoded and the previous frame already encoded, and thus, the temporal redundancy was effectively removed.

A generalized auto-regressive (AR) model is proposed for linear prediction based on adaptive spatio-temporal support region (ASTSR). The conventional AR model has the drawback that the prediction error increases in the edge region because the rectangular support region of the edge does not satisfy the stationary assumption, Thus the proposed approach puts an emphasis on the foundation of an adaptive spatio-temporal support region for the AR model, called ASTSR. The ASTSR consists of two pairs: 1) An adaptive spatial support region (ASSR) composed of pixels that are highly correlated with the current predicted pixel. 2) An adaptive temporal support region (ATSR) formed based on the existence of motion. The proposed AR model not only produces more accurate model parameters but also reduces the computational complexity in the motion picture restoration.

A frequency adaptive choice of the regularization parameter based on an unbiased estimate of the risk is investigated. The method is intended to be used in the restoration of images degraded by a shift invariant blur and additive white Gaussian noise. The results are compared with the more widely studied non adaptive case.

While high resolution images are required for various applications, aliased low-resolution images are only available due to the finite sensor pixels. In this paper a new approach is proposed to obtain superresolution using regularization. With the proposed approach based on the discrete cosine transform, high-resolution images can be efficiently reconstructed both in the underdetermined case and in the case with inaccurate motion estimates.

When approximating the shape of a region, a fixed bound on the tolerable distortion is set for approximating its contour points. An adaptive approximation bound for lossy coding of the contour points is proposed. A function representing the relative significance of the contour points is defined to adjust the distortion bound along the region contour allowing an adaptive approximation of the region shape. The effectiveness of the adaptive contour coding approach for a region-based coding system is verified through experiments.

In this paper a three dimensional anisotropic diffusion equation is proposed to remove noise in video sequences. The three dimensional anisotropic diffusion equation utilizes the fact that consecutive frames of high correlation can be obtained in video sequences. It will be shown that the three dimensional diffusion equation is more suitable for video sequences than the two dimensional diffusion equation.

This paper introduces an iterative regularized approach to increase the resolution of a video sequence. A multiple input smoothing convex functional is defined and used to obtain a globally optimal high resolution video sequence. A mathematical model of multiple inputs is described by using the point spread function between the original and bilinearly interpolated images in the spatial domain, and motion estimation between frames in the temporal domain. An iterative algorithm is utilized for obtaining the solution. The regularization parameter is updated at each iteration step from the partially restored video sequence. Experimental results demonstrate the capability of the proposed approach.

In this correspondence, a constrained least-squares multichannel image restoration approach is proposed, in which no prior knowledge of the noise variance at each channel or the degree of smoothness of the original image is required. The regularization functional for each channel is determined by incorporating both within-channel and cross-channel information. It is shown that the proposed smoothing functional has a global minimizer.

Laminated sediments from lakes are of great interest to paleoclimatologists interested in high resolution records of past climate change. Laminated sediments from Lake Malawi in East Africa are investigated by X-ray analysis. Due to the physical nature of sediments, X-ray images having penetrated through sediments are corrupted by signal dependent nonstationary noise, which affects the accuracy and precision of time-series and the spectral analysis of patterns. Therefore, a nonlinear pre-filtering algorithm is proposed to suppress the noise, so that more stable and reliable analysis can be obtained. The algorithm is based on the generalized weighted regularization.

The determination of the regularization parameter is an important issue in regularized image restoration, since it controls the trade-off between fidelity to the data and smoothness of the solution. A number of approaches have been developed in determining this parameter. In this paper, a new paradigm is adopted, according to which the required prior information is extracted from the available data at the previous iteration step, i.e., the partially restored image at each step. We propose the use of a regularization functional instead of a constant regularization parameter. The properties such a regularization functional should satisfy are investigated, and two specific forms of it are proposed. An iterative algorithm is proposed for obtaining a restored image. The regularization functional is defined in terms of the restored image at each iteration step, therefore allowing for the simultaneous determination of its value and the restoration of the degraded image. Both proposed iteration adaptive regularization functionals are shown to result in a smoothing functional with a global minimum, so that its iterative optimization does not depend on the initial conditions. The convergence of the algorithm is established and experimental results are shown.

Proposes a general form of the weighted smoothing functional for regularized image restoration. The weighting matrices which introduce the spatial adaptivity are defined as a function of the (partially) restored image. As a result no prior knowledge about the image is required but the smoothing functional to be minimized is nonlinear with respect to the unknown image. Conditions for the convexity of the functional are established. An iterative algorithm is proposed for obtaining its minimum. Sufficient conditions for the convergence of the algorithm are established. Various forms of the weighting matrices are proposed. Experimental results demonstrate the effectiveness of the approach.>

The determination of the regularization parameter is an important issue in regularized image restoration, since it controls the trade-off between fidelity to the data and smoothness of the solution. A number of approaches have been developed in determining this parameter. In this paper, we propose the use of a regularization functional instead of a constant regularization parameter. The properties such a regularization functional should satisfy are investigated, and two specific forms of it are proposed. An iterative algorithm is proposed for obtaining a restored image. The regularization functional is defined in terms of the restored image at each iteration step, therefore allowing for the simultaneous determination of its value and the restoration of the degraded image. Both proposed iteration adaptive regularization functionals are shown to result in a smoothing functional with a global minimum, so that its iterative optimization does not depend on the initial conditions. The convergence of the algorithm is established and experimental results are shown.

The authors consider a generalized acoustic tube model of the vocal tract, relating it to the pole-zero type linear prediction. The generalization is done by including the nasal cavity for the modeling, thus forming a three-branched model. The transfer function is obtained from the generalized model by conglomerating one of the three branches to the branch section at the junction of the three branches. It is also discussed how find coefficients for the pole-zero type linear prediction from the voiced sounds. Also discussed is how to evaluate the reflection coefficients by connecting the pole-zero type linear prediction algorithm to the transfer function of the generalized model.>