We address the challenge of generalizing video deblurring models to real-world scenarios, where traditional methods often fail due to a significant domain gap between synthetic and real blur. This work extends the image-to-image translation framework to the more complex domain of video deblurring, introducing a training procedure that effectively bridges this gap. Our method integrates a robust video deblurring backbone with realistic motion priors captured from gimbal-mounted cameras, enabling the model to generalize well across both synthetic and real-world datasets—without requiring paired real-world training data or dataset-specific tuning. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art methods on multiple real-world benchmarks, including datasets never seen during training. Importantly, the generalization capability stems not from the specific architecture, but from the modular training procedure itself, which can be readily applied to other deblurring backbones. This positions our method as a scalable and transferable framework for real-world video deblurring.

This article introduces a novel deep learning method for efficiently compressing truncated signed distance function (TSDF) volumes. Previous works divide TSDF volumes into blocks and encode each block into the same number of latent codes, regardless of the geometric complexity stored in each block. This results in higher bitrates and increased complexity in arithmetic coding, as both complex and simple geometric blocks require the same number of latent codes. To address these inefficiencies, we propose a Hyperprior-based Latent Code Selection (HyperLCS) that dynamically adjusts the number of latent codes based on the geometric complexity of TSDF blocks. Through geometry-complexity-adaptive selective coding, HyperLCS reduces unnecessary bit allocation and arithmetic coding complexity, leading to improved coding efficiency, lower bitrates, and faster compression times. Furthermore, we introduce the Sampling-Enhancing Residual Block (SERB), a modified residual block designed to compensate for feature loss during spatial sampling by calculating residuals at the input resolution and adjusting the sampled output. Experimentally, SERB demonstrated improved TSDF volume compression performance compared to conventional residual blocks with the same capacity in terms of weight parameters. By combining HyperLCS and SERB, our method achieves superior TSDF volume compression performance, maintaining high data fidelity even at high compression rates. Experimental results demonstrate substantial improvements over existing techniques.

This paper introduces a method for efficiently interpolating 3D dynamic sequences using truncated signed distance function (TSDF) volumes. The method calculates bi-directional motions between TSDF volumes of two frames and refines them to reconstruct intermediate frames. Unlike point cloud-based methods, which can suffer from varying and irregular point densities, the uniform and dense grid structure of TSDF offers a consistent framework for estimating the true motion of objects within a scene. In our experiments, the TSDF-based method offers more precise and reliable smooth motion prediction compared to the often error-prone surface depiction in point clouds. Experimental results demonstrate improved accuracy and reduced computational complexity, making it suitable for real-time applications.

This work presents the restoration of drawings of wooden built heritage. Hand-drawn drawings contain the most important original information but are often severely degraded over time. A novel restoration method based on the vector quantized variational autoencoders is presented. Latent space representations of drawings and noise are learned, which are used to map noisy drawings to clean drawings for restoration and to generate authentic noisy drawings for data augmentation. The proposed method is applied to the drawings archived in the Cultural Heritage Administration. Restored drawings show significant quality improvement and allow more accurate interpretations of information.

This paper presents a recurrent learning-based facial attribute recognition method that mimics human observers' visual fixation. The concentrated views of a human observer while focusing and exploring parts of a facial image over time are generated and fed into a recurrent network. The network makes a decision concerning facial attributes based on the features gleaned from the observer's visual fixations. Experiments on facial expression, gender, and age datasets show that applying visual fixation to recurrent networks improves recognition rates significantly. The proposed method not only outperforms state-of-the-art recognition methods based on static facial features, but also those based on dynamic facial features.

Applying alpha matting to large images is a challenging task because of its computational complexity. This paper provides a divide and conquer strategy for performing closed-form matting. The matting problem, defined for an entire image, is broken down into systems of linear equations defined for very small blocks of the image. The sizes of the small systems are small enough for us to find solutions efficiently using a direct sparse linear equation system solver. The small systems are solved following a sequential order such that the alpha matte grows from a user scribble. With the block sequential application, matting is performed on fan-shaped partitions in parallel on multiple processing cores. Experiments on large test images as well as on standard benchmark test images show that the proposed parallel block sequential matting provides high quality alpha mattes with good scalability.

In this paper we propose an efficient method for a convex optimization problem which involves a large nonsymmetric and non-Toeplitz matrix. The proposed method is an instantiation of the alternating direction method of multipliers applied in Krylov subspace. Our method offers significant advantages in computational speed for the convex optimization problems involved with general matrices of large size. We apply the proposed method to the restoration of spatially variant blur. The matrix representing spatially variant blur is not block circulant with circulant blocks (BCCB). Efficient implementation based on diagonalization of BCCB matrices by the discrete Fourier transform is not applicable for spatially variant blur. Since the proposed method can efficiently work with general matrices, the restoration of spatially variant blur is a good application of our method. Experimental results for total variation restoration of spatially variant blur show that the proposed method provides meaningful solutions in a short time.

Images acquired by a camera show lens blur due to imperfection in the optical system even when images are properly focused. Lens blur is non-stationary in a sense that the amount of blur depends on pixel locations in a sensor. Lens blur is also asymmetric in a sense that the amount of blur is different in the radial and tangential directions, and also in the inward and outward radial directions. This paper presents parametric blur kernel models based on the normal sinh-arcsinh distribution function. The proposed models can provide flexible shapes of blur kernels with a different symmetry and skewness to model complicated lens blur due to optical aberration in a properly focused images accurately. Blur of single focal length lenses is estimated, and the accuracy of the models is compared with the existing parametric blur models. An advantage of the proposed models is demonstrated through deblurring experiments.

Environmental illumination can degrade performance of face recognition systems. This paper presents a method to reduce the effect of environmental illumination for face recognition with near infrared (NIR) imaging systems. The camera response function that maps the image irradiance to image brightness is estimated via principal component analysis with pairs of facial images taken under different lighting conditions. The inverse of camera response function is used to map two facial images of a subject taken with and without frontal NIR lighting under ambient light. The irradiance of facial image with frontal NIR lighting in the absence of ambient light is estimated using the inverse camera response function. Then the brightness image with only the frontal NIR lighting is obtained from the irradiance to be used for face recognition. Since the brightness images are used by face recognition system when there is no ambient light, the same face recognition algorithm and database can be used without any modification. The proposed method is a pre-processing method and does not require additional hardware. Performance is evaluated and compared to existing methods at various false matching rates. The proposed method can enhance the performance of face recognition in the presence of ambient light at false matching rates at which practical biometric authentication systems operate.

A particle filtering based block-wise estimation method for estimation of motion in a video sequence and joint estimation of disparity and motion in a stereo video sequence is proposed. Parameters of motion and disparity of a block in a sequence are defined as a state, and evolution of the state with respect to the block index is tracked with particle filtering. The state is assumed to be dependent on the states of neighboring blocks. Estimated motion and disparity fields are consistent and suitable for intermediate frame or view generation. The particle filter provides a method to effectively sample the search space. The particles are concentrated in regions where the probability density function for the state has large values. Hence, the locations of the particles are good candidates for search points of a fast search method. The proposed method can estimate motion and disparity with a fraction of search points necessary for conventional estimation methods.

In this paper, we present a new format conversion methodology in which the interlaced 3:2 pulldown film is converted to the NTSC video. For display purposes, the 24 frames/s film is converted to 60 fields/s video by alternatively repeating each frame for 3 fields and 2 fields. This conversion results in severe motion judder when the film is displayed on the progressive TV. Although motion estimation and compensation (ME and MC) are employed to eliminate the motion judder, other artifacts may be resulted depending on the accuracy of ME. Such motion artifacts are including halo effects, block artifacts, and the severe distortion in periodic patterns when the block-based ME is used to interpolate the intermediate fields. To resolve those problems, we propose a robust full-search block motion algorithm using pseudo M-estimator, which is performed on field images to reduce the physical memory size (reducing motion search range) when implemented with hardware.

This paper presents an image size conversion method based on bilateral filtering. We formulate interpolation with bilateral filters by an optimization problem, with which we can enforce the interpolation constraint as well as smoothness constraint. The designed bilateral filter can be used in image size conversion, providing better pass-band characteristics and less artifacts from aliasing.

Block-wise processing of motion compensated interlaced-to-progressive conversion (MC-IPC) can cause spurious errors that degrade converted image quality significantly. This paper presents a novel IPC method based on motion adaptive IPC (MA-IPC) with its adaptation assisted by the motion compensation. The motion adaptation is performed with respect to a motion selected from a limited set based on the accuracy evaluated with a reduced and overlapped block. The use of the reduced and overlapped blocks makes evaluation of the accuracy more precise, and implementation less costly. The motion adaptation structure ensures that any possible error occurs pixel-wise so that the block-wise spurious errors can be eliminated. The estimation of the motion is performed efficiently with data already available for the motion adaptation. The proposed motion compensation assisted MA-IPC method provides image quality similar to the MC-IPC, for those images, the MC-IPC offers significant improvement over the MA-IPC, yet at considerably lower computational and implementation costs.

In this paper, we extend the results of the denoising in the wavelet domain to the lapped transform (LT) domain. The LT coefficients are rearranged into the octave based representation, and their statistics are modeled by the same methods as the wavelet coefficients. Since the part of the LT that follows the discrete cosine transform (DCT) is orthogonal, the problem of removing the quantization noise introduced in the DCT based codecs can be solved in the LT domain. The maximum a posteriori (MAP) estimator is obtained, which provides a novel approach to the image compression artifact reduction problem. The proposed method removes the additive noise and the quantization noise effectively with the significant improvement of image quality.

This paper presents a novel interlaced-to-progressive conversion (IPC) method based on the motion adaptive IPC (MA-IPC) with its adaptation assisted by the motion compensation. The motion adaptation is performed with respect to a motion selected from a limited set based on the accuracy evaluated with a reduced and overlapped block. The use of the reduced and overlapped blocks makes the evaluation of the accuracy more precise, and the implementation less costly. The motion adaptation structure ensures that any possible error occurs pixel-wise, so that block-wise spurious error can be eliminated. The estimation of the motion is performed efficiently with data already available for the motion adaptation. The proposed motion compensation assisted motion adaptive IPC (MoMA-IPC) method provides image quality similar to the motion compensated IPC (MC-IPC) for those images the MC-IPC offers significant improvement over the MA-IPC, yet at the considerably lower computational and implementation cost.

The histogram equalization (HE) is a widely used contrast enhancement method. But what is missing from the HE is a mechanism to control the rate of enhancement. The enhanced image always follows the uniform distribution. This paper presents a simple enhancement rate control mechanism for the HE. The gradient of the mapping function is controlled by putting constraints on the probability density function with the bin underflow (BU) and bin overflow (BO). The BUBO operation can provide the rate of enhancement from non to the full HE with a single parameter. With the enhancement rate control mechanism available, the HE can be used to perform image processing tasks such as black/white level stretch or automatic brightness control as well as variable rate contrast enhancement.

Motion compensated interlaced-to-progressive conversion (MC-IPC) methods employ implicit decision on whether to use the estimated motion vectors in order to prevent spurious errors due to poorly estimated motion vectors. This often limits the performance of the MC-IPC. We present an IPC method based on the pattern matching assisted motion estimation and motion vector histogram analysis. The proposed method employs an explicit decision on whether to use the estimated motion vectors. Extensive experiments show that high quality IPC can be achieved without spurious errors.

Image/video size conversion at variable rates requires that a large set of interpolation filters should be stored in a table. We present the interpolated Mth band filters as the interpolation filters. The interpolated Mth band filters are obtained from the cubic spline interpolation of a prototype M/sub p/th band eigenfilter. The proposed filter can be calculated in real time, eliminating the need for a large on-chip memory. Scaled images using the proposed filters show superb image quality.

The picture quality of video frames encoded at very low-bit rates often suffers from both blocking and ringing artifacts. We present two post-processing methods to mitigate the visual quality degradation caused by these artifacts. To reduce the blocking artifact of decoded images, we substitute IDCT for the lapped orthogonal transform embedded inverse discrete cosine transform (le-IDCT). On the other hand, we post-process the decoded video frames using a nonlinear robust filter to reduce the ringing artifact. Extensive simulation results indicated significant improvement in both objective and subjective visual qualities. The computation overhead incurred due to these quality enhancement operations is quite moderate, and can be easily optimized to achieve real-time operation.

Images and video sequences often have to be scaled to a different size. We present binary interpolation filters obtained from the Mth band eigenfilters via the lifting steps for the sampling rate alteration. The binary filters are implemented with only the shift and add operations, providing an efficient way to perform image size conversion. Scaled images using the proposed filters show superb image quality.

At low bit rates, image compression codecs based on overlapping transforms introduce spurious oscillations known as ringing artifacts in the vicinity of major edges. Unlike previous works, we present a maximum-likelihood approach to the ringing-artifact removal problem. Our approach employs a parameter estimation method based on the k-means algorithm with the number of clusters determined by a cluster-separation measure. The proposed algorithm and its simplified approximation are applied to JPEG2000 compressed images. Our results show effective and efficient removal of ringing artifacts.

At low bit rates, image compression codecs based on overlapping transforms introduce spurious oscillation known as ringing artifacts in the vicinity of major edges. The image quality can be enhanced considerably by removing the artifacts. We present a maximum likelihood approach to the ringing artifact removal problem. Our approach employs a parameter estimation method based on the k-means algorithm with the number of clusters determined by a cluster separation measure. The proposed algorithm and its simplified approximation are applied to JPEG2000 compressed images to demonstrate their effectiveness.

This paper presents the generalized lapped biorthogonal transform embedded inverse discrete cosine transform (ge-IDCT) as an alternative to the IDCT. The ge-IDCT with nonlinear weighting in the embedded transform domain can reconstruct the signal with alleviated blockishness. Additional complexity, imposed by the replacement, is trivial thanks to an efficient lattice structure. The proposed ge-IDCT is applied in the JPEG still image compression standard to demonstrate its validity.

The Gibbs distribution with spatially adaptive clique potential functions is used as the image prior distribution in the coding artifact reduction problem. Unlike previous approaches, the image is divided in regions based on the local statistics and a clique potential function is assigned for each region. A non-differentiable potential function that preserves edge in detailed image regions is introduced. Images recovered by the proposed approach contain reduced coding artifact and improved preservation of details.

Image compression codecs suffer from the various coding artifacts at low bit rate. The coding artifacts can be removed by way of post-processing. Most of the coding artifacts removal algorithms depend on a priori knowledge of the original image, which is about its smoothness. The resulting surface of the restored image is often too smooth to be the one in the natural scene. Dithering is based on the idea of adding controlled noise to the system to achieve the better results. An algorithm based on regularization and dithering is proposed to remove the coding artifacts. The results show that the algorithm removes the coding artifacts successfully and the surface of the restored image is more visually pleasing.

Biased anisotropic diffusion is applied to the coding artifacts removal of the DCT based codec. It is formulated as a cost minimization problem. The weighting factors of the cost function are controlled such that the solution removes the blocking effect and conceals the block losses. It has an advantage over other postprocessing schemes because it handles the discontinuity of the image, smoothes the image selectively, and takes the visual masking in to account. Features needed for the weighting factors are extracted directly from the DCT coefficients to reduce the computational complexity.