Few-shot learning aims to generalize deep models to novel categories with only a handful of labeled examples, but existing methods remain vulnerable to task-irrelevant noise, unstable prototype estimation, and limited adaptability under domain shift. To address these issues, we propose the Noise-Adaptive Prototype Perturbation Network (NAPP), a framework that enhances robustness and generalization for few-shot learning. NAPP introduces three key innovations: (1) a Noise Cancellation Mechanism embedded in Vision Transformer self-attention layers that dynamically suppresses spurious, task-irrelevant features. (2) a Mix-Perturbation Module that perturbs class prototypes through augmented feature combinations, producing more stable and transferable prototype representations. (3) an Adaptive Noise-Conditioned Meta-Learning scheme that finetunes less than 0.02% of noise-related parameters at metatest time, enabling efficient and rapid adaptation to unseen classes without eroding pretrained knowledge. Extensive experiments demonstrate that NAPP achieves competitive and superior performance compared to state-of-the-art fewshot classification methods across both in-domain and challenging cross-domain benchmarks.

Image inpainting restores missing regions in a visually plausible way, yet existing methods struggle with irregular holes and complex structures due to fixed kernels and static attention. In this paper, we propose Mask-Aware Deformable Inpainting Network (MADIN), which incorporates mask information for position-aware control. The proposed model employs a Two-branch Offset Estimator that jointly utilizes query features and mask signals to reliably predict reference positions even within masked regions. In addition, Adaptive Offset Range Scaling adjusts offset magnitude based on masking ratio to capture broader context when needed. Experiments on CelebA-HQ and Places2 show that MADIN achieves state-of-the-art results in PSNR, SSIM, LPIPS, and FID, while remaining lightweight and efficient. The code will be released at https://github.com/ohhhyeongsn/MADIN.

In multispectral object detection, integrating visible and thermal features offers significant advantages, particularly for autonomous driving in low-light environments. However, collecting a large dataset of pixel-aligned visible and thermal image pairs is labor-intensive, and achieving real-time alignment in operational driving systems remains a challenge. Additionally, the representation of thermal images varies across different camera types, complicating generalization, while visible images are often prone to environmental noise. To address these issues, we present CAMDet, a novel network that selectively fuses visible and thermal images based on day/night conditions. To compensate for missing modality information, we incorporate a diffusion model-based Visible-Thermal conversion method to synthesize the missing modality. An attention-based Modality Feature Refinement (MFR) module further enhances feature quality by reducing uncertainties in the generated images. Comprehensive experiments on the FLIR and LLVIP datasets show that CAMDet significantly outperforms single-modality detection methods and surpasses multi-modality baseline models that depend on visible-thermal image pairs.

This paper introduces a novel prototype-based incremental learning method designed to address the critical issue of catastrophic forgetting in incremental learning by leveraging prototypes. Incremental learning involves sessions containing multiple classes, each with numerous images. This study tackles catastrophic forgetting by utilizing class prototypes and session prototypes within a session. The proposed method defines class prototypes, representing the characteristic features of specific classes, and session prototypes, calculated as the average of all class prototypes within the session. Learning is performed based on the distances between these prototypes. The approach maximizes the distances between class prototypes within the same session while minimizing the distance between the session prototype and each class prototype. Additionally, when a new session is introduced, the method maximizes the distance between the session prototypes of the previous and new sessions to ensure clear differentiation between sessions. This strategy prevents the mixing of class prototypes, reduces interference between sessions, and effectively alleviates catastrophic forgetting. Experiments conducted on the ImageNet-100 and CIFAR-100 datasets validate the superior performance of the proposed method.

In autonomous driving, reliable object detection across varied environmental conditions is essential, particularly when transitioning between spectral domains like visible light (RGB) and infrared (IR). This paper introduces a novel, fully unsupervised approach for RGB-to-IR domain adaptive object detection. By utilizing generative models to synthesize IR images from RGB inputs, our method eliminates the need for direct IR data collection. The proposed method falls under unsupervised domain adaptation. It generates IR images from RGB inputs to train a domain adaptive object detection model, leveraging a Large Vision-Language Model (LVLM) to transfer the target domain style using text prompts. By generating IR images solely from RGB inputs, the approach eliminates the need for expensive and time-consuming IR data collection, making it highly efficient. This enhances the robustness of object detection across spectral domains, improving vehicle safety in challenging environments. Extensive experiments demonstrate significant improvements in IR detection accuracy, all achieved without direct IR data during training, offering a cost-effective and scalable solution.

This paper addresses the challenge of Single Domain Generalized Object Detection, where a network is trained solely on source images and must accurately detect and classify objects in unseen target domains. To improve object detection across varying weather conditions, it is essential to consider both the local regions where objects are present and the global context of the entire image. In this paper, we introduce a Fourier-Based Local and Global Amplitude MixUp called FLGAM, which uses Fourier transform to augment both the local and global components of a single source image, simulating diverse environmental conditions. By enhancing the training data through this augmentation, the model achieves better generalization to new and unseen environments. Consequently, the proposed approach outperforms other state-of- the-art networks in various weather scenarios encountered in driving situations.

Accurate object detection depends on the precise refinement of bounding box regression. Recent advancements in bounding box regression have introduced a variety of methodologies aimed at reducing the disparity between predicted and ground truth bounding boxes. The prevailing objective functions for bounding box regression typically encompass three key perspectives: i) Intersection over Union (IoU), ii) distance between central points, and iii) aspect ratio alignment. Nonetheless, these existing loss functions encounter two primary challenges including slow convergence of the distance term and aspect ratio variation irrelevant to bounding box localization. This paper presents two novel loss terms to address these challenges. Firstly, we introduce the concept of the Integral of Central-Gaussian, a novel approach that leverages the cumulative distribution function (CDF) derived from a closed-form Gaussian distribution based on the central points of bounding boxes. Secondly, we introduce an alternative aspect ratio representation by minimizing the angle between two bounding boxes in direct proportion to their IoU. We term this comprehensive loss function “Central-Gaussian Angle-IoU” (CA-IoU), seamlessly incorporating the Integral of Central-Gaussian with angle-based IoU. Extensive experiments on various models and benchmarks for object detection highlight the superior performance of CA-IoU loss compared to existing bounding box regression methods. The source code and the corresponding trained models will be made available.

Images obtained from different modalities can effectively enhance the accuracy and reliability of the detection model by complementing specialized information from visible (RGB) and infrared (IR) images. However, integrating information from multiple modalities faces the following challenges: 1) distinct characteristics of RGB and IR images lead to the problem of modality imbalance, 2) fusing multimodal information can greatly affect the detection accuracy, as some of the unique information provided by each modality is lost during the integration process, and 3) RGB and IR images are fused while preserving the noise of each modality. To address these issues, we propose a novel multi spectral object detection network which contains two main components; 1) Cross-modal Information Complementary (CIC) module, and 2) Cosine Similarity Channel Resampling (CSCR) module. The proposed method addresses the modality imbalance problem and efficiently fuses RGB and IR images in the feature level. Extensive experimental results on LLVIP, FLIR,$M^{3}FD$, VEDAI and KAIST benchmark datasets, verify the effectiveness and generalization performance of the proposed multispectral object detection network compared with other state-of-the-art methods.

Despite of pre-processing the raw point cloud is important, limited research has been conducted on learning-based approaches to point cloud upsampling. PU-EdgeFormer [1] model stands out for its exceptional performance, which is attributed to its unique inductive biases that seamlessly blend both local and global characteristics of point clouds through the integration of graph convolution and transformer mechanisms. In this paper, we present EdgeFormer++, an advanced hierarchical edge transformer module designed for arbitrary-scale point cloud upsampling. Our module employs crossattention and dense connections to integrate information from both feature and input point cloud while maintaining the structural elements of graph convolutions and transformers. Experimental results, both qualitative and quantitative, indicate that our proposed method outperforms existing techniques in point cloud upsampling. The official source code is accessible at https://github.com/dohoon2045/PU-EdgeFormer2.

High Dynamic Range (HDR) imaging seeks to enhance image quality by combining multiple Low Dynamic Range (LDR) images captured at varying exposure levels. Traditional deep learning approaches often employ reconstruction loss, but this method can lead to ambiguities in feature space during training. To address this issue, we present a new loss function, termed Gravitated Latent Space (GLS) loss, that leverages a metric tensor to introduce a form of virtual gravity within the latent space. This feature helps the model in overcoming saddle points more effectively. Easy to integrate, the GLS loss function fosters stable learning within a convex environment and demonstrates its performance in improving HDR image quality. Experimental data confirms that the proposed method outperforms existing state-of-the-art techniques in quantitative evaluations.

Detecting small objects in drone-captured images or aerial videos is challenging due to their minimal representation. As data traverses deep learning networks, the information about small objects can diminish, making high-resolution images essential for enhanced detection performance. However, high-resolution images increase computational load undesirably. Leveraging this fact, we propose a streamlined neural network designed specifically for small object detection in high-resolution images. The proposed network encompasses three main components: i) Enhanced High-Resolution Processing Module (EHRPM), ii) the Small Object Feature Amplified Feature Pyramid Network (SOFA-FPN) with its Edge Enhancement Module (EEM), Cross Lateral Connection Module (CLCM), and Dual Bottom-up Convolution Module (DBCM), and iii) the Sigmoid Re-weighting Module (SRM). Compared to several state-of-the-art networks, our method delivers superior performance with fewer parameters and a lower computational demand. The source code is available at https://github.com/datu0615/EHRPM.

In this paper, we present a novel method for point cloud registration in large-scale 3D scenes. Our approach is accurate and robust, and does not rely on unrealistic assumptions. We address the challenges posed by scanning equipment like LiDAR, which often produce point clouds with dense properties. Additionally, our method is effective even in scenes with low overlap rates, specifically less than 30%. Our approach begins by computing overlap region-based correspondences. This involves extracting deep geometric features from point cloud pairs, which is especially beneficial in enhancing registration performance in cases with low overlap ratios. We then construct efficient triplets that vote in the 6D Hough space, representing the transformation parameters. This process involves creating a quartet from overlap region-based correspondences and then forming a final triplet following a sampling process. To mitigate ambiguity during training, we use similarity values of the triplet as features of each vote when configuring votes for network input. Our framework incorporates the architecture of the Fully Convolutional Geometric Features (FCGF) network, augmented with a transformer’s attention mechanism, to reduce noise in the voting process. The final stage involves identifying the consensus of correspondence in the Hough space using a binning approach, which enables us to predict the final transformation parameters. Our method has demonstrated state-of-the-art performance on indoor datasets, including high overlap ratio data like 3DMatch and low overlap ratio data like 3DLoMatch. It has also shown comparable performance to leading methods on outdoor datasets like KITTI.

Despite the recent development of deep learning-based point cloud upsampling, most MLP-based point cloud upsampling methods have limitations in that it is difficult to train the local and global structure of the point cloud at the same time. To solve this problem, we present a combined graph convolution and transformer for point cloud upsampling, denoted by PU-EdgeFormer. The proposed method constructs EdgeFormer unit that consists of graph convolution and multi-head self-attention modules. We employ graph convolution using Edge-Conv, which learns the local geometry and global structure of point cloud better than existing point-to-feature method. Through in-depth experiments, we confirmed that the proposed method has better point cloud upsampling performance than the existing state-of-the-art method in both subjective and objective aspects. The code is available at https://github.com/dohoon2045/PU-EdgeFormer.

Visual camouflage is an effective means of protecting valuable assets that are vulnerable to theft, espionage, or other forms of malicious activity. To overcome the limitation of standardized camouflage patterns in certain environments, we need an innovative approach that adapts the camouflage pattern to the specific surroundings of the asset to be concealed. In this paper, we present a novel camouflage image generation method whose results change by the circumstances around the asset to be concealed using the style transfer. In order to remove the influence of stuff and objects that are not advantageous for camouflage in the surrounding, we propose a novel mechanism that introduces the contents-aware information into the calculation of style representation. Experimental results in various situations, including the snowy natural scene, show that the proposed method provides excellent adaptive camouflage outcomes while effectively suppressing conspicuous elements in the surrounding.

The image obtained using an image sensor with limited dynamic range cannot perfectly represent the various lighting conditions of the real world. Various HDR methods have been studied for expanding the dynamic range in a single image. However, it is difficult to avoid ghosting artifacts caused by the movement of the subject over time and the corresponding texture loss. To solve these problems, we present a novel HDR image acquisition method via dynamic range transformer (DrT) that learns enhanced log-perceptual information using Swin-Fourier convolutional neural network as a backbone. When training the DrT with Swin-Fourier network, it estimates the attention map to obtain an HDR image by minimizing the enhanced log-perceptual (ELP) loss. The Swin-Fourier network considers both local and global contexts simultaneously, which reduces ghosting and texture loss. By learning ELP, it also minimizes color distortion and restores fine details of the dynamic range. Experimental results demonstrate that the HDR results obtained using DrT show reduced color distortion, significantly decreased ghosting artifacts, and texture loss compared to conventional methods. We provide implementation code of our proposed methods in https://github.com/HeunSeungLim/DrT

This paper presents a convolutional neural network to automatically remove the haze distribution using a single multispectral remote sensing image in the raw file format. To train the proposed dehazing network, we synthesized multispectral hazy images using the haze thickness map (HTM) and relative scattering model representing the wavelength-dependent scattering property of the haze distribution. Since the raw multispectral hazy images have a low dynamic range, we cannot accurately estimate the haze distribution directly from them. To differently impose a proper amount of attention to hazy and haze-free regions, we used the HTM from the contrast-enhanced version of the input hazy image. The proposed dehazing network consists of four sub-networks: (i) shallow feature extraction network (SFEN), (ii) cascaded residual dense block network (CRDBN), (iii) multiscale feature extraction network (MFEN), and (iv) refinement network (RN). The densely connected convolutional layers and local residual learning allow the residual dense block (RDB) to extract the abundant local features, and the cascaded architecture further improves the propagation of the local information and gradients. The MFEN is used to extract multiscale local features representing the hierarchical information for the haze distribution and haze-free region. Experimental results demonstrated that the proposed method can achieve improved dehazing performance on Korea Multi-Purpose Satellite-3A (KOMPSAT-3A) multispectral remote sensing imagery without undesired artifacts. In the sense of quantitative assessment, the proposed method produced improved peak signal-to-noise ratio (PSNR) by 10%, structural similarity index measure (SSIM) by 1%, and spectral angle mapper (SAM) by 19% compared with the existing best method.

Early works for 3D point cloud completion primarily focused on revising deep learning networks, and showed an acceptable performance. However, the performance those methods is limited because of lack of geometric studies. To solve that problem, we used multiple loss functions that allow the deep learning networks to fully understand the geometric perspective of the object in both two-dimensional (2D) and three-dimensional (3D) spaces. As a result, the angle between the prediction and ground truth vectors is reduced for accurate local details in the 3D space, and at the same time, the difference in distance on a specific plane is reduced for 2D local details. For that reason, the proposed point cloud completion method can lead to clarify semantic information and structure of 3D objects. In addition, the proposed mathematical approach can give better understanding of object, which can improve the performance of 3D point cloud completion.

Most physical model-based dehazing methods are subject to contrast degradation in a dark or shadow region because of the mismatch between the physical model and real haze. This degradation decreases the quality of dehazed images. Furthermore, the retinex-haze combined models can cause the brightness saturation problem in a haze region. For this reason, the retinex-haze combined approaches are not appropriate to enhance the real-world haze images. To solve these problems, we present a novel region-based dehazing method via dual-supervised triple-convolutional network (TCN). More specifically, the proposed network first simulates the mismatch problem based on the region-model. Next, we then train the proposed triple-convolutional network, which can estimate the degraded regions. We then present a novel dual-supervised learning method to efficiently train the networks using a non-ideal dataset. Experimental results show that the proposed method outperforms state-of-the-art approaches in solving complex haze problems. The output of the proposed network has a high-similarity index in most cases for various benchmark dataset. Our approach also produces high-quality images in real haze image datasets.

Advanced driver-assistance systems need a camera calibration algorithm for various vision applications including surround-view monitoring (SVM) and lane departure warning (LDW). Although cameras mounted on a vehicle are calibrated in the manufacturing process, their orientation angles are subject to tilting because of continuing vibration and external impact. To solve the problem, this paper presents an online calibration algorithm for camera orientation estimation using motion vectors and three-dimensional geometry. The proposed algorithm consists of three steps: i) driving direction estimation by calculating an intersection of motion vectors, ii) camera orientation estimation based on 3-line random sample consensus (RANSAC) using the estimated intersection, and iii) final orientation decision using extended Kalman filter from the result of each frame. Experimental results demonstrate that the proposed algorithm stably estimates camera orientation angles from motion vectors and lines under the parallelism and orthogonality assumptions.

Most deep learning-based salient object detection (SOD) methods tried to manipulate the convolution block to effectively capture the context of object. In this paper, we propose a novel method, called progressive and mutl-task learning scheme, to extract the context of object by only manipulating the learning scheme without changing the network architecture. The progressive learning scheme is a method to grow the decoder progressively in the train phase. In other words, starting from easier low-resolution layers, it gradually adds high-resolution layers. Although the progressive learning successfully captures the context of object, its output boundary tends to be rough. To solve this problem, we also propose a multi-task learning (MTL) scheme that processes the object saliency map and contour in a single network jointly. The proposed MTL scheme trains the network in an edge-preserved direction through an auxiliary branch that learns contours. The proposed a learning scheme can be combined with other convolution block manipulation methods. Extensive experiments on five datasets show that the proposed method performs best compared with state-of-the-art methods in most cases.

Image filtering is a fundamental preprocessing step for accurate, robust computer vision applications such as image segmentation, object classification, and reconstruction. However, many convolutional neural network (CNN)‐based methods tend to lose significant edge information in the output layer, and generate undesired artefacts in the feature extraction layers. This study presents a deep CNN model for edge‐aware image filtering. The proposed network model consists of three sub‐networks: (i) feature extraction, (ii) convolution artefact removal, and (iii) structure extraction networks. The proposed network model has an end‐to‐end trainable architecture that does not need any post‐processing steps. Especially, the structure extraction network can successfully preserve significant edges. The proposed filter outperforms state‐of‐the‐art denoising filters in terms of both objective and subjective measures, and can be used for various image enhancement and restoration problems such as edge‐preserving smoothing, image denoising, deblurring, and deblocking.

Outdoor images are subject to degradation regarding contrast and color because atmospheric particles scatter incoming light to a camera. Existing haze models that employ model-based dehazing methods cannot avoid the dehazing artifacts. These artifacts include color distortion and overenhancement around object boundaries because of the incorrect transmission estimation from a depth error in the skyline and the wrong haze information, especially in bright objects. To overcome this problem, we present a novel optimization-based dehazing algorithm that combines radiance and reflectance components with an additional refinement using a structure-guided ℓ0-norm filter. More specifically, we first estimate a weak reflectance map and optimize the transmission map based on the estimated reflectance map. Next, we estimate the structure-guided ℓ0transmission map to remove the dehazing artifacts. The experimental results show that the proposed method outperforms state-of-the-art algorithms in terms of qualitative and quantitative measures compared with simulated image pairs. In addition, the real-world enhancement results demonstrate that the proposed method can provide a high-quality image without undesired artifacts. Furthermore, the guided ℓ0-norm filter can remove textures while preserving edges for general image enhancement algorithms.

Even though numerous visual tracking methods have been proposed to deal with image streams, it is a still challenging problem to facilitate a tracking method to accurately distinguish the target from the background without drifting under the severe appearance variation of target caused by distortion of local structures. For preserving local structures of target template datasets, we present a novel structure-preserving sparse learning algorithm by obtaining sparse coefficients under maximum margin projection-based subspace representation and by updating the sparse codes under multiple task feature selection framework. To reinforce local structures of targets, we adopted a novel optimization process using an accelerated proximal gradient shrinkage operation and an efficient stopping criterion. Experimental results demonstrate that the proposed method outperforms existing state-of-the-art tracking methods.

Recently, smart vehicle technology gains increasing attractions for the driving security. In this context a front-view camera plays an important role in avoiding collision between vehicles. This paper presents a distance measurement system for driver safety. The proposed method consists of three steps: (i) estimation of distortion function, (ii) tracking features in the license plate region, and (iii) distance estimation by geometry analysis. The major contribution of this work is an accurate estimation of distance using a single, wide field-of-view lens camera.

This paper presents a novel super-resolution (SR) algorithm using local self-examples. The proposed algorithm consists of three steps: i) generation of the patch dictionary using multiple-step image blurring, ii) search of the optimum patches using the magnitude and orientation of the image gradient, and iii) combination of the restored and original patches for reducing the patch-mismatching error. Example-based SR methods have a common disadvantage of unnaturally reconstructed edges. The proposed method can reconstruct realistic images by searching patches based on the edge strength in dictionary made by multiple-step degradations. Experimental results show that the proposed SR algorithm provides more natural images with less synthetic artifacts than existing methods. The proposed SR method provides significant improvement in both subjective and objective measures including peak-to-peak signal-to-noise ratio (PSNR) and structural similarity measure (SSIM).

In this paper, we present a novel contrast enhancement method for backlit images that consists of three steps: i) computation of the transmission coefficients using the dark channel prior, ii) generation of multiple images having different exposures based on the transmission coefficients, and iii) image fusion. Compared to global intensity transformation methods and spatially invariant contrast enhancement algorithms, our approach first extracts under-exposed regions using the dark channel prior map, and then performs spatially adaptive contrast enhancement. As a result, the contrast of the image is increased, especially for backlit scenes and those with very wide dynamic range, while still preserving image details and color.

In this paper, we present a novel method for simultaneous detection and tracking of multiple objects using dual-layer particle filtering. For detecting and tracking multiple moving objects, the proposed dual-layer particle filter (DLPF) consists of parent-particles (PPs) in the first layer for detecting multiple objects and child-particles (CPs) in the second layer for tracking objects that are detected in the first layer. In the first layer, PPs detect persons using a classifier prior trained by the intersection kernel support vector machine (IKSVM) at each particle under a randomly selected scale. If a certain PP detects a person, it generates CPs, and makes an object model in the detected object region for tracking the detected object. While PPs that have detected objects generate CPs for tracking, the rest of PPs still move for detecting objects. Experimental results show that the proposed method can automatically detect and track multiple objects and efficiently reduce the computational time using the sampled particles based on motion distribution in video sequences.

In this paper, we present a compressive sensing-based image denoising algorithm using spatially adaptive image representation and estimation of optimal error tolerance based on sparse signal analysis. The proposed method performs block-based multiple compressive sampling after decomposing the sparse signal into feature and non-feature regions using simple statistical analysis. For minimization of recovery error and number of iterations, the modified OMP method estimates the optimal error tolerance using the average variance in the recovery step. Experimental results demonstrate that the proposed denoising algorithm better removes noise without undesired artifacts than existing state-of-the-art methods in terms of both objective (PSNR/SSIM) and subjective measures. Processing time of the proposed method is 5 to 10 times faster than the standard OMP-based method.

In this paper, we present a dual off-axis color filtered aperture (DCA)-based computational imaging system for depth estimation in a single-camera framework. The DCA has one primary (red) color and its complementary (cyan) color filtered apertures located off the optical axis to generate misalignment between color channels depending on the distance of a region-of-interest from the camera. Disparity of color shifting values (CSVs) between color channels in the image acquired by the DCA-based imaging system is estimated using L1-norm minimization of energy functional considering both brightness and gradient constancies and only translation of x-axis. The two data terms both red to green (R-G) and red to blue (R-B) are combined in a single data term. The proposed imaging system can be implemented by simply inserting an appropriately resized DCA into any general optical system. Experimental results show that the proposed DCA can estimate the distance of the scene from the camera in the single-camera framework.

In this paper, we present an image enhancement algorithm based on the wavelength-adaptive image formation model and geometric classification for defogging UAV images. We first generate a labeled image using geometric class-based segmentation. We then generate a modified transmission map based on the wavelength-adaptive image formation model with scattering coefficients in the labeled image. We also estimate the atmospheric light from the modified transmission map instead of simply choosing the brightest pixel. The proposed method can significantly enhance the visibility of foggy UAV images compared with existing monochrome model-based defogging method. The proposed algorithm can enhance the visibility by removing atmospheric degradation factor in airborne images acquired by aerial platforms such as satellite, airplane, and UAV under critical weather conditions such as haze, fog, and smoke.

We present an antialiasing method using combined wavelet-Fourier transform and spatially adaptive shrinkage of the transform coefficients. Traditional antialiasing methods employ a simple low-pass filter onto the entire image, so the resulting image loses not only aliasing artifacts but also high-frequency components such as edges and ridges. The proposed algorithm analyzes the property of the LL subband of the discrete wavelet transform (DWT), and reduces aliasing artifacts using patch-adaptive shrinkage of the DWT coefficients. More specifically, an antialiased LL subband is obtained using adaptive patch-based aliasing reduction. To detect an aliased region, we subtract the discrete Fourier transform (DFT) coefficients of the LL subband from the DFT coefficients of antialiased LL subband. The detected aliasing artifacts in the LH, HL, and HH subbands are reduced by patch-wise adaptive shrinkage of the transform coefficients. The resulting antialiased image is obtained using the inverse DWT. The aliasing artifacts can be efficiently reduced by adaptively shrinking wavelet transform coefficients for preserving high-frequency image details. The proposed antialiasing algorithm is suitable for removing aliasing artifacts which frequently occur in imaging sensors with limited resolution.

This paper presents a novel real-time super-resolution (SR) method using directionally adaptive image interpolation and image restoration. The proposed interpolation method estimates the edge orientation using steerable filters and performs edge refinement along the estimated edge orientation. Bi-linear and bi-cubic interpolation filters are then selectively used according to the estimated edge orientation for reducing jagging artifacts in slanting edge regions. The proposed restoration method can effectively remove image degradation caused by interpolation using the directionally adaptive truncated constrained least-squares (TCLS) filter. The proposed method provides high-quality magnified images which are similar to or better than the result of advanced interpolation or SR methods without high computational load. Experimental results indicate that the proposed system gives higher peak-to-peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) values than the state-of-the-art image interpolation methods.

This study presents a spatially varying image registration method based on regions of the same depth. The proposed registration method uses phase correlation matching to measure colour shifting vectors (CSVs) between colour channels in a pre‐specified region of the same distance to the camera, and aligns colour channels of the corresponding region according to the CSV. The authors also present the foreground region detection method by using binary edge labelling and analysis of histograms of channel‐shifting features. The major contribution of this study is 2‐fold: (i) the proposed method can be considered as a region‐wise approximated version of fully non‐rigid registration, which is widely used in the medical imaging area, and (ii) it can compensate misalignment between red (R), green (G) and blue (B) colour channels caused by refraction and chromatic aberration of a multiple colour‐filtered aperture (MCA) camera, which has been proposed as a single camera‐based multifocusing system. Among various applications of non‐rigid image registration, the proposed region‐based registration method is particularly suitable for multifocusing images acquired by an MCA camera. In depth analysis of each step of the proposed algorithm is provided with experimental results, and its application to the MCA camera is also provided to realise efficient depth estimation and highly accurate multifocusing functions using a single camera. Without using joint histogram or geometric transformation, the proposed region‐adaptive approach successfully approximates the fully non‐rigid registration with significantly reduced amount of computation.

This letter presents a novel contrast enhancement approach based on dominant brightness level analysis and adaptive intensity transformation for remote sensing images. The proposed algorithm computes brightness-adaptive intensity transfer functions using the low-frequency luminance component in the wavelet domain and transforms intensity values according to the transfer function. More specifically, we first perform discrete wavelet transform (DWT) on the input images and then decompose the LL subband into low-, middle-, and high-intensity layers using the log-average luminance. Intensity transfer functions are adaptively estimated by using the knee transfer function and the gamma adjustment function based on the dominant brightness level of each layer. After the intensity transformation, the resulting enhanced image is obtained by using the inverse DWT. Although various histogram equalization approaches have been proposed in the literature, they tend to degrade the overall image quality by exhibiting saturation artifacts in both low- and high-intensity regions. The proposed algorithm overcomes this problem using the adaptive intensity transfer function. The experimental results show that the proposed algorithm enhances the overall contrast and visibility of local details better than existing techniques. The proposed method can effectively enhance any low-contrast images acquired by a satellite camera and is also suitable for other various imaging devices such as consumer digital cameras, photorealistic 3-D reconstruction systems, and computational cameras.

A multiple color-filter aperture (MCA) camera can provide depth information as well as color and intensity in the single-camera framework, where the MCA generates misalignment between color channels depending on the distance of a region-of-interest. In this paper, we present a single camera-based estimation of the full depth map using the color shifting property of the MCA. For estimating the color shifting vectors (CSVs) among red, green, and blue color channels, edges are extracted at each color channel. At the edge, we estimate CSVs using normalized cross correlation combined with color shifting mask map. A full depth map is then generated by depth interpolation using the matting Laplacian method from sparsely estimated CSVs at an edge location. Experimental results show that the proposed method can not only estimate the full depth map but also correct the misaligned color image to generate photorealistic color images using a single camera equipped with MCA.

This paper presents a spatially adaptive super-resolution (SR) algorithm using homogeneous region analysis for minimizing undesired interpolation artifacts such as aliasing and jagged edges. The proposed regularized SR algorithm incorporates two constraints enforcing (i) disconnected edges to be reconnected and (ii) orientation-adaptive regularization. By combining two constraints in the regularization framework, the proposed SR algorithm can significantly reduce aliasing artifacts and, as a result, produce edge-preserved high-resolution (HR) images. In addition to the formulation of the regularized SR algorithm with hybrid constraints, experimental results show that the proposed SR algorithm improves peak-to-peak signal-to-noise ratio (PSNR) of the reconstructed HR images by up to 5[dB]over existing state-of-the-art SR methods.

This paper presents a novel approach to depth estimation using a multiple color-filter aperture (MCA) camera and its application to multifocusing. An image acquired by the MCA camera contains spatially varying misalignment among RGB color channels, where the direction and length of the misalignment is a function of the distance of an object from the plane of focus. Therefore, if the misalignment is estimated from the MCA output image, multifocusing and depth estimation become possible using a set of image processing algorithms. We first segment the image into multiple clusters having approximately uniform misalignment using a color-based region classification method, and then find a rectangular region that encloses each cluster. For each of the rectangular regions in the RGB color channels, color shifting vectors are estimated using a phase correlation method. After the set of three clusters are aligned in the opposite direction of the estimated color shifting vectors, the aligned clusters are fused to produce an approximately in-focus image. Because of the finite size of the color-filter apertures, the fused image still contains a certain amount of spatially varying out-of-focus blur, which is removed by using a truncated constrained least-squares filter followed by a spatially adaptive artifacts removing filter. Experimental results show that the MCA-based multifocusing method significantly enhances the visual quality of an image containing multiple objects of different distances, and can be fully or partially incorporated into multifocusing or extended depth of field systems. The MCA camera also realizes single camera-based depth estimation, where the displacement between multiple apertures plays a role of the baseline of a stereo vision system. Experimental results show that the estimated depth is accurate enough to perform a variety of vision-based tasks, such as image understanding, description, and robot vision.

A multiple color-filter aperture (MCA) camera can provide depth information as well as color and intensity in the single-camera framework, where the MCA generates misalignment between color channels depending on the distance of a region-of-interest. In this paper, we present a simultaneous object tracking and depth estimation approach based on the color shifting property of an MCA camera. An object region is first extracted by using Markov Chain Monte Carlo (MCMC) sampling-based particle filter. The extracted object's region has color misalignment among RGB planes depending on the distance of the object. We estimate color shifting vectors (CSVs) between green-and-red (G-R) and green-and-blue (G-B) channels using a simplified elastic registration algorithm. Using the color shifting property of the MCA camera, the depth of the object's region is estimated from CSVs. From experimental results, we can show the MCA camera-based surveillance system can estimate depth information as well as track object.

In this paper, we propose a ghost artifacts removing algorithm for obtaining artifact-free high dynamic range (HDR) images in the presence of camera movement. Existing HDR methods work on condition that there is no camera movement when acquiring multiple low dynamic range (LDR) images. For overcoming such unrealistic restriction, the proposed algorithm first specifies the target and the source images in the set of acquired LDR images, and then the source image is transformed to fit the target image by estimating translation and rotation components in the affine matrix. Therefore, the proposed algorithm can reconstruct the HDR image without ghost artifacts because camera movement is completely removed in transformed images. Experimental results show that the proposed algorithm successfully removes ghost artifacts. For this reason, the proposed algorithm can extend application areas of HDR imaging to various mobile imaging devices, such as a handheld camcorder and a mobile phone camera.

This paper introduces a new fast motion estimation based on a maximum matching error constraint that can eliminate an impossible candidate block much earlier than a conventional partial distortion elimination (PDE) scheme. The main contributions of the proposed scheme are that 1) it can reduce the computational cost considerably for the matching error calculation; and 2) the proposed matching ideas can be applied to the conventional PDE algorithms without significant changes. In order to evaluate the proposed scheme, we compare the results of the proposed algorithm with several baseline approaches. The experimental results show that the proposed algorithm can reduce the computations more than 62% for motion estimation at the cost of 0.005 dB quality degradation versus the general PDE algorithm.

This paper presents a new robust digital image stabilization system, which involves a practical motion model based on a feature-histogram. The feature-histogram is used for adaptive selection of feasible motion estimation regions. By estimating the representative motion vector in the optimally selected region, the proposed algorithm can robustly remove undesired camera vibration regardless of object's movement. When compared with the traditional methods, experimental results show that the proposed algorithm can improve the performance by 7% with four times faster computation compared with the existing sum-absolute-difference (SAD) based method.

Most of the focusing techniques need to estimate depth information for ensuring that the object of interest is at an appropriate distance for full frontal focus. Computational cameras which can variably focus different regions of the scene with large depth of field have been proposed. In this paper we propose a full auto-focusing algorithm using computational camera without involving any digital image restoration methods and just one input. The proposed computational camera uses multiple filter apertures corresponding to each color channel which can acquire three shifted views of a scene in the RGB color planes. We can make any region focused by appropriately shifting each color channel to be aligned. Depth map estimation is carried out to extract different regions from these channel shifted images which is later fused to produce the final image without any focal blur. Experimental results show performance and feasibility of the proposed algorithm for auto-focusing images with one or more differently out-of-focused objects.

This letter presents an efficient video coding algorithm in the RGB color space. Currently, most video coding algorithms are based on a decorrelated color space such as YUV or YCbCr, which has reduced inter-color redundancy. For high-fidelity video applications, however, it is essential to maintain the original signal fidelity without any mismatch in color space conversion. In this context, the H.264/AVC High 4:4:4 Intra/Predictive profiles support the RGB color space to satisfy increasing demand for high-fidelity video coding. However, very little effort has been made to utilize inter-color correlation to increase RGB coding efficiency. In pursuit of both high-fidelity and coding efficiency, we propose an adaptive inter-plane-weighted prediction algorithm exploiting inter-color redundancy of the RGB signal. We integrate the proposed algorithm into H.264/AVC High 4:4:4 Intra/Predictive profile reference software for simulation, and show that the proposed algorithm increases RGB video coding efficiency at average 0.80 dB and 0.75 dB, compared with the existing H.264/AVC High 4:4:4 Intra profile RGB and YCbCr 4:4:4 coding, respectively.

Fusion-based image restoration is an effective way to remove multiple out-of-focus blurs in images. Although image restoration and image fusion have been successfully investigated and developed over the years, little effort has been made to combine them. In this paper, we present an integration method of the two approaches and make them benefit from each other to obtain significantly improved performance. Based on the proposed fusion approach, we present a novel digital auto-focusing algorithm, which restores an image with multiple, differently out-of-focused objects. To this end, an out-of-focused image is first restored by using a directionally regularized iterative restoration with multiple regularization parameters. By assembling multiple, restored regions from consecutive levels of iterations, a salient focus measure is formed as a new query using sum modified Laplacian (SML). An auto-focusing error metric (AFEM) is used as an appropriate termination criterion for iterative restoration. A novel soft decision fusion and blending (SDFB) algorithm combines images from restored by different point-spread functions (PSFs) and enables smooth transition across region boundaries for creating the finally restored image using a pseudo activity measure. Experimental results show that the proposed auto-focusing algorithm provides sufficiently high-quality restored images so that it can be used for devices such as a digital camera and a camcorder.

In this paper, we propose a personal verification method using both face and speech to improve the rate of single biometric verification. False acceptance rate (FAR) and false rejection rate (FRR) have been a fundamental bottleneck of real-time personal verification. The proposed multimodal biometric method is to improve both verification rate and reliability in real-time by overcoming technical limitations of single biometric verification methods. The proposed method uses principal component analysis (PCA) for face recognition and hidden markov model (HMM) for speech recognition. It also uses fuzzy logic for the final decision of personal verification. Based on experimental results, the proposed system can reduce FAR down to 0.0001%, which provides that the proposed method overcomes the limitation of single biometric system and provides stable personal verification in real-time.

Multiple camera based surveillance systems provide us with a more robust tracking of objects. To take advantage of additional cameras, it is necessary to establish geometrical relationship between the cameras and relationship between an object and a camera. In recent years several techniques have been proposed, which estimate only the relation of a dominant ground plane between different views instead of fully geometrical relation of camera-object-camera. They are however neither fully automatic nor suitable for a non-planar ground. We propose a fully automatic calibration algorithm which can cope with complex environment, including non-planar ground. The proposed algorithm automatically tracks and matches objects between different views, determines the overlapped region, and aligns each piece-wisely segmented plane between two views. The proposed calibration algorithm minimizes the effect of occlusion and improves the accuracy of 3D measurements by using multiple views. The algorithm can also provide us large field of view by concatenating a series of cameras.

This paper proposes a fully digital auto-focusing algorithm for restoring the image with differently out-of-focused objects, which can restore background as well as all objects. In this paper, we assume that out-of-focus blur is isotropic such as circle of confusion (COC) or two-dimensional Gaussian blur. Therefore, the proposed algorithm can segment and estimate the point spread function (PSF) by using the size of ramp in the one-dimensional step response. The proposed algorithm can be developed by object-based image segmentation and restoration algorithm. Experimental results show that the proposed object-based image restoration algorithm can efficiently remove the space-variant out of focus blur from the image with multiple blurred objects.

We present a combined real-time face region tracking and highly accurate face recognition technique for an intelligent surveillance system. High-resolution face images are very important to achieving accurate identification of a human face. Conventional surveillance or security systems, however, usually provide poor image quality because they use only fixed cameras to record scenes passively. We have implemented a real-time surveillance system that tracks a moving face using four pan-tilt-zoom (PTZ) cameras. While tracking, the region-of-interest (ROI) can be obtained by using a low-pass filter and background subtraction with the PTZ. Color information in the ROI is updated to extract features for optimal tracking and zooming. FaceIt/sup /spl reg//, which is one of the most popular face recognition software packages, is evaluated and then used to recognize the faces from the video signal. Experimentation with real human faces showed highly acceptable results in the sense of both accuracy and computational efficiency.

This paper proposes a new, efficient surface representation method for surface matching. A feature carrier for a surface point, which is a set of two-dimensional (2-D) contours that are the projections of geodesic circles on the tangent plane, is generated. The carrier is named point fingerprint because its pattern is similar to human fingerprints and plays a role in discriminating surface points. Corresponding points on surfaces from different views are found by comparing their fingerprints. The point fingerprint is able to carry curvature, color, and other information which can improve matching accuracy, and the matching process is faster than 2-D image comparison. A novel candidate point selection method based on the fingerprint irregularity is introduced. Point fingerprint is successfully applied to pose estimation of real range data.

Tracking and recognizing non-rigid objects in video image sequences are complex tasks of increasing importance to many applications. In this paper, we present a hierarchical realization of an enhanced active shape model for color video tracking and we study the performance of hierarchical and non-hierarchical implementations in the RGB, YUV, and HSI color spaces.

We propose a novel algorithm to smooth and simplify simultaneously range images and also triangle meshes derived from those images. These data sets often suffer from noise and over-sampling. To overcome these issues, smoothing from image processing and simplification from computer graphics attempt to minimize noise and reduce complexity, respectively. Typically, these algorithms are separate and distinct steps, but we combine them into one algorithm. We employ surface normal voting to generate robust orientation estimates and then extend the quadric error metric framework to smooth noise while simplifying the surface. We demonstrate the capabilities of this algorithm with both synthetic and real data. The proposed algorithm provides significant noise smoothing improvement when compared to the standard Garland and Heckbert (1998) quadric simplification algorithm.

Triangle meshes are widely used in representing surfaces in computer vision and computer graphics. Although 2D image processing-based edge detection techniques have been popular in many application areas, they are not well developed for surfaces represented by triangle meshes. This paper proposes a robust edge detection algorithm for triangle meshes and its applications to surface segmentation and adaptive surface smoothing. The proposed edge detection technique is based on eigen analysis of the surface normal vector field in a geodesic window. To compute the edge strength of a certain vertex, the neighboring vertices in a specified geodesic distance are involved. Edge information are used further to segment the surfaces with the watershed algorithm and to achieve edge-preserved, adaptive surface smoothing. The proposed algorithm is novel in robustly detecting edges on triangle meshes against noise. The 3D watershed algorithm is an extension from previous work. Experimental results on surfaces reconstructed from multi-view real range images are presented.

This paper presents a simple and efficient algorithm for part decomposition of compound objects based on Gaussian curvature analysis. The proposed algorithm consists of three major steps, Gaussian curvature estimation, boundary detection, and region growing. Boundaries between two articulated parts are composed of points with highly negative curvature based on the transversality regularity. These boundaries are therefore detected by thresholding estimated Gaussian curvatures for each vertex. A component labeling operation is then performed to grow non-boundary vertices into parts. The original contributions of this paper include: (i) novel, curvature analysis-based decomposition of 3-D models represented by triangle meshes into functional parts instead of surfaces and (ii) large mesh (over 100,000 triangles) handling capability with low computational cost and easy implementation. Experiments were conducted on a large number of both synthetic and real 3-D models. Experimental results demonstrated the performance and efficiency of the proposed algorithm.

In this paper, we describe a robust method for the estimation of curvature on a triangle mesh, where this mesh is a discrete approximation of a piecewise smooth surface. The proposed method avoids the computationally expensive process of surface fitting and instead employs normal voting to achieve robust results. This method detects crease discontinuities on the surface to improve estimates near those creases. Using a voting scheme, the algorithm estimates both principal curvatures and principal directions for smooth patches. The entire process requires one user parameter-the voting neighborhood size, which is a function of sampling density, feature size, and measurement noise. We present results for both synthetic and real data and compare these results to an existing algorithm developed by Taubin (1995).

Various image restoration methods have been studied for removing space-variant motion blur such as iterative and POCS (projection on to convex sets) method. However, the computational complexity of the methods, such as regularized iteration and POCS method, is so high that they can hardly be implemented in real-time. We address a method to reduce the computational complexity by selecting the region to be restored. The primary application area of the proposed method is a surveillance system which requires accurate object extraction, identification and tracking functions. To remove motion blur, we propose a new spatially adaptive regularized iterative image restoration algorithm. Experimental results show the the proposed algorithm can efficiently remove space-variant motion blur with significantly reduced computational overhead.

The wavelet-compressed images suffer from coding artifacts, such as ringing and blurring, due to quantizing the transform coefficients. To reduce such coding artifacts, we propose a new image enhancement algorithm based on regularized iterative image restoration. The proposed algorithm uses multiple constraints to simultaneously suppress the coding artifacts and preserve the original high-frequency components. We also present the hierarchical implementation for the proposed algorithm. Spatial adaptivity plays a role in keeping the edge from spreading out to other pixels which have different directional characteristics.

We propose an adaptive regularization algorithm for smoothing dense range images using a novel, first order stabilizing function. The stabilizer we suggest is based upon minimizing the reconstructed surface area and is derived in the native, spherical coordinate system of the range scanner. This allows adjustments to be made along only the direction of measurement, thereby preventing the data overlapping problem that can arise in dense images. Adaptation is achieved by adjusting the regularization parameter according to the results of 2D edge analysis. Results indicate effective noise suppression along with well preserved edges and details in the reconstructed, 3D surfaces.

In this peeper, we propose a degradation model for blocking artifacts in inter-frame coded images and corresponding iterative image restoration algorithm for reducing blocking artifacts. In inter-frame coded images, most restoration process is performed in differential image between the previous motion compensated image and the current image. In order to efficiently reduce blocking artifacts, edge direction of each block is classified by using the DCT coefficients and spatially adaptive regularized iterative algorithm is used to remove blocking artifacts. We used a constraint which assumes that in differential image there must be discontinuity in opposite direction to the motion compensated image. The proposed algorithm is suitable for postprocessing reconstructed image sequences in HDTV, DVD, or video conferencing systems.

When a digital image acquisition system captures a scene, image degradation due to motion blur is unavoidable. Motion blur is caused by an imperfect imaging geometry, and it makes the obtained image lose important information. In this work we propose a new degradation model for boundary region between multiple moving objects. Based on the proposed model, we also propose a segmentation-based spatially adaptive image restoration algorithm. Accordingly, we present a proper segmentation algorithm and a motion estimation method for moving object.

This paper presents a novel fast superresolution algorithm using motion adaptive relaxation parameters. By adopting the fast iterative algorithm based on pre-conditioner, the proposed algorithm can rapidly restore high resolution details in the original high resolution image. Moreover, the proposed algorithm provides a better interpolated images by using the motion adaptive relaxation parameter which controls the convergence rate. Finally, it can be applied to the general image sequence with differently moving objects. As a result, we can obtain high resolution frames from image sequences in almost real-time.

We propose an image interpolation algorithm based on the regularized iterative image restoration method. By adopting spatial adaptive constraints in each regularized iteration step, the proposed algorithm provides a better interpolated image from the image sequence than the conventional non-adaptive interpolation algorithms. Due to its spatially adaptive nature, it can be applied to the general image sequence with differently moving objects. Real-time implementation of the proposed algorithm is also considered by updating the motion compensated image frames.

A modified Hopfield neural network model for regularized image restoration is presented. The proposed network allows negative autoconnections for each neuron. A set of algorithms using the proposed neural network model is presented, with various updating modes: sequential updates; n-simultaneous updates; and partially asynchronous updates. The sequential algorithm is shown to converge to a local minimum of the energy function after a finite number of iterations. Since an algorithm which updates all n neurons simultaneously is not guaranteed to converge, a modified algorithm is presented, which is called a greedy algorithm. Although the greedy algorithm is not guaranteed to converge to a local minimum, the l (1) norm of the residual at a fixed point is bounded. A partially asynchronous algorithm is presented, which allows a neuron to have a bounded time delay to communicate with other neurons. Such an algorithm can eliminate the synchronization overhead of synchronous algorithms.

A modified Hopfield network model for image restoration is presented. The proposed neural network does not require zero autoconnections, which is one of the major drawbacks of the Hopfield network. A new number-representation scheme for implementing the proposed network is given. The proposed network with sequential update is shown to converge. The sufficient conditions for convergence of n-simultaneous updates are also given. When the image-restoration problem does not satisfy the convergence conditions, a greedy algorithm which guarantees convergence (at the expense of the image quality) is used.>

An edge detection algorithm using multistate ADALINES (adaptive linear neurons) is presented. The proposed algorithm can suppress noise effects without increasing the mask size. The input states are defined using the local mean in a predefined mask, and the one-dimensional edges are defined so that they are linearly separable from nonedges. The two-dimensional edges are obtained using the rotation invariant property of layered neural networks. The proposed algorithm requires much less computation compared with Marr and Hildreth's (1980) edge detector for similar performance. An application of the proposed edge detector to adaptive image restoration is also presented.>

Iterative techniques for restoring colour images are presented. These iterative algorithms have been developed for the restoration of monochromatic images. The colour images are modeled as three spatially related monochromatic images or channels. The interchannel correlation is incorporated into the iterative restoration algorithms in an indirect and in a direct way. Experimental results with simulated and real photographically blurred images are presented. Based on experiments the authors conclude that the incorporation of the interchannel correlation into the algorithms does not necessarily improve the quality of the restored images over the algorithms that ignore the interchannel correlation, but they result in considerable computation savings.>