Recent advances in image editing leverage latent diffusion models (LDMs) for versatile, text-prompt-driven edits across diverse tasks. Yet, maintaining pixel-level edge structures—crucial for tasks such as photorealistic style transfer or image tone adjustment—remains as a challenge for latent-diffusion-based editing. To overcome this limitation, we propose a novel Structure Preservation Loss (SPL) that leverages local linear models to quantify structural differences between input and edited images. Our training-free approach integrates SPL directly into the diffusion model’s generative process to ensure structural fidelity. This core mechanism is complemented by a post-processing step to mitigate LDM decoding distortions, a masking strategy for precise edit localization, and a color preservation loss to preserve hues in unedited areas. Experiments confirm SPL enhances structural fidelity, delivering state-of-the-art performance in latent-diffusion-based image editing. Our code will be publicly released at https://github.com/gongms00/SPL.

In this paper, we propose DeepDeblurRF, a novel radiance field deblurring approach that can synthesize high-quality novel views from blurred training views with significantly reduced training time. DeepDeblurRF leverages deep neural network (DNN)-based deblurring modules to enjoy their deblurring performance and computational efficiency. To effectively combine DNN-based deblurring and radiance field construction, we propose a novel radiance field (RF)-guided deblurring and an iterative framework that performs RF-guided deblurring and radiance field construction in an alternating manner. Moreover, DeepDeblurRF is compatible with various scene representations, such as voxel grids and 3D Gaussians, expanding its applicability. We also present BlurRF-Synth, the first large-scale synthetic dataset for training radiance field deblurring frameworks. We conduct extensive experiments on both camera motion blur and defocus blur, demonstrating that DeepDeblurRF achieves state-of-the-art novel-view synthesis quality with significantly reduced training time.

We present FloVD, a novel video diffusion model for camera-controllable video generation. FloVD leverages optical flow to represent the motions of the camera and moving objects. This approach offers two key benefits. Since optical flow can be directly estimated from videos, our approach allows for the use of arbitrary training videos without groundtruth camera parameters. Moreover, as background optical flow encodes 3D correlation across different viewpoints, our method enables detailed camera control by leveraging the background motion. To synthesize natural object motion while supporting detailed camera control, our framework adopts a two-stage video synthesis pipeline consisting of optical flow generation and flow-conditioned video synthesis. Extensive experiments demonstrate the superiority of our method over previous approaches in terms of accurate camera control and natural object motion synthesis.

In this paper, we present GyroDeblurNet, a novel single-image deblurring method that utilizes a gyro sensor to resolve the ill-posedness of image deblurring. The gyro sensor provides valuable information about camera motion that can improve deblurring quality. However, exploiting real-world gyro data is challenging due to errors from various sources. To handle these errors, GyroDeblurNet is equipped with two novel neural network blocks: a gyro refinement block and a gyro deblurring block. The gyro refinement block refines the erroneous gyro data using the blur information from the input image. The gyro deblurring block removes blur from the input image using the refined gyro data and further compensates for gyro error by leveraging the blur information from the input image. For training a neural network with erroneous gyro data, we propose a training strategy based on the curriculum learning. We also introduce a novel gyro data embedding scheme to represent real-world intricate camera shakes. Finally, we present both synthetic and real-world datasets for training and evaluating gyro-based single image deblurring. Our experiments demonstrate that our approach achieves state-of-the-art deblurring quality by effectively utilizing erroneous gyro data.

Cell-free massive MIMO (CF-mMIMO) ensures uniform service across large areas by jointly serving user equipments (UEs) with densely distributed access points (APs). Dynamic cooperation clustering (DCC) provides a scalable framework for CF-mMIMO, but extending it to multiple central processing units (CPUs) increases complexity to the order of the total APs, leading to excessive signaling overhead. To address this, we propose multi-CPU DCC (MCC), a scalable approach that reduces large-scale fading coefficient (LSFC) exchanges from the total APs to a single LSFC. Using this shared LSFC, MCC estimates remaining AP–UE LSFCs to form a user-centric AP cluster while limiting signaling. Experimental results show that MCC achieves 99% of the performance of an idealized DCC while reducing UE–AP, AP–CPU, CPU–CPU, and total signaling by 74.06%, 51.72%, 65.56%, and 57.93%, respectively. Thus, MCC enhances scalability in multi-CPU environments while maintaining performance and minimizing signaling overhead.

We present LocoGS, a locality-aware 3D Gaussian Splatting (3DGS) framework that exploits the spatial coherence of 3D Gaussians for compact modeling of volumetric scenes. To this end, we first analyze the local coherence of 3D Gaussian attributes, and propose a novel locality-aware 3D Gaussian representation that effectively encodes locally-coherent Gaussian attributes using a neural field representation with a minimal storage requirement. On top of the novel representation, LocoGS is carefully designed with additional components such as dense initialization, an adaptive spherical harmonics bandwidth scheme and different encoding schemes for different Gaussian attributes to maximize compression performance. Experimental results demonstrate that our approach outperforms the rendering quality of existing compact Gaussian representations for representative real-world 3D datasets while achieving from 54.6$\times$ to 96.6$\times$ compressed storage size and from 2.1$\times$ to 2.4$\times$ rendering speed than 3DGS. Even our approach also demonstrates an averaged 2.4$\times$ higher rendering speed than the state-of-the-art compression method with comparable compression performance.

In this paper, we propose VideoFrom3D, a novel framework for synthesizing high-quality 3D scene videos from coarse geometry, a camera trajectory, and a reference image. Our approach streamlines the 3D graphic design workflow, enabling flexible design exploration and rapid production of deliverables. A straightforward approach to synthesizing a video from coarse geometry might condition a video diffusion model on geometric structure. However, existing video diffusion models struggle to generate high-fidelity results for complex scenes due to the difficulty of jointly modeling visual quality, motion, and temporal consistency. To address this, we propose a generative framework that leverages the complementary strengths of image and video diffusion models. Specifically, our framework consists of a Sparse Anchor-view Generation (SAG) and a Geometry-guided Generative Inbetweening (GGI) module. The SAG module generates high-quality, cross-view consistent anchor views using an image diffusion model, aided by Sparse Appearance-guided Sampling. Building on these anchor views, GGI module faithfully interpolates intermediate frames using a video diffusion model, enhanced by flow-based camera control and structural guidance. Notably, both modules operate without any paired dataset of 3D scene models and natural images, which is extremely difficult to obtain. Comprehensive experiments show that our method produces high-quality, style-consistent scene videos under diverse and challenging scenarios, outperforming simple and extended baselines. Code is available at github.com/KIMGEONUNG/VideoFrom3D.

RAW images are rarely shared mainly due to its exces-sive data size compared to their sRGB counterparts ob-tained by camera ISPs. Learning the forward and inverse processes of camera ISPs has been recently demonstrated, enabling physically-meaningful RAW-level image processing on input sRGB images. However, existing learning-based ISP methods fail to handle the large variations in the ISP processes with respect to camera parameters such as ISO and exposure time, and have limitations when used for various applications. In this paper, we propose ParamISP, a learning-based method for forward and inverse con-version between sRGB and RAW images, that adopts a novel neural-network module to utilize camera parameters, which is dubbed as ParamNet. Given the camera param-eters provided in the EXIF data, ParamNet converts them into a feature vector to control the ISP networks. Extensive experiments demonstrate that ParamISP achieve superior RAW and sRGB reconstruction results compared to previous methods and it can be effectively used for a variety of applications such as deblurring dataset synthesis, raw deblur-ring, HDR reconstruction, and camera-to-camera transfer.

In this paper, we propose the first generalizable view synthesis approach that specifically targets multi-view stereocamera images. Since recent stereo matching has demonstrated accurate geometry prediction, we introduce stereo matching into novel-view synthesis for high-quality geometry reconstruction. To this end, this paper proposes a novel framework, dubbed StereoNeRF, which integrates stereo matching into a NeRF-based generalizable view synthesis approach. StereoNeRF is equipped with three key components to effectively exploit stereo matching in novel-view synthesis: a stereo feature extractor, a depth-guided plane-sweeping, and a stereo depth loss. Moreover, we propose the StereoNVS dataset, the first multi-view dataset of stereocamera images, encompassing a wide variety of both real and synthetic scenes. Our experimental results demonstrate that StereoNeRF surpasses previous approaches in generalizable view synthesis.

Recent image tone adjustment (or enhancement) approaches have predominantly adopted supervised learning for learning human-centric perceptual assessment. However, these approaches are constrained by intrinsic challenges of supervised learning. Primarily, the requirement for expertly-curated or retouched images escalates the data acquisition expenses. Moreover, their coverage of target styles is confined to stylistic variants inferred from the training data. To surmount the above challenges, we propose an unsupervised learning-based approach for text-based image tone adjustment, CLIPtone, that extends an existing image enhancement method to accommodate natural language descriptions. Specifically, we design a hyper-network to adaptively modulate the pretrained parameters of a back-bone model based on a text description. To assess whether an adjusted image aligns with its text description without a ground-truth image, we utilize CLIP, which is trained on a vast set of language-image pairs and thus encompasses the knowledge of human perception. The major advantages of our approach are threefold: (i) minimal data collection expenses, (ii) support for a range of adjustments, and (iii) the ability to handle novel text descriptions unseen in training. The efficacy of the proposed method is demonstrated through comprehensive experiments including a user study.

Recent image restoration methods can be broadly categorized into two classes: (1) regression methods that recover the rough structure of the original image without synthesizing high-frequency details and (2) generative methods that synthesize perceptually-realistic high-frequency details even though the resulting image deviates from the original structure of the input. While both directions have been extensively studied in isolation, merging their benefits with a single framework has been rarely studied. In this paper, we propose UGPNet, a universal image restoration framework that can effectively achieve the benefits of both approaches by simply adopting a pair of an existing regression model and a generative model. UGPNet first restores the image structure of a degraded input using a regression model and synthesizes a perceptually-realistic image with a generative model on top of the regressed output. UGPNet then combines the regressed output and the synthesized output, resulting in a final result that faithfully reconstructs the structure of the original image in addition to perceptually-realistic textures. Our extensive experiments on deblurring, denoising, and super-resolution demonstrate that UGPNet can successfully exploit both regression and generative methods for high-fidelity image restoration.

This study addresses the challenge of high power collision rates in Grant-Free Non-Orthogonal Random Access (GF-NORA) for ultra-massive machine-type communication (umMTC) in ultra-dense networks (UDN). We analyze the impact of power collision and inter-cell interference, defining the key factors affecting successive interference cancellation (SIC) decoding failure. To tackle power collision problem, we propose a multi-agent reinforcement learning (MARL) framework, QMIX algorithm, with joint optimization of access control and power-level design. We evaluate the performance of the proposed scheme with extensive random access simulations in an umMTC environment. Our approach outperforms state-of-the-art schemes, achieving at most 10% increase in successful SIC decoding rate with lower access delay.

Cell-free massive multiple-input multiple-output (CF-mMIMO) can provide uniformly great service to user equipment (UE) by utilizing a large number of distributed access points (APs) connected to a single central processing unit (CPU). For addressing scalability and feasibility issues inherent in single-CPU CF-mMIMO systems, multiple-CPU CF-mMIMO (MC-CF-mMIMO) has been considered. However, the MC-CF-mMIMO system inevitably encounters performance degradation at the boundary area of the CPUs, referred to as the CPU edge. Cooperation among CPUs can improve the performance in the CPU edge region; however, scalability problems resurface owing to inter-CPU information sharing during the cooperation process. In this study, we propose a scalable CPU cooperation scheme that focuses on power control to address the performance degradation issue in the CPU edge region. Initially, we propose a policy estimation scheme for other CPUs to reduce the overhead of information sharing. Based on these estimated policies, each CPU can independently derive power control policies for the max-min optimization problem without simultaneous information sharing. Simulation results show that the proposed power control scheme achieves a performance improvement of up to 4.3234 dB in terms of CPU edge region performance and minimum spectral efficiency compared to baseline schemes based on the degree of CPU cooperation.

We study human pose estimation in extremely low-light images. This task is challenging due to the difficulty of collecting real low-light images with accurate labels, and severely corrupted inputs that degrade prediction quality significantly. To address the first issue, we develop a ded-icated camera system and build a new dataset of real low-light images with accurate pose labels. Thanks to our camera system, each low-light image in our dataset is coupled with an aligned well-lit image, which enables accurate pose labeling and is used as privileged information during training. We also propose a new model and a new training strategy that fully exploit the privileged information to learn representation insensitive to lighting conditions. Our method demonstrates outstanding performance on real extremely low-light images, and extensive analyses validate that both of our model and dataset contribute to the success.

Cell-free massive MIMO (CF-mMIMO) provides flattened spectral efficiency throughout the network. Recently, multi-CPU CF-mMIMO (MC-CF-mMIMO) has been considered for addressing a scalability problem. However, MC-CF-mMIMO now suffers from a performance degradation problem in the boundary of the CPU overage, referred the CPU-edge. In this study, we analyze and improve the performance of MC-CF-mMIMO, focusing on the CPU-edge. First, we present a simple CPU cooperation scheme to mitigate the performance degradation problem on the CPU-edge. Then, we provide the closed-form expression of the downlink spectral efficiency according to the precoding schemes. Experimental results indicate that performance without CPU cooperation on the CPU-edge is degraded by 28.66% compared with CPU cooperation.

While recent 3D-aware generative models have shown photo-realistic image synthesis with multi-view consistency, the synthesized image quality degrades depending on the camera pose (e.g., a face with a blurry and noisy boundary at a side viewpoint). Such degradation is mainly caused by the difficulty of learning both pose consistency and photo-realism simultaneously from a dataset with heavily imbalanced poses. In this paper, we propose SideGAN, a novel 3D GAN training method to generate photo-realistic images irrespective of the camera pose, especially for faces of side-view angles. To ease the challenging problem of learning photo-realistic and pose-consistent image synthesis, we split the problem into two subproblems, each of which can be solved more easily. Specifically, we formulate the problem as a combination of two simple discrimination problems, one of which learns to discriminate whether a synthesized image looks real or not, and the other learns to discriminate whether a synthesized image agrees with the camera pose. Based on this, we propose a dual-branched discriminator with two discrimination branches. We also propose a pose-matching loss to learn the pose consistency of 3D GANs. In addition, we present a pose sampling strategy to increase learning opportunities for steep angles in a pose-imbalanced dataset. With extensive validation, we demonstrate that our approach enables 3D GANs to generate high-quality geometries and photo-realistic images irrespective of the camera pose.

We present ExBluRF, a novel view synthesis method for extreme motion blurred images based on efficient radiance fields optimization. Our approach consists of two main components: 6-DOF camera trajectory-based motion blur formulation and voxel-based radiance fields. From extremely blurred images, we optimize the sharp radiance fields by jointly estimating the camera trajectories that generate the blurry images. In training, multiple rays along the camera trajectory are accumulated to reconstruct single blurry color, which is equivalent to the physical motion blur operation. We minimize the photo-consistency loss on blurred image space and obtain the sharp radiance fields with camera trajectories that explain the blur of all images. The joint optimization on the blurred image space demands painfully increasing computation and resources proportional to the blur size. Our method solves this problem by replacing the MLP-based framework to low-dimensional 6-DOF camera poses and voxel-based radiance fields. Compared with the existing works, our approach restores much sharper 3D scenes from challenging motion blurred views with the order of 10× less training time and GPU memory consumption.

Modeling the spatial radiance distribution of light rays in a scene has been extensively explored for applications, including view synthesis. Spectrum and polarization, the wave properties of light, are often neglected due to their integration into three RGB spectral bands and their non-perceptibility to human vision. However, these properties are known to encompass substantial material and geometric information about a scene. Here, we propose to model spectro-polarimetric fields, the spatial Stokes-vector distribution of any light ray at an arbitrary wavelength. We present Neural Spectro-polarimetric Fields (NeSpoF), a neural representation that models the physically-valid Stokes vector at given continuous variables of position, direction, and wavelength. NeSpoF manages inherently noisy raw measurements, showcases memory efficiency, and preserves physically vital signals — factors that are crucial for representing the high-dimensional signal of a spectro-polarimetric field. To validate NeSpoF, we introduce the first multi-view hyperspectral-polarimetric image dataset, comprised of both synthetic and real-world scenes. These were captured using our compact hyperspectral-polarimetric imaging system, which has been calibrated for robustness against system imperfections. We demonstrate the capabilities of NeSpoF on diverse scenes.

We introduce POP3D, a novel framework that creates a full 360° -view 3D model from a single image. POP3D resolves two prominent issues that limit the single-view reconstruction. Firstly, POP3D offers substantial generalizability to arbitrary categories, a trait that previous methods struggle to achieve. Secondly, POP3D further improves reconstruction fidelity and naturalness, a crucial aspect that concurrent works fall short of. Our approach marries the strengths of four primary components: (1) a monocular depth and normal predictor that serves to predict crucial geometric cues, (2) a space carving method capable of demarcating the potentially unseen portions of the target object, (3) a generative model pre-trained on a large-scale image dataset that can complete unseen regions of the target, and (4) a neural implicit surface reconstruction method tailored in reconstructing objects using RGB images along with monocular geometric cues. The combination of these components enables POP3D to readily generalize across various in-the-wild images and generate state-of-the-art reconstructions, outperforming similar works by a significant margin. Project page: http://cg.postech.ac.kr/research/POP3D.

With the development of IoT technology, 6G defines ultra-massive machine type communication (umMTC) as a core service type. Since umMTC in 6G is composed of a huge number of devices and various IoT service types, an efficient random access (RA) scheme for massive devices is required. We study a scheme that maximizes the successful RA ratio by applying multi-agent reinforcement learning (MARL) in the delay-constrained 6G umMTC environment. We define the necessary information for the optimal RA strategy and describe how to obtain the RA information with machine-type communication device (MTCD) grouping and learning framework. We utilize the QMIX learning framework to solve the non-stationarity problem in MARL and design the learning framework to select optimal RA for each MTCD group. We conduct a simulation to verify the proposed scheme and simulation results show that a successful RA ratio can be improved up to 20% compared to the state-of-the-art in non-uniform device distribution.

One significant security challenge in vehicular networks is defending against malicious members’ attacks, including insiders and compromised authorities. Insiders are legitimate vehicles who have passed the registration process. Since they can exploit all the information related to the network and other members’ communication, it is easier to perform various attacks with a high impact. In addition, an authority takes charge of registering and managing legitimate vehicles. Thus, if the authority is compromised, it will cause significant damage to the system, including the leaking of private information, such as identity, location, and membership. Many authentication schemes have been proposed to protect vehicular communication from these security issues. However, most existing schemes still face the vulnerability of malicious members. Furthermore, most conventional schemes require additional interactions between the vehicles and infrastructure for authentication, which can cause communication overheads. To overcome these issues, we propose a novel blockchain-based one-time authentication scheme to protect vehicular communication against malicious members. One-time authentication provides higher security and efficiency as every message is authenticated with different proof at a time. We use publicly verifiable secret sharing with blockchain for this property, which brings two benefits. First, it prevents even an authority from obtaining members’ identities by distributing encrypted shares instead of their real identities. Second, it enables robust vehicular communication against insiders’ attacks by allowing a vehicle to send unique proof generated from its private information with messages. Receivers can authenticate the messages by comparing attached values to the information through the blockchain in a noninteractive manner. Security analysis shows that our scheme assures secure vehicle-to-everything communication against insider attacks, and efficiency analysis shows how both authentication and consensus delay change.

Space–air–ground-integrated networks (SAGINs) are receiving a lot of attention as a candidate for an extension to nonterrestrial networks beyond the limit of terrestrial networks. SAGIN can be a means to satisfy various Quality of Service (QoS) by providing several communication links in different characteristics. To fully utilize the advantage of SAGIN, a protocol for informing user equipment (UE) of an appropriate base station layer of SAGIN is needed according to QoS requirements. In addition, the development of the protocol should consider the characteristics of the UE. In particular, when the UE is a machine-type communication device (MTCD) constituting a massive Internet of Things network, random access congestion issue should be considered. To solve the problems, we propose a random access protocol. We propose an indicator named the virtual deadline indicator (VDI) to inform MTCD which layer to try for random access. MTCD attempts random access to one layer of SAGIN by comparing the remaining deadlines of traffic and the point of the VDI. To minimize the deadline expiration rate, we propose an algorithm to shift the VDI position. The base station estimates the number of MTCDs and remaining deadline distribution by using the number of idle preambles to find the VDI location. Finally, we compare the performance of the protocol through simulation with three benchmark schemes. We confirm that the proposed protocol reduces the deadline expiration rate without compromising the Age of Information and energy efficiency.

Group Key Exchange (GKE) is an important tool to develop secure multi-user applications such as group text messages, ad-hoc networks, and so on. Most of the currently deployed GKE schemes are synchronous, i.e., they require all the participants to be online during their execution. However, with more battery-powered devices being used in such applications, the synchronicity requirement is challenging to fulfill. To fill the gaps, asynchronous GKE schemes have been introduced in the literature. Nevertheless, the currently available asynchronous and synchronous GKE schemes rely on Trusted Third Parties (TTPs) for key establishment and management. To this end, reliance on TTPs is a serious shortcoming since TTPs are well known to be the single point of failure. Furthermore, the existing GKE schemes require participants to perform all computations, which can degrade the performance of resource-constrained devices such as Internet of Things (IoT) devices. To solve these problems, in this paper, we propose an asynchronous GKE scheme that uses blockchain and smart contracts to store the security keys-related material and reduce the computational load of the participants. Furthermore, our proposed scheme provides Perfect Forward Secrecy (PFS) and Post-Compromised Security (PCS). Our implementation on Ethereum shows that the proposed scheme can scale to more than 100 participants when combined with a distributed storage system.

We propose a novel approach to 3D scene painting using a configurable 3D scene layout. Our approach takes a 3D scene with semantic class labels as input and trains a 3D scene painting network that synthesizes color values for the input 3D scene. We exploit an off-the-shelf 2D seman-tic image synthesis method to teach the 3D painting net-work without explicit color supervision. Experiments show that our approach produces images with geometrically cor-rect structures and supports scene manipulation, such as the change of viewpoint, object poses, and painting style. Our approach provides rich controllability to synthesized images in the aspect of 3D geometry.

We propose the first reference-based video super-resolution (RefVSR) approach that utilizes reference videos for high-fidelity results. We focus on RefVSR in a triple-camera setting, where we aim at super-resolving a low-resolution ultra-wide video utilizing wide-angle and tele-photo videos. We introduce the first RefVSR network that re-currently aligns and propagates temporal reference features fused with features extracted from low-resolution frames. To facilitate the fusion and propagation of temporal reference features, we propose a propagative temporal fusion module. For learning and evaluation of our network, we present the first RefVSR dataset consisting of triplets of ultra-wide, wide-angle, and telephoto videos concurrently taken from triple cameras of a smartphone. We also propose a two-stage training strategy fully utilizing video triplets in the proposed dataset for real-world 4 × video super-resolution. We extensively evaluate our method, and the result shows the state-of-the-art performance in 4 × super-resolution.

While 3D GANs have recently demonstrated the high-quality synthesis of multi-view consistent images and 3D shapes, they are mainly restricted to photo-realistic human portraits. This paper aims to extend 3D GANs to a different, but meaningful visual form: artistic portrait drawings. However, extending existing 3D GANs to drawings is challenging due to the inevitable geometric ambiguity present in drawings. To tackle this, we present Dr.3D, a novel adaptation approach that adapts an existing 3D GAN to artistic drawings. Dr.3D is equipped with three novel components to handle the geometric ambiguity: a deformation-aware 3D synthesis network, an alternating adaptation of pose estimation and image synthesis, and geometric priors. Experiments show that our approach can successfully adapt 3D GANs to drawings and enable multi-view consistent semantic editing of drawings.

Few-shot domain adaptation to multiple domains aims to learn a complex image distribution across multiple domains from a few training images. A naïve solution here is to train a separate model for each domain using few-shot domain adaptation methods. Unfortunately, this approach mandates linearly-scaled computational resources both in memory and computation time and, more importantly, such separate models cannot exploit the shared knowledge between target domains. In this paper, we propose DynaGAN, a novel few-shot domain-adaptation method for multiple target domains. DynaGAN has an adaptation module, which is a hyper-network that dynamically adapts a pretrained GAN model into the multiple target domains. Hence, we can fully exploit the shared knowledge across target domains and avoid the linearly-scaled computational requirements. As it is still computationally challenging to adapt a large-size GAN model, we design our adaptation module to be lightweight using the rank-1 tensor decomposition. Lastly, we propose a contrastive-adaptation loss suitable for multi-domain few-shot adaptation. We validate the effectiveness of our method through extensive qualitative and quantitative evaluations.

Abstract While motion compensation greatly improves video deblurring quality, separately performing motion compensation and video deblurring demands huge computational overhead. This paper proposes a real‐time video deblurring framework consisting of a lightweight multi‐task unit that supports both video deblurring and motion compensation in an efficient way. The multi‐task unit is specifically designed to handle large portions of the two tasks using a single shared network and consists of a multi‐task detail network and simple networks for deblurring and motion compensation. The multi‐task unit minimizes the cost of incorporating motion compensation into video deblurring and enables real‐time deblurring. Moreover, by stacking multiple multi‐task units, our framework provides flexible control between the cost and deblurring quality. We experimentally validate the state‐of‐the‐art deblurring quality of our approach, which runs at a much faster speed compared to previous methods and show practical real‐time performance (30.99dB@30fps measured on the DVD dataset).

Unsupervised image clustering is a chicken-and-egg problem that involves representation learning and clustering. To resolve the inter-dependency between them, many approaches that iteratively perform the two tasks have been proposed, but their accuracy is limited due to inaccurate intermediate representations and clusters. To overcome this, this paper proposes ProFeat, a novel iterative approach to unsupervised image clustering based on progressive feature refinement. To learn discriminative features for clustering while avoiding adversarial influence from inaccurate intermediate clusters, ProFeat rigorously divides representation learning and clustering by modeling a neural network for clustering as a composition of an embedding and a clustering function and introducing an auxiliary embedding function. ProFeat progressively refines representations using confident samples from intermediate clusters using an extended contrastive loss. This paper also proposes ensemble-based feature refinement for more robust clustering. Our experiments demonstrate that ProFeat achieves superior results compared to previous methods.

In this paper, we present DRANet, a network architecture that disentangles image representations and transfers the visual attributes in a latent space for unsupervised cross-domain adaptation. Unlike the existing domain adaptation methods that learn associated features sharing a domain, DRANet preserves the distinctiveness of each domain’s characteristics. Our model encodes individual representations of content (scene structure) and style (artistic appearance) from both source and target images. Then, it adapts the domain by incorporating the transferred style factor into the content factor along with learnable weights specified for each domain. This learning framework allows bi/multi-directional domain adaptation with a single encoder-decoder network and aligns their domain shift. Additionally, we propose a content-adaptive domain transfer module that helps retain scene structure while transferring style. Extensive experiments show our model successfully separates content-style factors and synthesizes visually pleasing domain-transferred images. The proposed method demonstrates state-of-the-art performance on standard digit classification tasks as well as semantic segmentation tasks.

We propose a novel end-to-end learning-based approach for single image defocus deblurring. The proposed approach is equipped with a novel Iterative Filter Adaptive Network (IFAN) that is specifically designed to handle spatially-varying and large defocus blur. For adaptively handling spatially-varying blur, IFAN predicts pixel-wise deblurring filters, which are applied to defocused features of an input image to generate deblurred features. For effectively managing large blur, IFAN models deblurring filters as stacks of small-sized separable filters. Predicted separable deblurring filters are applied to defocused features using a novel Iterative Adaptive Convolution (IAC) layer. We also propose a training scheme based on defocus disparity estimation and reblurring, which significantly boosts the de-blurring quality. We demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively on real-world images.

Federated Learning (FL) is a promising technique for edge computing environments as it provides better data privacy protection. It enables each edge node in the system to send a central server a computed value, named gradient, rather than sending raw data. However, recent research results show that the FL is still vulnerable to an inference attack, which is an adversarial algorithm that is capable of identifying the data used to compute the gradient. One prevalent mitigation strategy is differential privacy which computes a gradient with noised data, but this causes another problem that is accuracy degradation. To effectively deal with this problem, this paper proposes a new digestive neural network (DNN) and integrates it into FL. The proposed scheme distorts raw data by DNN to make it unrecognizable then computes a gradient by a classification network. The gradients generated by edge nodes will be sent to the server to complete a trained model. The simulation results show that the proposed scheme has 9.31% higher classification accuracy and 19.25% lower attack accuracy on average than the differential private schemes.

For successful semantic editing of real images, it is critical for a GAN inversion method to find an in-domain latent code that aligns with the domain of a pre-trained GAN model. Unfortunately, such in-domain latent codes can be found only for in-range images that align with the training images of a GAN model. In this paper, we propose BDInvert, a novel GAN inversion approach to semantic editing of out- of-range images that are geometrically unaligned with the training images of a GAN model. To find a latent code that is semantically editable, BDInvert inverts an input out-of-range image into an alternative latent space than the original latent space. We also propose a regularized inversion method to find a solution that supports semantic editing in the alternative space. Our experiments show that BDInvert effectively supports semantic editing of out-of-range images with geometric transformations.

Single image camera calibration is the task of estimating the camera parameters from a single input image, such as the vanishing points, focal length, and horizon line. In this work, we propose Camera calibration TRansformer with Line-Classification (CTRL-C), an end-to-end neural network-based approach to single image camera calibration, which directly estimates the camera parameters from an image and a set of line segments. Our network adopts the transformer architecture to capture the global structure of an image with multi-modal inputs in an end-to-end manner. We also propose an auxiliary task of line classification to train the network to extract the global geometric information from lines effectively. Our experiments demonstrate that CTRL-C outperforms the previous state-of-the-art methods on the Google Street View and SUN360 benchmark datasets. Code is available at https://github.com/jwlee-vcl/CTRL-C.

This paper proposes a novel deep learning approach for single image defocus deblurring based on inverse kernels. In a defocused image, the blur shapes are similar among pixels although the blur sizes can spatially vary. To utilize the property with inverse kernels, we exploit the observation that when only the size of a defocus blur changes while keeping the shape, the shape of the corresponding inverse kernel remains the same and only the scale changes. Based on the observation, we propose a kernel-sharing parallel atrous convolutional (KPAC) block specifically designed by incorporating the property of inverse kernels for single image defocus deblurring. To effectively simulate the invariant shapes of inverse kernels with different scales, KPAC shares the same convolutional weights among multiple atrous convolution layers. To efficiently simulate the varying scales of inverse kernels, KPAC consists of only a few atrous convolution layers with different dilations and learns per-pixel scale attentions to aggregate the outputs of the layers. KPAC also utilizes the shape attention to combine the outputs of multiple convolution filters in each atrous convolution layer, to deal with defocus blur with a slightly varying shape. We demonstrate that our approach achieves state-of-the-art performance with a much smaller number of parameters than previous methods.

Federated Learning (FL) is an efficient and secure machine learning technique designed for decentralized computing systems such as fog and edge computing. Its learning process employs frequent communications as the participating local devices send updates, either gradients or parameters of their models, to a central server that aggregates them and redistributes new weights to the devices. In FL, private data does not leave the individual local devices, and thus, rendered as a robust solution in terms of privacy preservation. However, the recently introduced membership inference attacks pose a critical threat to the impeccability of FL mechanisms. By eavesdropping only on the updates transferring to the center server, these attacks can recover the private data of a local device. A prevalent solution against such attacks is the differential privacy scheme that augments a sufficient amount of noise to each update to hinder the recovering process. However, it suffers from a significant sacrifice in the classification accuracy of the FL. To effectively alleviate the problem, this paper proposes a Digestive Neural Network (DNN), an independent neural network attached to the FL. The private data owned by each device will pass through the DNN and then train the FL. The DNN modifies the input data, which results in distorting updates, in a way to maximize the classification accuracy of FL while the accuracy of inference attacks is minimized. Our simulation result shows that the proposed DNN shows significant performance on both gradient sharing- and weight sharing-based FL mechanisms. For the gradient sharing, the DNN achieved higher classification accuracy by 16.17% while 9% lower attack accuracy than the existing differential privacy schemes. For the weight sharing FL scheme, the DNN achieved at most 46.68% lower attack success rate with 3% higher classification accuracy.

MPEG-DASH is a video streaming standard that outlines protocols for sending audio and video content from a server to a client over HTTP. However, it creates an opportunity for an adversary to invade users’ privacy. While a user is watching a video, information is leaked in the form of meta-data, the size of data and the time the server sent the data to the user. After a fingerprint of this data is created, the adversary can use this to identify whether a target user is watching the corresponding video. Only one defense strategy has been proposed to deal with this problem: differential privacy that adds sufficient noise in order to muddle the attacks. However, that strategy still suffers from the trade-off between privacy and efficiency. This paper proposes a novel defense strategy against the attacks with rigorous privacy and performance goals creating a private, scalable solution. Our algorithm, “No Data are Alone” (NDA), is highly efficient. The experimental results show that our scheme is more than two times efficient in terms of excess downloaded video (represented as waste) compared to the most efficient differential privacy-based scheme. Additionally, no classifier can achieve an accuracy above 7.07% against videos obfuscated with our scheme.

For the success of video deblurring, it is essential to utilize information from neighboring frames. Most state-of-the-art video deblurring methods adopt motion compensation between video frames to aggregate information from multiple frames that can help deblur a target frame. However, the motion compensation methods adopted by previous deblurring methods are not blur-invariant, and consequently, their accuracy is limited for blurry frames with different blur amounts. To alleviate this problem, we propose two novel approaches to deblur videos by effectively aggregating information from multiple video frames. First, we present blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames. Second, for motion compensation, instead of aligning frames by warping with estimated motions, we use a pixel volume that contains candidate sharp pixels to resolve motion estimation errors. We combine these two processes to propose an effective recurrent video deblurring network that fully exploits deblurred previous frames. Experiments show that our method achieves the state-of-the-art performance both quantitatively and qualitatively compared to recent methods that use deep learning.

Vehicle to everything (V2X) in heterogeneous networks concurrently retains multiple communication links within a channel: such as vehicle to vehicle (V2V), Vehicle to macro base station (V2C), and cellular user equipment to femtocell base station (U2F). To provide high spectral efficiency, there were many efforts such as non-orthogonal multiple access (NOMA) and channel aggregation. However, combining these schemes on the top of NOMA-V2X-femtocell is extremely challenging as it increases the number of dimensions to be considered. To address this issue, this paper proposes a new genetic deep learning algorithm. It employs a genetic algorithm (GA) to find a pair of communication links per channel in a way to maximize the throughput and a neural network to reduce the dimension gradually. The neural network is trained to predicts which pair can be part of the final result. The suitable pairs are marked by deep learning, then they are not shuffled in the subsequent generations. The simulation results show that the proposed scheme achieved higher throughput greater than 20%, compared to the existing GA.

Network anomalies are correlated to activities that deviate from regular behavior patterns in a network, and they are undetectable until their actions are defined as malicious. Current work in network anomaly detection includes network-based and host-based intrusion detection systems. However, most of them suffer from high false detection rates due to the base rate fallacy. To overcome such a drawback, this paper proposes a superior behavior-based anomaly detection system (SuperB) that defines legitimate network behaviors of authorized users in order to identify unauthorized accesses. We define the network behaviors of the authorized users by training the proposed deep learning model with time-series data extracted from network packets of each of the users. Then, the trained model is used to classify all other behaviors (we define these as anomalies) from the defined legitimate behaviors. As a result, SuperB effectively detects all anomalies of network behaviors. Our simulation results show that the proposed algorithm needs at least five end-to-end conversations to achieve over 95% accuracy and over 93% recall rate. Some simulations show 100% accuracy and recall rate. Our simulations use live network data combined with the CICIDS2017 data set. The performance has an average of less than 1.1% false-positive rate with some simulations showing 0%. The execution time to process each conversation is 85.20±0.60 milliseconds (ms), and thus it takes about only 426 ms to process five conversations to identify anomaly.

Group Key Agreement (GKA) mechanisms play a crucial role in realizing various applications in different networks, such as sensor networks and the Internet of Things (IoT). To be suitable for IoT, a GKA must satisfy several critical requirements. First, a GKA must be robust against a compromised device attack and satisfy essential secrecy definitions without the existence of a Trusted Third Party (TTP). TTP is often used by IoT devices to establish ad hoc networks securely, and usually, these devices are resource-constrained. Second, the GKA must be able to distribute session keys successfully, even with offline devices. Third, a GKA must reduce the burden of heavy cryptographic computations for IoT devices. Based on these observations, we propose a new GKA scheme that satisfies all the requirements above. The proposed scheme leverages smart contracts to alleviate the computational and storage overheads on IoT devices induced by cryptographic functions. It also brings the advantage of asynchronism such that offline devices will be able to compute the group key once they are online.

Sensor clustering and trajectory optimization are a hot topic for last decade to improve energy efficiency of wireless sensor network (WSN). Most of existing studies assume that the sensor is uniformly deployed or all regions in the WSN coverage have the same level of interest. However, even in the same WSN, areas with high probability of disaster will have to form a “hotspot” with more sensors densely placed in order to be sensitive to environmental changes. The energy hole can be serious if sensor clustering and trajectory optimization are formulated without considering the hotspot. Therefore, we need to devise a sensor clustering and trajectory optimization algorithm considering the hotspots of WSN. In this paper, we propose an iterative algorithm to minimize the amount of energy consumed by components of WSN named ISCTO. The ISCTO algorithm consists of two phases. The first phase is a sensor clustering phase used to find the suitable number of clusters and cluster headers by considering the density of sensor and residual battery of sensors. The second phase is a trajectory optimization phase used to formulate suitable trajectory of multiple mobile sinks to minimize the amount of energy consumed by mobile sinks. The ISCTO algorithm performs two phases repeatedly until the amount of energy consumed by the WSN is not reduced. In addition, we show the performance of the proposed algorithm in terms of the total amount of energy consumed by sensors and mobile sinks.

This paper presents a novel approach for learning instance segmentation with image-level class labels as supervision. Our approach generates pseudo instance segmentation labels of training images, which are used to train a fully supervised model. For generating the pseudo labels, we first identify confident seed areas of object classes from attention maps of an image classification model, and propagate them to discover the entire instance areas with accurate boundaries. To this end, we propose IRNet, which estimates rough areas of individual instances and detects boundaries between different object classes. It thus enables to assign instance labels to the seeds and to propagate them within the boundaries so that the entire areas of instances can be estimated accurately. Furthermore, IRNet is trained with inter-pixel relations on the attention maps, thus no extra supervision is required. Our method with IRNet achieves an outstanding performance on the PASCAL VOC 2012 dataset, surpassing not only previous state-of-the-art trained with the same level of supervision, but also some of previous models relying on stronger supervision.

In this paper, we propose the first end-to-end convolutional neural network (CNN) architecture, Defocus Map Estimation Network (DMENet), for spatially varying defocus map estimation. To train the network, we produce a novel depth-of-field (DOF) dataset, SYNDOF, where each image is synthetically blurred with a ground-truth depth map. Due to the synthetic nature of SYNDOF, the feature characteristics of images in SYNDOF can differ from those of real defocused photos. To address this gap, we use domain adaptation that transfers the features of real defocused photos into those of synthetically blurred ones. Our DMENet consists of four subnetworks: blur estimation, domain adaptation, content preservation, and sharpness calibration networks. The subnetworks are connected to each other and jointly trained with their corresponding supervisions in an end-to-end manner. Our method is evaluated on publicly available blur detection and blur estimation datasets and the results show the state-of-the-art performance.In this paper, we propose the first end-to-end convolutional neural network (CNN) architecture, Defocus Map Estimation Network (DMENet), for spatially varying defocus map estimation. To train the network, we produce a novel depth-of-field (DOF) dataset, SYNDOF, where each image is synthetically blurred with a ground-truth depth map. Due to the synthetic nature of SYNDOF, the feature characteristics of images in SYNDOF can differ from those of real defocused photos. To address this gap, we use domain adaptation that transfers the features of real defocused photos into those of synthetically blurred ones. Our DMENet consists of four subnetworks: blur estimation, domain adaptation, content preservation, and sharpness calibration networks. The subnetworks are connected to each other and jointly trained with their corresponding supervisions in an end-to-end manner. Our method is evaluated on publicly available blur detection and blur estimation datasets and the results show the state-of-the-art performance.

This paper considers a multimedia broadcasting scenario in which two types of heterogeneous users with different display resolutions and different numbers of antennas stay in the service area. We propose an image broadcasting scheme that uses the image super-resolution (SR) techniques, spatial diversity, and diversity-multiplexing tradeoff (DMT) achieving codes. The proposed scheme broadcasts a low-resolution (LR) image to two types of users, along with residual pixel-error map containing high-frequency details of high-resolution (HR) image. Then, a user retaining an HR screen employs SR to reconstruct an HR image from the received LR image, and exploits the residual map to further enhance the image quality. Our scheme properly trains the neural network models of the deep learning-based SR by taking into account the source coding rates of the images. Considering the relationship between the number of antennas and screen resolution, based on hardware space of user devices, the proposed scheme encodes an LR image with spatial diversity, and encodes residual map with DMT-achieving codes. Numerical evaluation shows that our scheme significantly outperforms the baseline strategy that broadcasts either HR or LR images.

Abstract This paper proposes a deep learning‐based image tone enhancement approach that can maximally enhance the tone of an image while preserving the naturalness. Our approach does not require carefully generated ground‐truth images by human experts for training. Instead, we train a deep neural network to mimic the behavior of a previous classical filtering method that produces drastic but possibly unnatural‐looking tone enhancement results. To preserve the naturalness, we adopt the generative adversarial network (GAN) framework as a regularizer for the naturalness. To suppress artifacts caused by the generative nature of the GAN framework, we also propose an imbalanced cycle‐consistency loss. Experimental results show that our approach can effectively enhance the tone and contrast of an image while preserving the naturalness compared to previous state‐of‐the‐art approaches.

A common way to view a 360° video on a 2D display is to crop and render a part of the video as a normal field-of-view (NFoV) video. While users can enjoy natural-looking NFoV videos using this approach, they need to constantly make manual adjustment of the viewing direction not to miss interesting events in the video. In this paper, we propose an interactive and automatic navigation system for comfortable 360° video playback. Our system finds a virtual camera path that shows the most salient areas through the video, generates a NFoV video based on the path, and plays it in an online manner. A user can interactively change the viewing direction while watching a video, and the system instantly updates the path reflecting the intention of the user. To enable online processing, we design our system consisting of an offline pre-processing step, and an online 360° video navigation step. The pre-processing step computes optical flow and saliency scores for an input video. Based on these, the online video navigation step computes an optimal camera path reflecting user interaction, and plays a NFoV video in an online manner. For improved user experience, we also introduce optical flow-based camera path planning, saliency-aware path update, and adaptive control of the temporal window size. Our experimental results including user studies show that our system provides more pleasant experience of watching 360° videos than existing approaches.

Abstract We propose a novel approach for detecting two kinds of partial blur, defocus and motion blur, by training a deep convolutional neural network. Existing blur detection methods concentrate on designing low‐level features, but those features have difficulty in detecting blur in homogeneous regions without enough textures or edges. To handle such regions, we propose a deep encoder‐decoder network with long residual skip‐connections and multi‐scale reconstruction loss functions to exploit high‐level contextual features as well as low‐level structural features. Another difficulty in partial blur detection is that there are no available datasets with images having both defocus and motion blur together, as most existing approaches concentrate only on either defocus or motion blur. To resolve this issue, we construct a synthetic dataset that consists of complex scenes with both types of blur. Experimental results show that our approach effectively detects and classifies blur, outperforming other state‐of‐the‐art methods. Our method can be used for various applications, such as photo editing, blur magnification, and deblurring.

Images acquired in low-light conditions with handheld cameras are often blurry, so steady poses and long exposure time are required to alleviate this problem. Although significant advances have been made in image deblurring, state-of-the-art approaches often fail on low-light images, as a sufficient number of salient features cannot be extracted for blur kernel estimation. On the other hand, light streaks are common phenomena in low-light images that have not been extensively explored in existing approaches. In this work, we propose an algorithm that utilizes light streaks to facilitate deblurring low-light images. The light streaks, which commonly exist in the low-light blurry images, contain rich information regarding camera motion and blur kernels. A method is developed in this work to detect light streaks for kernel estimation. We introduce a non-linear blur model that explicitly takes light streaks and corresponding light sources into account, and pose them as constraints for estimating the blur kernel in an optimization framework. For practical applications, the proposed algorithm is extended to handle images undergoing non-uniform blur. Experimental results show that the proposed algorithm performs favorably against the state-of-the-art methods on deblurring real-world low-light images.

One popular approach for blind deconvolution is to formulate a maximum a posteriori (MAP) problem with sparsity priors on the gradients of the latent image, and then alternatingly estimate the blur kernel and the latent image. While several successful MAP based methods have been proposed, there has been much controversy and confusion about their convergence, because sparsity priors have been shown to prefer blurry images to sharp natural images. In this paper, we revisit this problem and provide an analysis on the convergence of MAP based approaches. We first introduce a slight modification to a conventional joint energy function for blind deconvolution. The reformulated energy function yields the same alternating estimation process, but more clearly reveals how blind deconvolution works. We then show the energy function can actually favor the right solution instead of the no-blur solution under certain conditions, which explains the success of previous MAP based approaches. The reformulated energy function and our conditions for the convergence also provide a way to compare the qualities of different blur kernels, and we demonstrate its applicability to automatic blur kernel size selection, blur kernel estimation using light streaks, and defocus estimation.

This paper proposes a MAC protocol for Radio Frequency (RF) energy harvesting in Wireless Sensor Networks (WSN). In the conventional RF energy harvesting methods, an Energy Transmitter (ET) operates in a passive manner. An ET transmits RF energy signals only when a sensor with depleted energy sends a Request-for-Energy (RFE) message. Unlike the conventional methods, an ET in the proposed scheme can actively send RF energy signals without RFE messages. An ET determines the active energy signal transmission according to the consequence of the passive energy harvesting procedures. To transmit RF energy signals without request from sensors, the ET participates in a contention-based channel access procedure. Once the ET successfully acquires the channel, it sends RF energy signals on the acquired channel during Short Charging Time (SCT). The proposed scheme determines the length of SCT to minimize the interruption of data communication. We compare the performance of the proposed protocol with RF-MAC protocol by simulation. The simulation results show that the proposed protocol can increase the energy harvesting rate by 150% with 8% loss of network throughput compared to RF-MAC. In addition, the proposed protocol can increase the lifetime of WSN because of the active energy signal transmission method.

Taking a picture has been traditionally a one-person task. In this paper we present a novel system that allows multiple mobile devices to work collaboratively in a synchronized fashion to capture a panorama of a highly dynamic scene, creating an entirely new photography experience that encourages social interactions and teamwork. Our system contains two components: a client app that runs on all participating devices, and a server program that monitors and communicates with each device. In a capturing session, the server collects in realtime the viewfinder images of all devices and stitches them on-the-fly to create a panorama preview, which is then streamed to all devices as visual guidance. The system also allows one camera to be the host and send direct visual instructions to others to guide camera adjustment. When ready, all devices take pictures at the same time for panorama stitching. Our preliminary study suggests that the proposed system can help users capture high quality panoramas with an enjoyable teamwork experience.

Images taken in low-light conditions with handheld cameras are often blurry due to the required long exposure time. Although significant progress has been made recently on image deblurring, state-of-the-art approaches often fail on low-light images, as these images do not contain a sufficient number of salient features that deblurring methods rely on. On the other hand, light streaks are common phenomena in low-light images that contain rich blur information, but have not been extensively explored in previous approaches. In this work, we propose a new method that utilizes light streaks to help deblur low-light images. We introduce a non-linear blur model that explicitly models light streaks and their underlying light sources, and poses them as constraints for estimating the blur kernel in an optimization framework. Our method also automatically detects useful light streaks in the input image. Experimental results show that our approach obtains good results on challenging real-world examples that no other methods could achieve before.

In this study, a decentralised ranging method for uplink orthogonal frequency division multiple access (OFDMA) systems with half‐duplex (HD) amplify‐and‐forward (AF) relay stations (RSs) is proposed. In the OFDMA systems with HD AF RSs, twice more resources and delays are required as ranging without RS. To reduce the required resources and delays for ranging, the authors propose a two‐phase ranging scheme based on the decentralised timing‐offset estimation at each ranging mobile station (MS). At the first phase, RS occasionally broadcasts timing reference signal, and at the second phase RS retransmits the collected ranging signals from the MSs. Then, each ranging MSs can individually estimate its own timing offset from the received signals. In the proposed ranging method, the base station does not need to send a timing‐adjustment message, and the overhead associated with ranging in the downlink resources, and computational complexity can be significantly reduced without degrading the timing‐offset‐estimation performance. Moreover, the delay associated with ranging can be maintained as same as ranging without RS.

Panning and tracking shots are popular photography techniques in which the camera tracks a moving object and keeps it at the same position, resulting in an image where the moving foreground is sharp but the background is blurred accordingly, creating an artistic illustration of the foreground motion. Such shots however are hard to capture even for professionals, especially when the foreground motion is complex (e.g., non-linear motion trajectories). In this work we propose a system to generate realistic, 3D-aware tracking shots from consumer videos. We show how computer vision techniques such as segmentation and structure-from-motion can be used to lower the barrier and help novice users create high quality tracking shots that are physically plausible. We also introduce a pseudo 3D approach for relative depth estimation to avoid expensive 3D reconstruction for improved robustness and a wider application range. We validate our system through extensive quantitative and qualitative evaluations.

State-of-the-art single image deblurring techniques are sensitive to image noise. Even a small amount of noise, which is inevitable in low-light conditions, can degrade the quality of blur kernel estimation dramatically. The recent approach of Tai and Lin [17] tries to iteratively denoise and deblur a blurry and noisy image. However, as we show in this work, directly applying image denoising methods often partially damages the blur information that is extracted from the input image, leading to biased kernel estimation. We propose a new method for handling noise in blind image deconvolution based on new theoretical and practical insights. Our key observation is that applying a directional low-pass filter to the input image greatly reduces the noise level, while preserving the blur information in the orthogonal direction to the filter. Based on this observation, our method applies a series of directional filters at different orientations to the input image, and estimates an accurate Radon transform of the blur kernel from each filtered image. Finally, we reconstruct the blur kernel using inverse Radon transform. Experimental results on synthetic and real data show that our algorithm achieves higher quality results than previous approaches on blurry and noisy images.

Blind image deconvolution, i.e., estimating a blur kernel k and a latent image x from an input blurred image y, is a severely ill-posed problem. In this paper we introduce a new patch-based strategy for kernel estimation in blind deconvolution. Our approach estimates a “trusted” subset of x by imposing a patch prior specifically tailored towards modeling the appearance of image edge and corner primitives. To choose proper patch priors we examine both statistical priors learned from a natural image dataset and a simple patch prior from synthetic structures. Based on the patch priors, we iteratively recover the partial latent image x and the blur kernel k. A comprehensive evaluation shows that our approach achieves state-of-the-art results for uniformly blurred images.

Methods to undo the effects of motion blur are the subject of intense research, but evaluating and tuning these algorithms has traditionally required either user input or the availability of ground-truth images. We instead develop a metric for automatically predicting the perceptual quality of images produced by state-of-the-art deblurring algorithms. The metric is learned based on a massive user study, incorporates features that capture common deblurring artifacts, and does not require access to the original images (i.e., is "noreference"). We show that it better matches user-supplied rankings than previous approaches to measuring quality, and that in most cases it outperforms conventional full-reference image-similarity measures. We demonstrate applications of this metric to automatic selection of optimal algorithms and parameters, and to generation of fused images that combine multiple deblurring results.

Abstract This paper proposes an algorithm which uses image registration to estimate a non‐uniform motion blur point spread function (PSF) caused by camera shake. Our study is based on a motion blur model which models blur effects of camera shakes using a set of planar perspective projections (i.e., homographies). This representation can fully describe motions of camera shakes in 3D which cause non‐uniform motion blurs. We transform the non‐uniform PSF estimation problem into a set of image registration problems which estimate homographies of the motion blur model one‐by‐one through the Lucas‐Kanade algorithm. We demonstrate the performance of our algorithm using both synthetic and real world examples. We also discuss the effectiveness and limitations of our algorithm for non‐uniform deblurring.

Videos captured by hand-held Cameras often contain significant camera shake, causing many frames to be blurry. Restoring shaky videos not only requires smoothing the camera motion and stabilizing the content, but also demands removing blur from video frames. However, video blur is hard to remove using existing single or multiple image deblurring techniques, as the blur kernel is both spatially and temporally varying. This paper presents a video deblurring method that can effectively restore sharp frames from blurry ones caused by camera shake. Our method is built upon the observation that due to the nature of camera shake, not all video frames are equally blurry. The same object may appear sharp on some frames while blurry on others. Our method detects sharp regions in the video, and uses them to restore blurry regions of the same content in nearby frames. Our method also ensures that the deblurred frames are both spatially and temporally coherent using patch-based synthesis. Experimental results show that our method can effectively remove complex video blur under the presence of moving objects and other outliers, which cannot be achieved using previous deconvolution-based approaches.

Non-blind deconvolution is a key component in image deblurring systems. Previous deconvolution methods assume a linear blur model where the blurred image is generated by a linear convolution of the latent image and the blur kernel. This assumption often does not hold in practice due to various types of outliers in the imaging process. Without proper outlier handling, previous methods may generate results with severe ringing artifacts even when the kernel is estimated accurately. In this paper we analyze a few common types of outliers that cause previous methods to fail, such as pixel saturation and non-Gaussian noise. We propose a novel blur model that explicitly takes these outliers into account, and build a robust non-blind deconvolution method upon it, which can effectively reduce the visual artifacts caused by outliers. The effectiveness of our method is demonstrated by experimental results on both synthetic and real-world examples.

We present a novel method for decomposing an image into base and texture layers. Our method is simple and effective, and can handle textures of high contrast, which traditional image filtering techniques may not handle efficiently. The method first removes high-frequency texture information using low-pass filtering, and then restores structural information of the image using a deconvolution operation. Experimental results demonstrate the effectiveness of our method.

This paper presents a simple and effective image retargeting method that preserves visually important parts while reducing unwanted distortions of an image. Our approach is based on a novel importance diffusion scheme, which propagates importance of removed pixels to their neighbors for preserving visual contexts and avoiding over-shrinkage of unimportant parts. Importance diffusion enables even a simple row/column removal method, which removes the least important rows/columns repeatedly, to produce visually pleasant results. It also provides control over the trade-off between uniform and non-uniform sampling for the row/column removal and seam carving methods. Experimental result demonstrates that importance diffusion successfully improves the retargeting results of row/column removal and seam carving.

This paper presents a fast deblurring method that produces a deblurring result from a single image of moderate size in a few seconds. We accelerate both latent image estimation and kernel estimation in an iterative deblurring process by introducing a novel prediction step and working with image derivatives rather than pixel values. In the prediction step, we use simple image processing techniques to predict strong edges from an estimated latent image, which will be solely used for kernel estimation. With this approach, a computationally efficient Gaussian prior becomes sufficient for deconvolution to estimate the latent image, as small deconvolution artifacts can be suppressed in the prediction. For kernel estimation, we formulate the optimization function using image derivatives, and accelerate the numerical process by reducing the number of Fourier transforms needed for a conjugate gradient method. We also show that the formulation results in a smaller condition number of the numerical system than the use of pixel values, which gives faster convergence. Experimental results demonstrate that our method runs an order of magnitude faster than previous work, while the deblurring quality is comparable. GPU implementation facilitates further speed-up, making our method fast enough for practical use.

A control frame can be piggybacked in a data frame to increase the channel efficiency in a wireless communication such as IEEE 802.11 WLAN. However, the piggyback scheme may cause the decrease of the channel efficiency and the increase of the frame transmission delay for other stations when any station has the low transmission rate and the control frame has the global control information such as the channel reservation time. It is similar to the anomaly phenomenon in the network which supports the multi-rate transmission. In this paper, we define this phenomenon as "the piggyback problem at low physical transmission rate" and evaluate the effect of this problem with respect to the physical transmission rate and the normalized traffic load. Then, we propose the delay-based piggyback algorithm which decides to use the piggyback scheme for a control frame based on the delay efficiency in IEEE 802.11e HCCA. In the simulation results, the proposed algorithm reduces the average frame transmission delay and the channel utilization about 24% and 25%, respectively if there is one station which has low physical transmission rate.

We propose a method for removing non-uniform motion blur from multiple blurry images. Traditional methods focus on estimating a single motion blur kernel for the entire image. In contrast, we aim to restore images blurred by unknown, spatially varying motion blur kernels caused by different relative motions between the camera and the scene. Our algorithm simultaneously estimates multiple motions, motion blur kernels, and the associated image segments. We formulate the problem as a regularized energy function and solve it using an alternating optimization technique. Real- world experiments demonstrate the effectiveness of the proposed method.

A data frame can piggyback a control frame to increase the channel efficiency in a wireless communication such as IEEE 802.11 WLAN. However, the piggyback scheme may cause the decrease of the channel efficiency and the increase of the frame transmission delay for other stations if the station has the low transmission rate and the control frame presents the global control information such as the channel reservation time. It is similar to the anomaly phenomenon in the network which supports the multi-rate transmission. In this paper, we define this phenomenon as "the piggyback problem at low physical transmission rate" and evaluate the effect of this problem with respect to the physical transmission rate and the normalized traffic load. Then, we propose the delay-based piggyback algorithm which decides to use the piggyback scheme for a control frame based on the delay efficiency in IEEE 802.11 WLAN. In the simulation results, the proposed algorithm reduces the average frame transmission delay and the channel utilization about 24% and 25%, respectively even if there is one station which has low physical transmission rate.

A novel multiuser scheduling and feedback strategy for the multiple-input multiple-output (MIMO) downlink is proposed in this paper. It achieves multiuser diversity gain without substantial feedback requirements. The proposed strategy uses per-antenna scheduling at the base station, which maps each transmit antenna at the base station (equivalently, a spatial channel) to a user. Each user has a number of receive antennas that is greater than or equal to the number of transmit antennas at the base station. Zero-forcing receivers are deployed by each user to decode the transmitted data streams. In this system, the base station requires users' channel quality on each spatial channel for scheduling. An opportunistic feedback protocol is proposed to reduce the feedback requirements. The proposed protocol uses a contention channel that consists of a fixed number of feedback minislots to convey channel state information. Feedback control parameters including the channel quality threshold and the random access feedback probability are jointly adjusted to maximize the average throughput performance of this system. Multiple receive antennas at the base station are used on the feedback channel to allow decoding multiple feedback messages sent simultaneously by different users. This further reduces the bandwidth of the feedback channel. Iterative search algorithms are proposed to solve the optimization for selection of these parameters under both scenarios that the cumulative distribution functions of users are known or unknown to the base station

This paper considers some scheduler structures that are executable in environments of multiple traffic classes and multiple frequency channels. In designing a scheduler structure for multiple traffic classes, we first propose a scheduler selection rule that uses the priority of traffic class and the urgency level of each packet. Then we relax the barrier of traffic class priority if a packet of higher priority has some room in waiting time. This gives us a chance to exploit multi user diversity, thereby giving more flexibility in scheduling. Our considered scheduler can achieve higher throughput compared to the simple extension of conventional modified largest weighted delay first (MLWDF) scheduler while maintaining the delay performance of QoS class traffic. We also design a scheduler structure for multiple frequency channels that chooses a good channel for each user as much as possible to exploit frequency diversity. The simulation results show that our proposed scheduler increases the total system throughput up to 50% without degrading the QoS performance of delay. Our schedulers are suited to be deployed for OFDMA systems like IEEE 802.16 systems that have plenty of frequency channels and use the adaptive modulation and coding (AMC) scheme.

In this paper, we investigate a multiple input multiple output (MIMO) multiuser relay channel, where a source with multiple antennas sends data to multiple users via a relay with multiple antennas. The relay applies linear processing to the received signal and forwards the processed signal to multiple users. In our system model, the direct links from the source to the users are neglected. We propose algorithms to compute achievable sum rates of this system based on dirty paper coding. An achievable sum rate defines a sum rate that can be achieved in the MIMO multiuser relay channel with zero error probability for any user, hence it is also a lower bound of the capacity of this channel. These algorithms also produce coefficients of the precoder at the source node and the coefficients of the linear processing unit at the relay. Simulations show that the proposed system architecture and algorithms achieve sum rate performance that is close to the derived performance upper bound

We consider hard handoff scheme based on uplink and downlink signals in IEEE 802.16e systems. To exploit the channel reciprocity in TDD systems, the proposed scheme triggers handoff initiation process using uplink signal. A serving base station monitors uplink traffic signal and triggers handoff process when the uplink signal strength or SINR becomes less than the predetermined level. This process can prevent mobile stations from periodically monitoring and reporting downlink signals of neighbor base stations at the non-handoff region. In the handoff decision process, the proposed scheme exploits not only uplink signal but also downlink signal as the handoff criteria to reduce ping-pong and outage probability. To efficiently combine uplink and downlink signal levels, the proposed scheme adopt an uplink and downlink joint hysteresis to determine a handoff direction. Using the joint hysteresis can reduce the outage probability compared with using uplink and downlink hysteresis independently. We evaluate the performance of the proposed scheme using computer simulation. The simulation results show that the proposed algorithm has better performance than the conventional mobile assisted handoff scheme of IEEE 802.16e in terms of the average number of handoff and the outage probability during handoff

At present, mobile multimedia services such as sports information and mobile TV services have been delivered over point-to-point connections. But it is clear that the point-to-multipoint connections can support these mobile multimedia services more efficiently because they can simultaneously transmit data packets from a single source to multiple destinations in a multicast group. Recently, the simulcast scheme for multimedia broadcast and multicast service (MBMS) in which the same signal is transmitted from all the cells within the system, has been proposed, but, in order to adopt this technology, the same channel bands or time slots for only broadcast/multicast service should be reserved at all cells. The reservation of these resources may not be suitable for services in a local region, and some operators may also not be easy to reserve these resources. So, in this paper, we propose multicast strategies which make multicast resources easily multiplex with unicast resources and enhance cell-capacity by improving the performance of cell-boundary users. In addition, we mathematically analyze the link-level performance and cell throughput of conventional and proposed schemes, and discuss their numerical results