In recent years, significant advancements have been made in the field of low-light image enhancement, especially with the emergence of transformer- and diffusion-based models, which have demonstrated remarkable enhancement capabilities. However, the introduction of these advanced but sophisticated neural network architectures often leads to algorithms that demand extensive computational resources and storage space, hindering their applicability on resource-constrained devices. In this paper, to address this challenge, we propose LightLLIE, a lightweight low-light image enhancement network. The proposed network employs a global-local dual-branch architecture: a global brightness adjustment branch designed to formulate a global brightness enhancement scheme by analyzing brightness channel information, and a local detail enhancement branch employing a lightweight design with dual attention blocks for effective local detail and texture restoration. Notably, structural re-parameterization is integrated into the global brightness adjustment branch, enabling the conversion of the reparameterizable multi-scale convolution into a simple structure during inference, thereby improving inference efficiency. Extensive quantitative and qualitative experiments demonstrate that LightLLIE outperforms state-of-the-art methods while achieving superior processing speeds.

Text-driven 3D scene generation techniques have made rapid progress in recent years. Their success is mainly at-tributed to using existing generative models to iteratively perform image warping and inpainting to generate 3D scenes. However, these methods heavily rely on the out-puts of existing models, leading to error accumulation in geometry and appearance that prevent the models from being used in various scenarios (e.g., outdoor and unreal sce-narios). To address this limitation, we generatively refine the newly generated local views by querying and aggregating global 3D information, and then progressively generate the 3D scene. Specifically, we employ a tri-plane features-based NeRF as a unified representation of the 3D scene to constrain global 3D consistency, and propose a generative refinement network to synthesize new contents with higher quality by exploiting the natural image prior from 2D dif-fusion model as well as the global 3D information of the current scene. Our extensive experiments demonstrate that, in comparison to previous methods, our approach supports wide variety of scene generation and arbitrary camera tra-jectories with improved visual quality and 3D consistency.

Low visibility and severe noise are two main degradations in extreme low-light images. Nevertheless, existing low-light image enhancement methods often fail to handle real low-light images with strong noise. To address this issue, We propose a deep unfolding network based on the robust Retinex model with an additional noise term. In particular, we design an optimization model with implicit priors and employ the proximal gradient descent (PGD) technique to alternately solve three iterative sub-problems of the optimization model in a data-driven manner. The proposed method combines the interpretability of model-based methods with the speed and strong fitting ability of learning-based methods. In addition, we collect an extreme low-light sRGB image dataset (E-LOL) containing noisy low/normal-light image pairs. Extensive experimental results demonstrate that our method outperforms state-of-the-art methods in enhancing noisy low-light images and obtains better-exposed illumination, richer colors and textures.

The low-light image enhancement has long been a critical need in practical applications. Existing methods require either paired or unpaired datasets. Zero-shot methods avoid the requirement of datasets, but they simply enhance the illumination component of the entire image with traditional gamma transformation, which causes color deviation and fails to process low-light images with uneven illumination. Also, these methods often do not take noise into account. We propose a zero-shot low-light image enhancement method. First, we decompose the image into illumination and reflectance according to the Retinex theory. The decomposed reflectance usually contains noise, so we estimate and remove the noise from the reflectance. To enhance the illumination, we design a hybrid illumination enhancement curve that combines gamma transformation and linear transformation. Also, we use a convolutional neural network to estimate the parametric maps in the curve to achieve pixel-level illumination enhancement, so our method can robustly process low-light images with uneven illumination. Extensive experiments demonstrate that our method outperforms recent state-of-the-art methods qualitatively and quantitatively.

When a facial image is blurred, it significantly affects high-level vision tasks such as face recognition. The purpose of facial image deblurring is to recover a clear image from a blurry input image, which can improve the recognition accuracy, etc. However, general deblurring methods do not perform well on facial images. Therefore, some face deblurring methods have been proposed to improve performance by adding semantic or structural information as specific priors according to the characteristics of the facial images. In this paper, we survey and summarize recently published methods for facial image deblurring, most of which are based on deep learning. First, we provide a brief introduction to the modeling of image blurring. Next, we summarize face deblurring methods into two categories: model-based methods and deep learning-based methods. Furthermore, we summarize the datasets, loss functions, and performance evaluation metrics commonly used in the neural network training process. We show the performance of classical methods on these datasets and metrics and provide a brief discussion on the differences between model-based and learning-based methods. Finally, we discuss the current challenges and possible future research directions.

Abstract Compared with natural images, geospatial images cover larger area and have more complex image contents. There are few algorithms for generating controllable geospatial images, and their results are of low quality. In response to this problem, this paper proposes Geospatial Style Generative Adversarial Network to generate controllable and high‐quality geospatial images. Current conditional generators suffer the mode collapse problem in geospatial field. The problem is addressed via a modified mode seeking regularization term with contrastive learning theory. Besides, the discriminator network architecture is modified to process global feature information and texture information of geospatial images. Feature loss in the generator is introduced to stabilize the training process and improve generated image quality. Comprehensive experiments are conducted on UC Merced Land Use Dataset, NWPU‐RESISC45 Dataset, and AID Dataset to evaluate all compared methods. Experiment results show our method outperforms state‐of‐the‐art models. Our method not only generates high‐quality and controllable geospatial images, but also enhances the discriminator to learn better representations.

High-fidelity reconstruction of dynamic fluids from sparse multiview RGB videos remains a formidable challenge, due to the complexity of the underlying physics as well as the severe occlusion and complex lighting in the captured data. Existing solutions either assume knowledge of obstacles and lighting, or only focus on simple fluid scenes without obstacles or complex lighting, and thus are unsuitable for real-world scenes with unknown lighting conditions or arbitrary obstacles. We present the first method to reconstruct dynamic fluid phenomena by leveraging the governing physics (ie, Navier -Stokes equations) in an end-to-end optimization from a mere set of sparse video frames without taking lighting conditions, geometry information, or boundary conditions as input. Our method provides a continuous spatio-temporal scene representation using neural networks as the ansatz of density and velocity solution functions for fluids as well as the radiance field for static objects. With a hybrid architecture that separates static and dynamic contents apart, fluid interactions with static obstacles are reconstructed for the first time without additional geometry input or human labeling. By augmenting time-varying neural radiance fields with physics-informed deep learning, our method benefits from the supervision of images and physical priors. Our progressively growing model with regularization further disentangles the density-color ambiguity in the radiance field, which allows for a more robust optimization from the given input of sparse views. A pretrained density-to-velocity fluid model is leveraged in addition as the data prior to avoid suboptimal velocity solutions which underestimate vorticity but trivially fulfill physical equations. Our method exhibits high-quality results with relaxed constraints and strong flexibility on a representative set of synthetic and real flow captures. Code and sample tests are at https://people.mpi-inf.mpg.de/~mchu/projects/PI-NeRF/.

Learning neural radiance fields of a scene has recently allowed realistic novel view synthesis of the scene, but they are limited to synthesize images under the original fixed lighting condition. Therefore, they are not flexible for the eagerly desired tasks like relighting, scene editing and scene composition. To tackle this problem, several recent methods propose to disentangle reflectance and illumination from the radiance field. These methods can cope with solid objects with opaque surfaces but participating media are neglected. Also, they take into account only direct illumination or at most one-bounce indirect illumination, thus suffer from energy loss due to ignoring the high-order indirect illumination. We propose to learn neural representations for participating media with a complete simulation of global illumination. We estimate direct illumination via ray tracing and compute indirect illumination with spherical harmonics. Our approach avoids computing the lengthy indirect bounces and does not suffer from energy loss. Our experiments on multiple scenes show that our approach achieves superior visual quality and numerical performance compared to state-of-the-art methods, and it can generalize to deal with solid objects with opaque surfaces as well.

Modern 3D printers are capable of printing large-size light-field displays at high-resolutions. However, optimizing such displays in full 3D volume for a given light-field imagery is still a challenging task. Existing light field displays optimize over relatively small resolutions using a few co-planar layers in a 2.5D fashion to keep the problem tractable. In this paper, we propose a novel end-to-end optimization approach that encodes input light field imagery as a continuous-space implicit representation in a neural network. This allows fabricating high-resolution, attenuation-based volumetric displays that exhibit the target light fields. In addition, we incorporate the physical constraints of the material to the optimization such that the result can be printed in practice. Our simulation experiments demonstrate that our approach brings significant visual quality improvement compared to the multilayer and uniform grid-based approaches. We validate our simulations with fabricated prototypes and demonstrate that our pipeline is flexible enough to allow fabrications of both planar and non-planar displays.

Abstract Importance sampling is one of the most widely used variance reduction strategies in Monte Carlo rendering. We propose a novel importance sampling technique that uses a neural network to learn how to sample from a desired density represented by a set of samples. Our approach considers an existing Monte Carlo rendering algorithm as a black box. During a scene‐dependent training phase, we learn to generate samples with a desired density in the primary sample space of the renderer using maximum likelihood estimation. We leverage a recent neural network architecture that was designed to represent real‐valued non‐volume preserving (“Real NVP”) transformations in high dimensional spaces. We use Real NVP to non‐linearly warp primary sample space and obtain desired densities. In addition, Real NVP efficiently computes the determinant of the Jacobian of the warp, which is required to implement the change of integration variables implied by the warp. A main advantage of our approach is that it is agnostic of underlying light transport effects, and can be combined with an existing rendering technique by treating it as a black box. We show that our approach leads to effective variance reduction in several practical scenarios.

Abstract Rendering with full lens model can offer images with photorealistic lens effects, but it leads to high computational costs. This paper proposes a novel camera lens model, NeuroLens, to emulate the imaging of real camera lenses through a data‐driven approach. The mapping of image formation in a camera lens is formulated as imaging regression functions (IRFs), which map input rays to output rays. IRFs are approximated with neural networks, which compactly represent the imaging properties and support parallel evaluation on a graphics processing unit (GPU). To effectively represent spatially varying imaging properties of a camera lens, the input space spanned by incident rays is subdivided into multiple subspaces and each subspace is fitted with a separate IRF. To further raise the evaluation accuracy, a set of neural networks is trained for each IRF and the output is calculated as the average output of the set. The effectiveness of the NeuroLens is demonstrated by fitting a wide range of real camera lenses. Experimental results show that it provides higher imaging accuracy in comparison to state‐of‐the‐art camera lens models, while maintaining the high efficiency for processing camera rays.