The goal of neural ISP is to convert the RAW data captured by camera sensors into display-referred sRGB images via an end-to-end neural network. Most methods are based on the regression, so they are prone to recovering images with fewer details due to the constraint of the regression loss. Some methods attempt to recover more details by resorting to GAN or diffusion model (DM) at the cost of hurting the signal-to-noise ratio and image structures. To circumvent such a trade-off, inspired by the recent progress in image deblurring, we propose a baseline that integrates the regression-based Transformer and DM. Specifically, DM is performed in a highly compact latent space to predict the latent prior that mimics the ground-truth latent prior compressed from the target sRGB image; then, the predicted latent prior is injected into the Transformer to guide the RAW-to-sRGB mapping process. Despite the superior performance achieved by this baseline, we conjecture that the latent prior predicted by DM is not good enough to mimic the ground-truth latent prior. Therefore, based on the baseline, we propose D&R in which the latent prior is diffused and refined with Transformers. The experimental results show that our D&R achieves state-of-the-art performance across multiple metrics on ZRR dataset and SID dataset, and also demonstrates the superiority over the baseline.

Autonomous driving requires an understanding of the static environment from sensor data. Learned Bird’s-Eye View (BEV) encoders are commonly used to fuse multiple inputs, and a vector decoder predicts a vectorized map representation from the latent BEV grid. However, traditional map construction models provide deterministic point estimates, failing to capture uncertainty and the inherent ambiguities of real-world environments, such as occlusions and missing lane markings. We propose MapDiffusion, a novel generative approach that leverages the diffusion paradigm to learn the full distribution of possible vectorized maps. Instead of predicting a single deterministic output from learned queries, MapDiffusion iteratively refines randomly initialized queries, conditioned on a BEV latent grid, to generate multiple plausible map samples. This allows aggregating samples to improve prediction accuracy and deriving uncertainty estimates that directly correlate with scene ambiguity. Extensive experiments on the nuScenes dataset demonstrate that MapDiffusion achieves state-of-the-art performance in online map construction, surpassing the baseline by 5% in single-sample performance. We further show that aggregating multiple samples consistently improves performance along the ROC curve, validating the benefit of distribution modeling. Additionally, our uncertainty estimates are significantly higher in occluded areas, reinforcing their value in identifying regions with ambiguous sensor input. By modeling the full map distribution, MapDiffusion enhances the robustness and reliability of online vectorized HD map construction, enabling uncertainty-aware decision-making for autonomous vehicles in complex environments.

Recently, the Neighboring Correlation-Aware (NeCA) noise model has achieved impressive performance on both noise synthesis and the downstream image denoising task. However, its design regarding noise-level prediction requires training NeCA separately for each camera type. To this end, by making use of an adaptive feature modulation technique, we improve NeCA’s noise-level prediction model to be unified for different camera types and thus enable a unified sRGB real noise synthesis method. We also find out that in the neigh-boring correlation network of NeCA, there is no mechanism to maintain the signal dependency of the synthesized noise. Therefore, we introduce another adaptive feature modulation technique to the neighboring correlation network to maintain the signal dependency of the noise.

This paper presents a photometric stereo method that works with unknown natural illumination without any calibration objects or initial guess of the target shape. To solve this challenging problem, we propose the use of an equivalent directional lighting model for small surface patches consisting of slowly varying normals, and solve each patch up to an arbitrary orthogonal ambiguity. We further build the patch connections by extracting consistent surface normal pairs via spatial overlaps among patches and intensity profiles. Guided by these connections, the local ambiguities are unified to a global orthogonal one through Markov Random Field optimization and rotation averaging. After applying the integrability constraint, our solution contains only a binary ambiguity, which could be easily removed. Experiments using both synthetic and real-world datasets show our method provides even comparable results to calibrated methods.

Radiometrically calibrating nonlinear images from Internet photo collections makes photometric analysis applicable not only to lab data but also to big image data in the wild. However, conventional calibration methods cannot be directly applied to such photo collections. This paper presents a method to jointly perform radiometric calibration for a set of nonlinear images in Internet photo collections. By incorporating the consistency of scene reflectance of corresponding pixels across nonlinear images, the proposed method first estimates radiometric response functions of all the nonlinear images up to a unique exponential ambiguity using a rank minimization framework. The ambiguity is then resolved using the linear edge color blending constraint. Quantitative evaluation using both synthetic and real-world data shows the effectiveness of the proposed method.

Classic photometric stereo is often extended to deal with real-world materials and work with unknown lighting conditions for practicability. To quantitatively evaluate non-Lambertian and uncalibrated photometric stereo, a photometric stereo image dataset containing objects of various shapes with complex reflectance properties and high-quality ground truth normals is still missing. In this paper, we introduce the 'DiLiGenT' dataset with calibrated Directional Lightings, objects of General reflectance with different shininess, and 'ground Truth' normals from high-precision laser scanning. We use our dataset to quantitatively evaluate state-of-the-art photometric stereo methods for general materials and unknown lighting conditions, selected from a newly proposed photometric stereo taxonomy emphasizing non-Lambertian and uncalibrated methods. The dataset and evaluation results are made publicly available, and we hope it can serve as a benchmark platform that inspires future research.

This paper presents a photometric stereo method that works with unknown natural illuminations without any calibration object. To solve this challenging problem, we propose the use of an equivalent directional lighting model for small surface patches consisting of slowly varying normals, and solve each patch up to an arbitrary rotation ambiguity. Our method connects the resulting patches and unifies the local ambiguities to a global rotation one through angular distance propagation defined over the whole surface. After applying the integrability constraint, our final solution contains only a binary ambiguity, which could be easily removed. Experiments using both synthetic and real-world datasets show our method provides even comparable results to calibrated methods.

Radiometrically calibrating the images from Internet photo collections brings photometric analysis from lab data to big image data in the wild, but conventional calibration methods cannot be directly applied to such image data. This paper presents a method to jointly perform radiometric calibration for a set of images in an Internet photo collection. By incorporating the consistency of scene reflectance for corresponding pixels in multiple images, the proposed method estimates radiometric response functions of all the images using a rank minimization framework. Our calibration aligns all response functions in an image set up to the same exponential ambiguity in a robust manner. Quantitative results using both synthetic and real data show the effectiveness of the proposed method.

Recent progress on photometric stereo extends the technique to deal with general materials and unknown illumination conditions. However, due to the lack of suitable benchmark data with ground truth shapes (normals), quantitative comparison and evaluation is difficult to achieve. In this paper, we first survey and categorize existing methods using a photometric stereo taxonomy emphasizing on non-Lambertian and uncalibrated methods. We then introduce the 'DiLiGenT' photometric stereo image dataset with calibrated Directional Lightings, objects of General reflectance, and 'ground Truth' shapes (normals). Based on our dataset, we quantitatively evaluate state-of-the-art photometric stereo methods for general non-Lambertian materials and unknown lightings to analyze their strengths and limitations.

Content-based image retrieval (CBIR) has been receiving intensive research attention for many applications. In order to provide the users with more precise retrieval results, relevance feedback (RF) methods have been incorporated into CBIR which take the user's feedbacks into account. In general, explicit RF methods demand too much user effort while implicit RF methods suffer from lower retrieval accuracy. As such, we propose a hybrid RF method, adaptive-weight hybrid relevance feedback (AHRF) for content-based image retrieval. AHRF integrates explicit user grading and implicit user browsing histories to build a user preference model. The model is refined iteratively and used to train a preference classifier for the users. Moreover, an adaptive-weight mechanism is proposed to achieve a personalized preference model. Our proposed method is tested on a subset of the Corel Database and the experimental results reveal that AHRF can achieve good retrieval precision with less user effort.