Dark optical flow estimation aims to predict pixel-wise displacement between consecutive noisy dark frames. Existing methods primarily focus on enhancing feature-specific representations before cross-image matching, with few attention devoted to the inherent dark degradation during flow decoding for achieving holistic motion understanding of a given dark scene. In this paper, we introduce the Position-surpassing Flow Estimator (PsFE), which integrates a global graph method into flow decoders to accentuate holistic motion discrimination and robustness. In detail, we incorporate a graph-based motion reconstruction into the decoding paradigm to adaptive aggregate motion-rich feature channels and suppress degraded ones from a more global view. This characteristic suppression retains the graph structure, which is a robust characteristic in the dark. To accurately encode long-range pixel connections, PsFE employs a novel masked global encoder to capture top-kimportant features by using a sparse masking strategy and dynamic inductive modulation that suppresses noise and interference that only exist under dark conditions. Experiments on challenging FCDN and VBOF benchmarks demonstrate the effectiveness of our PsFE with superior performance over advanced methods.

Due to the loss of 3D information, accurate and robust 2D image feature matching remains challenging for many computer vision applications. This paper introduces a 2.5D feature that uses the disparity value from the light field Fourier disparity layer (FDL) as a rough proxy of scene depth. Without explicit depth estimation, a parameterized depth-degraded projection is proposed to construct the geometric transformation of paired features between two light fields. Then, we propose a parameterized learning solution to calculate the depth-degraded projection. This solution estimates a global constant fundamental matrix, a variable disparity-guided translation vector, and a depth compensation term using a very simple network. Although the 0.5D relative disparity provided by the FDL does not represent precise depth, it can also significantly reduce the depth ambiguity in feature matching. Therefore, the proposed solution achieves accurate feature-matching results by minimizing the sum of reprojection errors across all matching candidates. On the public light field feature-matching dataset, the proposed solution outperforms existing 2D image feature-matching solutions and light field feature-matching algorithms in terms of matching accuracy and robustness. The code is available online.

Low-light image enhancement methods based on diffusion models, though effective in improving image quality, often overrely on noise sensitivity and neglect the reconstruction deviations due to the naive up- and down-sampling operations. To address this issue, we propose a novel diffusion model, MWT-Diff, which utilizes multi-layer wavelet transforms to replace up-and down-sampling based on convolutions for extracting high-order features of different scales while mitigating representation degradations. Specifically, MWT-Diff is based on the U-Net architecture; it encodes four local features after the frequency-based down-sampling at each layer and fuses the enhanced four components during the up-sampling process. Additionally, we incorporate global refinement branches to mitigate information loss and employ efficient soft gate aggregation for feature fusion and reconstruction. Extensive quantitative and qualitative experiments demonstrate that our model achieves state-of-the-art performance on benchmark datasets. Code is available at: https://github.com/lalalulao/MWT-Diff.

False correspondence removal is a persistent challenge in image feature-matching-based applications, especially in complex scenes. Traditional methods often rely on the consistency assumption to model the motion of correct correspondences, which neglects non-consistent correct correspondences, resulting in suboptimal performance. In this paper, we introduce the NCNet, a novel network designed to address this limitation by fitting the motion of both consistent and non-consistent correct correspondences using the learnable frequency response function. Unlike conventional approaches that focus on pixel-based 2D movements, NCNet utilizes the Motion Fitting Residual Module to estimate the high-dimensional motion, capturing the intricate 3D geometrical variations of matching pairs. To further enhance performance, we propose a loss function that balances the performance between the two types of correct correspondences. Extensive experiments demonstrate that NCNet significantly outperforms existing methods in terms of precision for false correspondence removal. The implementation of our approach is publicly available at: https://github.com/Livsdjo/NCNet-Code.

Accurate activation and optimization of autofocus (AF) functions are essential for capturing high-quality images and minimizing camera response time. Traditional contrast detection autofocus (CDAF) methods suffer from a trade-off between accuracy and robustness, while learning-based methods often incur high spatio-temporal computational costs. To address these issues, we propose a lightweight focus discriminative network (FDNet) tailored for AF tasks. Built upon the ShuffleNet V2 backbone, FDNet leverages a genetic algorithm optimization (GAO) strategy to automatically search for efficient network structures, and incorporates coordinate attention (CA) and multi-scale feature fusion (MFF) modules to enhance spatial, directional, and contextual feature extraction. A dedicated focus stack dataset is constructed with high-quality annotations to support training and evaluation. Experimental results show that FDNet outperforms mainstream methods by up to 4% in classification accuracy while requiring only 0.2 GFLOPs, 0.5 M parameters, a model size of 2.1 MB, and an inference time of 0.06 s, achieving a superior balance between performance and efficiency. Ablation studies further confirm the effectiveness of the GAO, CA, and MFF components in improving the accuracy and robustness of focus feature classification.

When restoring low-light images, most methods largely overlook the ambiguity due to dark noise and lack discrimination for region and shape representations, resulting in invalid feature enhancement. In this work, we propose a physically explainable and prior guidance model for low-light image enhancement, termed Dual-Conditional Guidance Sparse Diffusion (DCGSD). Specifically, we introduce an elaborately designed Luminance Structure Guidance Head, which can be easily plugged into the existing diffusion model to emphasize the value of the luminance and structural representation. Furthermore, for reliable noise analysis, we provide a novel Sparse Attention Enhancement Module that is adaptively empowered to exploit the most useful region-to-region dependencies. This dynamic selection makes the diffusion process from dense to sparse, thus improving the efficiency of the reasoning noise distributions. To avoid noise amplification, we further present a Skip Calibration Module, which can be used to refine the local neighborhood that contains noisy and structural information. Extensive experiments have been performed to verify the superiority of the proposed method. DCGSD shows that leveraging dual-conditional guidance can support the diffusion model to produce sharper and more realistic results.

Infrared and visible image fusion is currently an important research direction in the field of multimodal image fusion, which aims to utilize the complementary information between infrared images and visible images to generate a new image containing richer information. In recent years, many deep learning-based methods for infrared and visible image fusion have emerged.However, most of these approaches ignore the importance of semantic information in image fusion, resulting in the generation of fused images that do not perform well enough in human visual perception and advanced visual tasks.To address this problem, we propose a semantic prior knowledge-driven infrared and visible image fusion method. The method utilizes a pre-trained semantic segmentation model to acquire semantic information of infrared and visible images, and drives the fusion process of infrared and visible images through semantic feature perception module and semantic feature embedding module.Meanwhile, we divide the fused image into each category block and consider them as components, and utilize the regional semantic adversarial loss to enhance the adversarial network generation ability in different regions, thus improving the quality of the fused image.Through extensive experiments on widely used datasets, the results show that our approach outperforms current leading algorithms in both human eye visualization and advanced visual tasks.

Accurately computing optical flow in low-contrast and noisy dark images is challenging, especially when contour information is degraded or difficult to extract. This paper proposes CEDFlow, a latent space contour enhancement for estimating optical flow in dark environments. By leveraging spatial frequency feature decomposition, CEDFlow effectively encodes local and global motion features. Importantly, we introduce the 2nd-order Gaussian difference operation to select salient contour features in the latent space precisely. It is specifically designed for large-scale contour components essential in dark optical flow estimation. Experimental results on the FCDN and VBOF datasets demonstrate that CEDFlow outperforms state-of-the-art methods in terms of the EPE index and produces more accurate and robust flow estimation. Our code is available at: https://github.com/xautstuzfy.

Infrared and visible image fusion is an important multimodal image processing task that aims to enhance computer vision performance by effectively fusing infrared and visible images. Although in recent years, many deep learning-based methods for infrared and visible image fusion have emerged. Howeve, most of these methods ignore the important role of semantic information in image fusion. Therefore, this paper proposes a semantic priori guided infrared and visible image fusion network called SPGFusion. It uses an adversarial generative network framework based on semantic priors to guide the infrared and visible image fusion process by combining a semantic feature-aware module and semantic generative adversarial loss. Experimental results demonstrate that the SPG-Fusion method yields more visually appealing fusion results and outperform state-of-the-art image fusion algorithms in visual quality and quantitative evaluation. The source code is available at https://github.com/tianzhiya/SPGFusion.

Photographs taken in low-light conditions have a low signal-to-noise ratio and impaired visual quality. We observe that low-light images exhibit a lower signal-to-noise ratio, resulting in a mixture of fine details, textures, and noise, making it challenging to reconstruct small-scale textures in the image. Inspired by this observation, we propose a SNR-guided CNN-Transformer network for high frequency restoration during low light image enhancement. The proposed method first decomposes image into high-frequency and low frequency components by image decomposition module. The low-frequency image is processed by a trainable Low Frequency SNR Perception (LFSP) module, resulting in excellent denoising performance and generating SNR-enhanced images with clearer edge contours. Guided by the low-frequency SNR feature maps, the details and textures of the high-frequency components are enhanced using a combination of transformer networks and convolutional networks, thereby compensating the detail distortions in the high frequency components of the image. The subjective and objective experiments demonstrate that our proposed method outperforms existing approaches in terms of detail and structure preservation.

Dark Image Enhancement (DIE) aims to improve contrast and restore details for captured images under low illumination. Currently, traditional Transformer methods have achieved significant performance in the DIE problem; however, all-pairs correlation computation is redundant in learning key properties and restoring high-order representations. To alleviate this problem, we introduce a Channel-wise Guidance Sparse Transformer framework, namely CGSformer, which not only adaptively selects the key channel-wise representations through a threshold operator, but also keeps the most useful self-attention values for feature restoration guided by the selected information. Besides, we introduce a Bidirectional Gate Feed-Forward (BGFF) network to aggregate features to better facilitate high-quality image reconstruction. The experiments are conducted on representative datasets, showing that our CGSformer consistently achieves state-of-the-art performance on widely used benchmarks.

In the low light image enhancement, single exposure images contains a limited dynamic range, which hinders the restoration of contrast and texture. To address these problems, we propose a multi exposure generation and fusion method (MEGF) which simulates multi exposure images and perform feature fusion and enhancement on these images. First, we propose a Multi-Exposure Generation (MEG) method, which constructs the Gaussian Distribution for each exposure level based on multi exposure datasets. MEG can generate images with different exposure levels based on the constructed distribution. Then, the Perceptual Importance based Multi-Exposure Feature Enhancement (PIMEFE) block is developed to fuse the feature of generated multi exposure images using VGG-16 network. Before fusion, the generated images are input to Multi Scale Recursive Feature Enhancement (MSRFE) blocks and obtain the denoised and enhanced features. Finally, the fused feature are input to Curve Adjustment (CA) block for fine tuning and provide the color enhancement on fusion features. We propose the Multiple Exposure Recursive Fusion (MERF) block which estimates the adjusting factors for CA block. Experimental results demonstrate that our method outperforms other techniques in both subjective and objective evaluations on real and synthetic datasets.

In low-light images, contrast and brightness are corrupted, making it difficult to accurately percept detail and edge information with the naked eye. Because of the development of multi sensor imaging, RGB-IR image fusion can enhance the imaging quality in low light conditions. However, the existing fusion algorithms have insufficient enhancement, distorted detail and low contrast, which make it difficult to generate high-quality fusion results. In this paper, we propose a Transformer-CNN image fusion method which considers global-local features fusion for low light image enhancement. The ConvGRU module is developed to alternatively select the global and local features with Transformer and CNN network. To effectively improve the network performance, a learnable weight adaptive loss function is proposed to adjust the weight of loss functions during training. Numerous experiments prove that our method can enrich fusion image information, improve image contrast and edge in low-light scenes compared to state of the art methods.

False correspondence removal remains a challenge for many feature-matching-based applications. This paper proposes a solution that revisits the local semantic context via position-attentive learning. First, a cross-divisional module is introduced to extract semantic features from image-patch pairs. Then, through a position-attentive mechanism, the parametric positions and extracted semantic features are jointly utilized to compute the probabilities of a set of putative correspondences. The proposed approach is evaluated on several indoor and outdoor challenging datasets, containing up to 70%+ false correspondences. In most cases, the solution outperforms existing algorithms in terms of matching precision and F1-score. Furthermore, an ablation study indicates that revisiting the semantic context improves precision by nearly 5%. The code is available at the link below1.

In the low-light conditions, images are corrupted by low contrast and severe noise, but event cameras can capture event streams with clear edge structures. Therefore, we propose an Event-Guided Attention Network for Low-light Image Enhancement (EGAN) using a dual branch Network and recover clear structure with the guide of events. To overcome the lack of paired training datasets, we first synthesize the dataset containing low-light event streams, low-light images, and the ground truth (GT) normal-light images. Then, we develop an end-to-end dual branch network consisting of a Image Enhancement Branch (IEB) and a Gradient Reconstruction Branch (GRB). The GRB branch reconstructs image gradients using events, and the IEB enhances low-light images using reconstructed gradients. Moreover, we develops the Attention based Event-Image Feature Fusion Module (AEIFFM) which selectively fuses the event and low-light image features using the spatial and channel attention mechanism, and the fused features are concatenated into the IEB and GRB, which respectively generate the enhanced images with clear structure and more accurate gradient images. Extensive experiments on synthetic and real datasets demonstrate that the proposed EGAN produces visually more appealing enhancement images, and achieves a good performance in structure preservation and denoising over state-of-the-arts.

Abstract In order to generate a denser focal stack, a cooperative generative adversarial network is proposed to learn the refocusing ability from light field imaging. To keep the axial continuity, the proposed framework is designed to learn features of a focal stack in both axial directions. Different from the classic generative adversarial network, our generative module consists of a forward prediction sub‐network and a backward prediction sub‐network, taking the forward‐and‐backward focal stacks as their inputs, respectively. The bi‐directional predictions are then fused by a weighting process, which is guided by an adversarial module. The proposed network is trained on light field focal stacks conducted via digital refocusing. Without loss of the refocus continuity, one can axial super‐resolve a focal stack by using the trained model. The effectiveness of the proposed algorithm on different types of focal stacks produced by both light fields and traditional camera shootings is validated. The experimental results indicate that the refocus variation of a focal stack can be well learned and predicted without a complete light field. Therefore, the proposed algorithm outperforms the traditional digital refocusing in terms of run‐time.

Many computer vision applications rely on feature detection and description, hence the need for computationally efficient and robust 4D light field (LF) feature detectors and descriptors. In this paper, we propose a novel light field feature descriptor based on the Fourier disparity layer representation, for light field imaging applications. After the Harris feature detection in a scale-disparity space, the proposed feature descriptor is then extracted using a circular neighborhood rather than a square neighborhood. It is shown to yield more accurate feature matching, compared with the LiFF LF feature, with a lower computational complexity. In order to evaluate the feature matching performance with the proposed descriptor, we generated a synthetic stereo LF dataset with ground truth matching points. Experimental results with synthetic and real-world dataset show that our solution outperforms existing methods in terms of both feature detection robustness and feature matching accuracy.

Many computer vision applications rely on feature matching, hence the need for computationally efficient and robust 4D light field (LF) feature detectors and descriptors for applications using this imaging modality. In this paper, we propose a novel LF feature extraction method in the scale-disparity space, based on a Fourier disparity layer representation. The proposed feature extraction takes advantage of both the Harris feature detector and SIFT descriptor, and is shown to yield more accurate feature matching, compared with the LiFF light field feature with low computational complexity. In order to evaluate the feature matching performance with the proposed descriptor, we generated synthetic LF datasets with ground truth matching points. Experimental results with synthetic and real datasets show that, our solution outperforms existing methods in terms of both feature detection robustness and feature matching accuracy.

In the past decade, light field imaging has greatly extended the imaging capabilities of traditional photography. However, the applications of light field imaging are limited by the aliasing artifacts due to the plenoptic sampling trade-off between angular and spatial domains. We propose to use a coded aperture light field camera instead of the traditional one, which can get more angular information without losing spatial resolution. To that end, we exploit a theoretical model to explain the relationship between light field and the raw data captured by the sensor. Then, we design a mask to code the rays using compressive sensing. Last, the sparse characteristic of light field in gradient domain and the corresponding optimization methods are utilized to reconstruct the high angular resolution light field. Experimental results on synthetic data and real data demonstrate that our system can obtain high angular resolution light field by producing a low-aliasing refocused image and high PSNR multi-view images.

When using plenoptic camera for digital refocusing, angular undersampling can cause severe (angular) aliasing artifacts. Previous approaches have focused on avoiding aliasing by pre-processing the acquired light field via prefiltering, demosaicing, reparameterization, and so on. In this paper, we present a different solution that first detects and then removes angular aliasing at the light field refocusing stage. Different from previous frequency domain aliasing analysis, we carry out a spatial domain analysis to reveal whether the angular aliasing would occur and uncover where in the image it would occur. The spatial analysis also facilitates easy separation of the aliasing versus non-aliasing regions and angular aliasing removal. Experiments on both synthetic scene and real light field data sets (camera array and Lytro camera) demonstrate that our approach has a number of advantages over the classical prefiltering and depth-dependent light field rendering techniques.

When using plenoptic camera for digital refocusing, angular undersampling can cause severe (angular) aliasing artifacts. Previous approaches have focused on avoiding aliasing by pre-processing the acquired light field via prefiltering, demosaicing, reparameterization, etc. In this paper, we present a different solution that first detects and then removes aliasing at the light field refocusing stage. Different from previous frequency domain aliasing analysis, we carry out a spatial domain analysis to reveal whether the aliasing would occur and uncover where in the image it would occur. The spatial analysis also facilitates easy separation of the aliasing vs. non-aliasing regions and aliasing removal. Experiments on both synthetic scene and real light field camera array data sets demonstrate that our approach has a number of advantages over the classical prefiltering and depth-dependent light field rendering techniques.

Foreground occlusion is a significant challenge in 3D reconstruction. In the paper, we first characterize the differences between multiview reconstruction with and without foreground occlusion. Considering both scene depth and appearance are unknown, we propose a generalized model for scene reconstruction. Then, we propose an iterative reconstruction approach in the global optimization framework, which is well performed on the camera array system. Even when all views are partially occluded, our approach can recover accurate depth map as well as scene appearance. Experimental results have indicated that our approach is more robust to foreground occlusions and outperforms state-of-the-art approaches.