Image rectangling involves filling in the blanks created during image stitching through deformation techniques. However, existing methods still struggle with incomplete filling and distortion of content, ultimately affecting the overall visual impression and potentially hindering subsequent tasks such as recognition. In this work, we design a pixel-wise deformation framework that utilizes explicit edge guidance to maintain consistency of texture and structure, yielding rectangular images with natural structure. Specifically, we decouple motion into region-level and pixel-level components through uniform mesh warping and pixel-wise deformation to precisely rearrange the spatial distribution of all pixels. Uniform deformation preserves local structure within divided patches, while pixel-wise motion coordinates the consistency between patches. Their combination provides robust and accurate pixel-wise offsets for structure-preserved rectangling. To further bolster the consistency of structure and texture, we leverage edge information to establish structural constraints and design an edge-guided enhancement module to aid in restoring fine texture details. Additionally, stitched images encompass both meaningful content and blank spaces, we innovatively incorporate a mask predictor, which acts as a guiding beacon, directing the network's attention solely towards content-rich regions to facilitate precise pixel-wise motion estimation. Experimental results demonstrate that our approach achieves state-of-the-art performance in rectifying irregular boundaries while contributing to downstream visual perception tasks.

Homography describes the mapping relations of the same plane across views. In scenarios with multiple planes, single homography estimation aims to obtain the optimal solution generated by the largest consistent plane to obey the coplanar constraints. However, existing methods typically consider all planes equally, neglecting the negative impact of regions that differ significantly from the largest approximate planar areas (dominant plane). In this work, we propose a depth-guided dominant plane perception network to achieve unsupervised homography estimation with additional attention on the dominant plane. Specifically, we leverage the depth-wise prior to adaptively detecting the approximate dominant plane, invoking essential scene structures for unsupervised homography estimation. Then, we enhance the corresponding features of the dominant plane and explore their correlations through a specially designed perceptual module. Finally, we employ dominant plane perception on multi-scale features progressively to estimate the homography in a coarse-to-fine manner. Extensive experiments on a large parallax dataset demonstrate that our method improves the alignment performance by 10.29%, yielding more accurate alignment than previous competitive methods.

Homography estimation plays a pivotal role in aligning image pairs across multiple viewpoints. Existing methods focus mainly on texture alignment, whereas overlooking the influence of geometric structures, thereby resulting in inaccurate homography estimation. In this paper, we propose a novel recurrent homography estimation framework with joint edge detection learning. We find that edge detection explores extra anchors for homography estimation, and meanwhile homography provides complementary information of cross views for edge detection refinement. Unlike traditional edge detection applied to individual images, our approach establishes structural consistency constraints to reinforce mutual edges while suppressing unreliable structures. Specifically, the detected edges guide and enhance the texture features through a specifically designed edge-aware fusion module. Ultimately, we recurrently compute the correlation of fusion features from small to large scales for homography regression. Our experimental results demonstrate that the proposed method reduces the matching error by 41.7% than state-of-the-art methods. Furthermore, our network excels in detecting edges with extensive details even under dramatic perspective changes. Code is available at https://github.com/edmandzhao/edge-detection-for-RHE.

Homography estimation aligns image pairs in cross-views, which is a crucial and fundamental computer vision problem. Existing methods only consider correspondences of texture features for homography estimation, leading to unpleasant artifacts and misalignments introduced by mismatches, especially for low-texture image pairs. In contrast to others, we introduce intuitive structural information as an additional clue that is more sensitive to human vision and low-texture scenarios. In this paper, we propose an edge-aware unsupervised progressive network that couples texture and edge correlation to comprehensively explore potential matching features for homography estimation. To explore robust edge and texture features, we employ a multiscale network to capture feature pyramids with different receptive fields. Then, we design an edge-aware correlation module tailored for homography regression, which plugs in multiscale features to capture accurate correlation maps. Specifically, the edge-aware correlation module leverages the feature-selecting strategy for edge features to capture discriminative matching edges and further guides the texture correlation unit to focus on correctly matched textures. Finally, we leverage multiscale edge-aware correlation maps to predict homography progressively from coarse to fine. Experimental results demonstrate that our proposed method improves PSNR by 11.09% on the real large parallax dataset and reduces matching error by 32.04% on the synthetic COCO dataset, yielding more accurate alignment results than previous state-of-the-art methods.

Image stitching aims to align a pair of images in the same view. Generating precise alignment with natural structures is challenging for image stitching, as there is no wider field-of-view image as a reference, especially in non-coplanar practical scenarios. In this paper, we propose an unsupervised image stitching framework, breaking through the coplanar constraints in homography estimation, yielding accurate pixel-wise alignment under limited overlapping regions. First, we generate a global transformation by an iterative dense feature matching combined with an error control strategy to alleviate the difference introduced by large parallax. Second, we propose a pixel-wise warping network embedded within a large-scale feature extractor and a correlative feature enhancement module to explicitly learn correspondences between the inputs, and generate accurate pixel-level offsets upon novel constraints on both overlapping and non-overlapping regions. Notably, we leverage the pixel-level offsets in the overlapping area to guide the adjustment in the non-overlapping area upon content and structure consistency constraints, rendering a natural transition between two regions and distortions suppression over the entire stitched image. The proposed method achieves state-of-the-art performance that surpasses both traditional and deep learning approaches by a large margin. It also achieves the shortest execution time and has the best generalization ability on the traditional dataset.

Abstract Due to insufficient illumination in low‐light conditions, the brightness and contrast of the captured images are low, which affect the processing of other computer vision tasks. Low‐light enhancement is a challenging task that requires simultaneous processing of colour, brightness, contrast, artefacts and noise. To solve this problem, the authors apply the deep residual network to the low‐light enhancement task, and propose a hierarchical guided low‐light enhancement network. The key of this method is recombined hierarchical guided features through the feature aggregation module to realize low‐light enhancement. The network is based on the U‐Net network, and then hierarchically guided with the input pyramid branch in the encoding and decoding network. The input pyramid structure realizes multi‐level receptive fields and generates a hierarchical representation. The encoding and decoding structure concatenates the hierarchical features of the input pyramid and generates a set of hierarchical features. Finally, the feature aggregation module is used to fuse different features to achieve low‐light enhancement tasks. The effectiveness of the components is proved through ablation experiments. In addition, the authors are also evaluating on different data sets, and the experimental results show that the method proposed is superior to other methods in subjective and objective evaluation.

Low-light images have low brightness and contrast, which presents a huge obstacle to computer vision tasks. Low-light image enhancement is challenging because multiple factors (such as brightness, contrast, artifacts, and noise) must be considered simultaneously. In this study, we propose a neural network—a progressive-recursive image enhancement network (PRIEN)—to enhance low-light images. The main idea is to use a recursive unit, composed of a recursive layer and a residual block, to repeatedly unfold the input image for feature extraction. Unlike in previous methods, in the proposed study, we directly input low-light images into the dual attention model for global feature extraction. Next, we use a combination of recurrent layers and residual blocks for local feature extraction. Finally, we output the enhanced image. Furthermore, we input the global feature map of dual attention into each stage in a progressive way. In the local feature extraction module, a recurrent layer shares depth features across stages. In addition, we perform recursive operations on a single residual block, significantly reducing the number of parameters while ensuring good network performance. Although the network structure is simple, it can produce good results for a range of low-light conditions. We conducted experiments on widely adopted datasets. The results demonstrate the advantages of our method compared with other methods, from both qualitative and quantitative perspectives.

Improper functioning, or lack, of human cone cells leads to vision defects, making it impossible for affected persons to distinguish certain colors. Colorblind persons have color perception, but their ability to capture color information differs from that of normal people: colorblind and normal people perceive the same image differently. It is necessary to devise solutions to help persons with color blindness understand images and distinguish different colors. Most research on this subject is aimed at adjusting insensitive colors, enabling colorblind persons to better capture color information, but ignores the attention paid by colorblind persons to the salient areas of images. The areas of the image seen as salient by normal people generally differ from those seen by the colorblind. To provide the same saliency for colorblind persons and normal people, we propose a saliency-based image correction algorithm for color blindness. Adjusted colors in the adjusted image are harmonious and realistic, and the method is practical. Our experimental results show that this method effectively improves images, enabling the colorblind to see the same salient areas as normal people.