Recent research has focused on using convolutional neural networks (CNNs) as the backbones in two-view correspondence learning, demonstrating significant superiority over methods based on multilayer perceptrons. However, CNN backbones that are not tailored to specific tasks may fail to effectively aggregate global context and oversmooth dense motion fields in scenes with large disparity. To address these problems, we propose a novel network named SC-Net, which effectively integrates bilateral context from both spatial and channel perspectives. Specifically, we design an adaptive focused regularization module (AFR) to enhance the model's position-awareness and robustness against spurious motion samples, thereby facilitating the generation of a more accurate motion field. We then propose a bilateral field adjustment module (BFA) to refine the motion field by simultaneously modeling long-range relationships and facilitating interaction across spatial and channel dimensions. Finally, we recover the motion vectors from the refined field using a position-aware recovery module (PAR) that ensures consistency and precision. Extensive experiments demonstrate that SC-Net outperforms state-of-the-art methods in relative pose estimation and outlier removal tasks on YFCC100M and SUN3D datasets.

Accurate lung inflammation segmentation is essential for clinical decision-making, yet remains challenging due to the large variability in lesion appearance and location across different lung regions. Existing CNN-based models excel at local feature extraction, but they struggle to capture long-range dependencies and complex spatial relationships, such as those between the left and right lung lobes. Transformer-based models, while effective in modeling long-range dependencies, incur high computational costs and often fail to capture irregular anatomical relationships due to their reliance on Euclidean positional encodings. To overcome these challenges, we propose a novel Channel-Region Adaptive Unet (CRA-Unet) for accurate lung inflammation segmentation. Specifically, we design a Channel-Region Adaptive (CRA) layer that expands the recalibration process of the Squeeze-Excitation layer to include not only the channel dimension but also the height and width dimensions, enabling dynamical element-wise feature adjustment within different regions of interest across all three dimensions—channel, height, and width. Additionally, we propose a region-adaptive positional encoding strategy that learns dynamic weights for spatial locations, allowing the model to capture both intra-region and inter-region spatial relationships. Unlike traditional Euclidean positional encodings, which assume regular and grid-like spatial structures, our strategy can adapt to the irregular and asymmetric spatial relationships commonly found in anatomical structures such as the lungs. Experimental results on several datasets demonstrate that our CRA-Unet achieves state-of-the-art segmentation performance while maintaining high computational efficiency.

Accurately identifying correct correspondences in two images is a crucial task in computer vision. Current methods predominantly use PointCN blocks as feature extraction backbones and learn local-global consensus through a progressive learning strategy. However, such methods have two main drawbacks: First, PointCN blocks, composed of multilayer perceptrons and normalization layers, process spatial positions independently, leading to limited interaction between channel-wise and spatial-wise dimensions. Second, the progressive learning strategy primarily focuses on unidirectional transfer from local to global consensus, yet neglects the bidirectional interaction between local and global consensus. To address these issues, we propose the Channel-Spatial interaction and Bidirectional Consensus interaction-Based Network (CSBCNet), which contains three innovative blocks: Channel-Spatial Interaction (CSI), Local Consensus Mining (LCM), and Global Consensus-Aware Attention (GCAA). Specifically, CSI enhances interaction between channel-wise and spatial-wise dimensions through a dual-path attention mechanism, addressing the limited interaction caused by the independent processing of spatial positions in PointCN blocks. LCM extracts reliable local consensus by modeling geometric structures and spatial continuity within correspondences. GCAA captures global consensus by aggregating correspondences that are highly likely to be correct ones, and achieves bidirectional interaction between local and global consensus through cross attention. Experiments demonstrate our CSBCNet's superior performance in camera pose estimation and correspondence pruning. Notably, when the CSI block is applied to the existing OANet and MS2DGNet networks, it achieves significant performance improvements of 10.27% and 7.5%, respectively, on the mAP5° metric on the camera pose estimation task.

Summary Feature extraction is a fundamental step in the feature matching task. A lot of studies are devoted to feature extraction. Recent researches propose to extract features by pre‐trained neural networks, and the output is used for feature matching. However, the quality and the quantity of the features extracted by these methods are difficult to meet the requirements for the practical applications. In this article, we propose a two‐stage object‐aware‐based feature matching method. Specifically, the proposed object‐aware block predicts a weighted feature map through a mask predictor and a prefeature extractor, so that the subsequent feature extractor pays more attention to the key regions by using the weighted feature map. In addition, we introduce a state‐of‐the‐art model estimation algorithm to align image pair as the input of the object‐aware block. Furthermore, our method also employs an advanced outlier removal algorithm to further improve matching quality. Experimental results show that our object‐aware‐based feature matching method improves the performance of feature matching compared with several state‐of‐the‐art methods.

Object detection in scenes captured by unmanned aerial vehicles (UAV) is an active research area. However, the performance and efficiency of current small object detection models for UAV images are far from reaching the desired level. The inherent limitations of the features of the small objects themselves and the inconsistency of the contextual information in the feature maps lead to a degradation of the final detection performance. In this letter, to improve the performance of UAV image small object detection, we propose a multi-scale feature optimization network, named MFO-Net. We have designed three crucial modules: feature optimization fusion (FOF) module, multi-scale localized feature aggregation (MLFA) module, and feature enhancement (FE) module. FOF module enhances the fusion of features with inconsistent contexts at different levels by learning pixel-wise displacement, facilitating more effective feature fusion, which further helps focus on and capture critical information about small objects. MLFA module aggregates richer contextual information through multi-branch stripe convolution blocks, while the FE module extracts richer gradient flow information, suppresses incompatible information, and enhances feature representation capability. We conduct extensive experiments on the challenging VisDrone2019 dataset and compare the results against those obtained from the state-of-the-art methods. The experimental results show that MFO-Net performs better than other detectors. Specifically, MFO-Net achieves the best performance with 22.3% AP, 38.9% AP50, and 22.5% AP75on VisDrone2019. Code: https://github.com/Lanziyang121/MFO-Net.

Motion segmentation is an essential task in artificial intelligence and computer vision. However, scene motion in real-world intelligent systems usually integrates multiple types of models, so specifying only one type of basic model may lead to the failure of scene-motion segmentation tasks. In this paper, we propose a novel and efficient heterogeneous model-fitting-based motion segmentation method (HMFMS) to accurately segment moving objects. HMFMS includes a new co-attention-induced heterogeneous model construction algorithm (HMC), an adaptive heterogeneous model refinement algorithm (HMR), and a heterogeneous model segmentation algorithm (HMS). First, we propose HMC to generate high-quality accumulated correlation matrices, by evaluating the quality of heterogeneous model hypotheses, based on the density estimation technique. Next, we propose HMR to construct sparse affinity matrices from the accumulated correlation matrices by applying information theory, effectively suppressing the values of correlations between different objects. Finally, we fuse the sparse affinity matrices and perform motion segmentation by using HMS, to obtain more accurate segmentation results. Experimental results show that HMFMS obtains superior performance on four challenging datasets (i.e., Hopkins155, Hopkins12, MTPV62 and KT3DMoSeg), compared with several subspace-based and model-fitting-based motion segmentation methods. More remarkably, HMFMS outperforms the state-of-the-art MCMS method by 57.1% and 1.8 times in terms of accuracy and computational efficiency on the representative KT3DMoSeg, respectively.

The success of most robust model estimation methods heavily relies on their used data sampling algorithms. This paper proposes a novel sampling algorithm, called Guided Sampling by Neighborhood Information and Matching Scores (NIMS), to efficiently sample promising hypotheses for fitting multi-structure data. Specifically, NIMS follows a specific sampling process. First, the proposed NIMS randomly selects a data point. Then, NIMS selects the neighbors of the selected data by using the neighborhood information to remove most of the outlier neighbors. Finally, NIMS samples a data subset using matching scores from the selected neighbors, which encourages NIMS to sample inliers from the selected neighbors. Experimental results on the publicly availableAdelaideRMFdataset demonstrate that the proposed NIMS outperforms several state-of-the-artsampling algorithms.

Robust model fitting is a critical technique for artificial intelligence. The performance of most robust model fitting techniques heavily depends on the use of sampling algorithms. In this paper, we propose an efficient guided sampling algorithm for multi-structure data by using the neighborhood consensus and the residual sorting. Specifically, a Neighborhood Consensus based Strategy (NCS) is first proposed to select the first datum (i.e., seed datum) of a minimal subset, and then a Residual Sorting based Strategy (RSS) samples the rest data of the minimal subset based on the seed datum. This strategy effectively combines the benefits of neighborhood consensus and residual sorting, where neighborhood consensus can judge whether a selected data point is an inlier, and residual sorting encourages this strategy to select data points from the same structure of the first selected data point. Moreover, to achieve better fitting performance, the Markov Chain Monte Carlo process is used to combine NCS with the random selection strategy to select the seed datum, and an appropriate size is set to the initial block of randomly sampled hypotheses for RSS. Experimental results on three vision tasks (e.g., two-view motion segmentation and 3D motion segmentation) demonstrate that the proposed algorithm achieves superior performance to several state-of-the-art sampling algorithms.

Establishing superior-quality correspondences in an image pair is pivotal to many subsequent computer vision tasks. Using Euclidean distance between correspondences to find neighbors and extract local information is a common strategy in previous works. However, most such works ignore similar sparse semantics information between two given images and cannot capture local topology among correspondences well. Therefore, to deal with the above problems, Multiple Sparse Semantics Dynamic Graph Network (MS2DG-Net) is proposed, in this paper, to predict probabilities of correspondences as inliers and recover camera poses. MS2 DG-Net dynamically builds sparse semantics graphs based on sparse semantics similarity between two given images, to capture local topology among correspondences, while maintaining permutation-equivariant. Extensive experiments prove that MS2 DG-Net outperforms state-of-the-art methods in outlier removal and camera pose estimation tasks on the public datasets with heavy outliers. Source code:https://github.com/changcaiyang/MS2DG-Net

In this letter, we propose a remote sensing image matching method that is simple yet efficient to deal with different deformations. Inspired by the region growing strategy used in image segmentation, we integrate the motion consistency into the general region growing pipeline from a novel perspective. Specifically, we first obtain a subset with a high ratio inlier as the seed correspondence set. Then, to find more reliable correspondences, we formulate the motion consistency into the correspondence growing criterion, which is general to be suitable to many remote sensing applications. Extensive experimental results on the public available remote sensing data set show that our method achieves the best performance compared with state-of-the-art methods.

In this paper, a new robust model fitting method is proposed to efficiently segment multistructure data even when they are heavily contaminated by outliers. The proposed method is composed of three steps: first, a conventional greedy search strategy is employed to generate (initial) model hypotheses based on the sequential "fit-and-remove" procedure because of its computational efficiency. Second, to efficiently generate accurate model hypotheses close to the true models, a novel global greedy search strategy initially samples from the inliers of the obtained model hypotheses and samples subsequent data subsets from the whole input data. Third, mutual information theory is applied to fuse the model hypotheses of the same model instance. The conventional greedy search strategy is used to generate model hypotheses for the remaining model instances, if the number of retained model hypotheses is less than that of the true model instances after fusion. The second and the third steps are performed iteratively until an adequate solution is obtained. Experimental results demonstrate the effectiveness and efficiency of the proposed method for model fitting.

The performance of many robust model fitting techniques is largely dependent on the quality of the generated hypotheses. In this paper, we propose a novel guided sampling method, called accelerated guided sampling (AGS), to efficiently generate the accurate hypotheses for multistructure model fitting. Based on the observations that residual sorting can effectively reveal the data relationship (i.e., determine whether two data points belong to the same structure), and keypoint matching scores can be used to distinguish inliers from gross outliers, AGS effectively combines the benefits of residual sorting and keypoint matching scores to efficiently generate accurate hypotheses via information theoretic principles. Moreover, we reduce the computational cost of residual sorting in AGS by designing a new residual sorting strategy, which only sorts the top-ranked residuals of input data, rather than all input data. Experimental results demonstrate the effectiveness of the proposed method in computer vision tasks, such as homography matrix and fundamental matrix estimation.

Motion segmentation is an important task for intelligent transportation systems. In this paper, inspired by the fact that a feature point trajectory can be sparsely represented as a combination of several feature point trajectories that share coherent transformations, an efficient and effective motion segmentation method with a sparsity constraint is proposed. Specifically, we first propose an accumulated scheme to efficiently integrate motion information from all the frames of a video sequence to construct a correlation matrix. Then, a sparse affinity matrix is built on the correlation matrix by using information-theoretic principles, where the nonzero elements in the same row of the sparse affinity matrix correspond to the feature point trajectories more likely belonging to the same motion. Thereafter, a segment and merge procedure is proposed to effectively estimate the number of motions via the sparse affinity matrix. Finally, by applying spectral clustering on the sparse affinity matrix, different motions in the video sequence are accurately segmented based on the estimated number of motions. Experimental results on theHopkins 155and the62-clipdatasets demonstrate that the proposed method achieves superior performance compared with several state-of-the-art methods.

Hypothesis generation is crucial to many robust model fitting methods. In this paper, we propose an effective hypothesis generation method by adopting conditional sampling with local constraints. We choose data to generate hypotheses according to sampling weights, which are computed according to ordered residual indices. To sample a minimal subset, we randomly choose a seed datum, compute sampling weights of all data with regard to the seed datum, search the neighborhood set of the seed datum by using the sampling weights, and then sample the remaining data of the minimal subset from the neighborhood set. It has two advantages to consider the neighboring information in guided sampling: It raises the probability of generating all-inlier minimal subsets and it reduces the computational loads in hypotheses generation. The proposed method shows good performance in fundamental matrix estimation using real image pairs.