With the rapid development of intelligent transportation systems and the popularity of smart city infrastructure, Vehicle Re-ID technology has become an important research field. The vehicle Re-ID task faces an important challenge, which is the high similarity between different vehicles. Existing methods use additional detection or segmentation models to extract differentiated local features. However, these methods either rely on additional annotations or greatly increase the computational cost. Using attention mechanism to capture global and local features is crucial to solve the challenge of high similarity between classes in vehicle Re-ID tasks. In this paper, we propose LKA-ReID with large kernel attention. Specifically, the large kernel attention (LKA) utilizes the advantages of self-attention and also benefits from the advantages of convolution, which can extract the global and local features of the vehicle more comprehensively. We also introduce hybrid channel attention (HCA), which combines channel attention with spatial information, so that the model can better focus on channels and feature regions, and ignore background and other disturbing information. Experiments on VeRi-776 dataset demonstrated the effectiveness of LKA-ReID, with mAP reaches 86.65% and Rank-1 reaches 98.03%.

Monocular scene flow estimation has been a long-standing problem in computer vision. Methods based on the RAFT architecture are currently the mainstream approaches, while often overlooking the long-range dependencies in motion and texture features and fail to fully utilize the spatial information in texture features. In this paper, we consider that using Transformers introduces high computational complexity. Therefore, we propose the Mamba Motion Module based on State Space Models design, which first models long-range dependencies in motion and texture features and then fully leverages the spatial information in texture features to enhance the motion features, generating global motion features while maintaining low computational complexity. Additionally, texture features play a significant role in constraining motion boundaries. Therefore, we propose the Enhanced Texture Module, which predicts a set of channel weights from the global motion features to enrich the channel properties of the texture features and concatenates texture features with the global motion features along the channels, thereby improving scene flow accuracy. Experimental results show that our method achieves highly competitive results on the KITTI 2015 and Eigen Split datasets, increasing by 18.82% and 2.15% compared to the baseline, respectively.

Abstract Conventional optical flow estimation methods typically recover two‐dimensional motion from RGB image sequences. Recently, due to the rise and widespread use of spike cameras, learning optical flow from spiking cameras has become a hot topic in the field of two‐dimensional motion estimation. Although existing methods have been designed to learn optical flow by designing feature processing methods for spike streams, there is still insufficient consideration for flow field post‐processing, resulting in limited accuracy of optical flow estimation. To address this problem, an optical flow estimation method based on directional disassembly is proposed. Specifically, the estimated flow fields along the horizontal and vertical directions are disassembled and the motion vectors along the two directions are denoised separately to reduce the burden of post‐processing for complex two‐dimensional motion information. In addition, contextual information is introduced in the post‐processing so that the scene information can effectively contribute to the results of the flow post‐processing. Experimental results show that this proposed method is capable of achieving comparable performance on spike‐based public datasets.

Scene flow estimation from 4D radar sensors has become increasingly popular in recent years. In this paper, we propose a matching and refinement decoupling method to estimate scene flow from 4D radar point clouds. Since 4D radar point clouds are much sparser and noisier than LiDAR point clouds, it is challenging to effectively establish correspondences between two frames and properly refine flow fields in the 3D space. To address this issue, we present decoupled correlation fields and decoupled flow fields for scene flow estimation, named DMRFlow. On the one hand, we propose a position-velocity decoupled matching approach that decouples the positional features from the velocity features of two adjacent point clouds and matches them separately. On the other hand, we design a dynamic-static decoupled refinement approach that splits initial flow fields into two groups according to motion segmentation maps and refines them separately. By integrating the matching and refinement decoupling method, our DMRFlow is able to effectively reduce mutual interference between different features during the matching and refinement process. We evaluate the proposed approach on the View-of-Delft (VoD) dataset. Experimental results show that DMRFlow yields competitive performance in autonomous driving scenarios compared to recent 4D radar scene flow estimation methods.

Scene flow estimation from LiDAR sensors is a crucial task for dynamic environmental perception in autonomous driving scenarios. Recently, self-supervised approaches have gained attention for their ability to reduce the burden of point-wise annotation. Although existing methods have been able to generate initial flow fields by constructing point-to-point correspondences between adjacent frames of point clouds, the reliability of correlation extraction and initial motion measurement has not been adequately considered. To address this problem, we propose a novel deep neural network to estimate scene flow from LiDAR sensors. Unlike previous works, our approach incorporates a Statistical-based Correlation Compensation Module (SCCM) that leverages statistical features to capture more reasonable correspondences. Furthermore, we design a Holistic Correlation Compensation Module (HCCM) to capture the overall correspondence that reflects most of the rigid motion in dynamic environments. In addition, an Initial Motion Check Mechanism (IMCM) is introduced to calibrate the initial flow and provide more reliable motion priors for subsequent flow refinement. Extensive experimental results on public scene flow benchmarks show that the proposed approach achieves competitive performance in autonomous driving scenarios.

Recurrently refining the optical flow based on a single high-resolution feature demonstrates high performance. We exploit the strength of this strategy to build a novel architecture for the joint learning of optical flow and depth. Our pro-posed architecture is improved to work in the case of training on unlabeled data, which is extremely challenging. The loss is computed for the iterations carried out over a single high-resolution feature, where the reconstruction loss fails to optimize the accuracy particularity in occluded regions. Therefore, we propose to hierarchically refine the optical flow across multiple scales while feeding the rigid flow calculated from depth and camera pose to provide more refinement. We further propose a self-supervised patch-based similarity loss to be optimized with the reconstruction loss to improve accuracy in the occluded regions. Our proposed method demonstrates efficient performance on the KITTI 2015 dataset, with more improvement in the occluded regions.

Optical flow estimation from image sequences is a fundamental problem in computer vision. In recent years, some methods have utilized Transformer to model global dependencies and improve optical flow, achieving impressive performance. However, in these methods, Transformers typically treat two-dimensional image features as one-dimensional sequences. While position encoding partially mitigates the loss of position information between different feature patches, Transformer still lacks inherent biases for modeling local visual patterns and tend to overlook channel characteristics in image features. Therefore, this paper introduces a deformable large kernel attention module, combining the strengths of convolution and attention mechanisms, which can preserve feature channel adaptability while modeling global dependencies without compromising the two-dimensional structure of features, significantly enhancing optical flow estimation. Additionally, the introduced deformable mechanism allows the model to adapt appropriately to different data patterns. Experimental results demonstrate that our optical flow estimation method achieves competitive results on publicly available benchmarks such as Sintel and KITTI.

Scene flow describes the 3D motion in a scene. It can be modeled as a single task or as a composite of the auxiliary tasks of depth, camera motion, and optical flow estimation. Deep learning's emergence in recent years has broadened the horizons for new methodologies in estimating these tasks, either as separate tasks or as joint tasks to reconstruct the scene flow. The sequence of images that are either synthesized or captured by a camera is used as input for these methods, which face the challenge of dealing with various situations in images to provide the most accurate motion, such as image quality. Nowadays, images have been superseded by point clouds, which provide 3D information, thereby expediting and enhancing the estimated motion. In this paper, we dig deeply into scene flow estimation in the deep learning era. We provide a comprehensive overview of the important topics regarding both image-based and point-cloud-based methods. In addition, we cover the methodologies for each category, highlighting the network architecture. Furthermore, we provide a comparison between these methods in terms of performance and efficiency. Finally, we conclude this survey with insights and discussions on the open issues and future research directions.

Optical flow is the process of estimating motion in scenes. Each object in the scene has a homogeneous motion, i.e., moves in the same direction with the same velocity. Therefore, connecting the parts of an image globally provides an essential cue for learning accurate motion. Convolution-based methods estimate the motion features from the local regions, which miss this important cue. Recently, some methods used Transformer to model global dependencies to improve optical flow. However, Transformer suffers from excessive attention computations and still brings irrelevant parts into the region of interest. Therefore, we propose a deformable cross-attention for optical flow estimation, which provides two important advantages: connecting the parts of the image globally while deforming the attention to the objects’ shapes in the image and reducing the memory consumption. Our proposed method achieved competitive performance on Sintel and KITTI 2015 datasets in terms of accuracy and efficiency.

Scene flow estimation is the task of obtaining 3D motion from a dynamic scene. Due to the sparseness of point clouds, extracting features for a local group of points separately may result in different features that may all belong to the same object. This difference makes global correlation prone to producing an unacceptable flow. Local correlation restricts the algorithm to capturing limited movements and fails when fast movement or large deformation of an object occurs. Therefore, we propose a transformer-based scene flow method that can perform global feature modeling through a self-attention layer. Moreover, we propose a cross-attention-based flow embedding layer for global feature matching. We further propose a learnable attention-based up-sampling layer to up-sample the estimated flow to higher resolution based on a single feature scale, eliminating the need to model global dependencies at all scales. Experimental results show that our model produces competitive results on Flyingthings3D and KITTI datasets with efficient performance.

Scene flow estimation aims to extract the 3D motion vector between each surface point in two consecutive point clouds. Pseudo-label-based approaches usually exploit point-to-point relations and 3D geometry information to generate the pseudo label for self-supervised learning. However, unreasonable results are still obtained due to the unexploited information of negative samples. Moreover, previous approaches are limited by the fact that pseudo labels are only generated along the forward direction, ignoring the backward direction that has strong spatiotemporal correlations with the forward direction. In this paper, we address these issues in a simple yet effective manner. Specifically, we introduce a contrastive loss to exploit both the information of positive samples and negative samples. Furthermore, we design a dual pseudo labels generation strategy to provide a bidirectional self-supervision for scene flow estimation. Experiments on FlyingThings3D and KITTI datasets show that our method can achieve competitive performance compared to recent self-supervised methods.

Person search is a challenging retrieval problem which aims at matching pedestrians with the same identity over non-overlapping camera views. In this paper, we adopt Swin Transformer as the backbone network to extract discriminative features. We propose a symmetric online instance matching loss which transfers the symmetric idea from KL divergence to the online instance matching loss. The purpose is to strengthen the robustness of the person search model under the condition of limited training identities. We compared with the state-of-the-arts on two mainstream benchmarks: CUHK-SYSU and PRW datasets. Experimental results demonstrate the effectiveness of our method. Especially, we achieve better performance on the PRW dataset with an improvement of 6.5% and 3.5% at the mAP and top-1 accuracy, respectively.

In this work, we propose a self-supervised scene flow framework for joint learning of optical flow, stereo depth, camera pose, and rigidity map and handle the occlusion during training. Specifically, we propose a feature masking method to alleviate the occlusion impact on the correlation result and reduce the outliers in both optical flow and depth map. We use the improved optical flow and depth to estimate the camera motion directly using the Perspective-n-Point method, which improves it accordingly. Furthermore, we recursively update the optical flow in both occluded and non-occluded regions with self-supervised cues learned from the rigid and optical flows. Reducing the error in the occluded regions enhances the rigidity map and improves the final optical flow accordingly. Our model achieved the state-of-the-art performance on KITTI 2015 benchmark for optical flow and produced competitive results for the depth, pose, and segmentation tasks.

Change detection (CD) is an important means to monitor environmental changes on Earth. Recently, many methods based on deep learning have been proposed for CD tasks. However, existing CD methods still have difficulties in the detection of changing target edges and small changing targets. In this letter, we propose a novel CD method named FCDNet based on full-scale skip connections (FSC) and coordinate attention (CA). FCDNet makes full use of shallow information and high-level semantics through FSC between different levels of encoder features and decoder features, which can effectively alleviate the missed detection of small targets in the CD task. In addition, we design a multi-receptive field position enhancement module (MRPEM) based on CA. MRPEM enhances the local relationship of features through convolution operations of different kernel sizes, and establishes long-distance dependency of features with the application of CA, thus facilitate accurate detection of the edges of changing targets. We also introduce Depthwise Over-parameterized Convolutional Layer (DOConv) in our network architecture, which can improve model performance without increasing computational complexity during inference. The experimental results show that our method is comparable to state-of-the-art (SOTA) methods on the Season-Varying Change Detection (SVCD) dataset.

Person search, which requires both pedestrian detection and person re-identification, is a challenging computer vision task applied to real-world scenarios. The challenges faced by detection and re-identification, such as occlusion, poor illumination, confusing background, are still urgent for person search. In addition, one-step methods for person search need to deal with the divergence between two tasks. In this work, we propose an end-to-end model containing the feature extractor, the region proposal network, and the multi-task learning module. In order to process divergence between detection and re-identification, we introduce switchable normalization and gradient centralization to improve the stability of the model. To solve the imbalance problem of hard examples, we introduce focal loss as a classification loss in the multi-task learning module. The experimental results on two bench-marks, i.e., CUHK-SYSU and PRW, well demonstrate that our method outperforms the state-of-the-art one-step methods.

Abstract Graph is a data structure that can represent complex relationships among data. Graph signal processing, unlike traditional signal processing, explicitly considers the structure and relationship among the signal samples. Graph wavelet transform can provide a multiscale analysis for the graph signal. It is well known that texture is a region property in an image, which is characterized with the intensity and relationship among pixels. In this context of the graph signal processing framework, an image texture can be considered as the signal on the graph. Therefore, a texture classification method based on graph wavelet transform is proposed. Specifically, image textures are decomposed into multiscale components by using two‐channel graph wavelet filter banks. Then the local singular value decomposition is applied to each subband. In order to improve the noise‐resistant ability, the maximum, mean and median values of the local singular values of graph‐wavelet transformation coefficients are extracted. Finally, the Weibull distributions are used to model those extracted values to describe the image textures. The experiments on the benchmark texture datasets are conducted to demonstrate the effectiveness of the proposed method.

Many deep learning applications benefit from multi-task learning with several related objectives. In autonomous driving scenarios, being able to accurately infer motion and spatial information is essential for scene understanding. In this paper, we combine an adaptive feature refinement module and a unified framework for joint learning of optical flow, depth and camera pose estimation in an unsupervised manner. The feature refinement module is embedded into motion estimation and depth prediction sub-networks, which can exploit more channel-wise relationships and contextual information for feature learning. Given a monocular video, our network firstly estimates depth and camera motion, and calculates rigid optical flow. Then, we design an auxiliary flow network for inferring non-rigid flow fields. In addition, a forward-backward consistency check is adopted for occlusion reasoning. Extensive experiments on KITTI dataset demonstrate that the proposed method achieves potential results comparing to recent deep learning networks.

Automatic detection of pavement crack is an important task for conducting road maintenance. However, as an important part of the intelligent transportation system, automatic pavement crack detection is challenging due to the poor continuity of cracks, the different width of cracks, and the low contrast between cracks and the surrounding pavement. This study proposes a novel pavement crack detection method based on an end‐to‐end trainable deep convolution neural network. The authors build the network using the encoder–decoder architecture and adopt a pyramid module to exploit global context information for the complex topology structures of cracks. Moreover, they introduce a spatial‐channel combinational attention module into the encoder–decoder network for refining crack features. Further, the dilated convolution is used to reduce the loss of crack details due to the pooling operation in the encoder network. In addition, they introduce a lovász hinge loss function, which is suitable for small objects. They train the authors' network on the CRACK500 dataset and evaluate it on three pavement crack datasets. Among the methods they compare, their method can achieve the best experimental results.

Recently, optical flow estimation benefits greatly from deep learning based techniques. Most approaches use encoder-decoder architecture (U-Net) or spatial pyramid network (SPN) to learn optical flow. Both U-Net and SPN can extract multi-scale features and can predict optical flow directly. However, existing networks ignore to exploit the global information among channel features and inter-spatial relationship of features. In this paper, we propose a dual self-attention pyramid network, which adaptively integrates local features with their global dependencies and focuses on important features and suppresses unimportant features. Specifically, we introduce two types of attention modules into SPN, which emphasizes meaningful features along channel and spatial axes. The channel attention can adaptively re-weight channel-wise features by considering interdependencies among channels. Moreover, the spatial attention can utilize global contextual information to emphasize or suppress features in different spatial locations. In addition, two attention modules are embedded into each pyramidal level, which can refine features at different scale. We evaluate our method on MPI-Sintel and KITTI. The experimental results show that using the dual self-attention module can improve the representation power of network and further increase the accuracy of optical flow estimation.

Supervised depth prediction and optical flow estimation have achieved promising performance due to the advanced deep network architectures. Since the ground truths are difficult to be collected, many recent works try to learn the depth and flow in an unsupervised manner. However, existing methods only use features from convolutional layers or a simple aggregation of multi-level features to predict the depth and flow maps, which is insufficient to exploit context information. In this paper, we attempt to exploit object contextual information and investigate the effect of the object context for joint learning of depth and optical flow. Specifically, we present a novel combination of object context and the framework of joint learning depth and optical flow. Our proposed network can exploit and integrate the object context for both tasks by aggregating the context according to pair-wise similarities. Furthermore, we adopt the existing spatial pyramid network (SPN) to estimate the depth and flow in a coarse-to-fine strategy effectively. Given temporally adjacent stereo pairs, our network can be trained end-to-end in an unsupervised manner and can predict the depth and flow maps simultaneously. We conduct experiments on two publicly available datasets, KITTI2012 and KITTI2015. Our proposed approach yields comparable performance on both depth and flow tasks, compared to the recent deep learning-based approaches. Experimental results demonstrate that exploiting object contextual information is useful and beneficial for depth and optical flow estimation.

Variational models for optical flow estimation usually define an energy function that contains prior assumptions to explore rudimentary statistics of images. However, such methods cannot learn motion knowledge from the pre-prepared data and have many parameters that need to be set manually. Nowadays, convolutional neural networks (CNNs) have been used in optical flow estimation successfully, which can learn weights from the training dataset and can predict optical flow end-to-end. In this paper, we propose an attention guided network for learning optical flow, named AD-Net, which contains several attention units for modelling the relativities between the channels. Further, we introduce dilated convolution into supervised network for reducing the loss of motion details. In addition, some prior auxiliary constraints are embedded in the supervised network as auxiliary loss terms. Our proposed approach is tested on MPI-Sintel and KITTI2012 datasets and can preserve motion edges and details effectively.

Recently, learning to estimate optical flow via deep convolutional networks is attracting significant attention. In this paper, we introduce a spatial-channel attention module into optical flow estimation, which infers attention maps along two separated dimensions, channel and spatial, and then integrates these separated attention maps into a fusion attention map for feature refinement. We embed this module into spatial pyramid network, which can adaptively learn the channel and spatial attention maps at each level for modifying the different scaled features and can further improve the accuracy of optical flow estimation. Our network is trained on FlyingChairs and FlyingThings3D datasets with a supervised manner, and is further tested on MPI-Sintel benchmark. The experimental results show that using the spatial-channel attention unit is beneficial for dense flow estimation and our approach is comparable with the state-of-the-art methods.

In this paper, we proposed new framework for human action representation, which leverages the strengths of convolutional neural networks (CNNs) and the linear dynamical system (LDS) to represent both spatial and temporal structures of actions in videos. We make two principal contributions: first, we incorporate image-trained CNNs to detect action clip concepts, which takes advantage of different levels of information by combining the two layers in CNNs trained from images; Second, we further propose adopting a linear dynamical system (LDS) to model the relationships between these clip concepts, which captures temporal structures of actions. We have applied the proposed method on two challenging realistic benchmark datasets, and our method achieves high performance up to 86.16% on the YouTube and 82.76% UCF50 datasets, which largely outperforms most of the state-of-the-art algorithms with more sophisticated techniques.

Image-set classification has recently generated great popularity due to widespread application to challenging tasks in computer vision. The great challenges arise from measuring the similarity between image sets which usually exhibit huge inter-class ambiguity and intra-class variation. In this paper, based on the assumption that each image set as a linear subspace can be treated as a point on a Grassmann manifold, we propose discriminant Grassmann kernels (DGK) of principal angles between subspaces. To tackle the ambiguity and variation, we propose learning the DGK via kernel target alignment, which achieves kernels of great discrimination by maximizing correlations with class labels. The proposed DGK has been evaluated on two challenging datasets including the ETH-80 and UCSD datasets for object recognition and video-based traffic congestion recognition, respectively. Extensive experiments have shown that the proposed DGKs achieves state-of-the-art performance and surpasses most of previous methods, which demonstrates the great effectiveness of the DGKs for image-set classification.

High-dimensional feature representations have recently been widely used for image classification, which not only induce large storage requirement and high computational complexity, but also tend to be lack of discrimination due to redundant and noisy features. In this paper, we propose a novel algorithm named supervised locality analysis (SLA) for dimensionality reduction. In contrast to conventional dimensionality reduction methods, the proposed SLA incorporates supervision into locality analysis by fully exploring multi-class distributions, which can handle the non-linear data structure while preserving intrinsic discriminative information. The obtained compact and highly discriminative features by the SLA is enables more accurate and efficient classification. Moreover, the SLA can be used for supervised dimensionality reduction of both handcrafted and deep learning based features. We have conduced experiments to evaluate the proposed SLA on three datasets for image classification. The SLA has produced state-of-the-art performance and largely outperformed widely-used dimensionality reduction methods.

In this paper we propose to learn semantic kernels for scene classification. We first decompose the Object Bank representation into subspaces associated with each object, Anchor Objects are then created by clustering for each scene class separately. The Anchor Distances are computed to measure the distance between objects to scene classes. In order to take the advantage of the discriminative information from different scene classes, we propose semantic kernels based on the anchor distances to different classes for scene classification. Through extensive experiments on two benchmark datasets: UIUC-Sports dataset and 15-Scene dataset, we prove that the proposed Semantic Kernels can significantly improve the original Object Bank and achieve state-of-the-art performance.