Multi-view spectral clustering (MVSC) has garnered growing interest across various real-world applications, owing to its flexibility in managing diverse data space structures. Nevertheless, the fusion of multiple $n\times n$n×n similarity matrices and the separate post-discretization process hinder the utilization of MVSC in large-scale tasks, where $n$n denotes the number of samples. Moreover, noise in different similarity matrices, along with the two-stage mismatch caused by the post-discretization, results in a reduction in clustering effectiveness. To overcome these challenges, we establish a novel fast multi-view discrete clustering (FMVDC) model via spectral embedding fusion, which integrates spectral embedding matrices ($n\times c$n×c, $c\ll n$c≪n) to directly obtain discrete sample categories, where $c$c indicates the number of clusters, bypassing the need for both similarity matrix fusion and post-discretization. To further enhance clustering efficiency, we employ an anchor-based spectral embedding strategy to decrease the computational complexity of spectral analysis from cubic to linear. Since gradient descent methods are incapable of discrete models, we propose a fast optimization strategy based on the coordinate descent method to solve the FMVDC model efficiently. Extensive studies demonstrate that FMVDC significantly improves clustering performance compared to existing state-of-the-art methods, particularly in large-scale clustering tasks.

Traditional few-shot action recognition (FSAR) aims to address the problem of the scarcity of action videos, enabling the recognition of action categories with just a few labeled samples. It is generally believed that the samples in the meta-training phase and the meta-testing phase are all drawn from the same domain. However, in practical applications, they often come from different domains, which may lead to significant differences in the distribution of spatiotemporal features. Researchers have started to study the problem of cross-domain few-shot action recognition (CDFSAR). The current solution is to train the model by combining source domain video and unlabeled target domain video to improve the model’s generalization ability. In this paper, we follow this paradigm but make a more refined use of the unlabeled target domain videos to better extract transferable features. First, we decouple the source and target domain videos along the temporal dimension and extract the domain-irrelevant features in both the source and target domains. Second, in each episode, we calculate the centroid of the domain-irrelevant features of the target domain and perform a momentum update on this feature centroid. We use Cross-Attention to align the domain-irrelevant features of the source domain toward this dynamic centroid. Finally, we use these aligned source domain features for few-shot classification. Experimental results demonstrate that our approach significantly improves few-shot classification performance across diverse domain shifts, validating the effectiveness of our refined use of unlabeled target video. Our code has been published at the URL: https://github.com/cofly2014/MCA-TRD.git.

In light of their capability to capture structural information while reducing computing complexity, anchor graph-based multi-view clustering (AGMC) methods have attracted considerable attention in large-scale clustering problems. Nevertheless, existing AGMC methods still face the following two issues: 1) They directly embedded diverse anchor graphs into a consensus anchor graph (CAG), and hence ignore redundant information and numerous noises contained in these anchor graphs, leading to a decrease in clustering effectiveness; 2) They drop effectiveness and efficiency due to independent post-processing to acquire clustering indicators. To overcome the aforementioned issues, we deliver a novel one-step multi-view clustering method with adaptive low-rank anchor-graph learning (OMCAL). To construct a high-quality CAG, OMCAL provides a nuclear norm-based adaptive CAG learning model against information redundancy and noise interference. Then, to boost clustering effectiveness and efficiency substantially, we incorporate category indicator acquisition and CAG learning into a unified framework. Numerous studies conducted on ordinary and large-scale datasets indicate that OMCAL outperforms existing state-of-the-art methods in terms of clustering effectiveness and efficiency.

Understanding how selective attention influences human gaze behaviors is important for behavioral vision and visual psychology. However, most existing scanpath models ignore the internal sub-stages in visual search and do not imitate human-like selective attention. To bridge this gap, a novel inverse reinforcement learning model with a dual policy network (DPNet) is proposed to accurately predict how humans select and shift their attention at different stages of a task. Additionally, to establish the semantic correlation between objects for state representation modeling, this paper employs Augmented Belief Maps (ABMs). Besides, an Option-Viterbi module is introduced to infer sub-task options of the real human scanpaths, learning the sub-task switching during visual search. Experimental results on widely used datasets for visual search justify the effectiveness of our method in terms of scanpath similarity and sub-task switching.

Predicting the gaze patterns of the human eye when performing a visual search helps understand the mechanism of the human visual system. Most current scanpath models consider the physiological properties of human visual space when modeling gaze behaviors, and use image patches with different resolutions to represent the central and peripheral vision. However, the simplified method of lowering the image resolution to simulate the peripheral scene cannot accurately represent the visual information loss. On the other hand, few studies have focused on how peripheral information loss affects human attention shift during visual search. To address the above issues, we propose a novel scanpath prediction approach by designing a Multi-scale Visual Representation Module (MVRM) to process peripheral information of the human visual system. Besides, we introduce a Neural Interaction Module (NIM) with three interaction mechanisms of visual signals to predict the next fixation point. Experimental results on the benchmark datasets demonstrate the superiority of our method.

The ability to rapidly adapt to unseen scenarios for safe driving has long been a core challenge in autonomous driving. Rule-based methods are heavily reliant on labeled data and suffer from data biases. Many current reinforcement learning(RL)-based methods, on the other hand, are restricted to certain training scenarios, making it difficult for them to adapt to more complex and diverse traffic scenarios. In contrast, human drivers can quickly adapt to new driving situations based on their accumulated driving skills. Inspired by this, we propose the Vehicle Skill Graph (VSG), a novel framework for autonomous driving decision-making. By accumulating 1,000 diverse skills using RL and adopting knowledge graph embedding (KGE) techniques, we build a skill graph that offers a structured understanding of driving knowledge and discovers the potential relations between driving skills and new traffic scenarios. This enables rapid adaptation to new driving environments and addresses the issue of training scenario dependency in RL learning-based approaches. Experimental results demonstrate that VSG effectively captures the latent connections between traffic scenes and driving skills, facilitating efficient sequential decision-making in complex driving situations.

Spiking Neural Networks (SNNs), as a biologically inspired computational model, have garnered significant attention in object detection and image classification due to their event-driven mechanism and low-power characteristics. However, in object detection tasks, the residual structures in conventional networks introduce nonspiking operations, posing a critical challenge for SNNs. To address this issue, we propose a multi-layer feature cascade fusion SNN (MFCF-SNN) for object detection. During feature extraction, our novel multi-level cascaded feature extraction module replaces residual connections with cascade operations, eliminating nonspiking computations while enhancing gradient propagation to deeper layers. For downsampling, we introduce a pooling-convolution module that combines max-pooling and spiking convolution, effectively preserving feature information and improving gradient flow. These two modules collectively ensure pure spike-based computation while facilitating deep network training, thereby enhancing detection accuracy. Experimental results on the PASCAL VOC 2012 and SSDD datasets demonstrate state-of-the-art performance, validating the effectiveness of our approach in advancing SNN-based object detection.

Hypergraph clustering has garnered considerable attention in complex learning tasks due to its powerful capacity for modeling high-order relationships among samples. Nevertheless, existing methods encounter two fundamental challenges: 1) The need for an additional discretization step following low-dimensional spectral embedding, which introduces a suboptimal mismatch between continuous embeddings and discrete cluster assignments, thereby impairing clustering performance; and 2) the susceptibility to diverse and complex noise are commonly present in real-world scenarios, which significantly compromises clustering robustness. To address these issues, we propose a novel correntropy-induced hypergraph spectral clustering (CIHSC) model. Different from current spectral clustering methods, CIHSC integrates a correntropy-based framework to enable direct discrete spectral decomposition on hypergraphs, eliminating the need for post discretization and thereby enhancing clustering fidelity and robustness. To effectively address the non-convex optimization arising from the correntropy-induced objective, we develop a half-quadratic optimization strategy tailored to the CIHSC model. Extensive experiments conducted on both real-world and noise-contaminated datasets demonstrate that CIHSC consistently outperforms state-of-the-art clustering methods in terms of performance and robustness.

Multi-view spectral clustering has attracted considerable attention since it can explore common geometric structures from diverse views. Nevertheless, existing min-min framework-based models adopt internal minimization to find the view combination with the minimized within-cluster variance, which will lead to effectiveness loss since the real clusters often exhibit high within-cluster variance. To address this issue, we provide a novel scalable min-max multi-view spectral clustering (SMMSC) model to improve clustering performance. Besides, anchor graphs, rather than full sample graphs, are utilized to reduce the computational complexity of graph construction and singular value decomposition, thereby enhancing the applicability of SMMSC to large-scale applications. Then, we rewrite the min-max model as a minimized optimal value function, demonstrate its differentiability, and develop an efficient gradient descent-based algorithm to optimize it with linear computational complexity. Moreover, we demonstrate that the resultant solution of the proposed algorithm is the global optimum. Numerous experiments on different real-world datasets, including some large-scale datasets, demonstrate that SMMSC outperforms existing state-of-the-art multi-view clustering methods regarding clustering performance.

Due to its high computational complexity, graph-based methods have limited applicability in large-scale multiview clustering tasks. To address this issue, many accelerated algorithms, especially anchor graph-based methods and indicator learning-based methods, have been developed and made a great success. Nevertheless, since the restrictions of the optimization strategy, these accelerated methods still need to approximate the discrete graph-cutting problem to a continuous spectral embedding problem and utilize different discretization strategies to obtain discrete sample categories. To avoid the loss of effectiveness and efficiency caused by the approximation and discretization, we establish a discrete fast multiview anchor graph clustering (FMAGC) model that first constructs an anchor graph of each view and then generates a discrete cluster indicator matrix by solving the discrete multiview graph-cutting problem directly. Since the gradient descent-based method makes it hard to solve this discrete model, we propose a fast coordinate descent-based optimization strategy with linear complexity to solve it without approximating it as a continuous one. Extensive experiments on widely used normal and large-scale multiview datasets show that FMAGC can improve clustering effectiveness and efficiency compared to other state-of-the-art baselines.

Spiking reinforcement learning (SRL) has gradually received attention because of its ultra-low energy consumption, but the current SRL algorithms are almost online algorithms that are sample inefficient. It is well known that pure offline spiking reinforcement (offline RL) learning has limited performance. Therefore, we study SRL in the offline-to-online setting, which simultaneously possesses the advantages of low energy consumption and sample efficiency. To the best of our knowledge, this is the first study for offline-to-online SRL. Like offline-to-online RL, offline-to-online SRL also has the policy collapse issue. To overcome this problem, we adaptively adjust the penalty of behavior cloning term of spiking TD3+BC (SpikTD3+BC) based on the adaptability of policy to environment and propose a stable offline-to-online SRL, named AdaSpikTD3+BC. Experimental results on the D4RL benchmark tasks show that AdaSpikTD3+BC can not only avoid policy collapse but also cost about 10% energy to approach the performance of offline-to-online RL based on DNN.

Multi-view clustering has attracted a lot of attention due to its ability to integrate information from distinct views, but how to improve efficiency is still a hot research topic. Anchor graph-based methods and k-means-based methods are two current popular efficient methods, however, both have limitations. Clustering on the derived anchor graph takes a while for anchor graph-based methods, and the efficiency of k-means-based methods drops significantly when the data dimension is large. To emphasize these issues, we developed an efficient multi-view k-means clustering method with multiple anchor graphs (EMKMC). It first constructs anchor graphs for each view and then integrates these anchor graphs using an improved k-means strategy to obtain sample categories without any extra post-processing. Since EMKMC combines the high-efficiency portions of anchor graph-based methods and k-means-based methods, its efficiency is substantially higher than current fast methods, especially when dealing with large-scale high-dimensional multi-view data. Extensive experiments demonstrate that, compared to other state-of-the-art methods, EMKMC can boost clustering efficiency by several to thousands of times while maintaining comparable or even exceeding clustering effectiveness.

One of the hottest topics in unsupervised learning is how to efficiently and effectively cluster large amounts of unlabeled data. To address this issue, we propose an orthogonal conceptual factorization (OCF) model to increase clustering effectiveness by restricting the degree of freedom of matrix factorization. In addition, for the OCF model, a fast optimization algorithm containing only a few low-dimensional matrix operations is given to improve clustering efficiency, as opposed to the traditional CF optimization algorithm, which involves dense matrix multiplications. To further improve the clustering efficiency while suppressing the influence of the noises and outliers distributed in real-world data, an efficient correntropy-based clustering algorithm (ECCA) is proposed in this article. Compared with OCF, an anchor graph is constructed and then OCF is performed on the anchor graph instead of directly performing OCF on the original data, which can not only further improve the clustering efficiency but also inherit the advantages of the high performance of spectral clustering. In particular, the introduction of the anchor graph makes ECCA less sensitive to changes in data dimensions and still maintains high efficiency at higher data dimensions. Meanwhile, for various complex noises and outliers in real-world data, correntropy is introduced into ECCA to measure the similarity between the matrix before and after decomposition, which can greatly improve the clustering effectiveness and robustness. Subsequently, a novel and efficient half-quadratic optimization algorithm was proposed to quickly optimize the ECCA model. Finally, extensive experiments on different real-world datasets and noisy datasets show that ECCA can archive promising effectiveness and robustness while achieving tens to thousands of times the efficiency compared with other state-of-the-art baselines.

This paper proposes a method for head pose estimation from a single image. We employ a multi-stage regression strategy. To overcome the discontinuity of Euler angles and quaternions and avoid the additional constraints required to directly regress the rotation matrix, we apply a continuous 6D representation to the head pose estimation problem. Each stage of the network regresses two 1 × 3 vectors, which are then transformed into a 3 × 3 rotation matrix by this continuous 6D representation. To better perceive the difference in rotation angles, we adopt the Riemann distance to measure the closeness between the network-estimated rotation matrix and the ground truth rotation matrix corresponding to the head pose. Experiments show that our method achieves the state-of-the-art on BIWI dataset and performs favorably on AFLW2000 dataset.

In this paper, we present a new framework for view synthesis of novel view based on Neural Radiance Fields(NeRF). We aim to address two main limitations of NeRF. Firstly, we propose to combine multi-view stereo into NeRF to help construct general neural radiance fields across different scenes. Specifically, We build a MVS-Encoding Feature Volume with average groupwise correlation to aggregate the multi-view appearance and geometry feature for every source view. And then we use an MLP to encode neural radiance fields by using the scene-dependent features interpolated from the MVS-Encoding Feature Volumes. This makes our model can be applied to other unseen scenes without any per-scene fine-tuning, and render realistic images with few images. If more training images are provided, our method can be fine-tuned quickly to render more realistic images. In fine-tuning phase, we propose a depth priors guided sampling method, which can make the model represent more accurate geometry for corresponding scenes and so render high-quality images of novel view. We evaluate our method on three common datasets. The experiment results show that our method performs better than other baselines, neither without or with fine-tuning. And the depth priors guided sampling method can be easily applied on other methods based on Neural Radiance Fields to further improve the quality of rendered images.

Multi-view clustering has received widespread attention owing to its effectiveness by integrating multi-view data appropriately, but traditional algorithms have limited applicability to large-scale real-world data due to their high computational complexity and low robustness. Focusing on the aforementioned issues, we propose an efficient and robust multi-view clustering algorithm with anchor graph regularization (ERMC-AGR). In this work, a novel anchor graph regularization (ARG) is designed to improve the quality of the learned embedded anchor graph (EAG), and the obtained EAG is decomposed by nonnegative matrix factorization (NMF) under correntropy criterion to acquire clustering results directly. Different from the traditional graph regularization that needs to construct a large-scale Laplacian matrix pertaining to the all-sample graph, our lightweight AGR, constructed from the perspective of anchors, can reduce the computational complexity significantly while improving the EAG quality. Moreover, a factor matrix of NMF is constrained to be the cluster indicator matrix to omit additional k-means after optimization. Subsequently, correntropy is utilized to improve the effectiveness and robustness of ERMC-AGR owing to its promising performance to complex noises and outliers. Extensive experiments on real-world datasets and noisy datasets show that ERMC-ARG can improve the clustering efficiency and robustness while ensuring comparable or even better effectiveness.

Spectral clustering has been a hot topic in unsupervised learning owing to its remarkable clustering effectiveness and well-defined framework. Despite this, due to its high computation complexity, it is unable of handling large-scale or high-dimensional data, particularly multi-view large-scale data. To address this issue, in this paper, we propose a fast multi-view clustering algorithm with spectral embedding (FMCSE), which speeds up both the spectral embedding and spectral analysis stages of multi-view spectral clustering. Furthermore, unlike conventional spectral clustering, FMCSE can acquire all sample categories directly after optimization without extra k-means, which can significantly enhance efficiency. Moreover, we also provide a fast optimization strategy for solving the FMCSE model, which divides the optimization problem into three decoupled small-scale sub-problems that can be solved in a few iteration steps. Finally, extensive experiments on a variety of real-world datasets (including large-scale and high-dimensional datasets) show that, when compared to other state-of-the-art fast multi-view clustering baselines, FMCSE can maintain comparable or even better clustering effectiveness while significantly improving clustering efficiency.

The rapid growth of the number of data brings great challenges to clustering, especially the introduction of multi-view data, which collected from multiple sources or represented by multiple features, makes these challenges more arduous. How to clustering large-scale data efficiently has become the hottest topic of current large-scale clustering tasks. Although several accelerated multi-view methods have been proposed to improve the efficiency of clustering large-scale data, they still cannot be applied to some scenarios that require high efficiency because of the high computational complexity. To cope with the issue of high computational complexity of existing multi-view methods when dealing with large-scale data, a fast multi-view clustering model via nonnegative and orthogonal factorization (FMCNOF) is proposed in this paper. Instead of constraining the factor matrices to be nonnegative as traditional nonnegative and orthogonal factorization (NOF), we constrain a factor matrix of this model to be cluster indicator matrix which can assign cluster labels to data directly without extra post-processing step to extract cluster structures from the factor matrix. Meanwhile, the F-norm instead of the L2-norm is utilized on the FMCNOF model, which makes the model very easy to optimize. Furthermore, an efficient optimization algorithm is proposed to solve the FMCNOF model. Different from the traditional NOF optimization algorithm requiring dense matrix multiplications, our algorithm can divide the optimization problem into three decoupled small size subproblems that can be solved by much less matrix multiplications. Combined with the FMCNOF model and the corresponding fast optimization method, the efficiency of the clustering process can be significantly improved, and the computational complexity is nearly O(n) . Extensive experiments on various benchmark data sets validate our approach can greatly improve the efficiency when achieve acceptable performance.

In this paper, a real WiFi fingerprint based indoor localization system is considered for experiments, including three primary components: the APP in the smart phone, the server system and the embedded localization algorithm. As we all know, one of the main drawbacks in fingerprint based localization is the labor intensity and time consumption of data collection. This paper proposes an improved graph-based semi-supervised learning (I-GSSL) to better overcome this problem. Apart from taking advantage of the indoor propagation model, the I-GSSL algorithm is proposed to handle the existing out-of-sample problem where an elastic regularization is considered as an extra constraint. Meanwhile, due to unequal amount of location information in the received signal strength (RSS) from different access points (APs) and the redundancy of RSS at APs, a double weighted K nearest neighbor (DWKNN) algorithm is proposed for localization. Experimental results show the proposed scheme achieves a better label propagation and localization accuracy.

In this paper, we propose a robust point set registration algorithm which combines correntropy and point-to-plane distance, which can register rigid point sets with noises and outliers. Firstly, as correntropy performs well in handling data with non-Gaussian noises, we introduce it to model rigid point set registration problem based on point-to-plane distance; Secondly, we propose an iterative algorithm to solve this problem, which repeats to compute correspondence and transformation parameters respectively in closed form solutions. Simulated experimental results demonstrate the high precision and robustness of the proposed algorithm. In addition, LiDAR based localization experiments on automated vehicle performs satisfactory for localization accuracy and time consumption.

In this paper, a real WiFi fingerprint-based indoor localization system is considered, where three primary components including the APP in smart phone, the server system and the embedded localization algorithm, have been designed. This paper proposes a dedicated data preprocessing algorithm to solve the singular-collection problem. Furthermore, the issue of WiFi access point (AP) missing is discussed and the theoretical analysis is presented under the condition of a two-AP scenario. Finally, because of unequal amount of location information contained in received signal strength (RSS) from different AP, the weighted RSS (WRSS) and the iterative weighted K nearest neighbor (IWKNN) algorithm are proposed for localization. Experimental results shows the proposed scheme achieves a competitive localization accuracy.

The traditional iterative closest point (ICP) algorithm could register two points sets well, but it is easily affected by local dissimilar. To deal with this problem, this paper proposes an isotropic scaling ICP algorithm with corner point constraint. First, an objective function is proposed under the guidance of the corner points, as the corner points can preserve the similar of the whole shapes. Secondly, a new ICP algorithm is used to complete the isotropic scaling registration. At each step of this new algorithm, the correspondence is built based on the closest point searching, and then a closed-form solution of the transformation is computed. The experimental results demonstrate that our algorithm can prevent the influence of the local dissimilar and improve the registration precision compared with the traditional ICP algorithm.

Overtaking vehicle detection is vital for road safety, as the dangerous behavior of that vehicle may affect the safety of ego-vehicle and the time is not enough for the driver to attend and react. Therefore, it is one of the key components of the Advanced Driver Assistance Systems. Mostly, traditional methods only use local information, appearance or motion. In this paper, we build a novel CRF model to make use of the interaction between local regions, and the motion features from multiple scales as well. The whole model is based on the low-level optical flows. In order to increase the robustness to the noise in the flow, we divided the motion field into small blocks, and learned Mixture of Probabilistic Principle Analysis models for the common motion patterns of the background. Moreover, we also adopted an online scheme for updating the parameters. Results of testing on the real road images demonstrated the capability of the proposed algorithm.

Change detection is the foremost pre-attention process of visual motion analysis. It provides important preprocessing clues for the following complex visual attention selection and pattern recognition process. In this letter, a novel dual-probe adaptive model of the weak image change signal detection is advanced. Then its basic parameter constraints are analyzed and the numerical analysis of its characteristic is discussed. Simulation results show that the related change detector could capture the tiny change signals in synthetic and nature scenes with noisy background.

Road situation analysis in Interactive Intelligent Driver-Assistance and Safety Warning (I2DASW) systems involves estimation and prediction of the position and size of various on-road obstacles. Real-time processing, given incomplete and uncertain information, is a challenge for current object detection and tracking technologies. This paper proposed a development framework and novel algorithms for road situation analysis based on driving action behavior, where the safety situation is analyzed by simulating real driving action behaviors. First, we review recent development and trends in road situation analysis to provide perspective for the related research. Second, we introduce a road situation analysis framework, where onboard sensors provide information about drivers, traffic environment, and vehicles. Finally, on the basis of the previous frameworks, we proposed multiple-obstacle detection and tracking algorithms using multiple sensors including radar, lidar, and a camera, where a decentralized track-to-track fusion approach is introduced to fuse these sensors. In order to reduce the effect of obstacle shape and appearance, we cluster lidar data and then classify obstacles into two categories: static and moving objects. Future collisions are assessed by computation of local tracks of moving obstacles using extended Kalman filter, maximum likelihood estimation to fuse distributed local tracks into global tracks, and finally, computation of future collision distribution from the global tracks. Our experimental results show that our approach is efficient for road situation evaluation and prediction