The unstructured, unordered and inherent irregular sampling properties presents difficulties for accurate and efficient realizing semantic segmentation of large-scale 3D point cloud. The complexity and the long-distance information exploitation are the key challenges for large-scale 3D point cloud semantic segmentation. Therefore, in order to efficiently exploit long-distance global information and improve the segmentation accuracy of point cloud data located at the edges of distinct categories, a novel Trigonometric Bilinear attention and Global-aware Aggregation network (TBGA-Net) is designed to integrate and supplement the local-global contextual features for large-scale point clouds segmentation. The proposed Global-aware Context Aggregation block (GCA) can implicitly excavate the global context for each 3D point by utilizing its surface-to-volume ratio of the neighborhood to the global point cloud. Furthermore, aim at refining local-global features, the Trigonometric Bilinear Attention block (TBA) utilizes trigonometric functions to embed the point cloud coordinates of local regions, and applies bilinear attention to realize feature enhancement for obtaining more discriminative local-global features. Additionally, we further designed a novel Dynamic-adjusting Cross-entropy Loss (DCLoss) to incorporate with TBGA-Net for addressing the issue of class imbalance in training data for large-scale 3D point cloud semantic segmentation. The experimental results on three 3D point cloud datasets demonstrates that the proposed algorithm indicates better segmentation accuracy especially for the point located at the boundary of the distinct categories.

Memory stability and learning flexibility in continual learning (CL) is a core challenge for cross-scene Hyperspectral Anomaly Detection (HAD) task. Biological neural networks can actively forget history knowledge that conflicts with the learning of new experiences by regulating learning-triggered synaptic expansion and synaptic convergence. Inspired by this phenomenon, we propose a novel Biologically-Inspired Continual Learning Generative Adversarial Network (CL-BioGAN) for augmenting continuous distribution fitting ability for cross-domain HAD task, where Continual Learning Bio-inspired Loss (CL-Bio Loss) and self-attention Generative Adversarial Network (BioGAN) are incorporated to realize forgetting history knowledge as well as involving replay strategy in the proposed BioGAN. Specifically, a novel Bio-Inspired Loss composed with an Active Forgetting Loss (AF Loss) and a CL loss is designed to realize parameters releasing and enhancing between new task and history tasks from a Bayesian perspective. Meanwhile, BioGAN loss with L2-Norm enhances self-attention (SA) to further balance the stability and flexibility for better fitting background distribution for open scenario HAD (OHAD) tasks. Experiment results underscore that the proposed CL-BioGAN can achieve more robust and satisfying accuracy for cross-domain HAD with fewer parameters and computation cost. This dual contribution not only elevates CL performance but also offers new insights into neural adaptation mechanisms in OHAD task.

In the above article [1], in (19), the original formulation has been revised and replaced with the following: \begin{equation*} \textbf {R}_{t}\gets \ \textbf {R}_{t-1}\cup \ k\left ({{\textbf B_{t}}}\right ),\textbf {R}_{0}=\phi . {5pt}\tag {19}\end{equation*}

Occlusions and complex backgrounds are common factors that hinder many computer vision applications. In a street scene, the challenge of accurately predicting pedestrian trajectories comes from the complexity of human behavior and the diversity of the external environment. It is difficult, if not impossible, to extract relevant information to accurately predict pedestrian trajectories in dynamic scenes. Synthetic aperture imaging (SAI) uses an array of cameras to mimic a camera with a large virtual convex lens by projecting images of a scene from different views onto a virtual focal plane. It is commonly used to reconstruct occluded objects, and in a street scene, can provide observation of pedestrians occluded by other objects and pedestrians. In this paper, we propose a joint prediction method based on autofocusing of SAI to predict pedestrian trajectories in dynamic scenes. The main contributions of this paper include: 1) The task of pedestrian trajectory prediction in dynamic scenarios is redefined as pedestrian trajectory prediction and SAI autofocusing from a practical but more challenging perspective. 2) The proposed method is based on an existing SAI-based method to extract information in heavily occluded views, which can obtain more accurate results but with less computational cost and without using other sensors such as LiDAR or depth cameras. 3) A new pedestrian trajectory prediction model, an attention-based trajectory prediction variational autoencoder (ATP-VAE), is proposed to extract complex human behavior and social interactions in dynamic scenes through a new Intention Attention Unit. The experimental results on multiple public datasets show that the proposed method achieves state-of-the-art results in the first-person perspective and in aerial view.

Anomaly detection (AD) has attracted remarkable attention in hyperspectral image (HSI) processing fields, most existing deep learning (DL) based algorithms indicate dramatic potential for detecting anomaly samples through specific training process under current scenario. However, the limited prior information and the catastrophic forgetting problem indicate crucial challenges for existing DL structure in open scenarios cross-domain detection. In order to improve the detection performance, a novel capsule differential adversarial continual learning framework (CL-CaGAN) is proposed to elevate the cross-scenario learning performance for facilitating the real application of DL-based structure in hyperspectral anomaly detection (HAD) task. First, a modified capsule structure with adversarial learning network is constructed to estimate the background distribution for surmounting the deficiency of prior information. To mitigate the catastrophic forgetting phenomenon, clustering-based sample replay strategy and a designed extra self-distillation regularization are integrated for merging the history and future knowledge in continual AD task, while the discriminative learning ability from previous detection scenario to current scenario are retained by the elaborately designed structure with continual learning strategy. In addition, the differentiable enhancement is enforced to augment the generation performance of the training data for further stabilizing the training process with better convergence, this procedure further efficiently consolidates the reconstruction ability of background samples. To verify the effectiveness of our proposed CL-CaGAN, we conduct experiments on several real HSIs, the results indicate that the proposed CL-CaGAN demonstrates higher detection performance and continuous learning capacity for mitigating the catastrophic forgetting under cross-domain scenarios.

Deep learning-based methods have demonstrated significant breakthroughs in the application of hyperspectral image (HSI) classification. However, some challenging issues still exist, such as the overfitting problem caused by the limitation of training size with high-dimensional feature and the efficiency of spectral–spatial (SS) exploitation. Therefore, to efficiently model the relative position of samples within the generative adversarial network (GAN) setting, we proposed a dual-channel SS fusion capsule generative adversarial network (DcCapsGAN) for HSI classification. Dual channels (1-D-CapsGAN and 2-D-CapsGAN) are constructed by integrating the capsule network (CapsNet) with GAN for eliminating the mode collapse and gradient disappearance problem caused by traditional GAN. Meanwhile, octave convolution and multiscale convolution are integrated into the proposed model for further reducing the parameters of the CapsNet and extracting multiscale features. To further boost the classification performance, the SS channel fusion model is constructed to composite and switch the feature information of different channels, thereby facilitating the accuracy and robustness of the whole classification performance. Three commonly used HSI data sets are utilized to investigate the performance of the proposed DcCapsGAN model, and the performance of the experiment demonstrates that the proposed model can efficiently improve the classification accuracy and performance.

Deep learning (DL) has become a hot topic in the research field of hyperspectral image (HSI) classification. However, with increasing depth and size of deep learning methods, its application in mobile and embedded vision applications has brought great challenges. In this article, we address a network architecture search (NAS)-guided lightweight spectral–spatial attention feature fusion network (LMAFN) for HSI classification. The overall architecture of the proposed network is guided by several conclusions of NAS, which achieves fewer parameters and lower computation cost with deeper network structure by exploiting multiscale Ghost grouped with efficient channel attention (ECA) module for adaptively adjusting the weights of different channels. It helps fully extract spectral–spatial discriminant features to avoid information loss of the dimension reduction operation. Specifically, a multilayer feature fusion method is proposed to extract the fusion information of the spectral–spatial features of each layer by considering complementary information of different hierarchical structures. Therefore, high-lever spectral–spatial attributes are gradually exploited along with the increase in layers and the fusion of layers. The experimental verification on three real HSI data sets demonstrates that the proposed framework presents more satisfying classification performance and efficiency with deeper network structure and lower parameter size.

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral-spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral-spatial feature learning. Third, a sequential spectral-spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).