Point clouds contain rich spatial information, providing important supplementary clues for human action recognition . Recent methods for action recognition based on point cloud sequences primarily rely on complex spatiotemporal local encoding. However, these methods often utilize max-pooling operations to select features when extracting local features , restricting feature updates to local neighborhoods and failing to fully exploit the relationships between regions. Moreover, cross-frame encoding can also lead to the loss of spatiotemporal information. In this study, we propose PRG-Net, a Point Relation Guided Network, to further improve the learning of spatiotemporal features in point clouds. First, we designed two core modules: the Spatial Feature Aggregation (SFA) and the Spatial Feature Descriptor (SFD) modules. The SFA module expands the spatial structure between regions using dynamic aggregation techniques, while the SFD module guides the region aggregation process by Attention-Weighted Descriptors. They enhance the modeling of human spatial structure by expanding the relationships between regions. Second, we introduce inter-frame motion encoding techniques that can obtain the final spatiotemporal representation of the human body through the aggregation of cross-frame vectors, without relying on complex spatiotemporal local encoding. We evaluate PRG-Net on publicly available human action recognition datasets, including NTU RGB+D 60, NTU RGB+D 120, UTD-MHAD, and MSR Action 3D. Experimental results demonstrate that our method outperforms state-of-the-art point-based 3D action recognition methods significantly. Furthermore, we conduct extended experiments on the SHREC 2017 dataset for gesture recognition , and the results show that our method maintains competitive performance on that dataset as well.

Current LiDAR-only 3D detection methods are limited by the sparsity of point clouds. The previous method used pseudo points generated by depth completion to supplement the LiDAR point cloud, but the pseudo points sampling process was complex, and the distribution of pseudo points was uneven. Meanwhile, due to the imprecision of depth completion, the pseudo points suffer from noise and local structural ambiguity, which limit the further improvement of detection accuracy. This paper presents SQDNet, a novel framework designed to address these challenges. SQDNet incorporates two key components: the SQD, which achieves sparse-to-dense matching via grid position indices, allowing for rapid sampling of large-scale pseudo points on the dense depth map directly, thus streamlining the data preprocessing pipeline. And use the density of LiDAR points within these grids to alleviate the uneven distribution and noise problems of pseudo points. Meanwhile, the sparse 3D Backbone is designed to capture long-distance dependencies, thereby improving voxel feature extraction and mitigating local structural blur in pseudo points. The experimental results validate the effectiveness of SQD and achieve considerable detection performance for difficult-to-detect instances on the KITTI test.

Abstract The absence of local features and global shape constraints severely limits the performance of the hourglass network for facial landmark detection in unconstrained environments. Moreover, diverse feature types and scales may result in low accuracy. This paper proposes a probability‐guided hourglass network to enhance the shape constraints for robust facial landmark detection. Firstly, a multi‐scale pre‐processing module is designed to extract features at different scales. Secondly, based on the heatmaps generated by the stacked hourglass network, the coarse localizations are obtained, while the probability maps are generated with local features. Finally, a probability‐based boundary regression method is proposed and the hausdorff distance is modified as the loss function to constrain the feature shape. Adaptive weights are also added to the loss function, which can help relieve the data imbalance problem. Subjective and objective experimental results on the challenging datasets show that this method outperforms the state‐of‐the‐arts on unconstrained conditions.

Abstract Multimodal human behaviour recognition is a research hotspot in computer vision. To fully use both skeleton and depth data, this paper constructs a new multimodal network identification scheme combined with the self‐attention mechanism. The system comprises a transformer‐based skeleton self‐attention subnetwork and a depth self‐attention subnetwork based on CNN. In the skeleton self‐attention subnetwork, this paper proposes a motion synergy space feature that can integrate the information of each joint point according to the entirety and synergy of human motion and puts forward a quantitative standard for the contribution degree of each joint motion. In this paper, the results from the skeleton self‐attention subnetwork and the depth self‐attention subnetwork are integrated and they are verified on the NTU RGB+D and UTD‐MHAD datasets. The authors have achieved 90% recognition rate on UTD‐MHAD dataset, and the CS recognition rate of the authors’ method on the NTU RGB+D dataset reaches 90.5% and the recognition rate of CV is 94.7%. Experimental results show that the network structure proposed in this paper achieves a high recognition rate, and its performance is better than most current methods.

Real-time 3-D human action recognition has broad industrial applications, such as surveillance, human–computer interaction, and healthcare monitoring. By relying on complex spatio-temporal local encoding, most existing point cloud sequence networks capture spatio-temporal local structures to recognize 3-D human actions. To simplify the point cloud sequence modeling task, we propose a lightweight and effective point cloud sequence network referred to as SequentialPointNet for real-time 3-D action recognition. Instead of capturing spatio-temporal local structures, SequentialPointNet encodes the temporal evolution of static appearances to recognize human actions. First, we define a novel type of point data, hyperpoint, to better describe the temporally changing human appearances. A theoretical foundation is provided to clarify the information equivalence property for converting point cloud sequences into hyperpoint sequences. Second, the point cloud sequence modeling task is decomposed into a hyperpoint embedding task and a hyperpoint sequence modeling task. Specifically, for hyperpoint embedding, the static point cloud technology is employed to convert point cloud sequences into hyperpoint sequences, which introduces inherent frame-level parallelism; for hyperpoint sequence modeling, a hyperpoint-mixer module is designed as the basic building block to learning the spatio-temporal features of human actions. Extensive experiments on three widely-used 3-D action recognition datasets demonstrate that the proposed SequentialPointNet achieves a competitive classification performance with up to 10× faster than existing approaches.

Abstract Although Sparse Representation based Classifier (SRC), a non‐parametric model, can obtain an interesting result for pattern recognition , a reasonable interpretation has been lacked for its classification mechanism. What is more, the training samples are used as off‐the‐shelf dictionary directly in SRC, which can make the feature hidden in the training samples hard be extracted. At the same time, the complexity of the algorithm is increased because of too many atoms of the dictionary. The authors first explains in detail the classification mechanism of SRC from the view of probabilistic collaborative subspace and offer the process to improve the stability of the algorithm using the joint probability in the case of the multi‐subspace. Then, the authors introduce the dictionary learning (DL) and Fisher criterion into the model to further enhance the discrimination of the coding coefficient. In order to ensure the convexity of the discrimination term and further enhance the discrimination, the authors add the L 21 ‐norm term into the Fisher discrimination term and offer the proof for its convexity. Finally, the experimental result on a series of benchmark databases, such as AR, Extended Yale B, LFW3D‐hassner, LFW3D‐sdm and LFW3D‐Dlib, show that PCDDL outperforms existing classical classification models.

To accurately detect defects in patterned fabrics, a novel detection algorithm combining template correction with primitive decomposition (TCPD) method is proposed in this study. First of all, the fabric image is segmented into lattices according to variation regularity. Then, the authors propose an effective anisotropy correction method to reduce the interference of stretching and distortion between lattices. On the basis of the proposed PD method, the corrected lattice is further divided into graphic elements with smaller particle size. The smaller primitives make the boundary of the detection results more accurate. Moreover, a self‐supervised threshold selection strategy is presented, which utilises the defect‐free regions to obtain threshold. Furthermore, this strategy makes each primitive has corresponding criteria for judging defects. Extensive experiments demonstrate that TCPD method achieves 0.8127 true positive rate, 0.3889 positive predictive value and 0.5261 f value in star‐patterned fabrics.

In this study, the authors focus on the challenging problem of verifying faces captured under unconstrained conditions. Unconstrained face images often vary largely in poses, illuminations, expressions, occlusions, and ages. To address these challenges, they combine face frontalisation method with metric learning. To deal with the variations of poses, they apply an improved 3D face frontalisation method to generate the frontal view of the face images. Recent studies observed that bilinear similarity and Mahalanobis distance have a promising performance on measuring the similarity of two images. Based on these studies, they propose a weighted similarity and distance metric learning method which balances the role of bilinear similarity and Mahalanobis distance to better measure the similarity of an image pair. All the experiments are conducted based on the labelled faces in the wild database, and the experimental results show the effectiveness of their method.