Point clouds contain rich spatial information, providing important supplementary clues for human action recognition . Recent methods for action recognition based on point cloud sequences primarily rely on complex spatiotemporal local encoding. However, these methods often utilize max-pooling operations to select features when extracting local features , restricting feature updates to local neighborhoods and failing to fully exploit the relationships between regions. Moreover, cross-frame encoding can also lead to the loss of spatiotemporal information. In this study, we propose PRG-Net, a Point Relation Guided Network, to further improve the learning of spatiotemporal features in point clouds. First, we designed two core modules: the Spatial Feature Aggregation (SFA) and the Spatial Feature Descriptor (SFD) modules. The SFA module expands the spatial structure between regions using dynamic aggregation techniques, while the SFD module guides the region aggregation process by Attention-Weighted Descriptors. They enhance the modeling of human spatial structure by expanding the relationships between regions. Second, we introduce inter-frame motion encoding techniques that can obtain the final spatiotemporal representation of the human body through the aggregation of cross-frame vectors, without relying on complex spatiotemporal local encoding. We evaluate PRG-Net on publicly available human action recognition datasets, including NTU RGB+D 60, NTU RGB+D 120, UTD-MHAD, and MSR Action 3D. Experimental results demonstrate that our method outperforms state-of-the-art point-based 3D action recognition methods significantly. Furthermore, we conduct extended experiments on the SHREC 2017 dataset for gesture recognition , and the results show that our method maintains competitive performance on that dataset as well.

Noisy labels in Facial Expression Recognition (FER) datasets severely affect the performance of FER models. We propose a novel dual-branch noise extraction and suppression method to address this issue. This algorithm reduces the model’s impact from noisy labels by decreasing the dataset noise ratio and suppressing label-noisy samples. The method comprises three primary stages: sample extraction, pseudo-label generation, and re-training. The approach initially extracts label-noisy samples from the dataset by computing an exponential moving average of the model predictions and the joint probability distribution matrix of noisy and actual labels. The remaining samples form a clean dataset. Next, the training weights of the clean dataset are utilized to assign appropriate pseudo-labels to the label-noisy samples. Subsequently, the noisy labels are replaced with pseudo-labels to create a corrected dataset. The corrected and clean datasets are combined to create the reconstructed dataset, reducing noisy labels within the dataset. Finally, the model is retrained using the reconstructed dataset. Furthermore, this study introduces a novel gradient suppression smoothing function specifically designed to mitigate the impact of label-noisy samples in the dataset during the re-training process. The proposed algorithm is robust, with accuracies of 91.17%, 91.56%, and 91.58% on the RAF-DB dataset with 10%, 20%, and 30% noisy labels, and accuracies of 89.91%, 90.17%, and 89.69% on the corresponding FERPlus.

As personal photos are widely shared on social media, face anonymization becomes an effective solution to avoid identity leakage. Aiming at the problems of low quality and uncontrollable attributes of anonymized faces in existing algorithms, we propose a face anonymization algorithm PICLAnony based on parametric imitation contrast learning. It transfers the four visual information of identity, expression, pose and illumination from the source image to the generated anonymized face image by parametric imitation contrast learning. And it edits these attribute features that reflect sensitive behavioral intentions under the premise of controllable background. In the parameter imitation learning stage, high-quality and pose-controllable anonymized faces are generated by imitating the semantic parameters of source images. In the parameter contrast learning stage, the semantic parameters of the edited generated image and the source image are compared and learned, which solves the problem of insufficient decoupling of expression and illumination attributes. In addition, a background control module is designed to keep the background controllable during the editing process of anonymous face facial attributes. The subjective and objective results demonstrate that PICLAnony outperforms the state-of-the-art methods in terms of image quality and editing of facial attributes of anonymized faces.

The eye area, which can effectively convey social information and behavioral intentions, contains most of the information in facial images. However, the occlusion of glasses makes it difficult to accurately obtain the eye information in some facial images. Although the existing attribute editing method can remove glasses from face images, there are existing drawbacks lay in incomplete eye information and insufficient disentanglement after the glasses were removed (e.g. the background and hair color also changed when the glasses were removed), unable to get real and natural eye editing results. To address this problem, we propose an eye attribute editing model assisted by dual-coordinate system and multi-probability fusion prediction. The former one is designed to enhance the position awareness of inpainting to directly focus on the eyes area while the later one could integrate both local and global information of the target area. Specifically, the first stage of our method is to remove the glasses, and then use multi-probability fusion prediction method to inpaint the eye area to obtain a reasonably complete eye editing result supported by dual-coordinate system. On the one hand, we designed a dual-coordinate positioning module that uses a cascaded classifier and a dual-coordinate system to locate the human eye height area mask, which effectively solves the problem of inconsistent eye position height in facial images and determines the editing area for inpainting after the glasses are removed. On the other hand, we employed a multi-probability fusion prediction algorithm to supplement and enhance the ocular information, which can combine a distance-based local probability and an adaptive global probability, resulting in improved editing results. Quantitative and qualitative evaluations show that our method outperforms the state-of-the-art methods.

Abstract Improving the accuracy and disentanglement of attribute translation, and maintaining the consistency of face identity have been hot topics in face attribute translation. Recent approaches employ attention mechanisms to enable attribute translation in facial images. However, due to the lack of accuracy in the extraction of style code, the attention mechanism alone is not precise enough for the translation of attributes. To tackle this, we introduce a style translator module, which partitions the style code into attribute‐related and unrelated components, enhancing latent space disentanglement for more accurate attribute manipulation. Additionally, many current methods use per‐pixel loss functions to preserve face identity. However, this can sacrifice crucial high‐level features and textures in the target image. To address this limitation, we propose a multiple‐perceptual reconstruction loss to better maintain image fidelity. Extensive qualitative and quantitative experiments in this article demonstrate significant improvements over state‐of‐the‐art methods, validating the effectiveness of our approach.

Fabric defect detection methods based on deep networks are widely used in the textile industry , but they often suffer from poor model generalization and blurry edge detection. To resolve these challenges, we propose a novel network called “U-SMR Net”, which integrates global contextual features, defect detail features, and high-level semantic features through the combination of ResNet-50 and Swin Transformer modules. Our U-SMR network includes a lightweight multiscale feature extraction module, the dual-branch pyramid Module (DBPM), which is nested to preserve high-resolution, shallow semantic information. We propose a recursive multi-level residual decoding block for multiscale fusion to refine, filter, and enhance input characteristics, generating prediction maps at multiple stages, and by employing an improved binary cross entropy loss function to supervise saliency mapping. The experimental results based on four groups from ZJU-Leaper dataset demonstrate the superior performance of our approach compared to other competitive methods by achieving an average f m e a s u r e score of 75.33%, and finally testing results from both ZJU-Leaper-Total dataset and the HKU-Fabric dataset further support our U-SMR Net’s validity and generalization ability.

Abstract The absence of local features and global shape constraints severely limits the performance of the hourglass network for facial landmark detection in unconstrained environments. Moreover, diverse feature types and scales may result in low accuracy. This paper proposes a probability‐guided hourglass network to enhance the shape constraints for robust facial landmark detection. Firstly, a multi‐scale pre‐processing module is designed to extract features at different scales. Secondly, based on the heatmaps generated by the stacked hourglass network, the coarse localizations are obtained, while the probability maps are generated with local features. Finally, a probability‐based boundary regression method is proposed and the hausdorff distance is modified as the loss function to constrain the feature shape. Adaptive weights are also added to the loss function, which can help relieve the data imbalance problem. Subjective and objective experimental results on the challenging datasets show that this method outperforms the state‐of‐the‐arts on unconstrained conditions.

Abstract Multimodal human behaviour recognition is a research hotspot in computer vision. To fully use both skeleton and depth data, this paper constructs a new multimodal network identification scheme combined with the self‐attention mechanism. The system comprises a transformer‐based skeleton self‐attention subnetwork and a depth self‐attention subnetwork based on CNN. In the skeleton self‐attention subnetwork, this paper proposes a motion synergy space feature that can integrate the information of each joint point according to the entirety and synergy of human motion and puts forward a quantitative standard for the contribution degree of each joint motion. In this paper, the results from the skeleton self‐attention subnetwork and the depth self‐attention subnetwork are integrated and they are verified on the NTU RGB+D and UTD‐MHAD datasets. The authors have achieved 90% recognition rate on UTD‐MHAD dataset, and the CS recognition rate of the authors’ method on the NTU RGB+D dataset reaches 90.5% and the recognition rate of CV is 94.7%. Experimental results show that the network structure proposed in this paper achieves a high recognition rate, and its performance is better than most current methods.

Abstract Although Sparse Representation based Classifier (SRC), a non‐parametric model, can obtain an interesting result for pattern recognition , a reasonable interpretation has been lacked for its classification mechanism. What is more, the training samples are used as off‐the‐shelf dictionary directly in SRC, which can make the feature hidden in the training samples hard be extracted. At the same time, the complexity of the algorithm is increased because of too many atoms of the dictionary. The authors first explains in detail the classification mechanism of SRC from the view of probabilistic collaborative subspace and offer the process to improve the stability of the algorithm using the joint probability in the case of the multi‐subspace. Then, the authors introduce the dictionary learning (DL) and Fisher criterion into the model to further enhance the discrimination of the coding coefficient. In order to ensure the convexity of the discrimination term and further enhance the discrimination, the authors add the L 21 ‐norm term into the Fisher discrimination term and offer the proof for its convexity. Finally, the experimental result on a series of benchmark databases, such as AR, Extended Yale B, LFW3D‐hassner, LFW3D‐sdm and LFW3D‐Dlib, show that PCDDL outperforms existing classical classification models.

Abstract The traditional face alignment approaches based on cascade regression have achieved satisfactory result on the frontal face, but for the face with large changes in posture and expression, a single initial shape will lead to the result falling into local optimum. In order to solve this problem, a two‐stage cascade regression model for face alignment is proposed, which generates coarse initial shape from the aligned salient shape. The first stage is used to align the salient shape that contains some prominent landmarks. To enhance the robustness of authors' method, the fusion subspace is used to divide the samples, and each subset trains cascade regression model separately. The alignment results of the first stage are used to generate the coarse initial shapes for the second stage through 3D fitting. The second stage is still based on cascade regression, which is used to further predict the full shape. The experimental results demonstrate the proposed method can achieve state‐of‐art performance, especially in unconstrained conditions with various poses.

To accurately detect defects in patterned fabrics, a novel detection algorithm combining template correction with primitive decomposition (TCPD) method is proposed in this study. First of all, the fabric image is segmented into lattices according to variation regularity. Then, the authors propose an effective anisotropy correction method to reduce the interference of stretching and distortion between lattices. On the basis of the proposed PD method, the corrected lattice is further divided into graphic elements with smaller particle size. The smaller primitives make the boundary of the detection results more accurate. Moreover, a self‐supervised threshold selection strategy is presented, which utilises the defect‐free regions to obtain threshold. Furthermore, this strategy makes each primitive has corresponding criteria for judging defects. Extensive experiments demonstrate that TCPD method achieves 0.8127 true positive rate, 0.3889 positive predictive value and 0.5261 f value in star‐patterned fabrics.

In this study, the authors focus on the challenging problem of verifying faces captured under unconstrained conditions. Unconstrained face images often vary largely in poses, illuminations, expressions, occlusions, and ages. To address these challenges, they combine face frontalisation method with metric learning. To deal with the variations of poses, they apply an improved 3D face frontalisation method to generate the frontal view of the face images. Recent studies observed that bilinear similarity and Mahalanobis distance have a promising performance on measuring the similarity of two images. Based on these studies, they propose a weighted similarity and distance metric learning method which balances the role of bilinear similarity and Mahalanobis distance to better measure the similarity of an image pair. All the experiments are conducted based on the labelled faces in the wild database, and the experimental results show the effectiveness of their method.

High-quality optical flow computation algorithms are computationally intensive. The low computational speed of such algorithms causes difficulties for real-world applications. In this article, we propose an optimized implementation of the classical Combine-Brightness-Gradient (CBG) model on the Xilinx ZYNQ FPGA-SoC, by taking advantage of the inherent algorithmic parallelism and ZYNQ architecture. The execution time decreases to 0.82 second with a lower power consumption (1.881W). It is better than software implementation on PC (Intel i7-3520M, 2.9GHz), which costs 2.635 seconds and 35W. We use C rather than HDLs to describe the algorithm for rapid prototyping.

In this paper, we propose a new dictionary learning method to improve the performance of face recognition by learning a shared dictionary for all persons and a class-specific dictionary for each person. Such shared-dictionary is employed by computing the differences between samples and the centroid of neutral-samples, which improves the representation effort in the case of pose, expression and occlusion changes. Moreover, the discriminative power of learnt class-specific dictionary is further enhanced by enforcing structured dictionaries to be incoherence. Extensive experiments on three databases (AR, CMU-PIE, Extended Yale B) illustrate that the proposed framework achieves dramatic performance on face recognition even when existing big changes between training data and testing samples.

This study studies different lighting processing and feature extraction methods for efficient face recognition. The purpose is to find some robust face recognition methods by combining different feature extraction methods with illumination compensation. In this study, some typical illumination preprocessing approaches are reviewed including wavelet transformation, self‐quotient image, Retinex, smoothing, discrete cosine transform normalisation in logarithm domain, homomorphic filter and local contrast enhancement. As the main contribution, this study proposes two efficient feature extraction methods for face recognition. One is an adaptive feature extraction (AFE) based on curvelet transform. The other is a feature extraction technique named two‐dimensional principal component analysis (2DPCA) non‐parametric analysis of 2D subspace (2DPCA + 2DNSA). Two groups of experiments are designed to verify the proposed methods. The first group of experimental results show that the proposed AFE methods have better performance than conventional methods. In the second group of experiments, each feature extraction method is combined with nine different lighting processing methods. The results show that the proposed 2DPCA + 2DNSA method is more robust to lighting processing than other methods. Experimental results also show that lighting processing contribution to face recognition are quite different for different face databases.

Pixel-domain analysis methods are widely adopted in background modeling, some of which are not only concerned by academia but also coming into view of industry. However, as the increasing data volume of video, how to process and analysis videos in a fast and effective way has still been an intractable problem in practical applications. Under this circumstance, surveillance video analysis in the compressed domain is indeed of strategic importance from the angle of balancing visual perception and processing speed, especially in modeling background and segmenting moving objects. Therefore, a background modeling method in the compressed domain is proposed to quickly extract moving objects in this paper. Our main contributions are: 1) a method to calculate MVLBP features based on MV amplitude in the compressed domain is presented; 2) a background modeling and moving objects extraction method is designed in the compressed domain based on Local Binary Patterns of Motion Vector (MVLBP). Experimental results show that our approach gives a stable performance in a shorter time in H.264 compressed domain.