Edge sensor devices generate vast amounts of user data, but centralized processing poses privacy risks. Federated Learning addresses this by decentralizing training. However, applying Federated Learning directly to skeleton videos fails to preserve motion dynamics and suffers from client heterogeneity bias. To address these limitations, we propose CSAR-a Client-Unbiased Skeletal Action Recognizer for Federated Learning-which tackles two core challenges: motion dynamics preservation and classifier bias mitigation. Specifically, CSAR employs a Model Calibration Loss during client training to align client-server representations and reduce drift. On the server, it generates class-balanced spatiotemporal federated features through Prototypical Gaussian Sampling, subsequently refined via a Motion-aware Differential Loss to capture kinematic properties. These features enable retraining of a globally debiased recognizer that achieves accuracy comparable to real-data-trained models. Further stabilization is achieved through Knowledge Matching, which enhances global understanding. Experiments under natural and label heterogeneity confirm that CSAR outperforms state-of-the-art methods.

Zero-Shot Skeleton-Based Action Recognition (ZSSAR) is an emerging research field focused on developing alignment models that connect skeleton movements with action definitions, thus enabling generalization to unobserved actions. Current methods often employ generative models to reconstruct cross-modal features or enhance mutual information across modalities for alignment. However, when applied to unseen action categories, these models often neglect the inherent consistency among basic actions, thereby diminishing their generalization capabilities. Furthermore, imprecise annotations fail to capture the rich semantic details of actions, resulting in misalignment. Inspired by human cognitive processes and chain of thought, we argue that integrating prior information about human actions with intrinsic commonality knowledge of basic actions is essential for ZSSAR. To actualize this, we propose a novel method termed Prompt-guided Prototype-aware Commonality and Discrimination Learning (PP-CDL). This method utilize the comprehensive world knowledge contained in LLMs, employing tailored prompts to partition seen action categories into distinct, non-overlapping prototype spaces that embody the commonality knowledge of basic actions. Subsequently, we introduce the Inter- and Intra-Prototype Discriminating (I2PD) module and the Intra-Prototype Commonality Mining (IPCM) module. The I2PD amplifies the distinctiveness of knowledge within prototypes, furnishing a personalized search space for the recognition of unseen actions. In contrast, the IPCM models the shared commonality concept within prototypes, bolstering the consistency between skeleton action representations and corresponding text knowledge representations. Experiments on different skeleton action benchmarks demonstrate the significant improvement of our method over existing alternatives.

Despite recent advancements in masked skeleton modeling and visual-language pre-training, no method has yet been proposed to explore capturing and utilizing the rich semantic information embedded in both modalities for enhanced action recognition. To address this challenge, we propose a novel Motion-Aware Mask Feature Reconstruction (MMFR) method for the challenging task of skeleton-based action recognition. MMFR ingeniously integrates masked skeleton feature reconstruction with visual-language pre-trained model within a consolidated framework, aiming to leverage the synergistic potential of both domains. Specifically, It employs visual-language model to infuse semantic understanding into the skeleton feature reconstruction process via probability distribution distillation. Moreover, we introduce a multi-granularity semantic contrast module that refines vision-text alignment precision and augments contextual information for accurate mask reconstruction. Extensive experiments demonstrate MMFR’s superiority in skeleton-based action recognition, as well as its efficacy in zero-shot scenarios.

Abstract The sparse matrix regression (SMR) model for the feature selection method has attracted much attention. However, most existing models do not consider the globality and adaptively preserve the local structure of the image data in projection space. To settle such issues, an adaptive non‐negative low‐rank preserving SMR model for supervised image feature selection is proposed. It first uses the low‐rank representation with non‐negative constraint to capture the globality and more discriminative information of image data and makes the error matrix in self‐representation of training data sparse. Next, the non‐negative low‐rank representation coefficients are used to establish a graph matrix learning model to reveal the local manifold structure of the image data. Thus, the proposed model enhances the discriminative ability as well as performs feature selection by the obtained transformation matrix. Finally, an alternating iterative algorithm for solving this model is developed and its convergence and complexity are also analyzed. Experimental results on some image data sets show that the proposed algorithm is effective for images and its recognition ability is obviously superior to other existing methods. In addition, the proposed method is also applied to two scene image classifications to further verify its effectiveness.

Abstract Matrix regression has attracted much attention due to directly select some meaningful features from matrix data. However, most existing matrix regressions do not consider the global and local structure of the matrix data simultaneously. To this end, we propose a low‐rank nonnegative sparse representation and local preserving matrix regression (LNSRLP‐MR) model for image feature selection. Here, the loss function is defined by the left and right regression matrices. To capture the global structure and discriminative information of the training images and reduce the effect of heterogeneous data and noises, we impose the low‐rank constraint on the self‐representation error matrix and the nonnegative sparse constraint on the coefficient vector. The graph matrix can be learned adaptively through representation coefficients, so that accurate local structure information in samples can be revealed. Feature selection is performed by obtained row sparse transformation matrix. An optimization procedure and its performance are also present. Experimental results on several image datasets show that compared with the‐state‐of‐the‐art method, the average classification accuracy of the proposed method is improved by at least 1.2% and up to 3.3%. For images with noise or occlusion, the accuracy is improved significantly, up to 4%, which indicates that this method has strong robustness.