Understanding physical human-robot and human-human interactions is a challenging yet emerging topic in 3D vision. While most existing methods rely on skeleton sequences-effective in low-light and privacy-sensitive environments-they face two major challenges: 1) learning and effectively exploiting interaction cues from skeletal data, and 2) compensating for the lack of visual information absent in skeletons alone. To address these challenges, we propose skeletal token alignment and rearrangement (STAR) for human-robot and human-human interaction recognition. It learns interaction-specific skeleton features and enriches them using visual cues by aligning skeleton and RGB video representations in a shared latent space. Specifically, STAR consists of three key components. First, we design a skeleton encoder that captures fine-grained interdependencies using Entity Rearrangement (ER) and Interactive Spatiotemporal Tokens (ISTs). Second, we present Visual Interaction Encoding that introduces a Focus on Interactions (FoI) strategy to attend to spatiotemporal regions relevant to interactions in RGB videos. Finally, these representations are aligned via a contrastive learning objective, with a refinement head further refines predictions. During training, STAR leverages both skeleton and RGB video data to learn robust, discriminative interaction representations. At inference time, it operates on skeletons alone, retaining visual-informed benefits while preserving skeleton-only efficiency. Extensive experiments on Chico, HARPER, NTU Mutual 11 and 26 datasets consistently validate our approach by demonstrating superior performance over state-of-the-art methods. Our code is publicly available athttps://github.com/Necolizer/STAR.

Force-free Control (FFC) efficiently accomplishes trajectory generating through manual interaction with a robotic manipulator in physical space, and the digital twin (DT) provides a comprehensive simulation and understanding of physical systems. In this study, an approach for facilitating human-robot collaboration (HRC) combining FFC and DT is implemented on a six-degree-of-freedom (6-DoF) collaborative manipulator. FFC has been realized by adopting three-tier control architecture that incorporates admittance control. DT is established to accurately and synchronously reflect the bi-directional mapping between the physical entity and its virtual counterpart. Experimental findings show that the proposed FFC architecture supports seamless interaction between the manipulator and external forces exerted by humans. Test results also indicate that DT enables data monitoring and singularity alert during FFC task execution, promisingly guaranteeing personnel safety and sustainable operation of equipment on industrial production lines.

Force-free control (FFC) allows for flexible manipulator motion in response to external forces, making it a vital component of human-robot interaction (HRI). Manual intervention may cause uneven forces on the manipulator or frequencies close to the natural frequency, and mechanical resonance can occur due to the inertia of the manipulator and adjustable equivalent stiffness of the controller. This paper proposes an FFC approach for industrial manipulators using a six-axis force/torque sensor (F/T sensor), implemented through a three-layer control architecture, consisting of motion control layer, Admittance Control layer and force decoupling layer. To mitigate the effects of mechanical resonance, a high-pass filter (HPF) is integrated with the F/T sensor and its impact is investigated. Experimental validation is conducted using both a simulation model and an industrial manipulator. Test results indicate that the proposed FFC architecture enables the manipulator not only to interact smoothly with external forces, but also to distinguish load forces at different frequencies and potentially address the issue of mechanical resonance between the manipulator and the applied load forces.

Skeleton-based multi-entity action recognition is a challenging task aiming to identify interactive actions or group activities involving multiple diverse entities. Existing models for individuals often fall short in this task due to the inherent distribution discrepancies among entity skeletons, leading to suboptimal backbone optimization. To this end, we introduce a Convex Hull Adaptive Shift based multi-Entity action recognition method (CHASE), which mitigates inter-entity distribution gaps and unbiases subsequent backbones. Specifically, CHASE comprises a learnable parameterized network and an auxiliary objective. The parameterized network achieves plausible, sample-adaptive repositioning of skeleton sequences through two key components. First, the Implicit Convex Hull Constrained Adaptive Shift ensures that the new origin of the coordinate system is within the skeleton convex hull. Second, the Coefficient Learning Block provides a lightweight parameterization of the mapping from skeleton sequences to their specific coefficients in convex combinations. Moreover, to guide the optimization of this network for discrepancy minimization, we propose the Mini-batch Pair-wise Maximum Mean Discrepancy as the additional objective. CHASE operates as a sample-adaptive normalization method to mitigate inter-entity distribution discrepancies, thereby reducing data bias and improving the subsequent classifier's multi-entity action recognition performance. Extensive experiments on six datasets, including NTU Mutual 11/26, H2O, Assembly101, Collective Activity and Volleyball, consistently verify our approach by seamlessly adapting to single-entity backbones and boosting their performance in multi-entity scenarios. Our code is publicly available at https://github.com/Necolizer/CHASE .

Recognizing interactive action plays an important role in human-robot interaction and collaboration. Previous methods use late fusion and co-attention mechanism to capture interactive relations, which have limited learning capability or inefficiency to adapt to more interacting entities. With assumption that priors of each entity are already known, they also lack evaluations on a more general setting addressing the diversity of subjects. To address these problems, we propose an Interactive Spatiotemporal Token Attention Network (ISTA-Net), which simultaneously model spatial, temporal, and interactive relations. Specifically, our network contains a tokenizer to partition Interactive Spatiotemporal Tokens (ISTs), which is a unified way to represent motions of multiple diverse entities. By extending the entity dimension, ISTs provide better interactive representations. To jointly learn along three dimensions in ISTs, multi-head self-attention blocks integrated with 3D convolutions are designed to capture inter-token correlations. When modeling correlations, a strict entity ordering is usually irrelevant for recognizing interactive actions. To this end, Entity Rearrangement is proposed to eliminate the orderliness in ISTs for interchangeable entities. Extensive experiments on four datasets verify the effectiveness of ISTA-Net by outperforming state-of-the-art methods. Our code is publicly available at https://github.com/Necolizer/ISTA-Net.

This work presents a novel design framework of adaptive iterative learning control (ILC) approach for a class of uncertain nonlinear systems. By using the closed-loop reference model that can be viewed as an observer, the proposed adaptive ILC approach can be adapted to deal with the output tracking problem of nonlinear systems with unavailable system states. In the systems considered, two classes of uncertainties are taken into account, including parametric input disturbances and input distribution uncertainties. To facilitate the controller design and convergence analysis, the composite energy function (CEF) methodology is employed. The design framework in this brief is novel and widely applicable, which extends the CEF-based ILC approach to output tracking control of nonlinear systems without requiring full knowledge of state information and complicated observer design process. To show the effectiveness of the proposed design framework and control algorithms, two numerical examples are illustrated.

Device-to-device communications underlaying cellular networks have been recognized as one of the key technologies for the fifth generation (5G) cellular system to improve the spectrum efficiency and system capacity. In this paper, we investigate the potential of deep reinforcement learning (DRL) for joint subcarrier assignment and power allocation in a general form of D2D networks, where a subcarrier can be assigned to multiple D2D pairs and each D2D pair is permitted to utilize multiple subcarriers. We first formulate the above problem as a Markov decision process, and then propose a double deep Q-network (DQN)-based subcarrier-power allocation algorithm to learn the optimal policy in the absence of full instantaneous channel state information (CSI). Specifically, each D2D pair acts as a learning agent that adjusts its own subcarrier-power allocation strategy iteratively through interactions with the operating environment in a trial-and-error fashion. Simulation results confirm that the proposed algorithm achieves near optimal performance in real time. It is worth mentioning that the proposed algorithm is especially suitable for the case where the environmental dynamics is not accurate and the CSI delay cannot be ignored.