In flexible manufacturing, robots need to swiftly adapt to constantly changing production tasks. However, it remains a challenging problem for robots to grasp objects of specific categories through language instructions to complete production tasks in cluttered scenes. To address this issue, this article proposes a language-guided category push–grasp synergy network following a cognitive-decision framework. First, inspired by how humans can understand the world through interactions with the environment, we propose an environment state difference embodied self-supervision method that enables robots to autonomously collect embodied multimodal data and generate ground truths that eliminate annotation errors for cognition network training. Second, we develop a language-guided embodied multimodal object cognition network that fuses color and depth image information, enhancing the object cognition ability of robots in cluttered scenes and enabling dynamic semantic segmentation based on language commands. Finally, we propose an object manipulation space metric to measure the manipulable space of target objects, linking the reward function with metric changes before and after actions, thereby enhancing the system's perception of the manipulation space and improving operational performance. Experiments conducted in both simulated and real-world environments demonstrate that our proposed method outperforms existing state-of-the-art methods and can be generalized for grasping novel objects.

Accurate and real-time locomotion classification is crucial for exoskeletons to assist construction workers in completing multiple tasks. However, state-of-the-art algorithms for classifying multiple activities face multifaceted challenges in both accuracy and real-time capability. In addition, advanced studies typically provide a single solution based on certain sensor combinations, which may have an indirect impact on different assistive devices (e.g., an algorithm using feet IMUs is not suited for bilateral portable hip exoskeletons or unilateral knee exoskeletons), limiting its practicality and applicability in diverse applications. To fill these two gaps, first, we developed a novel hierarchical ensemble network framework that can accurately and real-time classify 11 typical lower limb activities of construction workers. Second, building upon this hierarchical ensemble network framework, we developed 6 configurations wearing IMU sensors on different body segments, which are potentially used for different wearable devices. Experimental results with leave-one-out cross-validation obtained from 10 able-bodied subjects validated the effectiveness of the proposed algorithm. Compared to the baseline ANN-based algorithm, our algorithm under 6 configurations on average was able to improve accuracy, precision, recall, and F1-score by 4.97%, 3.40%, 4.97%, and 5.31%, respectively, and reduce the number of parameters and inference time by 71.86% and 47.85%, respectively. This study showcases multiple solutions with different wearable sensor configurations, offering high accuracy and strong real-time performance for classifying multiple activities, which can be deployed to controllers for multiple types of assistive devices targeting construction workers.

Intelligent control methods have led to a significant simplification of the robotic arm modeling and control tuning process, and thus they have been widely used. To further improve the precision of robotic arm motion control, this paper proposes a robotic arm motion control strategy based on a cascaded feature-enhanced elastic-net broad learning system (CFE-EN-BLS). This will fully extract data features to improve motion control accuracy. Moreover, ElasticNet regression is introduced to reduce feature redundancy. Finally, Lyapunov stability theory is introduced to constrain the learning parameters of the proposed learning method to enhance the convergence of the control strategy. The simulation and experiment show that the proposed control strategy can realize high-precision trajectory tracking control of the robotic arm.

State-of-the-art exoskeletons are typically limited by low control bandwidth and small range stiffness of actuators which are based on high gear ratios and elastic components (e.g., series elastic actuators). Furthermore, most exoskeletons are based on discrete gait phase detection and/or discrete stiffness control resulting in discontinuous torque profiles. To fill these two gaps, we developed a portable lightweight knee exoskeleton using quasi-direct drive (QDD) actuation that provides 14 Nm torque (36.8% biological joint moment for overground walking). This paper presents 1) stiffness modeling of torque-controlled QDD exoskeletons and 2) stiffness-based continuous torque controller that estimates knee joint moment in real-time. Experimental tests found the exoskeleton had high bandwidth of stiffness control (16 Hz under 100 Nm/rad) and high torque tracking accuracy with 0.34 Nm Root Mean Square (RMS) error (6.22%) across 0-350 Nm/rad large range stiffness. The continuous controller was able to estimate knee moments accurately and smoothly for three walking speeds and their transitions. Experimental results with 8 able-bodied subjects demonstrated that our exoskeleton was able to reduce the muscle activities of all 8 measured knee and ankle muscles by 8.60%-15.22% relative to unpowered condition, and two knee flexors and one ankle plantar flexor by 1.92%-10.24% relative to baseline (no exoskeleton) condition.