Tele-homecare has become a promising approach to meet the growing demand for elderly and disability care. In such a context, ensuring human-robot interaction safety during teleoperation poses a critical challenge. Existing teleoperation control approaches focus solely on the robot’s end-effector trajectory, failing to handle inevitable or even desirable contacts on other robot links. This paper proposes a teleoperated shared-control strategy to deal with this challenge. A lightweight exoskeleton is developed to teleoperate the robot and give force feedback to the operator. Additionally, an exoskeleton-based shared-control strategy is proposed to integrate operator commands with real-time proximity sensing information, allowing the robot to avoid collisions while executing tasks. To react to inevitable contact, the force feedback function is incorporated into the proposed strategy to enable the operator to experience intuitive contact. Comparative experiments and a demonstration are designed to evaluate the feasibility and reliability of the proposed strategy in a tele-homecare scenario. Compared to the traditional teleoperation strategy, the proposed method can greatly reduce the contact forces on the robot’s links, indicating the potential of the proposed strategy in advancing safety in tele-homecare systems.

Unmanned Aerial Vehicles (UAVs) equipped with multiple tasking capabilities will play an increasingly important role in future warfare scenarios. As a battlefield environment may have various of tasks and corresponding constraints, the strategic selection of tasks is important to improve the performance of the group of UAVs. This paper addresses this problem through the Environment Classes, Agents, Roles, Groups, and Objects (E-CARGO) model. Considering constraints of task requirements and resource limitations, we propose a hybrid approach that integrates genetic algorithms and linear programming to generate an optimal set of task choices. Our experimental results highlight the effectiveness of linear programming in efficiently achieving optimal solutions within small teams. At the same time, the combination of genetic algorithms and linear programming proves effective in ensuring satisfactory solutions within an acceptable time for larger teams. This research paves the way for optimizing the task selection in complex operational environments.

Soft actuators are intrinsically safe to surroundings and adaptable when operated in unstructured environments, thereby offering capabilities beyond conventional actuators based on rigid components. However, the challenge of achieving both lightweight and rapid response still remains. Inspired by the Mimosa pudica, we propose a novel approach to developing a biomimetic soft actuator, utilizing hydraulically amplified self-healing electrostatic (HASEL) technology. We proposed the fabrication procedure of the actuator based on widely available materials. Experimental results show that the HASEL Mimosa actuators could replicate the plant’s nature of significant deformation and rapid response, underscoring their application prospects in soft robotics, bioengineering, and human-robot interaction.

Filter pruning is a widely adopted technique for compressing convolutional neural networks (CNNs). However, common filter pruning algorithms often overlook varying importance and feature distributions across different layers, leading to suboptimal results. Additionally, they frequently require intricate fine-tuning processes and lack fine-grained control over intragroup coefficients, potentially failing to fully exploit correlations among individual features within a group. To overcome these limitations, we introduce a sparse penalty based on Sparse Group Lasso, encouraging the model to engage in more stringent feature selection within groups, thus flexibly adapting to intra-group sparsity. We propose an adaptive sparse weights scheme, dynamically allocating sparsity rates for each layer in the network based on evaluations of feature distribution and importance. Upon completion of training, the algorithm directly eliminates redundant parameters, eliminating the need for a fine-tuning process. The efficacy of the proposed methodology has been substantiated through both theoretical derivations and experimental validations by the research team.

In this paper, we introduce an Feature Aggregate Side Network (FASN), a simple, efficient, and easy-to-train method for open-vocabulary semantic segmentation. Building upon existing models based on the CLIP-Side Network framework, we address the issue of CLIP-generated features lacking pixel-level recognition capability by layering a novel fast graph representation learning layer between the CLIP and side networks. This integration introduces an inductive bias for the aggregation of local features, thereby better addressing the challenges in semantic segmentation. Through validation on five distinct datasets and extensive ablation studies, we have demonstrated the effectiveness of our modifications. Our findings indicate that with a slight increase in the number of parameters, there is a significant enhancement in performance.

Despite the success of 6D pose estimation in bin-picking scenarios, existing methods still struggle to produce accurate prediction results for symmetry objects in real-world scenarios. The primary bottlenecks include 1) the ambiguity in keypoints caused by object symmetries; and 2) the domain gap between real and synthetic data. To circumvent these problems, we propose a novel 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net). SD-Net builds on point-wise keypoint regression and deep hough voting to perform reliable keypoint detection under clutter and occlusion. Specifically, at the keypoint prediction stage, we propose a robust 3D keypoint selection strategy considering the symmetry class of objects and equivalent keypoints, which facilitate locating 3D keypoints even in highly occluded scenes. Additionally, we build an effective filtering algorithm on predicted keypoints to dynamically eliminate multiple ambiguity and outlier key-point candidates. At the domain adaptation stage, we propose the self-training framework using a student-teacher training scheme. To carefully distinguish reliable predictions, we harness tailored heuristics for 3D geometry pseudo labelling based on semi-chamfer distance. On the public Siléane dataset, SD-Net achieves state-of-the-art results, obtaining an average precision of 96%. Testing learning and generalization abilities on public Parametric datasets, SD-Net is 8% higher than the state-of-the-art method.

Convolutional Neural Networks (CNNs) have exhibited remarkable success in various vision tasks. The allocation of channels within each layer significantly influences CNN performance. Despite the acknowledged importance of channel configurations, achieving an optimal distribution that balances computational efficiency and model accuracy remains challenging. In this paper, we initially transform the CNN training problem into an analogous knapsack optimization problem, incorporating the loss sensitivity criteria. Subsequently, we introduce the concept of value density, employing greedy algorithms, to accurately quantify the improvement in accuracy achievable by increasing the unit FLOPs on every layer. This fosters a proficient exploration and optimization of channel configurations within convolutional layers. Additionally, the efficacy of the proposed methodology has been substantiated through both theoretical derivations and experimental validations by the research team, outperforming popular pruning methods under equal-scale FLOPs conditions. We hope that the integration of value density into channel search algorithms will contribute to the development of more powerful CNNs.

Accurately simulating physical processes is an extremely challenging task, but the rapid development of machine learning and the availability of large datasets have made Graph Neural Networks (GNNs) a powerful tool for effectively simulating physical systems. Currently, GNNs-based methods are primarily used in simple scenarios such as the free fall and collision of objects, fluid flow, and gravitational interactions among atoms. However, in complex industrial environments, there are always intricate interference factors such as friction, bearing connections, and torque affecting the motion between objects. Consequently, GNNs-based methods largely fail to solve practical physical problems related to complex multi-body dynamics. In this paper, to address the current lack of multi-body dynamics datasets in this field, we first introduce a multi-body dynamics dataset comprising eight different scenarios, each embodying distinct physical principles. Furthermore, we explore Graph Neural Simulators (GNSs) structure and physical priors and propose an efficient novel model, the Graph Neural Network with Restrgints (GOAT), that can directly learn the relationships between systems from multi-body trajectories, thereby enhancing performance. Our results have shown significant improvements compared to other state-of-the-art baselines, demonstrating strong generalization capabilities and data efficiency.

Accurate measuring and modeling of dynamic robot manipulation (e.g., tossing and catching) is particularly challenging, due to the inherent nonlinearity, complexity, and uncertainty in high-speed robot motions and highly dynamic robot–object interactions happening in very short distances and times. Most studies leverage extrinsic sensors such as visual and tactile feedback toward task or object-centric modeling of manipulation dynamics, which, however, may hit bottleneck due to the significant cost and complexity, e.g., the environmental restrictions. In this work, we investigate whether using solely the on-board proprioceptive sensory modalities can effectively capture and characterize dynamic manipulation processes. In particular, we present an object-agnostic strategy to learn the robot toss dynamics of arbitrary unknown objects from the spatio-temporal variations of robot toss movements and wrist-force/torque (F/T) observations. We then propose TossNet, an end-to-end formulation that jointly measures the robot toss dynamics and predicts the resulting flying trajectories of the tossed objects. Experimental results in both simulation and real-world scenarios demonstrate that our methods can accurately model the robot toss dynamics of both seen and unseen objects, and predict their flying trajectories with superior prediction accuracy in nearly real-time. Ablative results are also presented to demonstrate the effectiveness of each proprioceptive modality and their correlations in modeling the toss dynamics. Case studies show that TossNet can be applied on various real robot platforms for challenging tossing-centric robot applications, such as blind juggling and high-precise robot pitching.

Recently, visuotactile sensors have shown promising potential in robotics due to their high-resolution sensing ability. Unfortunately, the majority of available visuotactile sensors are limited to flat shapes, which severely limits their application possibilities. In this article, we propose a novel curved visuotactile sensor, the GelStereo Palm, which senses the 3-D contact geometry on a curved surface using a binocular vision system. Meanwhile, to solve the light refraction problem in the binocular stereo vision system under a curved medium, a refractive stereo ray tracing model for GelStereo Palm is presented. Moreover, a 3-D tactile point cloud sensing pipeline is introduced to reconstruct the 3-D contact geometry in real-time. Finally, extensive experiments are conducted to verify the accuracy and robustness of the 3-D contact geometry sensing of our GelStereo Palm sensor.

Object rearranging is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a goal configuration. Previous studies focus on designing an expert system for each specific task by model-based or data-driven approaches and the application scenarios are therefore limited. Some research has been attempting to design a general framework to obtain more advanced manipulation capabilities for deformable rearranging tasks, with lots of progress achieved in simulation. However, transferring from simulation to reality is difficult due to the limitation of the end-to-end CNN architecture. To address these challenges, we design a local GNN (Graph Neural Network) based learning method, which utilizes two representation graphs to encode keypoints detected from images. Self-attention is applied for graph updating and cross-attention is applied for generating manipulation actions. Extensive experiments have been conducted to demonstrate that our framework is effective in multiple 1-D (rope, rope ring) and 2-D (cloth) rearranging tasks in simulation and can be easily transferred to a real robot by fine-tuning a keypoint detector.

Rearranging deformable objects is a long-standing challenge in robotic manipulation for the high dimensionality of configuration space and the complex dynamics of deformable objects. We present a novel framework, Graph-Transporter, for goal-conditioned deformable object rearranging tasks. To tackle the challenge of complex configuration space and dynamics, we represent the configuration space of a deformable object with a graph structure and the graph features are encoded by a graph convolution network. Our framework adopts an architecture based on Fully Convolutional Network (FCN) to output pixel-wise pick-and-place actions from only visual input. Extensive experiments have been conducted to validate the effectiveness of the graph representation of deformable object configuration. The experimental results also demonstrate that our framework is effective and general in handling goal-conditioned deformable object rearranging tasks.

This work addresses the problem of planning a robot configuration and grasp to position a shared object during forceful human-robot collaboration, such as a puncturing or a cutting task. Particularly, our goal is to find a robot configuration that positions the jointly manipulated object such that the muscular effort of the human, operating on the same object, is minimized while also ensuring the stability of the interaction for the robot. This raises three challenges. First, we predict the human muscular effort given a human-robot combined kinematic configuration and the interaction forces of a task. To do this, we perform task-space to muscle-space mapping for two different musculoskeletal models of the human arm. Second, we predict the human body kinematic configuration given a robot configuration and the resulting object pose in the workspace. To do this, we assume that the human prefers the body configuration that minimizes the muscular effort. And third, we ensure that, under the forces applied by the human, the robot grasp on the object is stable and the robot joint torques are within limits. Addressing these three challenges, we build a planner that, given a forceful task description, can output the robot grasp on an object and the robot configuration to position the shared object in space. We quantitatively analyze the performance of the planner and the validity of our assumptions. We conduct experiments with human subjects to measure their kinematic configurations, muscular activity, and force output during collaborative puncturing and cutting tasks. The results illustrate the effectiveness of our planner in reducing the human muscular load. For instance, for the puncturing task, our planner is able to reduce muscular load by \( 69.5\% \) compared to a user-based selection of object poses.

The teleoperation system can transmit human intention to the remote robot, so that the system combines excellent robot operation performance and human intelligence. In this article, we have established a bilateral teleoperation system with force feedback from the arm and gripper. That is, the slave robot system can provide force feedback on both the wrist and the fingers, while the master robot system can render the slave feedback force and human interaction force, and control the slave robot accordingly. In addition, this paper also proposes the framework of the robot’s four-channel bilateral teleoperation control system, which is attributed to two situations: impedance control or admittance control. Finally, single-arm/single-arm, dual-arm/dual-arm bilateral teleoperation experiments prove the effectiveness of the bilateral teleoperation system and the four-channel controller architecture proposed in this paper.

We present a manipulation planning algorithm for a robot to keep an object stable under changing external forces. We particularly focus on the case where a human may be applying forceful operations, e.g. cutting or drilling, on an object that the robot is holding. The planner produces an efficient plan by intelligently deciding when the robot should change its grasp on the object as the human applies the forces. The planner also tries to choose subsequent grasps such that they will minimize the number of regrasps that will be required in the long-term. Furthermore, as it switches from one grasp to the other, the planner solves the problem of bimanual regrasp planning, where the object is not placed on a support surface, but instead it is held by a single gripper until the second gripper moves to a new position on the object. This requires the planner to also reason about the stability of the object under gravity. We provide an implementation on a bimanual robot and present experiments to show the performance of our planner.

In this paper, a novel pseudo gradient based DE approach is proposed, which takes advantage of both the differential evolutionary (DE) and the gradient-based algorithm. The gradient information, which is called pseudo gradient, is generated through randomly selected two vectors and their fitness function values. This work is to investigate the effect of proposed pseudo gradient on differential evolutionary algorithm. The simulation results show that DE with pseudo gradient can obtain better performance overall in comparison with classical DE variants. The pseudo gradient based DE with adaptive parameter section is compared with the existing adaptive DE algorithms. Also, the control parameter, step size are investigated to understand the mechanism of pseudo gradient in detail.