3-D language Gaussian splatting has recently shown strong potential for open-vocabulary scene understanding and robotic manipulation. However, most existing methods require dense multiview observations to achieve accurate geometry reconstruction and reliable semantic alignment, which limits their applicability in scenarios where only sparse-view observations are available. In this work, we propose SparseGrasper, a framework for language-guided zero-shot robotic grasping under sparse-view conditions. SparseGrasper constructs a 3-D Gaussian language field from as few as three RGB images, enabling joint reasoning over geometry and semantics without the need for dense observations. To improve representation learning under sparse observations, we introduce a dual feature distillation module that fuses local object features with global contextual cues. We further design a language-guided grasp pose generation strategy that incorporates semantic grounding into grasp candidate selection, encouraging grasps that are both semantically relevant and geometrically feasible. Real-world experiments on a 7-DoF robotic manipulator validate that SparseGrasper effectively performs language-guided grasping of diverse, previously unseen objects from sparse observations.

Heavy-duty glass installation is a high-risk, precision-critical task in modern construction, traditionally performed through labor-intensive and error-prone manual methods. This paper presents a novel robotic framework that leverages diffusion-based self-supervised imitation learning from imperfect visual servoing demonstrations to achieve safe and precise glass installation. Specifically, our approach employs noisy and suboptimal demonstration data obtained via visual servoing to train a Denoising Diffusion Probabilistic Model (DDPM). This model iteratively refines installation trajectories, transforming them into smooth, precise, and collisionfree movements. Extensive experiments demonstrate that our method significantly surpasses conventional visual servoing and standard imitation learning baselines in terms of success rate, precision, and installation efficiency, while markedly improving operational safety. Our results establish a new benchmark for automating complex, high-risk tasks in construction robotics.

Affordance refers to the functional properties that an agent perceives and utilizes from its environment, and is key perceptual information required for robots to perform actions. This information is rich and multimodal in nature. Existing multimodal affordance methods face limitations in extracting useful information, mainly due to simple structural designs, basic fusion methods, and large model parameters, making it difficult to meet the performance requirements for practical deployment. To address these issues, this paper proposes the BiT-Align image-depth-text affordance mapping framework. The framework includes a Bypass Prompt Module (BPM) and a Text Feature Guidance (TFG) attention selection mechanism. BPM integrates the auxiliary modality depth image directly as a prompt to the primary modality RGB image, embedding it into the primary modality encoder without introducing additional encoders. This reduces the model’s parameter count and effectively improves functional region localization accuracy. The TFG mechanism guides the selection and enhancement of attention heads in the image encoder using textual features, improving the understanding of affordance characteristics. Experimental results demonstrate that the proposed method achieves significant performance improvements on public AGD20K and HICO-IIF datasets. On the AGD20K dataset, compared with the current state-of-the-art method, we achieve a 6.0% improvement in the KLD metric, while reducing model parameters by 88.8%, demonstrating practical application values. The source code will be made publicly available at https://github.com/DAWDSE/BiT-Align.

Deformable object manipulation in robotics presents significant challenges due to uncertainties in component properties, diverse configurations, visual interference, and ambiguous prompts. These factors complicate both perception and control tasks. To address these challenges, we propose a novel method for One-Shot Affordance Grounding of Deformable Objects (OS-AGDO) in egocentric organizing scenes, enabling robots to recognize previously unseen deformable objects with varying colors and shapes using minimal samples. Specifically, we first introduce the Deformable Object Semantic Enhancement Module (DefoSEM), which enhances hierarchical understanding of the internal structure and improves the ability to accurately identify local features, even under conditions of weak component information. Next, we propose the ORB-Enhanced Keypoint Fusion Module (OEKFM), which optimizes feature extraction of key components by leveraging geometric constraints and improves adaptability to diversity and visual interference. Additionally, we propose an instance-conditional prompt based on image data and task context, which effectively mitigates the issue of region ambiguity caused by prompt words. To validate these methods, we construct a diverse real-world dataset, AGDDO15, which includes 15 common types of deformable objects and their associated organizational actions. Experimental results demonstrate that our approach significantly outperforms state-of-the-art methods, achieving improvements of 6.2%, 3.2%, and 2.9% in KLD, SIM, and NSS metrics, respectively, while exhibiting high generalization performance. Source code and benchmark dataset are made publicly available at https://github.com/Dikay1/OS-AGDO.

A primary challenge in robotic tool use is achieving precise manipulation with dexterous robotic hands to mimic human actions. It requires understanding human tool use and allocating specific functions to each robotic finger for fine control. Existing work has primarily focused on the overall grasping capabilities of robotic hands, often neglecting the functional allocation among individual fingers during object interaction. In response to this, we introduce a semantic knowledge-driven approach to distribute functions among fingers for tool manipulation. Central to this approach is the finger-to-function (F2F) knowledge graph, which captures human expertise in tool use and establishes relationships between tool attributes, tasks, and manipulation elements, including functional fingers, components, required force, and gestures. We also develop a manipulation element-oriented prediction algorithm using knowledge graph semantic embedding, enhancing the prediction of manipulation elements' speed and accuracy. Additionally, we propose the functionality-integrated adaptive force feedback manipulation (FAFM) module, which integrates manipulation elements with adaptive force feedback to achieve precise finger-level control. Our framework does not rely on extensive annotated data for supervision but utilizes semantic constraints from F2F to guide tool manipulation. The proposed method demonstrates superior performance and generalizability in real-world scenarios, achieving an 8% higher success rate in grasping and manipulation of representative tool instances compared to the existing state-of-the-art methods. The dataset and code are available at https://github.com/yangfan293/F2F.

To enable robots to use tools, the initial step is teaching robots to employ dexterous gestures for touching specific areas precisely where tasks are performed. Affordance features of objects serve as a bridge in the functional interaction between agents and objects. However, leveraging these affordance cues to help robots achieve functional tool grasping remains unresolved. To address this, we propose a granularity-aware affordance feature extraction method for locating functional affordance areas and predicting dexterous coarse gestures. We study the intrinsic mechanisms of human tool use. On the one hand, we use fine-grained affordance features of object-functional finger contact areas to locate functional affordance regions. On the other hand, we use highly activated coarse-grained affordance features in hand-object interaction regions to predict grasp gestures. Additionally, we introduce a model-based postprocessing module that transforms affordance localization and gesture prediction into executable robotic actions. This forms GAAF-Dex, a complete framework that learns granularity-aware affordances from human-object interaction to enable tool-based functional grasping with dexterous hands. Unlike fully supervised methods that require extensive data annotation, we employ a weakly supervised approach to extract relevant cues from exocentric (Exo) images of hand-object interactions to supervise feature extraction in egocentric (Ego) images. To support this approach, we have constructed a small-scale dataset, functional affordance hand (FAH)-object interaction dataset, which includes nearly 6k images of functional hand-object interaction Exo images and Ego images of 18 commonly used tools performing six tasks. Extensive experiments on the dataset demonstrate that our method outperforms state-of-the-art methods, and real-world localization and grasping experiments validate the practical applicability of our approach. The source code and the established dataset are available at https://github.com/yangfan293/GAAF-DEX.

Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To tackle these challenges, we propose a Temporal-Logic-guided Hybrid policy framework (HyTL) which leverages three-level decision layers to improve the agent’s performance. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy selects which behavior primitives to execute, and the low-level policy specifies the corresponding parameters to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. Our project is available at: https://sites.google.com/view/hytl-0257/.

This paper addresses the generation mechanism and avoidance method of negative stiffness during in-Hand manipulation with underactuated compliant hands. Firstly, a planar hand with two three-jointed fingers manipulating a rectangular is set, and a quasi-static underactuated operation model is established. Secondly, based on this simulation model, we investigated the stiffness evolution during in-hand manipulation, and analyze the influence factors of system stiffness. Finally, a stiffness regulation method is developed to avoid negative stiffness during in-hand manipulation. The method is validated by simulation. The research results are beneficial to improve the performance of underactuated in-hand manipulation.

Hand synergy from neuroscience provides an effective tool for anthropomorphic hands to realize versatile grasping with simple planning and control. This paper aims to extend the synergy-inspired design from anthropomorphic hands to multi-fingered robot hands. The synergy-inspired hands are not necessarily humanoid in morphology but perform primary characteristics and functions similar to the human hand. At first, the biomechanics of hand synergy is investigated. Three biomechanical characteristics of the human hand synergy are explored as a basis for the mechanical simplification of the robot hands. Secondly, according to the synergy characteristics, a three-fingered hand is designed, and its kinematic model is developed for the analysis of some typical grasping and manipulation functions. Finally, a prototype is developed and preliminary grasping experiments validate the effectiveness of the design and analysis.

This article analyzes and designs the transmission structure for underactuated compliant mechanisms (UCMs). The transmission structure of UCMs consists of serial and parallel transmission chains. At first, the UCMs are classified systematically according to the number and distribution of the serial and parallel transmissions. Next, the active and passive transmission properties of motion and force in UCMs are analyzed on the defined four subspaces of tangent and cotangent spaces of joint space. Synthesizing the classification and the transmission properties of UCMs, the congruent relationship between mechanical structure and transmission function is established, and different cases of UCMs are discussed and compared. A novel type of UCMs can achieve the independent regulation of passive stiffness, active force, and active motion that is useful for improving the transmission performance in robotic and prosthetic hands. Finally, a functional oriented design method is proposed and used to design a single-actuator two-fingered gripper for enveloping and precision grasps. The results demonstrate the validity of the proposed method.

How to design an anthropomorphic hand with a few actuators to replicate the grasping functions of the human hand is still a challenging problem. This paper aims to develop a general theory for designing the anthropomorphic hand and endowing the designed hand with natural grasping functions. A grasping experimental paradigm was set up for analyzing the grasping mechanism of the human hand in daily living. The movement relationship among joints in a digit, among digits in the human hand, and the postural synergic characteristic of the fingers were studied during the grasping. The design principle of the anthropomorphic mechanical digit that can reproduce the digit grasping movement of the human hand was developed. The design theory of the kinematic transmission mechanism that can be embedded into the palm of the anthropomorphic hand to reproduce the postural synergic characteristic of the fingers by using a limited number of actuators is proposed. The design method of the anthropomorphic hand for replicating human grasping functions was formulated. Grasping experiments are given to verify the effectiveness of the proposed design method of the anthropomorphic hand.

There are few mathematical indices to quantize the adaptability, although it is mentioned repeatedly as a highlight of underactuated mechanisms in literatures. This paper discusses the grasp adaptability of underactuated mechanisms. Different from compliance, namely the reciprocal of stiffness, the adaptability is presented as the ability of adaptive motion. Two measures of adaptability are proposed from two aspects respectively: the impact on the grasped object and the enforcement cost of the underactuated mechanism. Based on the measures, the trend of the adaptive grasping process is predicted in enveloping grasp with a simple gripper, and a corresponding experimental prototype is set up to verify the calculated results. As an application, we apply the two measures to analyze and compare the adaptability of three typical underactuated mechanisms.

How to design a robotic hand reflecting human hand motion information as much as possible is a constantly exploring problem. In this paper, we propose an approach to mechanical design of compliant underactuated finger for prosthetic hand based on the decomposition of human hand movements. Hand movements are decomposed into primary and secondary motion in PCA coordinate system. The primary motion is achieved in free motion via actuators, and the secondary motion is implemented with mechanical compliance matching statistics characteristic of human motion data. Although analysis and design of single finger is always throughout this paper, the same method can be generalized to the whole hand design and the parameters design of other mechanical configuration.