Keyframe selection plays a crucial role in balancing computational efficiency and localization accuracy in Visual Simultaneous Localization and Mapping (VSLAM) systems. Existing keyframe selection methods often struggle to capture high-level semantic information in environments where multiple semantic dimensions interact. In this paper, we propose the Multi-dimensional Semantic Analysis (MSA) module based on Visual-Language Models (VLMs). By leveraging the capability of VLMs to extract rich semantic features, we compute the similarity between each image frame and a set of textual descriptions, generating a scene descriptor that quantifies the semantic distance between frames across multiple dimensions (e.g., object count, texture, and lighting). We then introduce the Scene Change Assessment (SCA) module based on Bayesian On-line Changepoint Detection (BOCD), which identifies keyframes with significant semantic information gain, thereby reducing the total number of keyframes. Extensive experiments on an open dataset demonstrate that our method not only significantly reduces the number of keyframes but also maintains high localization accuracy. Furthermore, the inference speed of the MSA module satisfies the real-time requirements of VSLAM. These results underscore the potential of our approach to enhance the efficiency of keyframe selection.

Visual Simultaneous Localization and Mapping (VSLAM) is a critical technology for autonomous driving and mobile robotics. Traditional VSLAM methods based on discrete representations, such as point clouds, offer high computational efficiency and excellent localization accuracy, but they exhibit limited robustness. In contrast, methods employing field representations, like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3D GS), provide greater robustness at the expense of increased computational demands and reduced localization accuracy. Hybrid VSLAM approaches that attempt to combine these representations typically rely on serial, synchronous cascades, which compromise robustness, computational efficiency, and GPU memory usage. This paper introduces a novel adaptive cross-representation VSLAM framework that applies different representation modeling techniques to distinct regions of an image sequence and adopts asynchronous parallel modeling in overlapping regions. A Vision Language Model (VLM) is used to analyze the image sequence, enabling the detection of representation modeling regions and adaptive switching between representations. Cross-representation data association is performed through a coarse-to-fine feature selection process, resulting in a globally consistent map. The proposed method is evaluated on both public and custom-collected datasets, where experimental results show that it surpasses state-of-the-art methods in terms of robustness, computational efficiency, localization accuracy, and GPU memory usage.

3D object localization and intention prediction have become crucial components in autonomous system applications, such as self-driving car. However, there still faces a lot of challenges, especially for complex and dynamic scenarios where a single modality information is insufficient to effectively and precisely localize the position and analyze the intention of objects. An improved method based on multimodal information fusion has been proposed via leveraging the advantages of 2D image segmentation and 3D geometrical characteristics of LiDAR point cloud. Extensive comparative experiments have been conducted and the results demonstrate that the proposed method significantly enhances both localization and prediction accuracy, comparing with the method where 2D bounding box of object instead of segmentation information is used to be fused with point cloud.

Graph matching refers to establishing correspondence between two sets of point while keeping consistency between their edge sets. Recent works in learning-based graph matching have attempted to solve the problem either by linear assignment, which transfers local structure information into node embedding at individual graphs, or by quadratic assignment through vertex classification over their association graph. However, the former embedding-based pipeline methods often neglect second-order edge similarity, leading to decreased accuracy; while the latter quadratic assignment solvers consume significant memory due to huge computation on the association graph. To addressthese issues, our key idea is to integrate a factorized embedding module to efficiently propogate information over the association graph. To this end, we propose a novel factorized embedding-based network, namely FEGM, which takes into account the secondorder edge similarity, as well as a factorization model of GCN network, so that we extend the embedding-based pipeline for learning the Lawler’s QAP while reducing memory consumption. Experimental results show that FEGM achieves a competitive matching accuracy while being superior in time and space efficiency.

Limited by the working principles, LiDAR-SLAM systems suffer from the degeneration phenomenon in environments such as long corridors and tunnels, due to the lack of sufficient geometric features for frame-to-frame matching. The accuracy and sensitivity of existing degeneracy detection methods need to be further improved. In this paper, we propose a novel method for degeneracy detection using local geometric models based on point-to-distribution matching. To obtain an accurate description of local geometric models, an adaptive adjustment of voxel segmentation according to the point cloud distribution and density is designed. The codes of the proposed method is open-source and available at https://github.com/jisehua/Degenerate-Detection.git. Experiments with public datasets and self-build robots were conducted to evaluate the methods. The results exhibit that our proposed method achieves higher accuracy than the other existing approaches. Applying our proposed method is beneficial for improving the robustness of the LiDAR-SLAM systems.

Large-scale point cloud consists of a multitude of individual objects, thereby encompassing rich structural and underlying semantic contextual information, resulting in a challenging problem in efficiently segmenting a point cloud. Most existing researches mainly focus on capturing intricate local features without giving due consideration to global ones, thus failing to leverage semantic context. In this paper, we propose a Similarity-Weighted Convolution and local-global Fusion Network, named SWCF-Net, which takes into account both local and global features. We propose a Similarity-Weighted Convolution (SWConv) to effectively extract local features, where similarity weights are incorporated into the convolution operation to enhance the generalization capabilities. Then, we employ a downsampling operation on the K and V channels within the attention module, thereby reducing the quadratic complexity to linear, enabling Transformer to deal with large-scale point cloud. At last, orthogonal components are extracted in the global features and then aggregated with local features, thereby eliminating redundant information between local and global features and consequently promoting efficiency. We evaluate SWCF-Net on large-scale outdoor datasets SemanticKITTI and Toronto3D. Our experimental results demonstrate the effectiveness of the proposed network. Our method achieves a competitive result with less computational cost, and is able to handle large-scale point clouds efficiently. The code is available at https://github.com/Sylva-Lin/SWCF-Net.

Robust and accurate object association is essential for precise 3D object landmark inference in semantic Simultaneous Localization and Mapping (SLAM), and yet remains challenging due to the detection deficiency caused by high miss detection rate, false alarm, occlusion and limited field-of-view, etc. The 2D location of an object is a crucial complementary cue to the appearance feature, especially in the case of associating objects across frames under large viewpoint changes. However, motion model or trajectory pattern based methods struggle to infer object motion reliably with a moving camera. In this paper, by exploiting the local projective warping consistency, a local homography based 2D motion inference method is proposed to sequentially estimate the object location along with uncertainty. By integrating the deep appearance feature and semantic information, an object association method, named HOA, which is robust to detection deficiency is proposed. Experimental evaluations suggest that the proposed motion prediction method is capable of maintaining a low cumulative error over a long duration, which enhances the object association performance in both accuracy and robustness. Note to Practitioners—This work aims to consistently associate 2D detection boxes corresponding to the same 3D object across images. In tasks of landmark-based navigation, collision avoidance, grasping and manipulation, objects in the task space are commonly simplified into 3D enveloping surfaces (e.g. cuboid or ellipsoid) by using 2D object detection boxes from multiple image views, and accurate data association is a prerequisite for precise enveloping surface reconstruction. This problem remains challenging considering the imperfect object detections, the appearance similarity of objects and the unpredictable trajectory of the moving camera. This work proposes a long-term reliable 2D location prediction algorithm that is capable of handling the complex motion of the target. Along with the appearance feature extracted by a retrain-free deep learning based model, this work proposes an object association method that can simultaneously deal with multiple objects with unknown object categories under the moving camera scenario.

The utilization of cloud infrastructure and its extensive range of Internet-accessible resources holds significant potential for advancing intelligent transportation and robotics. Over the past two decades, interest in cloud-edge collaborative simultaneous localization and mapping (SLAM) has grown markedly. Consequently, a comprehensive review of current trends in this field is crucial for both novice and experienced researchers. This paper examines robots and automation systems that rely on network-based data or code, particularly in the context of SLAM development. Applying SLAM to mobile robots with limited computing power is essential for achieving autonomous navigation, and cloud-edge collaborative SLAM has emerged as an efficient solution. The review is structured around four key benefits of cloud-edge collaborative SLAM: Assisted Cloud Computing, which provides access to cloud computation and reduces the burden on edge devices; Total Cloud Computing, where the majority of computation is offloaded to the cloud, while edge devices primarily handle sensing and low-cost pre-processing; Data Storage, enabling access to large datasets, such as high-resolution environment maps and extensive training datasets, enhancing overall performance; and Data Transmission, involving cloud-edge communication for efficient data transfer and data association. Additionally, we address the challenges in existing work and the development of asynchronous collaboration and implicit representation transmission, which could mitigate transmission latency in communication-constrained environments. We believe that this review will bridge the gap between SLAM systems and deployed robotic systems, promoting the advancement of cloud-edge collaborative SLAM.

Visual simultaneous localization and mapping (VSLAM) is a crucial technology in intelligent vehicles that relies on either explicit or implicit representations. Explicit methods are prevalent in real-time systems, offer precise geometric control, and are easy to visualize. However, they struggle with complex, dynamic environments and require high storage capacity. On the other hand, implicit techniques excel in handling intricate, changing shapes due to their compact representation and inference ability while requiring more complex display and rendering processes. A combination of both types of representations could significantly enhance the performance of VSLAM, but the cross-data association method for standalone explicit and implicit representations is still lacking. To this end, this paper proposes a data association scheme that bridges the gap between explicit and implicit representations by individually modeling the uncertainties in each representation. Our approach features a multi-level feature selection process tailored for data association. It initially extracts coarse-level features during explicit representation generation based on Bayesian estimation and refines them using the implicit representation based on ray sampling, which enhances robustness while reducing rendering costs. We rigorously evaluated our proposed methodology against current state-of-the-art approaches using public datasets and real robot scenes. The results show that our coarse-to-fine feature selection method outperforms existing techniques both quantitatively and qualitatively, suggesting its potential to significantly boost the contemporary VSLAM system performance.

Visual relocalization is a fundamental problem in computer vision and robotics. Recently, regression-based methods become popular and they can be categorized into two classes: absolute pose regression and scene coordinate regression. In this work, we present a combined regression network that jointly learns scene coordinate regression and absolute pose regression for single-image visual relocalization. The proposed network composes of a feature encoder and two regression branches with uncertainty modeling. In particular, we design a deep feature conditioning module, aiming at propagating the coarse pose information in absolute pose regression to inform the predictions in scene coordinate regression. The proposed network is trained in an end-to-end fashion to learn both regression tasks. Moreover, we propose an uncertainty-driven RANSAC algorithm that incorporates the predicted scene coordinates and their uncertainties to solve the camera pose during inference. To the best of our knowledge, this work is the first to combine scene coordinate regression and pose regression in a hierarchical framework for visual relocalization. Experiments on indoor and outdoor benchmarks demonstrate the effectiveness and the superiority of the proposed method over the state-of-the-art methods.

As a hard high-altitude work, power transmission line inspection increasingly demands robots to conduct in place of the human being. A variety of robots have been developed to this end, with basic locomotion and inspection implemented on the lines. However, most current line inspection robots (LIRs) are only mobile platforms with complex structures and large weights, lacking sufficiently dexterous locomotion on lines, especially for obstacle overcoming and line transition. Also with sensors fixed on the platform, the inspection range is largely limited. For higher mobility and a larger inspection range, a novel biped robot that can roll and climb on a power transmission line for inspection, called Climbot-L, is proposed in this paper. While the rolling mode has the advantage of high locomotion efficiency, the biped climbing mode makes it possible to easily overcome obstacles on the line, transition to adjacent cables, and have multi-view detection. In this paper, the design of this novel robot is first introduced, the working principle of the wheel-gripper modules is then analyzed, and obstacle overcoming gaits are stated. The effectiveness and high maneuverability of the presented robot are verified by a series of experiments.

Loop closure detection is a key technology for long-term robot navigation in complex environments. In this paper, we present a global descriptor, named Normal Distribution Descriptor (NDD), for 3D point cloud loop closure detection. The descriptor encodes both the probability density score and entropy of a point cloud as the descriptor. We also propose a fast rotation alignment process and use correlation coefficient as the similarity between descriptors. Experimental results show that our approach outperforms the state-of-the-art point cloud descriptors in both accuracy and efficency. The source code is available and can be integrated into existing LiDAR odometry and mapping (LOAM) systems.

Point cloud is a discrete and unordered expression of 3D data. A lot of methods have been proposed to solve the problem in 3D object classification and scene recognition. To handle the huge amount of unordered point cloud, down-sampling before processing is needed. The shortage of existing sampling methods is the lack of geometry information consideration, which is essential for point cloud classification and segmentation tasks. Our method is mainly motivated by the observation that points with a high curvature variation can depict the outlines of objects. Thus, we propose a curvature variation based sampling method for point cloud classification and segmentation tasks. We aim to sample points with high curvature variations, which are considered to be more suitable for classification and segmentation tasks than the traditional sampling method. We combine the proposed sampling algorithm with the existing sampling method for multiple information fusion, and a higher accuracy and mean IoU can be achieved. The experimental results verify the advantage of considering curvature variation in classification and segmentation tasks.

Robot motion control aims to generate control inputs for a robotic system to track a planned trajectory. Feedback provided by sensors plays an essential role in motion control by improving system performance when external disturbances and/or initial errors exist. However, feedback signals, such as images are often of a large size, which imposes a heavy computational burden on the system. In this paper, a new robot motion control scheme is proposed based on compressive feedback to improve feedback rate. The controller is designed in non-vector space using compressive feedback. As an application, visual servoing is formulated under the proposed framework by considering a feedback image as a set, instead of a traditional feature vector. Experiments are conducted to validate the proposed scheme.

Recent works on semantic Simultaneous Localization and Mapping (SLAM) utilizing object landmarks have shown superiority in terms of robustness and accuracy in tracking and localization. 3D object landmarks represented by a cubic or quadric surface are inferred from 2D object bounding boxes which are typically captured from multiple views by an object detector. Nevertheless, bounding box noises and small camera baseline may lead to an inaccurate 3D object landmark inference. Inspired by the dual quadric enveloping property, in this work, we introduce the horizontal support assumption to constrain rotation w.r.t. roll and pitch for a quadric representation. As the result, we reduce the number of quadric parameters and narrow down the solution space, and ultimately produce a relatively accurate inference. Extensive experimental evaluations under both simulated and real scenarios are conducted in this paper. Quantitative results demonstrate that our approach outperforms the state-of-the-art.

Spatio-temporal information is key to resolve occlusion and depth ambiguity in 3D human pose estimation. Previous methods have focused on either temporal contexts or local-to-global architectures that embed fixed-length spatiotemporal information. To date, there have not been effective proposals to simultaneously and flexibly capture varying spatiotemporal sequences and effectively achieves real-time 3D human pose estimation. In this work, we improve the learning of kinematic constraints in the human skeleton: posture, local kinematic connections, and symmetry by modeling local and global spatial information via attention mechanisms. To adapt to single- and multi-frame estimation, the dilated temporal model is employed to process varying skeleton sequences. Also, importantly, we carefully design the interleaving of spatial semantics with temporal dependencies to achieve a synergistic effect. To this end, we propose a simple yet effective graph attention spatio-temporal convolutional network (GAST-Net) that comprises of interleaved temporal convolutional and graph attention blocks. Experiments on two challenging benchmark datasets (Human3.6M and HumanEva-I) and YouTube videos demonstrate that our approach effectively mitigates depth ambiguity and self-occlusion, generalizes to half upper body estimation, and achieves competitive performance on 2D-to-3D video pose estimation. Code, video, and supplementary information is available at: http://www.juanrojas.net/gast/

Soft climbing robots have been attracting increasing attention in soft robotics community, and a lot of prototypes been proposed with basic climbing function implemented. Climbing on poles is a challenge with soft robots, and the capability of current pole-climbing soft robots needs to be improved in terms of adaptability to various poles and deformation controllability or constraining of the soft body. In this paper, a rigid-compliant coupling or constraint method is proposed in design of pole climbing robots. Specifically, a novel gripper with inner expanding bubbles and a rigid-compliant belt is presented and designed, featured with excellent shape/size-adaptability and grip-reliability. To constrain undesired deformation of the soft body, rigid-compliant belts are also adopted. Three rigid-compliant constrained actuators are connected to implement an Ω-shaped deformation of the body for climbing motion. The kinematic feature of the body is established, and the two main features of the inner expanding gripper are analyzed. A series of experiments, where the robot climbs on poles with different shapes and sizes and in different poses, have verified the effectiveness of the presented rigid-compliant constraint and the shape/size adaptability and grip-reliability of the novel inner expanding grippers.

This article aims to extend the theory of Lie subgroups and symmetric subspaces for studying an important class of submanifolds of the special Euclidean group SE(3) whose tangent space at each point on the submanifold relates to that at the identity by an adjoint map. These submanifolds, which we call adjoint-invariant submanifolds in this article, are known in the literature as persistent submanifolds, since they are strictly related to the concept of persistent screw systems. The difference is that in this article, just as Lie subgroups and symmetric subspaces, we put forward adjoint-invariant submanifolds as independent geometric objects from mechanisms and their associated local screw systems. Adjoint invariance relaxes the strict left and right invariance of Lie subgroups and the reflective invariance of symmetric subspaces by allowing generic moving reference frame in the aforementioned adjoint map. It turns out such adjoint invariance can be studied under the framework of distributions on manifolds, which allows us to explore global geometric properties of adjoint-invariant submanifolds. We classify adjoint-invariant submanifolds into reflective-type and product-type submanifolds and derive the conditions for their adjoint invariance. We then propose geometric methods and algorithms for synthesizing the kinematic generators for reflective-type submanifolds, as demonstrated with a number of examples.

In order to improve the robot's perception ability in the complicated environment, especially the unstructured environment, a framework of 3D object detection and pose estimation using single shot detector (SSD) and modified LineMOD template matching is proposed, which can detect multiple objects and estimate their pose simultaneously. Firstly, the initial object detection (the first detection) is realized by single shot detector network and therefore the region of interest (RoI) of target objects are generated. LineMOD template matching is then applied to provide candidate templates. These calculated templates are grouped by the designed clustering algorithm. After sorting the clusters according to the descending order of the average similarity, non-maximum suppression removes the similar results and provide the further multiple detection results (the second detection). Finally, based on the results from the second detection, the pose of the object is estimated by using iterative closest point (ICP) algorithm. The object detection experiments show that on Tejani dataset, the average recognition rate of six objects reaches 99.25%. For the object pose estimation, F1 of the proposed method is 21.7% higher than the conventional method in the pose estimation experiments. Also, F1 of the presented algorithm is 9.5% higher than Deep-6Dpose method. Both comparison experiments verify the effectiveness of the proposed framework. Further, this framework for object detection and pose estimation is employed to do robotic grasping. In particular, the workpiece of steel plates is grabbed, which is a necessary procedure of the polishing technique.

We propose an efficient spectral clustering method for large-scale data. The main idea in our method consists of employing random Fourier features to explicitly represent data in kernel space. The complexity of spectral clustering thus is shown lower than existing Nyström approximations on largescale data. With m training points from a total of n data points, Nyström method requires O(nmd + m3+ nm2) operations, where d is the input dimension. In contrast, our proposed method requires O(nDd + D3+ n'D2), where n' is the number of data points needed until convergence and D is the kernel mapped dimension. In large-scale datasets where n ≪ n hold true, our explicitly mapping method can significantly speed up eigenvector approximation and benefit prediction speed in spectral clustering. For instance, on MNIST (60000 data points), the proposed method is similar in clustering accuracy to Nyström methods while its speed is twice as fast as Nyström.

Climbing or crawling robots may be widely applied in agriculture, forestry, military, construction industry, disaster searching and rescuing, and so on. Soft robots possess better safety, flexibility, dexterity, portability, and adaption to complex environments than traditional robots. However, there are big challenges in system development, modeling and control of soft climbing robots. To address system development of a soft robot as a new type climbing robot, we present a pneumatic soft robot capable of inchworm-like locomotion, PISRob. The presented robot is composed of three soft parts in H-shaped configuration. Each part is able to perform 2D bending. While the middle part, as the main body, can bend in Ω -shape for actuation, the two end parts as legs can conduct simple bending motion for grasping or anchoring during locomotion. The system design and fabrication process of the soft robot is presented in details in this paper. A control system is developed for pneumatic actuation of the robot. Tests are carried out to get the relationship between the actuating air pressure and the step length in locomotion. Experiments of crawling on a floor and climbing on a pole are performed to verify the feasibility of development of the new soft robot and the effectiveness of the control method for the pneumatic system.

For VSLAM (Visual Simultaneous Localization and Mapping), localization is a challenging task, especially for some challenging situations: textureless frames, motion blur, etc. To build a robust exploration and localization system in a given space, a submap-based VSLAM system is proposed in this paper. Our system uses a submap back-end and a visual front-end. The main advantage of our system is its robustness with respect to tracking failure, a common problem in current VSLAM algorithms. The robustness of our system is compared with the state-of-the-art in terms of average tracking percentage. The precision of our system is also evaluated in terms of ATE (absolute trajectory error) RMSE (root mean square error) comparing the state-of-the-art. The ability of our system in solving the “kidnapped” problem is demonstrated. Our system can improve the robustness of visual localization in challenging situations.

Event detection is a critical feature in data-driven systems as it assists with the identification of nominal and anomalous behavior. Event detection is increasingly relevant in robotics as robots operate with greater autonomy in increasingly unstructured environments. In this work, we present an accurate, robust, fast, and versatile measure for skill and anomaly identification. A theoretical proof establishes the link between the derivative of the log-likelihood of the HMM filtered belief state and the latest emission probabilities. The key insight is the inverse relationship in which gradient analysis is used for skill and anomaly identification. Our measure showed better performance across all metrics than all but one related state-of-the-art works. The result is broadly applicable to domains that use HMMs for event detection. Supplemental information, code, data, and videos can be found at [1].

Robots are increasingly entering uncertain and unstructured environments. Within these, robots are bound to face unexpected external disturbances like accidental human or tool collisions. Robots must develop the capacity to respond to unexpected events. That is not only identifying the sudden anomaly, but also deciding how to handle it. In this work, we contribute a recovery policy that allows a robot to recovery from various anomalous scenarios across different tasks and conditions in a consistent and robust fashion. The system organizes tasks as a sequence of nodes composed of internal modules such as motion generation and introspection. When an introspection module flags an anomaly, the recovery strategy is triggered and reverts the task execution by selecting a target node as a function of a state dependency chart. The new skill allows the robot to overcome the effects of the external disturbance and conclude the task. Our system recovers from accidental human and tool collisions in a number of tasks. Of particular importance is the fact that we test the robustness of the recovery system by triggering anomalies at each node in the task graph showing robust recovery everywhere in the task. We also trigger multiple and repeated anomalies at each of the nodes of the task showing that the recovery system can consistently recover anywhere in the presence of strong and pervasive anomalous conditions. Robust recovery systems will be key enablers for long-term autonomy in robot systems. Supplemental information including videos, code, and result analysis can be found at [1].

Re-configurable modular robotic (RMR) systems are advantageous for their reconfigurability and versatility. A new modular robot can be built for a specific task by using modules as building blocks. However, constructing a kinematic model for a newly conceived robot requires significant work. Due to the finite size of module-types, models of all module-types can be built individually and stored in a database beforehand. With this priori knowledge, the model construction process can be automated by detecting the modules and their corresponding interconnections. Previous literature proposed theoretical frameworks for constructing kinematic models of modular robots, assuming that such information was known a priori. While well-devised mechanisms and built-in sensors can be employed to detect these parameters, they significantly complicate the module design and thus are expensive. In this paper, we propose a vision-based method to identify kinematic chains and automatically construct robot models for modular robots. Each module is affixed with augmented reality (AR) tags that are encoded with unique IDs. An image of a modular robot is taken and the detected modules are recognized by querying a database that maintains all module information. The poses of detected module-links are used to compute: (i) the connection between modules and (ii) joint angles of joint-modules. Finally, the robot serial-link chain is identified and the kinematic model is constructed and visualized. Our experimental results validate the effectiveness of our approach. While implementation with only our RMR is shown, our method can be applied to other RMRs where self-identification is not possible.

An important problem in robot simultaneous localization and mapping (SLAM) is loop closure detection. Recent studies of the problem have led to successful development of methods that are based on images captured by the robot. These methods tackle the issue of efficiency through data structures such as indexing and hierarchical (tree) organization of the image data that represent the robot map. In this paper, we offer an alternative approach and present a novel method for visual loop-closure detection. Our approach uses an extremely simple image representation, namely, a down-sampled binarized version of the original image, combined with a highly efficient image similarity measure - mutual information. As a result, our method is able to perform loop closure detection in a map with 20 million key locations in about 2.38 seconds on a commodity computer. The excellent performance of our method in terms of its low complexity and accuracy in experiments establishes it as a promising solution to loop closure detection in large-scale robot maps.

Combining the advantages of active and passive walking robots, we have developed a novel active biped walking robot with only six DoFs. The robot is built with six 1-DoF joint modules and two wheels as the feet. It achieves locomotion in special gaits different from those of traditional biped robots. In this paper, this novel biped robot is introduced, and four walking gaits, namely turning-around gait, foot-wheel hybrid gait, side-stepping gait and turning-over gait are proposed, and their walking patterns and motion planning are presented and analyzed. Walking experiments are carried out to verify the locomotion function, the effectiveness of the presented gaits and to illustrate the features of this novel biped robot. It has been shown that biped active walking may be achieved with only a few DoFs and simple kinematic configuration.

High-rise tasks in agriculture, forestry and building industry requires robots possessing climbing function. Motivated by these potential applications and inspired by the climbing motion of animals such as inchworms, we have developed a novel biped climbing robot - Climbot. Built with a modular approach, the robot consists of five 1-DoF joint modules connected in series and two special grippers mounted at the ends. With this configuration, Climbot is able not only to climb a variety of media, but also to grasp and manipulate objects, and hence is a ¿mobile¿ manipulator. In this paper, we first introduce the development of this novel robot, and then illustrate three climbing gaits based on the unique configuration of the robot. Experiments of climbing poles are carried out to verify the climbing functions and to demonstrate potential application of the proposed robot.

Modules have been widely used in the development of re-configurable robots and snake-like robots. Modular methodology can also be applied in design of other robots. To build robots flexibly and quickly with low costs, we have developed two basic joint modules and several functional modules including grippers, suckers and wheels/feet as end-effectors. In this paper, we introduce the development of these modules, and present several novel robots built using them. Specifically, we show how to use them to set up a manipulator, a 6-DoF biped walking robot, a wheeled mobile robot, a biped tree-climbing robot, and a biped wall-climbing robot. It has been shown that a few modules can easily spawn a variety of novel robots with modular methodology.

In this paper, we propose a numerical approach to generate the workspace of a multifingered robotic hand manipulating an object in the 3-D case. Based on feasibility analysis of grasps, the proposed approach uses a numerical optimization technique to first compute discretely the boundary of the possible motion of the grasped object, and then the limits of rotation about various axes at a specified feasible position of the object. Finally the boundaries of the linear motion and rotation are visualized in 3-D coordinates separately. This approach provides an effective solution to the challenging problem of workspace analysis of 3-D multifingered manipulation.

For planning and implementation of manipulation tasks using one or two arms/hands of a humanoid robot, which would be the most common application of the robot in the future, it is undoubtedly important to know the workspace of the robot. In this paper, we study the reachable space or boundary of an arm/hand when the humanoid robot stands on the ground with its two feet fixed. Since a humanoid robot has more degrees of freedom than a traditional robot and special characters that the latter does not possess, it would be very difficult or impractical to use conventional methods to analyze and obtain its reachable space. Here we propose a numerical approach to analyze and generate humanoid reachable space. Taking into account the robot balance, the geometric and kinematic constraints, we use optimization technique to build mathematic models for the reachable boundary. This discrete method gives rise to an approximation of the reachable space though, it is adequate for practical application. The proposed algorithm is illustrated by an example conducted on the humanoid platform HRP-2, and can be extended to more complex cases

In view of the importance of workspace to robotic motion planning and control, we study the reachable space of a humanoid robot in standing postures. Since it would be very difficult or impractical to use conventional analytical method to analyze and obtain the workspace of a humanoid robot, we present a numerical approach in this paper using the Monte Carlo method. As a random sampling numerical method, the Monte Carlo method is relatively simple and flexible to apply, and hence is suitable for the workspace generation of a humanoid robot. After formulating the basic constraints that a humanoid robot must satisfy in a manipulation task to enable the Monte Carlo method in generation of the reachable space, we present the algorithm and a method to build a database for utilization of the numerical results in application. We illustrate this method by an example conducted on the humanoid platform HRP-2. The results show that this method is feasible and practical for the generation and visualization of workspace of a humanoid robot

The wide potential applications of humanoid robots require that the robots can walk in complex environments and overcome various obstacles. To this end, we address the problem of humanoid robots stepping over obstacles in this paper. We focus on two aspects, which are feasibility analysis and motion planning. The former determines whether a robot can step over a given obstacle, and the latter discusses how to step over, if feasible, by planning appropriate motions for the robot. We systematically examine both of these aspects. In the feasibility analysis, using an optimization technique, we cast the problem into global optimization models with nonlinear constraints, including collision-free and balance constraints. The solutions to the optimization models yield answers to the possibility of stepping over obstacles under some assumptions. The presented approach for feasibility provides not only a priori knowledge and a database to implement stepping over obstacles, but also a tool to evaluate and compare the mobility of humanoid robots. In motion planning, we present an algorithm to generate suitable trajectories of the feet and the waist of the robot using heuristic methodology, based on the results of the feasibility analysis. We decompose the body motion of the robot into two parts, corresponding to the lower body and upper body of the robot, to meet the collision-free and balance constraints. This novel planning method is adaptive to obstacle sizes, and is, hence, oriented to autonomous stepping over by humanoid robots guided by vision or other range finders. Its effectiveness is verified by simulations and experiments on our humanoid platform HRP-2.

The wide and potential applications of humanoid robots require that the robots can walk in complex environments cluttered with various obstacles. In such environments, the robots often need to overcome obstacles by stepping-over, stepping-on/down, or stepping-around. Before the implementation, it must be made sure whether the robots can do so, which is the so-called feasibility. In this paper, we give a more thorough feasibility analysis and more practical results of stepping-over and stepping-on/down, by extending our previous analyses in 2-D case to 3-D motion of the robot. Considering the geometry of the obstacle, the shapes and sizes, and kinematics of the robot legs, we use optimization technique to build corresponding models with nonlinear constraints (including those on collision avoidance and robot balance) to find the maximum height of the obstacle that the robot can step over or on/down. The results can be used as a prior knowledge and a database for motion planning of obstacle overcoming. The feasibility also provides an approach for evaluating the humanoid capability of obstacle overcoming.

In this paper, we address the problem of how a humanoid robot can step over a given obstacle. Obstacle stepping-over has two aspects, namely, feasibility analysis and motion planning. The former determines whether the robot can step over the obstacle, and the latter discusses how to realize the stepping-over, if it is feasible, by trajectory planning. The paper focuses on the latter. Specifically, based on our previous analysis of feasibility, we present a novel algorithm to plan suitable trajectories for obstacle stepping-over, taking into account two basic requirements. The first requirement is to avoid any collision between the robot and the obstacle, and the second to maintain stability or balance of the robot. To meet them, we decompose the whole body motion of the robot into two parts, corresponding to the upper body motion and the lower body, respectively. We first plan collision-free trajectories of the feet and the waist for lower body motion, and then adjust upper body motion by resolved momentum control to guarantee the robot stability. This novel planning method is adaptive to obstacle sizes and hence oriented to autonomous stepping-over of humanoid robots guided by vision or other range finders. Its effectiveness is shown by simulation and experiment on our humanoid platform HRP-2.

It is believed that humanoid robots have better mobility than other mobile robots. However, little work has been reported on this mobility for nontrivial motion such as walking on rough terrains and overcoming obstacles. In this paper, we address the problem of obstacle overcoming by stepping-over, focusing on the feasibility analysis. That is, given an obstacle to overcome, we determine if the robot can step over it. In our analysis, we present a technique of global optimization under constraints to obtain the maximum height or width of an obstacle that the robot can step over while maintains the stability and avoids any collision between the robot legs and the obstacle. This analysis is a basis of motion planning for the robots to step over obstacles. It also provides an approach for evaluating the capability of obstacle overcoming of humanoid robots.

In this paper, we develop a novel approach to evaluating the feasibility of a planar grasp of an arbitrary polygonal object by a multifingered hand. Our definition of a feasible grasp takes into account both kinematic constraints and force constraints, and the algorithm is capable of handling power grasps involving finger tips as well as inner links of a dextrous hand. The proposed approach is based on first expressing the kinematic and force constraints in terms of equalities and inequalities in variables of the object and hand configurations, and then formulating and solving grasp feasibility as a constrained global optimization problem. An example is provided to illustrate the algorithm.

The knowledge of the workspace of a multi-fingered hand is very important in planning a dexterous manipulation task. In this paper, we propose a novel approach to compute and visualize the workspace of a multifingered robotic hand manipulating an object in the planar case. Based on optimization models, our approach is numerical, in which the kinematic feasibility of a grasp at a given position is determined first, and then the range of rotation at this position is computed. The algorithm and its effectiveness are illustrated by examples. It is possible to extend our approach to the spatial case.

Planning of a dextrous manipulation task for a multifingered hand requires the feasibility of all the grasps involved throughout the manipulation process. In this paper, we address the problem of determining whether a desired grasp of a polyhedral object is kinematically feasible. In our study, we define a grasp in terms of a system of contact pairs between the topological features of the hand and the object, and formulate the grasp feasibility analysis as a set of equality and inequality constraints in the variables of the hand and object configurations. The feasibility of a grasp then becomes equivalent to the simultaneous satisfaction of all the constraints. This allows us to cast the feasibility analysis conveniently as a constrained nonlinear optimization problem and solve it numerically with commercially available software. The effectiveness of our approach is illustrated with an example of grasping a cuboid using a three-fingered robotic hand.

To enhance the dexterity of a tele-manipulation system and to provide a more efficient and intuitive user interface, we have developed an integrated system for remote dextrous manipulation using a multifingered robotic hand through the Internet. This system consists of a three-fingered hand for real-time execution and a graphic simulator for manipulation simulation and command input. We describe the main issues in the development of this system, including system architecture, functions, integration and user interface. We also provide some experiments of remote manipulation of typical objects with this system. Our work verifies the feasibility of such an integrated system, and provides a new approach to the potential application of a multifingered robotic hand.

We present a solution to the problem of determining if it is possible for a given dexterous hand to grasp a polyhedral object at a desired topological configuration, subject to kinematic constraints. We refer to this problem as kinematic feasibility analysis. In our study, we define a desired grasp in terms of a set of contact pairs between the topological features of the hand and the object, and formulate a general algorithm that makes use of constrained optimization with nonlinear constraints to determine the feasibility. We first derive the conditions in order for the hand to make the desired contact and avoid undesirable collision. These conditions are expressed in terms of equalities and inequalities in the configuration variables of the hand and object, which can then easily be transformed into a constrained nonlinear optimization problem. Numerical examples are provided to illustrate the solution.

We describe a solution to the problem of determining the kinematic feasibility of a dextrous hand to grasp a given object at a desired configuration in the two dimensional space. We refer to this problem as kinematic graspability. The kinematic configuration of a grasp is defined in terms of a set of contact pairs between the topological features of the hand and the object. We derive the sufficient conditions in order for the hand to make the desired contact and avoid collision. These conditions are then formulated as a constrained nonlinear global optimization problem, whose solution yields a definitive answer to kinematic graspability for a grasp configuration. Numerical examples are provided to illustrate the method.