The biased scene graph generation problem arises from the inherent long-tailed distributions of predicates, which are challenging to handle effectively with a single network. In this paper, we introduce a novel framework called peer learning, designed to address the issue of unbiased scene graph generation (USGG) through a divide-and-vote approach. To address the long-tailed problem, our framework operates in three steps. Firstly, we partition the heavily long-tailed distribution into subsets of more balanced sub-distribution groups, including head, body, and tail classes with a predicate sampling module. Next, we establish a peer network consisting of multiple peers, where each peer receives a combination of sub-distributions. This division enables peers to focus on different aspects of the scene graph generation task. Then, a novel peer learning loss function is introduced to cultivate the learning process among peer networks. Lastly, we employ the voting strategies for making final predictions within the peer network, boosting the influence of the majority’s opinion while downplaying the minority’s perspective. To illustrate the applicability of the proposed framework in intelligent transportation systems (ITSs), we further conduct qualitative evaluations on traffic scene understanding tasks. The results demonstrate that peer learning markedly enhances the reliability of interpreting complex traffic scenarios. Experimental results on the Visual Genome and Open Images V6 datasets further verify the effectiveness of our proposed model. These results highlight that the peer learning framework is well-suited for addressing the challenges of unbiased scene graph generation, offering practical benefits for ITS applications such as traffic analysis and monitoring. The code is available at: PL.

Image classification plays a pivotal role across diverse robotic applications, yet challenges persist when models are deployed in real-world scenarios. These models often fail to detect out-of-distribution (OOD) samples, classes not included in their training. This makes OOD detection a significant challenge for safe and effective real-world use. While existing techniques, like max logits, aim to leverage logits for OOD identification, they often disregard the intricate interclass relationships that underlie effective detection. This paper presents an innovative class relevance learning (CRL) method tailored for OOD detection. Our method establishes a comprehensive class relevance learning framework, strategically harnessing interclass relationships within the OOD pipeline. This framework significantly augments OOD detection capabilities. Extensive experimentation on diverse datasets, encompassing generic image classification datasets (Near OOD and Far OOD datasets), demonstrates the superiority of our method over state-of-the-art alternatives for OOD detection. The code is available on GitHub at: CRL.

Recently, learning-based stereo matching networks have advanced significantly. However, they often lack robustness and struggle to achieve impressive cross-domain performance due to domain shifts and imbalanced disparity distributions among diverse datasets. Leveraging Vision Foundation Models (VFMs) can intuitively enhance the model's robustness, but integrating such a model into stereo matching cost-effectively to fully realize their robustness remains a key challenge. To address this, we propose SMoEStereo, a novel framework that adapts VFMs for stereo matching through a tailored, scene-specific fusion of Low-Rank Adaptation (LoRA) and Mixture-of-Experts (MoE) modules. SMoEStereo introduces MoE-LoRA with adaptive ranks and MoE-Adapter with adaptive kernel sizes. The former dynamically selects optimal experts within MoE to adapt varying scenes across domains, while the latter injects inductive bias into frozen VFMs to improve geometric feature extraction. Importantly, to mitigate computational overhead, we further propose a lightweight decision network that selectively activates MoE modules based on input complexity, balancing efficiency with accuracy. Extensive experiments demonstrate that our method exhibits state-of-the-art cross-domain and joint generalization across multiple benchmarks without dataset-specific adaptation. The code is available at \textcolor{red}{https://github.com/cocowy1/SMoE-Stereo}.

Room segmentation plays a significant role in scene understanding, semantic mapping, and scene coverage for robots navigating in real-world indoor environments. However, most previous works take a passive segmentation that requires a complete and uncluttered grid map as input, often resulting in lower segmentation accuracy and cannot be deployed in unknown environments. In this paper, we propose an active room segmentation framework that can enable a robot to incrementally and autonomously perform room segmentation in cluttered indoor environments. Our framework consists of three key components: i) a door extraction module where a visual semantic feature, specifically, door, is extracted to better identify rooms in cluttered environments, ii) a within-room exploration module that detects frontiers within the currently exploring room, and iii) a topological module that represents connectivity between rooms and determines next room for exploration. We show through experiments that the proposed method depicts two distinct advantages against existing methods in segmentation accuracy and autonomy. The code is available at https://github.com/FreeformRobotics/Active_room_segmentation.

Spherical Modular Self-reconfigurable Robots (SMSRs) have been popular in recent years. Their Self-reconfigurable nature allows them to adapt to different en-vironments and tasks, and achieve what a single module could not achieve. To collaborate with each other, relative localization between each module and assembly is crucial. Existing relative localization methods either have low accuracy, which is unsuit-able for short-distance collaborations, or are designed for fixed-shape robots, whose visual features remain static over time. This paper proposes the first visual relative localization method for SMSRs. We first detect and identify individual modules of SMSRs, and adopt visual tracking to improve the detection and identification robustness. Using an optimization-based method, tracking result is then fused with odometry to estimate the relative pose between assemblies. To deal with the non-convexity of the optimization problem, we adopt semi-definite relaxation to transform it into a convex form. The proposed method is validated and analysed in real-world experiments. The overall localization performance and the performance under time-varying configuration are evaluated. The result shows that the relative position estimation accuracy reaches 2%, and the orientation estimation accuracy reaches 6.64°, and that our method surpasses the state-of-the-art methods.

Freeform modular self-reconfigurable robot (MSRR) systems overcome traditional docking limitations, enabling rapid and continuous connections between modules in any direction. Recent advancements in freeform MSRR technology have significantly enhanced connectivity and mobility. However, limitations in connector strength and operational efficiency in existing designs restrict performance. This paper proposes a rigid freeform connector and a rigid magnetic track design to improve the connection and motion performance of the SnailBot. Each SnailBot is equipped with a multi-channel rope-driven gripper, a metal spherical shell with densely distributed circular holes on the back, and a rigid chain design conforming to the spherical surface. This combination allows each SnailBot to move precisely along the surface of a peer, facilitated by the ferromagnetic spherical shell and magnetic track. The integration of the gripper and spherical shell hole array provides robust inter-module connections in any position and orientation. The effectiveness of these designs has been validated through a series of experiments and analyses, demonstrating improved connection and motion performance in the SnailBot dual-mode connector system and expanding its potential applications and functional capabilities.

The object navigation task requires robots to understand the semantic regularities in their environments. However, existing modular object navigation frameworks rely on instance segmentation models trained at fixed camera height viewpoints, limiting generalization performance and increasing labeling costs for new height viewpoints. To tackle this issue, we propose a semi-supervised method that transfers knowledge from a source height to a target height, minimizing the need for additional labels. Our approach introduces three key innovations: i) a projection policy to enhance the teacher model's detection capabilities at the target height, ii) a dynamic weight mechanism that emphasizes high-confidence pseudo-labels to reduce overfitting, and iii) a prototype contrast transferring method to transfer knowl-edge effectively. Experiments on the Habitat- Matterport 3D (HM3D) dataset show our method outperforms state-of-the-art semi-supervised techniques, improving both segmentation accuracy and navigation performance. The code is available at: https://github.com/FreeformRobotics/TransferKnowledge.

Modular Self-Reconfigurable Robot (MSRR) systems are a class of robots capable of forming higher-level robotic systems by altering the topological relationships between modules, offering enhanced adaptability and robustness in various environments. This paper presents a novel MSRR called MODUR, featuring dual-level reconfiguration capabilities designed to integrate reconfigurable mechanisms into MSRR. Specifically, MODUR can perform high-level self-reconfiguration among modules to create different configurations, while each module is also able to change its shape to execute basic motions. The design of MODUR primarily includes a compact connector and scissor linkage groups that provide actuation, forming a parallel mechanism capable of achieving both connector motion decoupling and adjacent position migration capabilities. Furthermore, the workspace, considering the interdependent connectors, is comprehensively analyzed, laying a theoretical foundation for the design of the module’s basic motion. Finally, the motion of MODUR is validated through a series of experiments.

Human-robot collaborative manipulation with mobile, multiple manipulators is crucial for expanding robotic applications, requiring precise handling of coupled force-position constraints between partners. Current systems, however, exhibit end-effector oscillations and instability during dynamic interactions. To overcome these limitations, this work develops a collaborative framework integrating a collaborative controller and a whole-body controller. The collaborative controller employs the object’s center-of-mass dynamics model with real-time contact forces and motion states to predict trajectories while coordinating with an attitude stabilization controller to adjust the desired end-effector poses. The whole-body controller utilizes model predictive control to generate coordinated motions that strictly follow pose commands from the collaborative controller, ensuring stable transportation. Simulation and physical experiments validate the proposed framework’s effectiveness in real-world scenarios.

Temporally consistent depth estimation from stereo video is critical for real-world applications such as augmented reality, where inconsistent depth estimation disrupts the immersion of users. Despite its importance, this task remains challenging due to the difficulty in modeling long-term temporal consistency in a computationally efficient manner. Previous methods attempt to address this by aggregating spatio-temporal information but face a fundamental trade-off: limited temporal modeling provides only modest gains, whereas capturing long-range dependencies significantly increases computational cost. To address this limitation, we introduce a memory buffer for modeling long-range spatio-temporal consistency while achieving efficient dynamic stereo matching. Inspired by the two-stage decision-making process in humans, we propose a Pick-and-Play Memory (PPM) construction module for dynamic Stereo matching, dubbed as PPMStereo. PPM consists of a pick process that identifies the most relevant frames and a play process that weights the selected frames adaptively for spatio-temporal aggregation. This two-stage collaborative process maintains a compact yet highly informative memory buffer while achieving temporally consistent information aggregation. Extensive experiments validate the effectiveness of PPMStereo, demonstrating state-of-the-art performance in both accuracy and temporal consistency.Codes are available at \textcolor{blue}{https://github.com/cocowy1/PPMStereo}.

The significance of depth estimation has spurred recent endeavors to enhance it through Multi-Sensor Fusion (MSF). However, prevailing MSF methods exhibit limitations concerning accuracy and resilience when confronted with sensor degradations. While certain forms of degradation, such as suboptimal lighting and adverse weather conditions, can be mitigated by collecting pertinent data in data-driven learning, this approach proves ineffective for Out-of-Distribution (OOD) sensor degradations. In this paper, we propose a novel approach termed Combinable and Separable Multi-Sensor Fusion (CSMSF) designed to bolster depth estimation robustness against multiple sensor degradations. CSMSF hinges on four core principles: i) improved performance is achieved with an increased number of valid sensors, ii) a single valid sensor can independently enable its own depth estimation, iii) maintaining a judicious equilibrium between accuracy and model complexity, and iv) autonomous diagnosis of sensor observation failure. Leveraging these advantages, CSMSF identifies and rejects degraded sensors, allowing autonomous selection of valid sensors for scene depth estimation. The experimental results demonstrate the superior robustness of the proposed CSMSF, underscoring its efficacy in addressing challenges associated with sensor degradations across diverse environmental conditions.

We explore the novel challenge of drone perception across varying low AGL (above ground level) heights, a task essential for dynamic tasks, unlike the fixed ground viewpoint in autonomous driving. Supervised learning for this incurs high annotation costs, and current semi-supervised methods struggle with viewpoint differences. In this paper, we introduce ground-to-aerial perception knowledge transfer and propose a progressive semi-supervised learning framework for drone perception using only labeled data from the ground viewpoint and unlabeled data from flying viewpoints. The framework hinges on four key components: 1) a dense viewpoint sampling strategy, segmenting the vertical flight height range into evenly distributed intervals; 2) nearest neighbor pseudo-labeling, inferring labels of the nearest neighbor viewpoint using a model learned on the preceding viewpoint; 3) MixView, generating augmented images among different viewpoints to mitigate viewpoint differences; and 4) a progressive distillation strategy, gradually learning until reaching the maximum flying height. To validate our approach, we create both synthesized and real-world datasets. Extensive experimental analyses reveal a remarkable relative accuracy improvement of 25.7% and 16.9% for the synthesized dataset and the real world, respectively. Code and datasets are available on https://github.com/FreeformRobotics/Progressive-Self-Distillation-for-Ground-to-Aerial-Perception-Knowledge-Transfer.

With the rapid advancements in autonomous driving and robot navigation, there is a growing demand for lifelong learning (LL) models capable of estimating metric (absolute) depth. LL approaches potentially offer significant cost savings in terms of model training, data storage, and collection. However, the quality of RGB images and depth maps is sensor-dependent, and depth maps in the real world exhibit domain-specific characteristics, leading to variations in depth ranges. These challenges limit existing methods to LL scenarios with small domain gaps and relative depth map estimation. To facilitate lifelong metric depth learning, we identify three crucial technical challenges that require attention: 1) developing a model capable of addressing the depth scale variation through scale-aware depth learning; 2) devising an effective learning strategy to handle significant domain gaps; and 3) creating an automated solution for domain-aware depth inference in practical applications. Based on the aforementioned considerations, in this article, we present 1) a lightweight multihead framework that effectively tackles the depth scale imbalance; 2) an uncertainty-aware LL solution that adeptly handles significant domain gaps; and 3) an online domain-specific predictor selection method for real-time inference. Through extensive numerical studies, we show that the proposed method can achieve good efficiency, stability, and plasticity, leading the benchmarks by 8%-15%. The code is available at https://github.com/FreeformRobotics/Lifelong-MonoDepth.

This study introduces a novel climbing strategy, reconfigurable parallel-type cable-driven climbing designed for long-span, large-scale bridge stay cable robotic applications, which has the potential to revolutionize the stay cable inspection and maintenance practice. The proposed methodology features the development of a Collaborative Climbing Robot Squad (CCRobot-S), which builds upon the design principles of the previous CCRobot series. In this study, CCRobot-S implements a parallel-type cable-driven manipulation design, allowing for reconfigurable kinematic morphology by its movable anchor bases and realizing the capacity of crossing over the stay cables for its flying platform. The collaborative robot squad design liberates the dimensions and scales of the robot's reachable workspace and moves the part of the robotic system that indeed needs to be moved, enhancing the working efficiency and climbing agility. This strategy also utilizes controllable adhesion instead of friction to interact with the bridge cable surface for the flying platform, realizing force multiplication for forceful manipulation. Toward bringing high efficiency and heavy-duty capacity, we propose the applicable climbing frameworks (zero-downtime climbing gait for cable inspection and spider-like climbing gait for cable maintenance) and the optimization frameworks (optimal anchor configuration for the movable anchor bases and optimal grasp arrangement for the flying gripper). This article includes the exploration of the design and climbing gaits of CCRobotS, the formulation of the CCRobot-S model, a comprehensive analysis of its workspace, and its climbing strategy and optimization. Extensive experiments have assessed the proposed climbing strategy's effectiveness and showcased CCRobot-S' capabilities.

Accurate full-body pose estimation across diverse actions in a user-friendly and location-agnostic manner paves the way for interactive applications in realms like sports, fitness, and healthcare. This task becomes challenging in real-world scenarios due to factors like the user’s dynamic positioning, the diversity of actions, and the varying acceptability of the pose-capturing system. In this context, we present PepperPose, a novel companion robot system tailored for optimized pose estimation. Unlike traditional methods, PepperPose actively tracks the user and refines its viewpoint, facilitating enhanced pose accuracy across different locations and actions. This allows users to enjoy a seamless action-sensing experience. Our evaluation, involving 30 participants undertaking daily functioning and exercise actions in a home-like space, underscores the robot’s promising capabilities. Moreover, we demonstrate the opportunities that PepperPose presents for human-robot interaction, its current limitations, and future developments.

This paper presents a decentralized cooperative motion planning approach for surface inspection of 3D structures which includes uncertainties like size, number, shape, position, using multi-robot systems (MRS). Given that most of existing works mainly focus on surface inspection of single and fully known 3D structures, our motivation is two-fold: first, 3D structures separately distributed in 3D environments are complex, therefore the use of MRS intuitively can facilitate an inspection by fully taking advantage of sensors with different capabilities. Second, performing the aforementioned tasks when considering uncertainties is a complicated and time-consuming process because we need to explore, figure out the size and shape of 3D structures and then plan surface-inspection path. To overcome these challenges, we present a meta-learning approach that provides a decentralized planner for each robot to improve the exploration and surface inspection capabilities. The experimental results demonstrate our method can outperform other methods by approximately 10.5%-27% on success rate and 70%-75% on inspection speed.

There is a growing interest in applying large language models (LLMs) in robotic tasks, due to their remarkable reasoning ability and extensive knowledge learned from vast training corpora. Grounding LLMs in the physical world remains an open challenge as they can only process textual input. Recent advancements in large vision-language models (LVLMs) have enabled a more comprehensive understanding of the physical world by incorporating visual input, which provides richer contextual information than language alone. In this work, we proposed a novel paradigm that leveraged GPT-4V(ision), the state-of-the-art LVLM by OpenAI, to enable embodied agents to perceive liquid objects via image-based environmental feedback. Specifically, we exploited the physical understanding of GPT-4V to interpret the visual representation (e.g., time-series plot) of non-visual feedback (e.g., F/T sensor data), indirectly enabling multimodal perception beyond vision and language using images as proxies. We evaluated our method using 10 common household liquids with containers of various geometry and material. Without any training or fine-tuning, we demonstrated that our method can enable the robot to indirectly perceive the physical response of liquids and estimate their viscosity. We also showed that by jointly reasoning over the visual and physical attributes learned through interactions, our method could recognize liquid objects in the absence of strong visual cues (e.g., container labels with legible text or symbols), increasing the accuracy from 69.0%—achieved by the best-performing vision-only variant—to 86.0%.

Energy sharing among modular robots enables sustainable operation of the system by maintaining energy balance among the modules. In this paper, we propose a novel energy sharing mechanism for FreeSN, a modular self-reconfigurable robot consisting of node and strut modules. Utilizing the feature that our modules are connected in a face-to-face manner, our method successfully establishes an energy sharing channel at almost any point on a sphere by placing transmission intermediaries at the interfacing face between modules, which is facilitated by the combination of brush contact and shell decomposition. Such mechanism also allows the utilization of the node module’s inner space for extra energy storage. A prototype of this energy sharing system has been implemented on FreeSN and rigorously tested. Our findings indicate that energy sharing is reliably established between modules; for strut modules positioned randomly on a node module’s surface, the probability of forming a valid connection is 56.6%. With orientation adjustment, a connection is achievable at nearly any position on the sphere, barring a few exceptional points. As a result, the operational endurance of the strut modules, which provide all the driving forces in the system, is markedly enhanced. This technique also holds potential for broader application across other freeform robotic platforms that incorporate conductive spherical surfaces for sliding connections.

In this paper, we extend the blind-spot based self-supervised denoising by using affinity learning to remove noise from affected pixels. Inspired by inpainting, we introduce a novel Mask Guided Residual Convolution (MGRConv) to learn a neighboring image pixel affinity map that gradually removes noise and refines blind-spot denoising process. We show that mask convolution plays an important role in blind-spot denoising since it is theoretically aligned with $\mathcal{J} - invariance$, which blind-spot based self-supervised denoising frameworks are built upon. The theoretical analysis further shows the motivation behind using more adaptive mask convolutions. Our MGRConv not only enables dynamic mask learning without external trainable parameters, but also preserves appropriate mask constraints by sigmoid activation and residual summation. Our MGRConv is a balance between partial convolution and learnable attention maps, and boosts denoising performance better than other inpainting convolutions with similar or even less parameters, memory, and training/inference time. Extensive experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising networks to reach promising results on both existing single-image based and dataset based benchmarks.

Collaborative localization (CL) has garnered substantial attention in the field of robotics in recent years. Nonetheless, conventional CL algorithms have faced challenges when dealing with practical issues such as spurious sensor data and limited or discontinued observation and communication in real-world settings. This paper proposes a fault-tolerant practical estimated cross-covariance minimum variance update method (FPECMV) designed to tackle these challenges under limited connectivity. The proposed algorithm uses a CNN-based method to evaluate confidence, along with a fault isolation module to identify faults and manage spurious data in real time. The proposed fault isolation module utilizes relative measurement information that randomly occurs, without requiring high observation and communication prerequisites. Notably, the algorithm takes into account correlations among agents to maintain consistency in localization filters and attain accurate localization despite constraints posed by limited connectivity. To evaluate the performance of the proposed algorithm, experiments were conducted in a collaborative multi-robot environment with spurious sensor data and limited connectivity, using both the BULLET simulation and physical mobile robots. The experimental results indicate that the overall localization performance of the proposed algorithm is improved by 21.0% compared to the state of the art. The experiment results demonstrate the effectiveness of our algorithm in localizing group agents in challenging and intricate scenarios with limited connectivity and spurious sensor data.

Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the first time, we provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods. Besides, we present a quantitative comparison of model performance on three widely used benchmarks, including indoor and outdoor datasets. Finally, we discuss the challenges of prior works and provide readers with some insights for future research directions.

Natural image matting aims to precisely separate foreground objects from backgrounds using alpha mattes. Fully automatic natural image matting without external annotations is challenging. Well-performed matting methods usually require accurate labor-intensive handcrafted trimap as an extra input while the performance of automatic trimap generation method, e.g., erosion/dilation manipulation on foreground segmentation, fluctuates with segmentation quality. Therefore, we argue that how to produce a high-quality trimap using coarse segmentation is a major issue in automatic matting. In this paper, we present a two-stage trimap-free natural image matting pipeline that does not need trimap and background as input. Specifically, guided by a coarse segmentation, Trimap Generation Network (TGN) estimates a trimap where the coarse segmentation can be produced by segmentation/salient object detection/matting approaches, which enables more flexibility for matting to adapt into different scenarios. Then, with an estimated trimap as guidance, our Sampling Propagation Attention Matting Network (SPAMattNet) estimates an alpha matte. Different from previous propagation-based matting networks, inspired by traditional sampling/propagation matting approaches, we propose Sampling Propagation Attention (SPA) for matting network to incorporate sampling and propagation procedures in deep learning based manner for network explainability and performance improvement. It explicitly investigates local spatial and global semantic relationships to reconstruct alpha features. To better harvest sampling/propagation and local/global information, a Cross-Fusion Contextual Module (CFC) is introduced to aggregate features from different sources. Extensive experiments are conducted to show that our matting approach is competitive compared to other state-of-the-art methods in both trimap-free and trimap-needed aspects on several challenging matting benchmarks.

Aiming to develop a more robust and intelligent heterogeneous system for adversarial catching in security and rescue tasks, in this article, we discuss the specialities of applying asymmetric self-play and curriculum learning techniques to deal with the increasing heterogeneity and number of different robots in modern heterogeneous multirobot systems (HMRS). Our method, based on actor-critic multiagent reinforcement learning, provides a framework that can enable cooperative behaviors among heterogeneous multirobot teams. This leads to the development of an HMRS for complex catching scenarios that involve several robot teams and real-world constraints. We conduct simulated experiments to evaluate different mechanisms' influence on our method's performance, and real-world experiments to assess our system's performance in complex real-world catching problems. In addition, a bridging study is conducted to compare our method with a state-of-the-art method called S2M2 in heterogeneous catching problems, and our method performs better in adversarial settings. As a result, we show that the proposed framework, through fusing asymmetric self-play and curriculum learning during training, is able to successfully complete the HMRS catching task under realistic constraints in both simulation and the real world, thus providing a direction for future large-scale intelligent security & rescue HMRS.

Dexterous joints have attracted interest in the field of robotics. This article presents a driving-integrated spherical gear (DISG) that enables entire spherical meshing and active driving between two spherical gears, forming a dexterous multidegrees of freedom rolling contact joint. The DISG consists of a pair of spherical gears and an omnidirectional internal driver. The spherical gear shape is a combined projection of the conventional planar gear profile in the longitudinal and latitudinal directions, which can mesh and be driven over the entire sphere. An actively driving magnet and a passively following magnet are magnetically connected across the spherical gear and together form the internal driver, enabling arbitrary connection points throughout the sphere. In all configurations, one spherical gear can roll in all directions on the surface of the other. Furthermore, we analyze the kinematics of DISG and prove that the DISG-based dexterous joint has good kinematic characteristics, such as singularity-free and full-range workspace. We verify the theoretical and physical characteristics of DISG in a series of experiments on the prototype. We also compare DISG-based joints to other joint actuators and show that DISG-based joints have advantages in dexterity, motion range, compactness, and lightweight.

Chain-type modular robots are capable of self-reconfiguration (SR), where the connection relationship between modules is changed according to the environment and tasks. This article focuses on the connection planning of SR based on multiple in-degree single out-degree (MISO) modules. The goal is to calculate the optimal connection planning solution: the sequence with the fewest detachment and attachment actions. To this end, we propose an auto-optimizing connection planning method that contains a polynomial-time algorithm to calculate near-optimal solutions and an exponential-time algorithm to further optimize the solutions automatically when some CPUs are idle. The method combines rapidity and optimality in the face of an NP-complete problem by using configuration pointers, strings that uniquely specify the robot's configuration. Our polynomial-time algorithm, in-degree matching (IM) uses the interchangeability of connection points to reduce reconfiguration steps. Our exponential-time algorithm, tree-based branch and bound (TBB) further optimizes the solutions to the optimum by a new branching strategy and stage cost. In the experiments, we verify the feasibility of the auto-optimizing method combining IM and TBB, and demonstrate the superiority of IM over Greedy-CM in the SR of MISO modules and the near-optimality of IM compared to the optimal solutions of TBB.

Modular self-reconfigurable robotic systems are potentially more robust and adaptive than conventional systems. This article proposes a novel freeform and truss-structured modular self-reconfigurable robot called FreeSN, containing node and strut modules. A node module contains a low-carbon steel spherical shell. A strut module contains two magnetic-based freeform connectors, which can connect to any position of the node module and provide spherical motions. Accurate configuration identification is essential for the automation of modular robot systems. This article presents a novel configuration identification system for FreeSN, including connection point magnetic localization, module identification, module orientation fusion, and system configuration fusion. A magnetic sensor array is integrated into the node module. A graph convolutional network-based magnetic localization algorithm is proposed, which can efficiently locate a variable number of magnet arrays under ferromagnetic material distortion. The module relative orientation is then estimated by fusing the magnetic localization result with the inertia moment unit and wheel odometry. Finally, the system configuration can be estimated, including the connection topology graph and the poses of modules. The configuration identification system is validated by a series of accuracy evaluation experiments and two library-based automation demonstrations based on closed-loop control.

Rolling contact joints are attracting increasing interest in applications to robotic fingers and manipulators, due to the potential of the absence of abrasion wear, the simplification of the controller, and the enlargement of reachable configurations. This article first proposes a novel two-degree-of-freedom (DOF) spherical rolling contact (SRC) joint, with the joint model elements being formulated, includingrotation matrix,position vector, andfree modes, as with those of classic joints. As an application, a new kind of serial manipulator formed by the SRC joints is presented, and its forward and inverse kinematics are modeled. The motions of the two-DOF SRC joint and manipulator are implemented using the FreeBOT, and the control method is proposed for the FreeBOT, such that the SRC joint and manipulator realize the motion control. The kinematics and control of the two-DOF SRC joint and manipulator are validated using physics simulations and on a real manipulator formed by FreeBOTs.

The occupancy grid map is a critical component of autonomous positioning and navigation in the mobile robotic system, as many other systems' performance depends heavily on it. To guarantee the quality of the occupancy grid maps, researchers previously had to perform tedious manual recognition for a long time. This work focuses on automatic abnormal occupancy grid map recognition using the residual neural network with novel attention mechanism modules. We propose an effective channel and spatial Residual Squeeze-and-Excitation (csRSE) attention module, which contains a residual block for producing hierarchical features, followed by both channel SE (cSE) block and spatial SE (sSE) block for the sufficient information extraction along the channel and spatial pathways. To further summarize the occupancy grid map characteristics and experiments with our csRSE attention modules, we constructed a dataset called occupancy grid map dataset (OGMD) for our experiments. On this OGMD test dataset, we tested a few variants of our proposed structure and compared them with other attention mechanisms. Our experimental results show that the proposed attention network can infer the abnormal map with state-of-the-art (SOTA) accuracy of 96.23% for abnormal occupancy grid map recognition.

Energy sharing in modular self-reconfigurable robots ensures the energy balance of the modules, thus allowing the system to work sustainably. This paper proposes an energy sharing mechanism for a novel modular self-reconfigurable robot that allows free connections among modules, termed as FreeBOT, such that each FreeBOT can share energy with peers through surface contact. Corresponding energy sharing rules are proposed to achieve an energy sharing network structure without invalid components. As alternative choices, several types of networks subjected to the above requirements are provided, which also maximize the number of FreeBOTs joining to share energy. We implement and test the prototype of the energy sharing mechanism on FreeBOT. The experimental results show that the mechanism can effectively achieve energy sharing among FreeBOTs.

This paper proposes a novel freeform strut-node structured modular self-reconfigurable robot (MSRR) called FreeSN, consisting of strut and node modules. A node module is mainly a low-carbon steel spherical shell. A strut module contains two freeform connectors, which provide strong magnetic connections and flexible spherical motions. The FreeSN system shares the benefits of freeform connection and strut-node structures. The freeform connection brings good adaptability to the environment. The triangle substructures inside the system configuration significantly improve the structural stability. The parallel execution of module motions can superpose the module capabilities and makes the system more scalable. The modules can combine these robot features by selecting the system configuration and better fit different circumstances and tasks. Four demonstrations, including assembly, obstacle crossing, transportation, and object manipulation, are designed to show the capabilities of the FreeSN system in different aspects. The results show the great performance and versatility of this MSRR system.

This paper proposes a novel modular self-assembling, self-reconfiguring robot with the 3D continuous dock called “SnailBot”. SnailBot mainly consists of a spherical ferromagnetic shell and a six-wheel rocker chassis with embedded magnets. Unlike many other existing modular self-reconfigurable robots with fixed docking locations, SnailBot uses the 3D continuous dock to attach to its peers regardless of alignment. This freeform docking mechanism can greatly improve the efficiency of self-reconfiguration and reduce docking failures because there is nearly no constraint in the location of the connector. Compared with the existing freeform MSRR, SnailBot can form a more structurally stable connection to its peers without loss of connection efficiency. Owing to the excellent obstacle crossing ability of the rocker-bogie suspension, the robot can freely crawl on other modules in the form of a sliding sphere. Experiments demonstrate the basic actions of a single module and some applications of SnailBots, such as a manipulator.

Multi-agent path finding in dynamic environments is of great academic and practical value for multi-robot systems in the real world. To improve the effectiveness and efficiency of the learning process during path planning in dynamic environments, we introduce an algorithm called Attention and BicNet based Multi-agent path planning with effective reinforcement (AB-Mapper) under the actor-critic reinforcement learning framework. In this framework, on one hand, we design an actor-network that can utilize the BicNet with communication function to achieve the intra-team coordination. On the other hand, we propose a critic network that can selectively allocate attention weights to surrounding agents. This attention mechanism allows an individual agent to automatically learn a better evaluation of actions by considering the behaviours of its surrounding agents. Compared with the SOTA method Mapper in crowded environments with dynamic obstacles, our AB-Mapper is more effective (90.27±0.06% vs. 61.65±13.90% in terms of mean success rate) in solving the general multi-agent path finding problem.

Transferring tools and objects to human hands is an important ability of collaborative robots. Most of the existing approaches focus on handover affordance, however, the comfort of receiving objects with human hands is often neglected. In this paper, we use advanced deep learning models to pre-generate handover target configurations that are convenient for human grasping based on the characteristics of the objects and tools, and then the robot grasps and passes the objects to the human. Experimental results on a mobile collaborative robot show that our proposed framework can robustly and efficiently deliver different shapes and types of objects to a human hand of any pose within the robot's field of view in a target pose that is convenient for grasping and can quickly deliver objects to a new target location even after the human hand moves to a new position.

For wave-driven unmanned surface vehicles (WUSVs), utilizing oscillating foils is the most straightforward and common wave energy conversion mechanism. Improving the thrust of the oscillating foil to increase its speed can help WUSVs improve their maneuverability and shorten the completion of ocean missions. This paper proposes a novel transformable two-segment foil, improving the wave energy-converting efficiency to provide more average thrust in every wave cycle. We estimate their working effectiveness numerically with a simple model to verify that the design enhances foils' thrust force. The thrust enhancement was further confirmed by computational fluid dynamic (CFD) simulations, and we estimated the suitable values of parameters of the foils in several different common sea conditions in coastal waters by CFD simulations. We design and make two wave gliders with traditional and transformable two-segment foils and finish the speed enhancement experiments. The speed enhancement is verified, and transformable two-segment foils can increase the speed of WUSVs by 10% in similar sea conditions in experiments.

The width of a neural network matters since increasing the width will necessarily increase the model capacity. However, the performance of a network does not improve linearly with the width and soon gets saturated. In this case, we argue that increasing the number of networks (ensemble) can achieve better accuracy-efficiency trade-offs than purely increasing the width. To prove it, one large network is divided into several small ones regarding its parameters and regularization components. Each of these small networks has a fraction of the original one's parameters. We then train these small networks together and make them see various views of the same data to increase their diversity. During this co-training process, networks can also learn from each other. As a result, small networks can achieve better ensemble performance than the large one with few or no extra parameters or FLOPs, i. e., achieving better accuracy-efficiency trade-offs. Small networks can also achieve faster inference speed than the large one by concurrent running. All of the above shows that the number of networks is a new dimension of model scaling. We validate our argument with 8 different neural architectures on common benchmarks through extensive experiments.

Stable walking in real-world environments is a challenging task for humanoid robots, especially when considering the dynamic disturbances, e.g., caused by external perturbations that may be encountered during locomotion. The varying nature of disturbance necessitates high adaptability. In this paper, we propose an enhanced Nonlinear Model Predictive Control (NMPC) approach for robust and adaptable walking – we term it versatile locomotion, by limiting both the Center of Pressure (CoP) and Divergent Component of Motion (DCM) movements. Due to utilization of the Nonlinear Inverted Pendulum plus Flywheel model, the robot is endowed with the capabilities of CoP manipulation (if equipped with finitesized feet), step location adjustment, upper body rotation, and vertical height variation. Considering the feasibility constraints, especially the usage of relaxed CoP constraints, the NMPC scheme is established as a Quadratically Constrained Quadratic Programming problem, which is solved efficiently by Sequential Quadratic Programming with enhanced solvability. Simulation experiments demonstrate the effectiveness of our method to recruit optimal hybrid strategies in order to realize versatile locomotion, for the robot with finite-sized or point feet.

In the Wave-driven unmanned surface vehicles (WUSVs), oscillating-foils are the most straightforward and widely used wave energy conversion mechanism. In this paper, a kind of novel asymmetric foil is proposed, which improves the wave energy-converting efficiency to provide a more significant thrust in every wave cycle. We break down the movement of the foils in the wave and build the corresponding kinetic model to analyze their working effectiveness numerically. Through computational fluid dynamic (CFD) simulations, we determine the optimal values of critical parameters of the foils, which are suitable for a wide range of wave conditions. The thrust enhancement of the asymmetric foil is verified in both CFD simulations and hydrodynamic experiments, and the result shows a similar enhancement trend. Comparing with the traditional symmetric foil, our asymmetric foil can provide at least 13.75% more thrust to the WUSVs.

Modular self-reconfigurable robotic (MSRR) systems are potentially more robust and more adaptive than conventional systems. Following our previous work where we proposed a freeform MSRR module called FreeBOT, this paper presents a novel configuration detection system for FreeBOT using a magnetic sensor array. A FreeBOT module can be connected by up to 11 modules, and the proposed configuration detection system can locate a variable number of connection points accurately in real-time. By equipping FreeBOT with 24 magnetic sensors, the magnetic field density produced by magnets and steel spherical shells can be monitored. The connectable area is split into 199 non-uniform regions, including 84 uniform regions. Using a Graph Convolutional Network (GCN) based algorithm, the connection points can be located accurately under ferromagnetic environments. The system can locate a variable number of connection points for such a region division with only single connection point training data. Finally, the localization algorithm can run faster than 40 Hz on FreeBOT. With the real-time configuration detection system, the FreeBOT system has the potential to reconfigure automatically and accurately.

This paper introduces a novel task-space decomposed motion planning framework for multi-robot simultaneous locomotion and manipulation. When several manipulators hold an object, closed-chain kinematic constraints are formed, and it will make the motion planning problems challenging by inducing lower-dimensional singularities. Unfortunately, the constrained manifold will be even more complicated when the manipulators are equipped with mobile bases. We address the problem by introducing a dual-resolution motion planning framework which utilizes a convex task region decomposition method, with each resolution tuned to efficient computation for their respective roles. Concretely, this dual-resolution approach enables a global planner to explore the low-dimensional decomposed task-space regions toward the goal, then a local planner computes a path in high-dimensional constrained configuration space. We demonstrate the proposed method in several simulations, where the robot team transports the object toward the goal in the obstacle-rich environments.

Hand gesture recognition plays an essential role in the human-robot interaction (HRI) field. Most previous research only studies hand gesture recognition in a short distance, which cannot be applied for interaction with mobile robots like unmanned aerial vehicles (UAVs) at a longer and safer distance. Therefore, we investigate the challenging long-range hand gesture recognition problem for the interaction between humans and UAVs. To this end, we propose a novel attention-based single shot multibox detector (SSD) model that incorporates both spatial and channel attention for hand gesture recognition. We notably extend the recognition distance from 1 meter to 7 meters through the proposed model without sacrificing speed. Besides, we present a long-range hand gesture (LRHG) dataset collected by the USB camera mounted on mobile robots. The hand gestures are collected at discrete distance levels from 1 meter to 7 meters, where most of the hand gestures are small and at low resolution. Experiments with the self-built LRHG dataset show our methods reach the surprising performance-boosting over the state-of-the-art method like the SSD network on both short-range (1 meter) and long-range (up to 7 meters) hand gesture recognition tasks.

Dynamic objects in the environment, such as people and other agents, lead to challenges for existing simultaneous localization and mapping (SLAM) approaches. To deal with dynamic environments, computer vision researchers usually apply some learning-based object detectors to remove these dynamic objects. However, these object detectors are computationally too expensive for mobile robot on-board processing. In practical applications, these objects output noisy sounds that can be effectively detected by on-board sound source localization. The directional information of the sound source object can be efficiently obtained by direction of sound arrival (DoA) estimation, but the depth estimation is difficult. Therefore, in this paper, we propose a novel audio-visual fusion approach that fuses sound source direction into the RGB-D image and thus removes the effect of dynamic obstacles on the multi-robot SLAM system. Experimental results of multirobot SLAM in different dynamic environments show that the proposed method uses very small computational resources to obtain very stable self-localization results.

The RGB-Thermal (RGB-T) information for semantic segmentation has been extensively explored in recent years. However, most existing RGB-T semantic segmentation usually compromises spatial resolution to achieve real-time inference speed, which leads to poor performance. To better extract detail spatial information, we propose a two-stage Feature-Enhanced Attention Network (FEANet) for the RGB-T semantic segmentation task. Specifically, we introduce a Feature-Enhanced Attention Module (FEAM) to excavate and enhance multi-level features from both the channel and spatial views. Benefited from the proposed FEAM module, our FEANet can preserve the spatial information and shift more attention to high-resolution features from the fused RGB-T images. Extensive experiments on the urban scene dataset demonstrate that our FEANet outperforms other state-of-the-art (SOTA) RGB-T methods in terms of objective metrics and subjective visual comparison (+2.6% in global mAcc and +0.8% in global mIoU). For the 480 × 640 RGB-T test images, our FEANet can run with a real-time speed on an NVIDIA GeForce RTX 2080 Ti card.

Accurate perception of the surrounding scene is helpful for robots to make reasonable judgments and behaviours. Therefore, developing effective scene representation and recognition methods are of significant importance in robotics. Currently, a large body of research focuses on developing novel auxiliary features and networks to improve indoor scene recognition ability. However, few of them focus on directly constructing object features and relations for indoor scene recognition. In this paper, we analyze the weaknesses of current methods and propose an Object-to-Scene (OTS) method, which extracts object features and learns object relations to recognize indoor scenes. The proposed OTS first extracts object features based on the segmentation network and the proposed object feature aggregation module (OFAM). Afterwards, the object relations are calculated and the scene representation is constructed based on the proposed object attention module (OAM) and global relation aggregation module (GRAM). The final results in this work show that OTS successfully extracts object features and learns object relations from the segmentation network. Moreover, OTS outperforms the state-of-the-art methods by more than 2% on indoor scene recognition without using any additional streams. Code is publicly available at: https://github.com/FreeformRobotics/OTS.

Dynamic environments that include unstructured moving objects pose a hard problem for Simultaneous Localization and Mapping (SLAM) performance. The motion of rigid objects can be typically tracked by exploiting their texture and geometric features. However, humans moving in the scene are often one of the most important, interactive targets – they are very hard to track and reconstruct robustly due to non-rigid shapes. In this work, we present a fast, learning-based human object detector to isolate the dynamic human objects and realise a real-time dense background reconstruction framework. We go further by estimating and reconstructing the human pose and shape. The final output environment maps not only provide the dense static backgrounds but also contain the dynamic human meshes and their trajectories. Our Dynamic SLAM system runs at around 26 frames per second (fps) on GPUs, while additionally turning on accurate human pose estimation can be executed at up to 10 fps.

Scene recognition is a fundamental task in robotic perception. For human beings, scene recognition is reasonable because they have abundant object knowledge of the real world. The idea of transferring prior object knowledge from humans to scene recognition is significant but still less exploited. In this paper, we propose to utilize meaningful object representations for indoor scene representation. First, we utilize an improved object model (IOM) as a baseline that enriches the object knowledge by introducing a scene parsing algorithm pretrained on the ADE20K dataset with rich object categories related to the indoor scene. To analyze the object co-occurrences and pairwise object relations, we formulate the IOM from a Bayesian perspective as the Bayesian object relation model (BORM). Meanwhile, we incorporate the proposed BORM with the PlacesCNN model as the combined Bayesian object relation model (CBORM) for scene recognition and significantly outperforms the state-of-the-art methods on the reduced Places365 dataset, and SUN RGB-D dataset without retraining, showing the excellent generalization ability of the proposed method. Code can be found at https://github.com/FreeformRobotics/BORM.

This paper presents a novel Cable Climbing Robot CCRobot-III, which is the third version designed for bridge cable inspection tasks, aiming at surpassing previous versions in terms of climbing speed and payload capacity. Benefiting from Split-type Wire-driven design, CCRobot-III can climb along a 90-110mm diameter bridge cable in inchworm-like gait at a speed of up to 12m/min, and carrying more than 40kg payload at the same time. CCRobot-III consists of a climbing precursor and a main-body frame. The two parts are connected and driven by steel wires. The climbing precursor, acting as a mobile anchor, moves quickly on a bridge cable. The mainbody frame, acting as a mobile winch, carries payload and pulls itself to a certain position with steel wires. Both parts have one or two pairs of palm-based gripper, which is the key component for providing strong adhesion to support the robot climbing. Experimental results have shown that CCRobotIII possesses outstanding climbing performance, high payload capacity, and good adaptability to complex conditions of cable surface. Moreover, it has potential engineering applications on the cable-stayed bridge for fieldwork.

Traditional solar trackers often adopt motors to automatically adjust the attitude of the solar panels towards the sun for maximum power efficiency. In this paper, a novel design of solar tracker for the ocean environment is introduced. Utilizing the fluctuations due to the waves, electromagnetic brakes are utilized instead of motors to adjust the attitude of the solar panels. Compared with the traditional solar trackers, the proposed one is simpler in hardware while the harvesting efficiency is similar. The desired attitude is calculated out of the local location and time. Then based on the dynamic model of the system, the angular acceleration of the solar panels is estimated and a control algorithm is proposed to decide the release and lock states of the brakes. In such a manner, the adjustment of the attitude of the solar panels can be achieved by using two brakes only. Experiments are conducted to validate the acceleration estimator and the dynamic model. At last, the feasibility of the proposed solar tracker is tested on the real water surface. The results show that the system is able to adjust 40° in two dimensions within 28 seconds.

In this paper, the 2D robot-to-robot relative pose (position and orientation) estimation problem based on ego-motion and noisy distance measurements is considered. We address this problem using an optimization-based method, which does not require complicated numerical analysis while yields no inferior relative localization (RL) results compared to existing approaches. In particular, we start from a state-of-the-art method named square distances weighted least square (SD-WLS), and reformulate it as a non-convex quadratically constrained quadratic programming (QCQP) problem. To handle its non-convex nature, a semidefinite programming (SDP) relaxation optimization-based method is proposed, and we prove that the relaxation is tight when measurements are free from noise or just corrupted by small noise. Further, to obtain the optimal solution of the relative pose estimation problem in the sense of maximum likelihood estimation (MLE), a theoretically optimal WLS method is developed to refine the estimate from the SDP optimization. Comprehensive simulations and well-designed experiments are presented for validating the tightness of the SDP relaxation, and the effectiveness of the proposed algorithm is highlighted by comparing it to the existing approaches.

This paper proposes a novel modular selfreconfigurable robot (MSRR) "FreeBOT", which can be connected freely at any point on other robots. FreeBOT is mainly composed of two parts: a spherical ferromagnetic shell and an internal magnet. The connection between the modules is genderless and instant, since the internal magnet can freely attract other FreeBOT spherical ferromagnetic shells, and not need to be precisely aligned with the specified connector. This connection method has fewer physical constraints, so the FreeBOT system can be extended to more configurations to meet more functional requirements. FreeBOT can accomplish multiple tasks although it only has two motors: module independent movement, connector management and system reconfiguration. FreeBOT can move independently on the plane, and even climb on ferromagnetic walls; a group of FreeBOTs can traverse complex terrain. Numerous experiments have been conducted to test its function, which shows that the FreeBOT system has great potential to realize a freeform robotic system.

This paper introduces an obstacle-crossing strategy, and the self-reconfiguration algorithm for a new class of modular robots called the rolling sphere, which can fit obstacles represented by cubes of different sizes due to the chain connection of multiple spheres. For the self-reconfiguration of the rolling spheres, a large gradient is obtained by classifying its action types and hierarchically minimizing the distance between the initial configuration and the final configuration. The most direct use of this large gradient is the fast crossing of various obstacles, by jointing multiple self-reconfigurations according to the OctoMap of the obstacles. It is verified in simulation that the self-reconfiguration takes full advantage of the parallel movement of multiple modules to reduce the total time steps, and the obstacle-crossing strategy can adapt to a variety of obstacles.

Towards long range and high endurance sailing, energy is of utmost importance. Moreover, benefiting from the dominance of the sailboat itself, it is energy-saving and environment-friendly. Thus, the sailboat with energy planning problem is meaningful. However, until now, the sailboat energy optimization problem has rarely been considered. In this paper, we focus on the energy consumption optimization of an autonomous sailboat. It has been formulated as a Nonlinear Programming problem (NLP). We deal with it with a hybrid control scheme, in which pseudo-spectral (PS) optimal control method is used in heading control, and a model-free framework guided by Extreme Seeking Control (ESC) is used in sail control. The optimal path is generated with the optimal input motor torques in time series. As a result, both simulation and experiments have validated motion planning and energy planning performance. Notably, about 7% of energy is saved on average. Our proposed method can make sailboats sailing longer and sustainable.

Automatic latching for charging in a disturbed environment for Unmanned Surface Vehicle (USVs) is always a challenging problem. In this paper, we propose a two-stage automatic latching system for USVs charging in berth. In Stage I, a vision-guided algorithm is developed to calculate an optimal latching position for charging. In Stage II, a novel latching mechanism is designed to compensate the movement misalignments from the water disturbance. A set of experiments have been conducted in real-world environments. The results show the latching success rate has been improved from 40% to 73.3% in the best cases with our proposed system. Furthermore, the vision-guided algorithm provides a methodology to optimize the design radius of the latching mechanism with respect to different disturbance levels accordingly. Outdoor experiments have validated the efficiency of our proposed automatic latching system. The proposed system improves the autonomy intelligence of the USVs and provides great benefits for practical applications.

In this paper, a novel soft inflatable arm is proposed for telepresence robots. The new proposed structure of the arm is achieved by a very common and low-cost inflatable material and it is very light, weighing only about 50 g. However, it can realize agile movement by driving six tiny cables installed in the shoulder and elbow joints. The soft inflatable arm can work by pumping air at a very low pressure (7.32 ± 3.45 kPa) and allows direct and soft human contact without any external force sensors. This paper proposes joint compressed models, joint kinematic models, and kinematic models for the whole inflatable robot arm. These models can be easily applied to multijoint arms. In addition, a new redundancy resolution method is also developed for the inflatable arm, which makes it easier to control resolution and is less complex than other traditional approaches. Numerous experiments have been conducted, including performances of accuracy, repeatability, motion trajectories, human-safe interaction, and remote interaction. Results are satisfactory and validate the expected performance of the proposed robotic arm.

In this paper, a longitudinal wheel-slip controller for four wheel independent steering and drive (4WISD) vehicles is proposed to suppress longitudinal wheel slip in varying road conditions. Different from conventional methods that consider single driving source and zero steering angle, the proposed controller considers all independent traction sources from each driving wheel and omnidirectional steering command so as to eliminate slip detection errors in 4WISD vehicles. The proposed controller requires low cost sensing equipment, including merely wheel speed sensor and accelerometer, which makes the system practical to be utilized. The proposed wheel-slip controller can be applied to vehicles with arbitrary quantity of driving wheels and different steering configurations such as traditional two-front-wheel steering and two-rear-wheel steering. Numerical simulation results are presented to demonstrate the efficiency of the proposed longitudinal wheel-slip controller.

In this paper, a novel soft inflatable arm is proposed for telepresence robots. It is capable of imitating human arms to realize remote interaction. The new proposed arm using a very common and low cost inflatable material, and it is very light, which weight is only about 50 grams, but can well realize agile movement by driving three tiny cables installed in shoulder joint and elbow joint, respectively. Meanwhile, the proposed cable driven mechanism also allows connecting numbers of joints easily. The soft inflatable can work just by pumping air with very low pressure (7.32 ± 3.45 kPa), and allows human directly and safely contact without any external sensors. Moreover, to solve the challenge problems of soft joint deformation, the kinematic modeling of the joint with deformation compensation is also developed. Experimental results show that the soft inflatable arm can agilely move for remote interaction. The workspace and velocity are also close to an adult's arm movement space and normal motion speed.

In this paper, the kinematic modeling and control for a multi-joint inflatable robot arm with cable-driven mechanism are proposed. The soft inflatable robot arm is capable of imitating human arms to realize remote interaction. The weight of the arm is only about 50 grams, and collision safe. To solve the challenge problems of kinematics of the soft inflatable arm, new approaches are proposed, including redundant rigid arm and soft inflatable joint models. The approaches have a good advantage of applying to muti-joint arms. As our knowledge it is the first time to solve the kinematics of muti-joint soft inflatable arm in Three-Dimensional coordinate space. Numerous experiments have been conducted, including movement space and positioning accuracy. The workspace and velocity are close to an adult's arm movement space and normal motion speed.

In this paper, mechanical design and implementation of a low-cost and lightweight inflatable robot finger are proposed. The proposed soft inflatable robot finger is different from traditional designs. It uses a common and low cost inflatable material and can be easily and massively manufactured. The proposed soft inflatable finger only weighs 0.8 grams, but can well realize swift movement which is actuated by low pressure air. Numerous analyses and experiments have been conducted for key parameters selection of the mechanical design. The performances of the proposed finger including flexing and extending have also been evaluated, and results are satisfactory.

In this paper, a traction/braking force distribution algorithm for omni-directional all-wheel-independent-drive vehicles is proposed as a tool to enhance driving stability. In the proposed algorithm, the amount of the traction or braking force on each driving wheel can be determined so as to generate a desired tangential force, yaw moment and centripetal force independently. The algorithm considers omni-directional steering command and is capable of handling both traction and braking force commands. The algorithm is applicable on vehicles with at least three independent driving wheels. Simulations have been conducted to illustrate the use of the proposed force distribution method in enhancing vehicles stability.

In this paper, a deterministic yaw moment controller for four wheel independent steering and drive vehicles is proposed to enhance driving stability and controllability. Different to conventional methods that track a desired yaw rate, the proposed controller stabilizes a vehicle by additionally tracking the heading angle of a vehicle which is more efficient and robust. The heading angle of a vehicle is obtained by a novel method which is based on centripetal force detection. It eliminates the prerequisite knowledge of the characteristics between wheels and road surface which are time varying and difficult to be measured in real time. The proposed system only requires low cost sensing equipment such as wheel speed sensor and accelerometer that makes the system practical to be utilized. The proposed heading angle detection method can be generally applied to any kind of vehicle. The deterministic yaw moment controller is also applicable to any type of four wheel independent drive vehicles.

Collision avoidance of industrial robot arms in varying environment is a challenging task which has been a tough problem for decades. It often requires a large number of sensors and high computational power. Moreover, since the sensors are often mounted on the surface of robot arms, they may affect the appearance of the robot arms and may be vulnerable to damage. This video presents a cost-effective invisible sensitive skin that can cover a large area without utilizing a large number of sensors and it is built inside the robot arm. By using only 5 contactless capacitive sensors and specially designed antennas, collision avoidance of a 6-DOF industrial robot arm is attained.

This paper proposed an autonomous tree climbing algorithm for a novel tree climbing robot named Treebot. Making a robot realize an environment and climb on a tree autonomously is a challenging task as the shape of tree is complex and irregular. To our best knowledge, this is the first paper dealing with the autonomous climbing problem in an unknown tree environment. The proposed method is aimed to use minimal sensing resources to achieve autonomous climbing. It reconstructs the shape of tree by using tactile sensors and guides the robot to climb along an optimal path. Numerous experiments have been carried out and the results are satisfactory.

This paper proposed a novel tree climbing robot "Treebot" that has high maneuverability on an irregular tree environment and surpasses the state of the art tree climbing robots. Treebot's body is a novel continuum maneuver structure that has high degrees of freedom and superior extension ability. Treebot also equips with a pair of omni-directional tree grippers that enable Treebot to adhere on a wide variety of trees with a wide range of gripping curvature. By combining these two novel designs, Treebot is able to reach many places on trees including branches. Treebot can maneuver on a complex tree environment, but only five actuators are used in the mechanism. As a result, Treebot can keep in compact size and lightweight. Although Treebot weighs only 600 grams, it has payload capability of 1.75 kg which is nearly three times of its own weight. On top of that, the special design of the gripper permits zero energy consumption in static gripping. Numerous experiments have been conducted on real trees. Experimental results reveal that Treebot has excellent climbing performance on a wide variety of trees.

In this paper, an novel tree gripping mechanism has been proposed for miniature tree-climbing robots. It is capable of attaching on a wide variety of trees with a wide range of gripping curvature. In addition, it is lightweight and simple in control. The gripper is simple in control as it is actuated by one actuator only. In addition, the omni-directional gripping ability also simplifies the use of the gripper as there is no extra actuator needed for orientation control. The special mechanism and optimized settings make the gripper able to attach on different sizes of tree tightly. The mechanism also allows zero energy consumption in static gripping. Numerous on-tree experiments of the proposed mechanism have been conducted and the results are satisfied.

In this paper, we propose an autonomous tree climbing strategy for a novel tree climbing robot that is named Treebot. The proposed algorithm aims to guide Treebot in climbing along an optimal path by the use of minimal sensing resources. Inspired by inchworms, the algorithm reconstructs the shape of a tree simply by the use of tactile sensors. It reveals how the realization of an environment can be achieved with limited tactile information. An efficient nonholonomic motion planning strategy is also proposed to make Treebot climb on an optimal path. This is accomplished by the prediction of the future shape of the tree. The study that is presented in this paper also includes the formulation of Treebot kinematics and an analysis of the workspace of Treebot on different shapes of a tree. Numerous experiments have been conducted to evaluate the proposed autonomous climbing algorithm and to unveil the ability of Treebot.

This paper proposes a path and motion planning algorithm for a tree climbing problem. This problem is challenging as the shape of tree is complex and irregular. To our best knowledge, this is the first paper dealing with the path planning problem on natural tree environment. Different from conventional motion planning approach that requires constructing a complex configuration space, this paper divides the planning problem into two parts, i.e., path and motion planning problem so as to reduce the dimension of the problem. An intuitive method to represent a climbing space is proposed that highly simplifies the path planning problem. With the use of a dynamic programming algorithm, an optimal path to reach a target position can be acquired in linear time. In addition, an efficient motion planning algorithm for a tree climbing robot named TreeBot is developed to make TreeBot follow the planned path.

The promising electric vehicle (EV) technology is a direction to tackle the global non-renewable energy problem. However, the efficiency to use the electric energy still needs deliberate research. Traditional EV has no choice to manage its energy flow, because it has only one traction motor. With the robotic research in 4 wheel independent drive (4WID), the driving task of the single traction motor can be shared by 4 independent in-wheel motors. By exploring the motor efficiency map, we propose the energy management strategy based on optimal driving torque distribution (ODTD). The total input power of the 4 motors can be minimized while the driving performance is still maintained, and electric energy consumption can be reduced compared with traditional single motor driving EV. Simulation results validate the proposed strategy. The energy management strategy can also be applied to multi-driving-wheel mobile robots.

In this paper, an omni-directional steer-by-wire interface for four wheel independent steering vehicle is presented. The proposed steering interface is an extension of a traditional steering interface that provides three steering inputs. By combination of which, driver can control the vehicle in traditional way or omni-directionally without any mode switching operation. The reservation of the conventional steering behavior makes driver easy to adapt the novel steering interface. The force feedback controller is designed to synchronize the extended steering interface and the orientations of wheels so as to improve vehicle handling. Hardware-in-the-loop simulations are conducted to verify the hardware prototype and examine the proposed algorithms.

This paper proposes an optimal traction force distribution for omni-directional four wheel independent steering (4WIS) and four wheel independent drive (4WID) vehicle. The proposed force distribution algorithm is aimed to enhance the vehicle stability with minimum cost. The algorithm avoids the use of any feedback information of vehicle motion such as linear velocity as this information is difficult to measure accurately and the price of the measuring equipment is very high. As a result, the implementation cost can be reduced and at the same time avoid improper force distribution due to the inaccurate measured information. Moreover, the proposed algorithm does not involve any parameter tuning. It makes the algorithm easy to implement. The proposed algorithm can also be applied to any steering types of 4WID vehicle such as typical two wheel steering (2WS) as 4WIS is the general case of any steering configuration. Simulation results reveal that the performance of the proposed force distribution is superior to the uniform force distribution which is commonly used in 4WID vehicle.