Semantic segmentation takes a pivotal role in various applications such as autonomous driving and medical image analysis. When deploying segmentation models in practice, it is critical to test their behaviors in varied and complex scenes in advance. In this paper, we construct an automatic data generation pipeline Gen4Seg to stress-test semantic segmentation models by generating various challenging samples with different attribute changes. Beyond previous evaluation paradigms focusing solely on global weather and style transfer, we investigate variations in both appearance and geometry attributes at the object and image level. These include object color, material, size, and position, as well as image-level variations such as weather and style. To achieve this, we propose to edit visual attributes of existing real images with precise control of structural information, empowered by diffusion models. In this way, the existing segmentation labels can be reused for the edited images, which greatly reduces the labor costs of constructing datasets. Using our pipeline, we construct two new benchmarks, Pascal-EA and COCO-EA. We benchmark a broad variety of semantic segmentation models, spanning from conventional close-set models to recent open-vocabulary large models. We have several key findings: 1) advanced open-vocabulary models do not exhibit greater robustness compared to closed-set methods under geometric variations; 2) traditional data augmentation techniques, such as CutOut and CutMix, are limited in enhancing robustness against appearance variations; 3) our generation pipeline can also be employed as a data augmentation tool and improve both in-distribution and out-of-distribution performances. Our work suggests the potential of generative models as effective tools for automatically analyzing segmentation models, and we hope our findings will assist practitioners and researchers in developing more robust and reliable segmentation models.

Biorobots involve embedding artificial components into a living insect to transform it into a controllable robot, which comes with the advantages of low energy consumption, quietness, and flexibility. However, no models have yet been developed to describe the randomness-filled motions of biorobots, and their unique biological properties are also underutilized. In this work, the locomotion responses of a biorobot to electrical stimuli were summarized. The constant-current signal was found to have a higher capacity for sustained stimuli than existing methods. The joint influence of wall and current factors on cockroach behavior was explored to establish an environment-adaptive control model describing their locomotion. This model was then used to propose an navigation method enhanced by the inherent nature of a biorobot. In an indoor scenario, the proposed method achieved a success rate of 78.6% with minimal runtime and stimulus number while the control group using conventional navigation methods achieved a success rate of 30%, with two or three times the runtime and stimulus number. Compared with other approaches, the proposed method facilitates high-precision and long-range navigation with a short navigation time and very low control costs.

The design and control of hexapod robots have become an active research field due to the ability to achieve adaptive and stable multi-terrain locomotion. However, existing hexapod robots focus on the integration of flexible pitch joints to enhance their obstacle-crossing and slope-climbing abilities, and few biological observations have been made to gain insight into the agile steering mechanisms of hexapod insects. Herein, we observed the steering movements of Madagascar cockroaches. Observations showed that cockroaches exhibited specific phase relationships in addition to regular tripod gait pattern during steering. Moreover, we also found that a smaller steering radius resulted in a larger lateral bending angle of the thoracic segments. Inspired by this, a hexapod robot with a flexible torso (F-RHex) was designed and fabricated. Bio-inspired gait patterns were abstracted and simplified into two steering strategies: gait-based and mix-based. Compared to the purely gait-based strategy, the F-RHex testing results demonstrated a ~27.4% reduction in turning radius and ~40% enhancement in steering velocity, implying that the mix-based strategy offers superior steering capability.

Robotic graspers are essential for enhancing the efficiency and versatility of robots in grasping tasks. In this paper, we propose a novel inflatable deployable origami grasper with a rigid-flexible coupling structure. The proposed grasper can achieve multiple deployment configurations under a single pneumatic actuation, enabling both deployment and grasping operations while also allowing for passive self-folding during deflation. The design and fabrication of the grasper are presented. Then, the stiffness model for the inflatable deployable origami unit is developed based on the equivalent truss method. Experimental results show that the grasper successfully grasps objects of various shapes and sizes in both enveloping and fingertip grasping modes, using either two or four fingers. With its simple mechanical system and high deploy/fold ratio, the proposed grasper holds significant potential for applications in industrial automation and space exploration.

Pelvic fracture reduction surgery (PFRS) has always been one of the most challenging procedures in trauma orthopedics. Excellent preoperative planning is crucial for surgery, especially with the increasingly mature robot-assisted surgical systems. However, current preoperative reduction planning heavily relies on surgeons’ experience. This paper proposes an automatic preoperative planning framework for PFRS. Firstly, an enlarged RRT$^\ast$(ERRT$\ast)$algorithm is proposed to search feasible paths, which adopts a synchronized exploration and asynchronous adjustment strategy in 6D space to change the position and orientation of fragments during the reduction process. Secondly, a collision detection method based on surface point cloud is proposed to improve the safety of the reduction path by taking into account the actual volume of fragments. Finally, a post-processing method combining path shortening (PS) algorithm and cubic spline interpolation is proposed to optimize and smooth the reduction path. The clinical case simulation results show that the ERRT$^\ast$algorithm can find a feasible reduction path within a few seconds ($<$10s), and the length of the path is reduced by an average of 11.11% with the PS algorithm. Furthermore, repeated experimental results demonstrate that the method has good consistency. The proposed preoperative planning method can serve as a powerful tool to provide references for surgeons and also provide a quantitative basis for robot-assisted PFRS.Note to Practitioners—This work addresses the challenge of automating preoperative planning for pelvic fracture reduction surgery, specifically in determining the target reduction pose and operative path for repositioning the fragment. The motion of the fragment is decoupled into translation along a point and rotation through the coordinate system of that point to quantify the difference between the initial pose and target pose and determine the planning requirements. An enlarged RRT$^\ast$algorithm based on six-dimensional generalized coordinates is proposed, combined with an efficient collision avoidance algorithm, to quickly find a safe and feasible reduction operation path. The proposed planning method not only provides guidance to physicians but also establishes a basis for robot-assisted fracture reduction surgery.

When deploying segmentation models in practice, it is critical to evaluate their behaviors in varied and complex scenes. Different from the previous evaluation paradigms only in consideration of global attribute variations (e.g. adverse weather), we investigate both local and global attribute variations for robustness evaluation. To achieve this, we construct a mask-preserved attribute editing pipeline to edit visual attributes of real images with precise control of structural information. Therefore, the original segmentation labels can be reused for the edited images. Using our pipeline, we construct a benchmark covering both object and image attributes (e.g. color, material, pattern, style). We evaluate a broad variety of semantic segmentation models, spanning from conventional close-set models to recent open-vocabulary large models on their robustness to different types of variations. We find that both local and global attribute variations affect segmentation performances, and the sensitivity of models diverges across different variation types. We argue that local attributes have the same importance as global attributes, and should be considered in the robustness evaluation of segmentation models. Code: https://github.com/PRIS-CV/Pascal-EA.

A small-scale jumping robot can reach obstacles much larger than its size. It is important for a jumping robot to perform intermittent jumps to cross through rough terrains. However, the limitations of conventional structures hinder the further integration of functions to a miniature (sub-50 g) jumping robot. No sub-50 g jumpers could perform intermittent jumps with adjustable jumping trajectories. In this work, we proposed an origami-based miniature jumper, which performed intermittent jumps with adjustable omni-directional trajectories. The intermittent jumps were achieved by the jumping and self-righting mechanisms, which were actuated by a single motor. The clockwise and counterclockwise rotation of the motor actuated the loading, self-righting and triggering process, respectively. The jumping height was adjustable by adjusting the rotation angle of the motor. Meanwhile, the take-off pitch & yaw angle adjustment methods were integrated into the robot. Therefore, we demonstrated a 9 cm, 13.5 g prototype with functions of re-loading, self-righting, jumping height adjustment and take-off pitch & yaw angle adjustment. The robot could adjust jumping height from 16 to 34 cm and self-right for the next jump. The results revealed that our robot could jump across different obstacles with different scales and directions. The mobility was greatly increased compared with other miniature jumping robots.

Robot-assisted decompressive laminectomy is a new strategy in clinical applications. However, a stable contact force between the bone-cutting device and lamina is needed to keep it working correctly, which may be affected by respiration and deformation. The surgeon can adapt to this dynamic process quickly, but the robot could cause a considerable force that may damage the patient. This paper proposes a compensation control method based on a respiration-spine model to first improve the stability of the contact force. The model is established based on human morphology and ventilator parameters for anaesthetised patients. The control method is a combination of a surgery sleeve and active fuzzy control to improve the robustness of the robot. The control of the sleeve is related to the thickness of the bone layer, which can be calculated from the image. Furthermore, the lower boundary of the lamina in the CT image is extracted as a safety constraint that can protect the spinal nerves. Finally, an experiment is conducted to verify the safety constraint and compare the changes in contact force with or without the control method. The statistical experiment shows that the control error is 2.47 N without the force control method, while the force control error is 0.223 N when the target control force is 2 N. The robot will hover on the surface of the spine after completing the laminectomy of the planned area. These results show that the robot can be controlled safely and stably. Note to Practitioners—The purpose of this paper is to propose a stable and safe control method for lamina grinding robots under the influence of breathing and deformation factors. Most previous research only considered one of the two factors, and both factors are considered in this paper. Unlike the surgeon, who has the ability to adapt to physiological movements, the robot may cause a considerable force to be exerted on the device, which may damage the patient. Therefore, a controller based on human experience is longitudinally designed, and position constraints are added in three directions. This application can improve the stability and safety of robot-assisted surgery, which may be suitable for remote surgery and can help rural areas solve the problem of a lack of medical resources. Spinal model bone experiments have been completed and will be tested on animals in the future.

Jumping motion is an effective way to overcome large obstacles, especially for the miniature robots. However, controlling of the jumping trajectory on a centimeter scale robot is not easy due to the limitation of size and payload. None of the jumping robots lighter than 90 g achieved the feedback control of their jumping height and take-off angle independently. In this work, we proposed a miniature 6 g jumping robot that ensured the feedback control of jumping trajectory. Two simple PD controllers were used in take-off angle and jumping height control, respectively. The robot can control its jumping height from 0 to 73cm, take-off angle from −20° to +20° with respect to the vertical direction. The control errors of the jumping height and the take-off angle were less than 5 cm and 2°, respectively. The robot can hop upon different obstacles exactly, greatly increased the controllability of the micro jumping robot.

Beetles can walk smoothly on the meshed surface without slipping or getting stuck in the meshed surface due to its stiffness-variable tarsi and expandable hooks on the tip of tarsi. In this study, we find that beetles bend and open their claws proactively to walk freely. Inspired by the mechanism, we designed a centimeter-scale climbing robot, equipping an artificial claw to open and bend in the same cyclic manner as the natural beetles. The robot can climb freely on the mesh surface of 30° without being stuck at a speed of 26.18 mm/s (0.3 body length per second), and the speed was 37.5 mm/s on the 55-degree rough slop. This is the first demonstration of a centimeter-scale robot that can climb on the mesh surface.

Jumping is beneficial for microrobots because they have to face obstacles larger than their height frequently. However, compacting a jumping mechanism into the small body of a microrobot is exceptionally challenging. Instead of assembling a bio-inspired microrobot, the insect itself can be transformed into a jumping robot. Herein, we demonstrated the first-ever biohybrid jumping robot that retained the natural jumping ability of a locust. The fast kicking of the locust's hindleg was mainly induced by two muscles, flexor muscle, and extensor muscle. The elaborate structure and accurate collaboration of the muscles are critical for leg kicking, contributing to the co-contraction process. In this article, we investigated the sequences of muscular activities and demonstrated the co-contraction control exogenously. The kicking control of the hindleg relies on the accurately overlapped stimulation and the independently modulated waveform. With the help of a tiny wireless stimulator, the cyborg locust was remotely controlled to jump an average of 10.4 cm high and 42.6 cm far. Moreover, the cyborg locust retained its internal body righting mechanism, which means the robot can quickly recover its posture for consecutive jumping. This work is a foundational step towards a fully controllable biohybrid jumping robot.

The development and control of biological actuators have been an active research field. Biological actuators revealed high mobility with compact dimensions, which is critical for the design of microrobots. The powerful kicking motion of the locust is important for its quick jumping. Herein, we examined the kicking process of the locust’s hindleg and controlled the flexion and extension motions via exogenous stimulation. Unlike a simple extension of the leg, co-contraction is adopted by locust to store energy and increase jumping power. Thus, we imitated the co-contraction process and transformed the locust’s hindleg into a biological jumping actuator. Through coordinating the kicking of bilateral hindlegs together, we achieved the jumping control of a locust.

In this study, a novel parallel learning particle swarm optimizer (PLPSO) is proposed. The evolutionary strategy of the algorithm is quite different from that of the existing PSO algorithms. To enhance the global search capability of the particle swarm, the original particle swarm is divided into two parallel evolving independent particle subpopulations to explore the search space simultaneously. In addition, according to the evolutionary factor in each iteration, the poorly performing particles in the two subpopulations are spurred and learned to improve the search speed of the particle swarm. In the process of particle learning, delay information is added to the velocity update item to reduce the occurrence of local capture. Twenty-eight benchmark functions of CEC2013 are used to evaluate the performance of the proposed PLPSO algorithm. Numerous comparisons demonstrate that the performance of the PLPSO algorithm is better than that of other nine PSO algorithms. Afterwards, the PLPSO algorithm is applied to the inverse kinematics (IK) solution of a robotic manipulator. By comparing with other three intelligent algorithms, the PLPSO algorithm shows an excellent performance in solving the IK problem. Finally, tests are carried out on an UR5 manipulator to confirm the practicability of the proposed PLPSO algorithm further. Thus, the feasibility of using the PLPSO algorithm for solving the IK problem of a robotic manipulator is verified.

Unlike biomimetic methods, the insect–computer hybrid is an alternative approach to developing insect-scale robots. Insect–computer hybrid is a technique that transforms a living insect into a controllable robot by embedding it with artificial devices. In this article, a beetle (Mecynorrhina torquata) was transformed into a hybrid flying system by mounting an electronic backpack and implanting electrodes on it. Wing trajectories during fictive flight climbing and flight diving were captured to investigate the natural flight-height control of beetles. Compare with the electrically induced wing trajectories via basalar, subalar, and third axillary (3Ax) muscle stimulations, we hypothesized that the basalar muscles are involved in ascending flight, whereas the 3Ax muscles function to reduce the flight-height. By reproducing the electrical stimulations on bilateral muscle pairs during free flights, we found that stimulations of the basalar muscle pair increased vertical accelerations. In contrast, the 3Ax muscle pair's stimulation decreased vertical accelerations gradually as a function of the electrical stimulation frequency. Accordingly, a proportional-derivative feedback controller was proposed to maintain the beetles' flight-height using frequency-dependent electrical pulses on the basalar and 3Ax muscles. In this article, the altitude control of a free-flying beetle was demonstrated for the first time.

According to the automobile industry's flexible manufacturing requirements, a novel flexible fixturing system for sheet metal assembly is proposed with parallel robots. A methodology of the structure synthesis is presented by taking account simultaneously several performance indices. Taking the 3UPU/UPS parallel robot in the system as an example, models of inverse kinematics, Jacobian matrix and design space are developed. Thus, the structure synthesis of the parallel robot is simplified to a two-dimensional problem. Once the performance indices (workspace, singularity and stiffness) are established, the corresponding indices atlases are expressed in the design space. The structure parameters of the parallel robot are obtained by analyzing these atlases. The prototype of the fixturing system is developed, and the relevant clamping and stiffness experiments are conducted. The experimental results generally agree well with the simulation results and satisfy the flexible fixturing systems' requirements.

In this paper, we consider the distributed containment control problem for multiagent systems with unknown nonlinear dynamics. More specifically, we focus on multiple second-order nonlinear systems and networked Lagrangian systems. We first study the distributed containment control problem for multiple second-order nonlinear systems with multiple dynamic leaders in the presence of unknown nonlinearities and external disturbances under a general directed graph that characterizes the interaction among the leaders and the followers. A distributed adaptive control algorithm with an adaptive gain design based on the approximation capability of neural networks is proposed. We present a necessary and sufficient condition on the directed graph such that the containment error can be reduced as small as desired. As a byproduct, the leaderless consensus problem is solved with asymptotical convergence. Because relative velocity measurements between neighbors are generally more difficult to obtain than relative position measurements, we then propose a distributed containment control algorithm without using neighbors' velocity information. A two-step Lyapunov-based method is used to study the convergence of the closed-loop system. Next, we apply the ideas to deal with the containment control problem for networked unknown Lagrangian systems under a general directed graph. All the proposed algorithms are distributed and can be implemented using only local measurements in the absence of communication. Finally, simulation examples are provided to show the effectiveness of the proposed control algorithms.

This paper investigates the H∞ consensus control problem of nonlinear multiagent systems under an arbitrary topological structure. A novel Takagi-Sukeno (T-S) fuzzy modeling method is proposed to describe the problem of nonlinear follower agents approaching a time-varying leader, i.e., the error dynamics between the follower agents and the leader, whose dynamics is evolving according to an isolated unforced nonlinear agent model, is described as a set of T-S fuzzy models. Based on the model, a leader-following consensus algorithm is designed so that, under an arbitrary network topology, all the follower agents reach consensus with the leader subject to external disturbances, preserving a guaranteed H(∞) performance level. In addition, we obtain a sufficient condition for choosing the pinned nodes to make the entire multiagent network reach consensus. Moreover, the fuzzy modeling method is extended to solve the synchronization problem of nonlinear systems, and a fuzzy H(∞) controller is designed so that two nonlinear systems reach synchronization with a prescribed H(∞) performance level. The controller design procedure is greatly simplified by utilization of the proposed fuzzy modeling method. Finally, numerical simulations on chaotic systems and arbitrary nonlinear functions are provided to illustrate the effectiveness of the obtained theoretical results.