The unique optical characteristics of the underwater environment, such as light refraction and loss of salient features, pose a significant challenge to traditional vision sensors, especially in the swarm operation scenario where multiple autonomous underwater vehicles (AUVs) cooperate with the mother ship for positioning. To address these challenges, this study proposes the application of soft constraints on mirror position to improve the robustness of optical target positioning in shallow water. During the mapping phase, we establish and optimize the pose relationships between ArUco markers and their mirrors, thereby expanding the locatable space for AUVs (Autonomous Underwater Vehicles) . With the arrangement of ArUco markers unchanged, the number of usable markers doubles. Surface mirror-assisted positioning provides more visual features and additional computed corner points, enhancing the reliability of camera observations and improving positioning accuracy. Experimental results demonstrate that, compared to classical algorithms from the Artificial Vision Applications (A.V.A) laboratory, our method improves position accuracy by 25.8% in single-marker scenarios and by 14.7% in multi-marker scenarios. Therefore, our method provides enhanced mapping and positioning for AUVs in shallow water areas where optical markers can be deployed. This study provides a new mapping paradigm and multi-body localization solution for optical marker-guided underwater swarm operations.

Marker-based optical motion capture (MoCap) is a vital tool in applications such as virtual production, and movement sciences. However, reconstructing scattered MoCap data into real motion sequences is challenging, and data processing is time-consuming and labor-intensive. Here we propose a novel framework for MoCap auto-labeling and matching. In this framework, we designed novel clusters of reflective markers called auto-labeling encoded marker clusters (AEMCs), including clusters with an explicit header (AEMCs-E) and an implicit header (AEMCs-I). Combining cluster design and coding theory gives each cluster a unique codeword for MoCap auto-labeling and matching. Moreover, we provide a method of mapping and decoding for cluster labeling. The labeling results are only determined by the intrinsic characteristics of the clusters instead of the skeleton structure or posture of the subjects. Compared with commercial software and data-driven methods, our method has better labeling accuracy in heterogeneous targets and unknown marker layouts, which demonstrates the promising application of motion capture in humans, rigid or flexible robots.

Towards flexible object-centric visual perception, we propose a one-shot instance-aware object keypoint (OKP) extraction approach, AnyOKP, which leverages the powerful representation ability of pretrained vision transformer (ViT), and can obtain keypoints on multiple object instances of arbitrary category after learning from a support image. An off-the-shelf petrained ViT is directly deployed for generalizable and transferable feature extraction, which is followed by training-free feature enhancement. The best-prototype pairs (BPPs) are searched for in support and query images based on appearance similarity, to yield instance-unaware candidate keypoints. Then, the entire graph with all candidate keypoints as vertices are divided into sub-graphs according to the feature distributions on the graph edges. Finally, each sub-graph represents an object instance. AnyOKP is evaluated on real object images collected with the cameras of a robot arm, a mobile robot, and a surgical robot, which not only demonstrates the cross-category flexibility and instance awareness, but also show remarkable robustness to domain shift and viewpoint change.

Aquatic organisms, due to soft body structure and high agility, have inspired many biomimetic robots. However, considering the issues of insulation and waterproofing, as well as the driving module of soft materials, their control systems are usually larger and heavier. Therefore, small underwater robots often tethered, i.e., it cannot integrate energy and control systems onto the body, which greatly limited in its working range and activity mode. This paper presents a small untethered bionic manta ray. The robotic fish is driven by dielectric elastomer actuators (DEA), which controls the double wing structure on both sides by the central muscle part to simulate the process of the manta ray’s lateral fins fanning to propel itself forward. And the flexible printed circuit board (FPC) constitutes the body of the fish and is also an independent energy control system. The electronic components are evenly distributed on the double-wing structure of the robotic fish to realize the integration of the energy control system. This circuit system can be powered by a small lithium battery and output a periodic voltage to drive the motion of the robotic fish. The masses of our tethered and untethered fish are 1.9g and 5.1g respectively. The swimming speed of these two types of fish can reach 42.5mm/s and 17.0 mm/s. And this design principle can be extended to the research and design of various flexible devices and soft robots.

Abstract The precise perception of the surrounding environment in traffic scenes is an important part of an intelligent transportation system. The event camera could provide complementary information to traditional frame‐based cameras, such as high dynamic range, and high time resolution, in the perception of traffic targets. To improve the precision and reliability of perception as well as facilitate lots of RGB camera‐based studies introduced to event cameras directly, a refined registration method for event‐based cameras and RGB cameras on the basis of pixel‐level region segmentation is proposed, to provide a fusion method at pixel level. A total of eight sequences and a dataset containing 260 typical traffic scenes are contained in the experiment dataset, both selected from DSEC, a traffic event‐based dataset. The registered event image shows a better spatial consistency with RGB images visually. Compared to the baseline, the evaluation indicators, such as the performance of the contrast, the proportion of overlapping pixels, and average registration accuracy have been improved. In the traffic object segmentation task, the average boundary displacement error of our method has decreased and the max decline value has reached 79.665%, compared to the boundary displacement error between ground truth and baseline. These results indicate prospective applications in the perception of intelligent transportation systems combined with event and RGB cameras. The traffic dataset with pixel‐level semantic annotations will be provided soon.

Lower-limb assisted exoskeletons are widely researched for movement assistance or rehabilitation training. Due to advantages of compliance with human body and lightweight, some cable-driven prototypes have been developed, but most of these can assist only unidirectional movement. In this paper we present an untethered cable-driven ankle exoskeleton that can achieve plantarflexion-dorsiflexion bidirectional motion bilaterally using a pair of single motors. The main weights of the exoskeleton, i.e., the motors, power supplement units, and control units, were placed close to the proximity of the human body, i.e., the waist, to reduce the redundant rotation inertia which would apply on the wearer’s leg. A cable force transmission system based on gear-pulley assemblies was designed to transfer the power from the motor to the end-effector effectively. A cable self-tension device on the power output unit was designed to tension the cable during walking. The gait detection system based on a foot pressure sensor and an inertial measurement unit (IMU) could identify the gait cycle and gait states efficiently. To validate the power output performance of the exoskeleton, a torque tracking experiment was conducted. When the subject was wearing the exoskeleton with power on, the muscle activity of the soleus was reduced by 5.2% compared to the state without wearing the exoskeleton. This preliminarily verifies the positive assistance effect of our exoskeleton. The study in this paper demonstrates the promising application of a lightweight cable-driven exoskeleton on human motion augmentation or rehabilitation.

Aquatic-aerial multimodal vehicle is a new concept aircraft that can freely shuttle between water and air. Some of the natural organisms provide the inspiration to realize this multi-locomotion. Most of current prototypes use rigid link mechanisms or hinges to morph the structure thus to adapt to the aquatic-aerial environment, which is commonly complicated and bulky. In this paper, we present a novel prototype with pneumatically-driven soft fins and arms that can fold and spread just like the flying squid. The fins and arms can augment the lift force during flying by spreading and reduce drag force during swimming by folding. The performance of the morphable structures was investigated in wind and water tunnel. The results explain the tradeoff strategies of multimodal-locomotion between water and air, and verify the feasibility of the novel aquatic-aerial vehicle with soft morphable structures.