Exploring high hydrostatic pressure environments such as deep sea presents significant challenges to robotic devices, for they often rely on strong yet heavy and costly protective structures to shield components from being crushed by the extreme pressure. To dismiss the need for bulky protection shells for actuation devices, we reported an extreme-hydrostatic-pressure resilient rotary dielectric elastomer actuator (DEA) for propulsion application in deep-sea pressure condition. DEAs are inherently resistant to damage caused by external pressure, due to their uniform and cavity-free structure. In this study, we analyzed the material properties of the DEA’s elastomer, evaluated the rotary actuator’s lifespan at up to 110 MPa high-pressure liquid conditions, and output performance under both ambient and 30 MPa (equivalent to 3,000 m underwater). Our results show that the rotary actuator maintained functionality at such hydrostatic pressure, with a lifespan exceeding 300,000 cycles and a high rotational output speed of 820 rpm. The rotary actuator was subsequently used to drive the robot with a propeller in a simulated deep-sea pressure fluidic environment, demonstrating our DEA’s performance as well as design simplicity for deep-sea applications without protection structures. While high hydrostatic pressure negatively impacted the actuator’s lifespan and slightly reduced its dynamic performance, our results confirmed that the DEA was a viable solution for deep-sea exploration, laying a solid foundation for the further development of DEA-powered devices for underwater missions.

This paper presents the design, modeling, and experimental validation of a biomimetic robotic butterfly (BRB) that integrates a compliant mechanism to achieve coupled wing-abdomen motion. Drawing inspiration from the natural flight dynamics of butterflies, a theoretical model is developed to investigate the impact of abdominal undulation on flight performance. To validate the model, motion capture experiments are conducted on three configurations: a BRB without an abdomen, with a fixed abdomen, and with an undulating abdomen. The results demonstrate that abdominal undulation enhances lift generation, extends flight duration, and stabilizes pitch oscillations, thereby improving overall flight performance. These findings underscore the significance of wing-abdomen interaction in flapping-wing aerial vehicles (FWAVs) and lay the groundwork for future advancements in energy-efficient biomimetic flight designs.

Scene understanding enables intelligent agents to interpret and comprehend their environment. While existing large vision-language models (LVLMs) for scene understanding have primarily focused on indoor household tasks, they face two significant limitations when applied to outdoor large-scale scene understanding. First, outdoor scenarios typically encompass larger-scale environments observed through various sensors from multiple viewpoints (e.g., bird view and terrestrial view), while existing indoor LVLMs mainly analyze single visual modalities within building-scale contexts from humanoid viewpoints. Second, existing LVLMs suffer from missing multidomain perception outdoor data and struggle to effectively integrate 2D and 3D visual information. To address the aforementioned limitations, we build the first multidomain perception outdoor scene understanding dataset, named SVM-City, deriving from multi-Scale scenarios with multi-View and multi-Modal instruction tuning data. It contains 420k images and 4, 811M point clouds with 567k question-answering pairs from vehicles, low-altitude drones, high-altitude aerial planes, and satellite. To effectively fuse multimodal data in the absence of one modality, we introduce incomplete multimodal learning to model outdoor scene understanding and design the LVLM named City-VLM. Multimodal fusion is realized by constructed as a joint probabilistic distribution space rather than implementing directly explicit fusion operations (e.g., concatenation). Experimental results on three typical outdoor scene understanding tasks show City-VLM achieves 18.14 % performance surpassing existing LVLMs in question-answering tasks averagely. Our method demonstrates pragmatic and generalization performance across multiple outdoor scenes.

The utilization of a well-established flapping wing robot simulation holds significant importance in the advancement of flapping wing mechanisms and algorithms. This research paper introduces a pioneering application-oriented flapping wing robot simulation platform that exhibits high compatibility with diverse mechanical designs and adaptability to various robotic tasks. Initially, the paper presents the blade element theory and the quasi-steady model as computational approaches for determining the aerodynamics of flapping wings based on their kinematics. The computation incorporates translational lift, translational drag, rotational lift, added mass force, and clap-and-fling mechanism. The simulation platform is validated through flight control tasks, providing a comprehensive assessment of its performance. Moreover, this study addresses the challenges of attitude tracking and trajectory tracking control for a specially designed flapping wing robot. The proposed control strategies are evaluated through real flight experiments, offering practical insights into the robot flight capabilities. Note to Practitioners—One of the primary contributions of this research is the introduction of a novel and robust flapping wing flight controller that effectively addresses the challenges associated with attitude tracking control and positional trajectory tracking. The proposed controller demonstrates superior performance compared to existing algorithms, as evidenced by comparative simulations conducted on the developed simulation platform. Furthermore, real flight experiments are performed on a self-made flapping wing robot using the same control algorithm and parameters employed in the simulations. Another highlight lies in the transferability from simulation to reality. The utilization of a high-fidelity simulation platform enables in-depth exploration and scrutiny of complex behaviors manifested in diverse flight tasks. This, in turn, allows for meticulous analysis and examination to gain valuable insights into the practical implementation of flapping wing robot flight.

Robotic grasping is a fundamental ability for a robot to interact with the environment. Current methods focus on how to obtain a stable and reliable grasping pose in object level, while little work has been studied on part (shape)-wise grasping which is related to fine-grained grasping and robotic affordance. Parts can be seen as atomic elements to compose an object, which contains rich semantic knowledge and a strong correlation with affordance. However, lacking a large part-wise 3D robotic dataset limits the development of part representation learning and downstream applications. In this paper, we propose a new large Language-guided SHape grAsPing datasEt (named LangSHAPE) to promote 3D part-level affordance and grasping ability learning. From the perspective of robotic cognition, we design a two-stage fine-grained robotic grasping framework (named LangPartGPD), including a novel 3D part language grounding model and a partaware grasp pose detection model, in which explicit language input from human or large language models (LLMs) could guide a robot to generate part-level 6-DoF grasping pose with textual explanation. Our method combines the advantages of humanrobot collaboration and LLMs’ planning ability using explicit language as a symbolic intermediate. To evaluate the effectiveness of our proposed method, we perform 3D part grounding and fine-grained grasp detection experiments on both simulation and physical robot settings, following language instructions across different degrees of textual complexity. Results show our method achieves competitive performance in 3D geometry fine-grained grounding, object affordance inference, and 3D part-aware grasping tasks. Our dataset and code are available on our project website https://sites.google.com/view/lang-shape.

Twisted and Coiled Actuators (TCAs) are promising in soft robotics for their high energy density, light weight, and low voltage. However, current TCA-based soft robots face challenges of limited deformation and control precision, mainly due to the preloading requirement of TCAs and the lack of suitable intrinsic sensing capabilities. To address these issues, we designed a TCA with high load capacity, free stroke, and self-sensing capabilities, proposed flexible optical fiber-based posture and tactile sensing methods, and developed a multi-loop feedback controller. Collectively, these enable millimeter-level tracking accuracy in a soft tentacle robot. The TCA, with an optimized manufacturing process, achieves a 30% free stroke without preloading, a 32% improvement in ultimate stress, and temperature self-sensing capabilities with a maximum error of less than 6%. Combining the TCAs with compliant macro-bend optical fibers and soft optical waveguides, we created a soft robotic tentacle with intrinsic posture and tactile sensing, and designed a multi-input multi-output closed-loop and feedforward controller. Experiments demonstrate that the model-based feedforward significantly improves the control performance, reducing the rise time by 15.3%. The trajectory tracking error remains within the millimeter range, and the repetitive positioning error for hexagonal trajectories reaches sub-millimeter precision. The tactile sensor of the robot enables real-time perception of the object's modulus and pressing states. These findings highlight the soft robotic tentacle's potential for various applications, including underwater exploration, detection, and sampling.

Visual language navigation (VLN) is one of the important research in embodied AI. It aims to enable an agent to understand the surrounding environment and complete navigation tasks. VLN instructions could be categorized into coarse-grained and fine-grained commands. Fine-grained command describes a whole task with subtasks step-by-step. In contrast, coarse-grained command gives an abstract task description, which more suites human habits. Most existing work focuses on the former kind of instruction in VLN tasks, ignoring the latter abstract instructions belonging to daily life scenarios. To overcome the above challenge in abstract instruction, we attempt to consider coarse-grained instruction in VLN by event knowledge enhancement. Specifically, we first propose a prompt-based framework to extract an event knowledge graph (named VLN-EventKG =) for VLN integrally over multiple mainstream benchmark datasets. Through small and large language model collaboration, we realize knowledge-enhanced navigation planning (named EventNav) for VLN tasks with coarse-grained instruction input. Additionally, we design a novel dynamic history backtracking module to correct potential error action planning in real time. Experimental results in various public benchmarks show our knowledge-enhanced method has superiority in coarse-grained-instruction VLN using our proposed VLN-EventKG with over 5% improvement in success rate. Our project is available at https://sites.google.com/view/vln-eventkg

3D multimodal question answering (MQA) plays a crucial role in scene understanding by enabling intelligent agents to comprehend their surroundings in 3D environments. While existing research has primarily focused on indoor household tasks and outdoor roadside autonomous driving tasks, there has been limited exploration of city-level scene understanding tasks. Furthermore, existing research faces challenges in understanding city scenes, due to the absence of spatial semantic information and human-environment interaction information at the city level.To address these challenges, we investigate 3D MQA from both dataset and method perspectives. From the dataset perspective, we introduce a novel 3D MQA dataset named City-3DQA for city-level scene understanding, which is the first dataset to incorporate scene semantic and human-environment interactive tasks within the city. From the method perspective, we propose a Scene graph enhanced City-level Understanding method (Sg-CityU), which utilizes the scene graph to introduce the spatial semantic. A new benchmark is reported and our proposed Sg-CityU achieves accuracy of 63.94 % and 63.76 % in different settings of City-3DQA. Compared to indoor 3D MQA methods and zero-shot using advanced large language models (LLMs), Sg-CityU demonstrates state-of-the-art (SOTA) performance in robustness and generalization.