Vision-language-action models have emerged as a crucial paradigm in robotic manipulation. However, existing VLA models exhibit notable limitations in handling ambiguous language instructions and unknown environmental states. Furthermore, their perception is largely constrained to static two-dimensional observations, lacking the capability to model three-dimensional interactions between the robot and its environment. To address these challenges, this paper proposes GraphCoT-VLA, an efficient end-to-end model. To enhance the model's ability to interpret ambiguous instructions and improve task planning, we design a structured Chain-of-Thought reasoning module that integrates high-level task understanding and planning, failed task feedback, and low-level imaginative reasoning about future object positions and robot actions. Additionally, we construct a real-time updatable 3D Pose-Object graph, which captures the spatial configuration of robot joints and the topological relationships between objects in 3D space, enabling the model to better understand and manipulate their interactions. We further integrates a dropout hybrid reasoning strategy to achieve efficient control outputs. Experimental results across multiple real-world robotic tasks demonstrate that GraphCoT-VLA significantly outperforms existing methods in terms of task success rate and response speed, exhibiting strong generalization and robustness in open environments and under uncertain instructions.

Recent advancements in Large Language Models (LLMs) have catalyzed numerous efforts to apply these technologies to embodied tasks, with a particular focus on high-level task planning and task decomposition. LLMs face challenges in understanding the physical world, especially regarding spatial, temporal, and causal relationships among objects and actions. Moreover, the current benchmarks for evaluating these relationships are limited. To further investigate this domain, we introduce a novel embodied task planning benchmark, ET-Plan-Bench. This benchmark features a controllable and diverse array of embodied tasks, varying in levels of difficulty and complexity. It is designed to evaluate two critical dimensions of LLMs’ application in embodied task understanding: spatial understanding (including relation constraints and occlusion of target objects) and temporal and causal comprehension of sequences of actions within an environment. Utilizing multi-source simulators as the backend simulator, ET-Plan-Bench provides immediate environmental feedback to LLMs, enabling dynamic interaction with the environment and the capacity for re-planning as necessary. We evaluated state-of-the-art open-source and closed-source foundational models, including GPT-4, Llama, and Mistral, using our proposed benchmark. While these models perform adequately on simple navigation tasks, their performance significantly deteriorates when con-fronted with tasks that demand a deeper understanding of spatial, temporal, and causal relationships. Consequently, our benchmark distinguishes itself as a large-scale, quantifiable, highly automated, and fine-grained diagnostic framework that presents a substantial challenge to the latest foundational models. We hope it will inspire and propel further research in embodied task planning utilizing foundational models. Code available at: https://github.com/ET-Plan-Bench/ET-Plan-Bench

Recent advances in LVLMs have improved vision-language understanding, but they still struggle with spatial perception, limiting their ability to reason about complex 3D scenes. Unlike previous approaches that incorporate 3D representations into models to improve spatial understanding, we aim to unlock the potential of VLMs by leveraging spatially relevant image data. To this end, we introduce a novel 2D spatial data generation and annotation pipeline built upon scene data with 3D ground-truth. This pipeline enables the creation of a diverse set of spatial tasks, ranging from basic perception tasks to more complex reasoning tasks. Leveraging this pipeline, we construct SPAR-7M, a large-scale dataset generated from thousands of scenes across multiple public datasets. In addition, we introduce SPAR-Bench, a benchmark designed to offer a more comprehensive evaluation of spatial capabilities compared to existing spatial benchmarks, supporting both single-view and multi-view inputs. Training on both SPAR-7M and large-scale 2D datasets enables our models to achieve state-of-the-art performance on 2D spatial benchmarks. Further fine-tuning on 3D task-specific datasets yields competitive results, underscoring the effectiveness of our dataset in enhancing spatial reasoning.

Leveraging diverse robotic data for pretraining remains a critical challenge. Existing methods typically model the dataset’s action distribution using simple observations as inputs. However, these inputs are often incomplete, resulting in a dispersed conditional action distribution—an issue we refer to as coordinate system chaos and state chaos. This inconsistency significantly hampers pretraining efficiency. To address this, we propose 4D-VLA, a novel approach that effectively integrates 4D information into the input to mitigate these sources of chaos. Our model introduces depth and temporal information into visual features with sequential RGB-D inputs, aligning the coordinate systems of the robot and the scene. This alignment endows the model with strong spatiotemporal reasoning capabilities while minimizing training overhead. Additionally, we introduce Memory bank sampling, a frame sampling strategy designed to extract informative frames from historical images, further improving effectiveness and efficiency. Experimental results demonstrate that our pretraining method and architectural components substantially enhance model performance. In both simulated and real-world experiments, our model achieves a significant increase in success rate over OpenVLA.To further assess spatial perception and generalization to novel views, we introduce MV-Bench, a multi-view simulation benchmark. Our model consistently outperforms existing methods, demonstrating stronger spatial understanding and adaptability.

Playing table tennis with a human player is a challenging robotic task due to its dynamic nature. Despite a number of researches being devoted to developing robotic table tennis systems, most of the works have demanding hardware requirements and ignore safety measures when generating the swing stoke. To address these issues, we propose a safe motion planning framework that fully pushes the robotic hardware performance limits to play table tennis. In particular, we propose a pipeline to generate manipulator joint trajectories with environmental safety constraints and scale the trajectories to satisfy joint movement limitations. We use three different agents to validate the planning algorithm with our handmade robot platform in both simulation and real-world environments.

Flexible operation and maintenance (O&M) is critical for telecommunication (telecom) service providers due to the ever-growing communication networks. Currently, most O&M operations still rely on rule-based strategies, which only cover limited scenarios and are costly to extend for novel applications as expert knowledge is intensively involved. To build a more flexible O&M system, we propose a language model to extract useful representations out of massive network signaling messages and use the representations to perform downstream O&M tasks. Given that a vanilla language model is not directly applicable for the structured signaling messages, we develop an expert-knowledge-inspired statistical approach to preprocess the messages and a hierarchical network architecture to extract message relations among different levels. Moreover, network messages in the real world are often contaminated, which can mislead the language model to learn incorrect message patterns. To mitigate data contamination, we propose a reverse training method that prevents the language model from learning the contaminated data. We collected hundreds of thousands of signaling message flows to train the proposed signaling language model and applied the trained model to O&M tasks. Offline experiments show that our proposed language model captures various signaling protocols and the extracted representations enable us to achieve expert-level performance in network anomaly detection and service recognition. Our language model has been deployed online at Huawei and significantly improved O&M efficiency.