Vision-language-action models have emerged as a crucial paradigm in robotic manipulation. However, existing VLA models exhibit notable limitations in handling ambiguous language instructions and unknown environmental states. Furthermore, their perception is largely constrained to static two-dimensional observations, lacking the capability to model three-dimensional interactions between the robot and its environment. To address these challenges, this paper proposes GraphCoT-VLA, an efficient end-to-end model. To enhance the model's ability to interpret ambiguous instructions and improve task planning, we design a structured Chain-of-Thought reasoning module that integrates high-level task understanding and planning, failed task feedback, and low-level imaginative reasoning about future object positions and robot actions. Additionally, we construct a real-time updatable 3D Pose-Object graph, which captures the spatial configuration of robot joints and the topological relationships between objects in 3D space, enabling the model to better understand and manipulate their interactions. We further integrates a dropout hybrid reasoning strategy to achieve efficient control outputs. Experimental results across multiple real-world robotic tasks demonstrate that GraphCoT-VLA significantly outperforms existing methods in terms of task success rate and response speed, exhibiting strong generalization and robustness in open environments and under uncertain instructions.

Significant disparities between the features of natural images and those inherent to histopathological images make it challenging to directly apply and transfer pre-trained models from natural images to histopathology tasks. Moreover, the frequent lack of annotations in histopathology patch images has driven researchers to explore self-supervised learning methods like mask reconstruction for learning representations from large amounts of unlabeled data. Crucially, previous mask-based efforts in self-supervised learning have often overlooked the spatial interactions among entities, which are essential for constructing accurate representations of pathological entities. To address these challenges, constructing graphs of entities is a promising approach. In addition, the diffusion reconstruction strategy has recently shown superior performance through its random intensity noise addition technique to enhance the robust learned representation. Therefore, we introduce H-MGDM, a novel self-supervised Histopathology image representation learning method through the Dynamic Entity-Masked Graph Diffusion Model. Specifically, we propose to use complementary subgraphs as latent diffusion conditions and self-supervised targets respectively during pre-training. We note that the graph can embed entities' topological relationships and enhance representation. Dynamic conditions and targets can improve pathological fine reconstruction. Our model has conducted pretraining experiments on three large histopathological datasets. The advanced predictive performance and interpretability of H-MGDM are clearly evaluated on comprehensive downstream tasks such as classification and survival analysis on six datasets.

The efficient segmentation of histopathological tissues plays a crucial role in aiding diagnostics and prognosis. However, the existing segmentation methods not only typically demand extensive human annotation and/or time-consuming model fine-tuning, but also exhibit limited generalization capabilities. To address these, we propose a novel fine-tuning-free method SuperPromptSeg, which can use the Segment Anything Model (SAM) as a foundation model and only needs a few point prompts. SuperPromptSeg consists of three main components: prompt selection, pseudo point generation, and mask selection. First, Simple Linear Iterative Clustering (SLIC) is employed to partition a patch into superpixels and K-means clustering is used to select point prompts. Then, pseudo points are generated from these point prompts and superpixels, serving together with the point prompts as inputs of SAM. Finally, two novel penalties are proposed to select predicted mask results using SAM’s outputs. Extensive experiments are conducted on three benchmark datasets to demonstrate the robust zero-shot segmentation capabilities of SuperPromptSeg. SuperPromptSeg achieves an average increase of 8.4% in Dice score, with a maximum improvement of 19.6%, greatly improving SAM’s segmentation capability on digital pathology images.

International audience