Recent advances in large language models (LLMs) have demonstrated exceptional reasoning capabilities in natural language processing, sparking interest in applying LLMs to task planning problems in robotics. Most studies focused on task planning for clear natural language commands that specify target objects and their locations. However, for more user-friendly task execution, it is crucial for robots to autonomously plan and carry out tasks based on abstract natural language commands that may not explicitly mention target objects or locations, such as ‘Put the food ingredients in the same place.’ In this study, we propose an LLM-based autonomous task planning framework that generates task plans for abstract natural language commands. This framework consists of two phases: an environment recognition phase and a task planning phase. In the environment recognition phase, a large vision-language model generates a hierarchical scene graph that captures the relationships between objects and spaces in the environment surrounding a robot agent. During the task planning phase, an LLM uses the scene graph and the abstract user command to formulate a plan for the given task. We validate the effectiveness of the proposed framework in the AI2THOR simulation environment, demonstrating its superior performance in task execution when handling abstract commands.

This paper presents a novel method for state estimation of rigid body attitude system evolving on the manifold S3, which is crucial in robotics and drone applications. We introduce a particle filter with stable embedding that extends the system into Euclidean space while ensuring stability of the manifold. Our particle filter with stable embedding enables accurate state estimation by maintaining estimated state values in close proximity to the manifold, while requiring significantly fewer computational resources than the standard exponential-map-based method that keeps state estimates on the manifold. Furthermore, our method facilitates the application of usual techniques designed for particle filters in Euclidean spaces, to the manifold system, as is, without any modification. The accuracy and the efficiency of our particle filter are confirmed both by simulation and by real drone experiments.

Multi-object tracking (MOT) presents a crucial challenge in robotics. Due to limited resources embedded in robots, one time step per processing time for algorithms can be considerably large. This scenario necessitates the operation of MOT at a low frame rate. However, algorithms within the MOT research field have been constructed around datasets functioning at 10–30 frames per second (fps) which can be difficult to operate in the limited resources. In response to it, we introduce a new algorithm, called FocoTrack, which maintains tracking ability in four situations, one of which is when objects are overlapped by each other. Our algorithm exhibits remarkable performance without using any deep appearance descriptor, surpassing existing MOT methods which even use the deep appearance descriptor on a 2.5 fps dataset. We also demonstrate strong results with our algorithm on DanceTrack dataset at 20 fps and provide comprehensive insights through detailed analysis of our tracking model.

Percutaneous coronary intervention (PCI) is a frequently used surgical treatment for cardiovascular disease, one of the leading cause of death in the world. In traditional PCI, a doctor navigates a thin guidewire in a patient's vessel toward a target location by looking into live X-ray angiogram images of the patient. Recently, researchers are using reinforcement learning to automate this guidewire navigation process without attaching any sensor to the guidewire tip. These researchers use a real vessel phantom to train their behavior policy using reinforcement learning. Training a reinforcement learning algorithm on a real setup can give a good guidewire control on that setup, but it is under question whether the trained algorithm can be applied to other vessel structures. We can make various vessel phantoms and train the algorithm on the setups, but it can be really time and money consuming. In this paper, we devise a method for sim-to-real transfer of a guidewire navigation trained by reinforcement learning using only images. We pretrain our behavior policy using data collected by running an expert algorithm in the virtual environment. Then, we train the behavior policy by deep deterministic policy gradient (DDPG) in a virtual environment. With behavior cloning, our method learns to successfully navigate a guidewire in much shorter time than training DDPG from scratch without behavior cloning. After done with the training, we transfer the behavior policy trained in the virtual environment to the guidewire navigation in a real vessel phantom. Our trained behavior policy navigates the guidewire to destinations successfully in all test episodes and navigates faster than the expert algorithm. Experiment video is available at: https: //youtu.be/HCEbIhZsXqw

With the growing application of various robots in real life, the need for an automatic anomaly detection system for robots is necessary for safety. In this paper, we develop an anomaly detection method using a stacked LSTM that can be applied to any robot controlled by a feedback control. Our method does not need installation of additional sensors. Our method is model-free and unsupervised because it does not require the analytical model of the system and the training data does not require faulty operation conditions. We validate our method on real fixed-wing unmanned aerial vehicle flight data containing control surface failure scenarios. We demonstrate the superiority of the proposed algorithm over existing anomaly detection methods in the literature. Our code is available at https://github.com/superhumangod/Model-free-unsupervised-anomaly-detection.