Large Language Models (LLMs) have demonstrated remarkable abilities in various language tasks, making them promising candidates for decision-making in robotics. Inspired by Hierarchical Reinforcement Learning (HRL), we propose Retrieval-Augmented Hierarchical in-context reinforcement Learning (RAHL), a novel framework that decomposes complex tasks into sub-tasks using an LLM-based high-level policy, in which a complex task is decomposed into sub-tasks by a high-level policy on-the-fly. The sub-tasks, defined by goals, are assigned to the low-level policy to complete. To improve the agent's performance in multi-episode execution, we propose Hindsight Modular Reflection (HMR), where, instead of reflecting on the full trajectory, we let the agent reflect on shorter sub-trajectories using intermediate goals to improve reflection efficiency. We evaluated the decision-making ability of the proposed RAHL in three benchmark environments, ALFWorld, Webshop, and HotpotQA, where the results show that RAHL can achieve an improvement in the performance of, respectively, 9%, 42%, and 10% in 5 execution episodes compared to state-of-the-art baselines. We also implemented RAHL on the Boston Dynamics SPOT robot, which is shown to effectively scan the environment, find entrances, and navigate to new rooms controlled by the LLM policy.

Recent advances in segmentation models such as the Segment Anything Model (SAM) have made it possible to generate high-quality masks for arbitrary regions in an image. However, these models lack semantic reasoning and cannot assign meaningful labels without predefined class prompts. Similarly, models like YOLO-Seg are restricted to fixed vocabularies, and CLIP-based segmentors struggle with abstract or hierarchical concepts. In this paper, we propose a Large Vision-Language Model (LVLM)-assisted information-rich semantic segmentation framework that bridges this gap by integrating SAM with open-vocabulary segmentation and multimodal language understanding. Our approach begins by generating precise masks using SAM and querying an LVLM to extract semantically relevant object names via carefully designed prompts. We then use a CLIP-based segmentor to produce rough masks corresponding to the object list and apply a voting-based mechanism to align rough and fine masks, enriching the final output with detailed and hierarchically organized semantic information. Through extensive evaluations on the COCO 2017 dataset, we demonstrate that our method not only outperforms traditional models in abstract and implicit labeling tasks but also provides semantically rich outputs that support human-in-the-loop and open-world applications.

To better enhance the network service for different user devices in various scenarios, autonomous aerial vehicles (AAVs) are increasingly used as aerial base stations (ABSs). However, optimizing coverage for user devices via AAV team control is an NP-hard problem and escalates exponentially in complexity with the growing number of user devices. To address this challenge, researchers have turned to reinforcement learning (RL) for a more practical solution. With the growing prevalence of the Internet of Things (IoT), the diversity of user devices increases, posing challenges for traditional RL, as 1) the spatial distribution of devices becomes more complex; 2) variations in device types and device mobility increase the training latency; 3) the high-speed movement of IoT devices can lead to performance deterioration in widely used RL algorithms with discrete action space; and 4) traditional RL struggles to adapt to new environments. To solve these problems, we propose a new meta-RL framework, Meta-RL with explicit task inference (Meta-ETI). Then, we apply this framework to efficiently train an energy-efficient AAV control policy for fair and effective coverage in 3-D dynamic environments. Meta-ETI is evaluated in both theoretical and application-related aspects and demonstrates superior performance compared to the baseline frameworks. The result shows that Meta-ETI demonstrates 2–3 times faster adaptation speed and a decent performance in sample efficiency. Furthermore, in the AAV-IoT coverage application, Meta-ETI shows 30%–50% better in energy efficiency and 40%–60% more served devices because of the fair coverage.

Advancements in Intelligent Transportation Systems (ITS) have led to innovative solutions for planning optimization, efficiency enhancement, and resource allocation in transportation networks, which are demonstrated in applications such as smart parking lot management and electric vehicle (EV) charging station allocation, where improved decision-making and system-wide optimization have been achieved. However, as these systems evolve, the demand for better adaptability and coordination continues to grow to maximize their overall effectiveness and efficiency. To achieve this, we propose the Multi-Personality Multi-Agent Meta-Reinforcement Learning (MPMA-MRL) framework. This approach incorporates multiple meta-trained, meta-tested explainable personality policies, which are deployed to each agent. A personality selector is trained and deployed on each agent to optimize the overall performance. MPMA-MRL is superior than traditional methods in terms of the adaptability and coordination in ITS by leveraging improved information from the environment, more practical coordination among agents, faster adaptation speed to intermediate tasks, and more appropriate allocation and planning. The proposed framework is evaluated in the applications of parking lot optimization and EV charging station allocation. Its broader impact on multi-agent smart systems is analyzed to demonstrate its generalizability. The results demonstrate that in parking lot optimization, MPMA-MRL significantly reduces the time required to direct all vehicles to available parking spots. In EV charging station allocation, MPMA-MRL effectively minimizes waiting times at charging stations. Moreover, in both applications, MPMA-MRL exhibits enhanced adaptability to previously unseen scenarios, improving its applicability.

Unmanned Aerial Vehicles (UAVs) have become prevalent in Search-And-Rescue (SAR) missions. However, existing solutions to the control and coordination of UAV s are mostly limited to specific environments and are not robust to handle unreliable/unstable communications. To deal with these challenges, Hierarchical Multi-Agent Actor-Critic (HMAAC) framework is proposed where a high-level policy is placed on top of individual low-level actor-critic policies to relax the inter-dependency among the agents. The low-level policies are considered conditionally independent given the coordination action, which is generated by the high-level policy. A Central-ized Training Decentralized Execution (CTDE) would not work because it cannot be assumed that communication is always perfect during training and that the whole system can rely on stable communications during deployment. The proposed framework is evaluated in AirSim, a realistic multi-UAV simula-tor, and is compared against two existing algorithms, i.e., Multi- Agent Actor-Critic (MAAC) and decentralized REINFORCE, in two scenarios, (a) when packet drop is modeled as a Bernoulli process and (b) when shadow zones are created in the search space and communication will be lost if the agents are in these zones. Results show that HMAAC is scalable and robust to unreliable communication and outperforms the other algorithms in terms of exploration and coordination when the number of agents is large and communications are not stable.

With the rapid growth of the use of Machine Learning (ML) techniques in Unmanned Aerial Vehicles (UAVs), there is an opportunity to use ML techniques to detect and prevent anomalous behavior in drones. However, limited drone power thwarts successful implementation of contemporary power-hungry ML techniques. Therefore, we propose a hybrid analog-digital system to solve the problem of continuous anomaly detection. In this paper, a series of pure analog ML methods including SVMs (linear, polynomial, Radial Basis Function (RBF) and Sigmoid kernels) as well as pure analog fully-connected Neural Network (NN) are presented. We validate our method with sensor data from a series of drone experiments to detect and identify causes of failure in real-time. The results show that RBF kernel provides at least 88.17 % and at most 99.99 % accuracy under different time window and crash-like scenarios with an extremely low False Negative (FN) ratio with sensor data especially in the z-axis.