Large Language Models (LLMs) have demonstrated remarkable abilities in various language tasks, making them promising candidates for decision-making in robotics. Inspired by Hierarchical Reinforcement Learning (HRL), we propose Retrieval-Augmented Hierarchical in-context reinforcement Learning (RAHL), a novel framework that decomposes complex tasks into sub-tasks using an LLM-based high-level policy, in which a complex task is decomposed into sub-tasks by a high-level policy on-the-fly. The sub-tasks, defined by goals, are assigned to the low-level policy to complete. To improve the agent's performance in multi-episode execution, we propose Hindsight Modular Reflection (HMR), where, instead of reflecting on the full trajectory, we let the agent reflect on shorter sub-trajectories using intermediate goals to improve reflection efficiency. We evaluated the decision-making ability of the proposed RAHL in three benchmark environments, ALFWorld, Webshop, and HotpotQA, where the results show that RAHL can achieve an improvement in the performance of, respectively, 9%, 42%, and 10% in 5 execution episodes compared to state-of-the-art baselines. We also implemented RAHL on the Boston Dynamics SPOT robot, which is shown to effectively scan the environment, find entrances, and navigate to new rooms controlled by the LLM policy.

Previous studies on cooperative perception often assumed homogeneous sensors across vehicles, which is unrealistic due to the incremental deployment of technology and vehicle budgets. Heterogeneous sensors pose challenges, as they generate raw data with different modalities, resolutions, locations, and orientations. We propose a Location-Aware and Modality-Aware Data Fusion architecture (LAMAF) for heterogeneous data fusion in distributed cooperative perception. LAMAF’s Reduce Layer utilizes latent Bird’s Eye View (BEV) coordinates encoding and modality embedding to capture spatial dependencies and extract input data modality. Experiments on the Vehicle-to-Vehicle (V2V) perception dataset, OPV2V, demonstrate LAMAF’s outstanding performance and generalizability to different downstream tasks. Furthermore, we introduce the first configurable Communication-In-the-Loop (CommIL) V2V dataset, which significantly reduces the gap between real-world environments and cooperative V2V datasets, allowing researchers to develop cooperative V2V-based frameworks while considering realistic communication constraints. The dataset generation code and pre-generated data are available at: CommIL-Dataset.

Recent advances in segmentation models such as the Segment Anything Model (SAM) have made it possible to generate high-quality masks for arbitrary regions in an image. However, these models lack semantic reasoning and cannot assign meaningful labels without predefined class prompts. Similarly, models like YOLO-Seg are restricted to fixed vocabularies, and CLIP-based segmentors struggle with abstract or hierarchical concepts. In this paper, we propose a Large Vision-Language Model (LVLM)-assisted information-rich semantic segmentation framework that bridges this gap by integrating SAM with open-vocabulary segmentation and multimodal language understanding. Our approach begins by generating precise masks using SAM and querying an LVLM to extract semantically relevant object names via carefully designed prompts. We then use a CLIP-based segmentor to produce rough masks corresponding to the object list and apply a voting-based mechanism to align rough and fine masks, enriching the final output with detailed and hierarchically organized semantic information. Through extensive evaluations on the COCO 2017 dataset, we demonstrate that our method not only outperforms traditional models in abstract and implicit labeling tasks but also provides semantically rich outputs that support human-in-the-loop and open-world applications.

To better enhance the network service for different user devices in various scenarios, autonomous aerial vehicles (AAVs) are increasingly used as aerial base stations (ABSs). However, optimizing coverage for user devices via AAV team control is an NP-hard problem and escalates exponentially in complexity with the growing number of user devices. To address this challenge, researchers have turned to reinforcement learning (RL) for a more practical solution. With the growing prevalence of the Internet of Things (IoT), the diversity of user devices increases, posing challenges for traditional RL, as 1) the spatial distribution of devices becomes more complex; 2) variations in device types and device mobility increase the training latency; 3) the high-speed movement of IoT devices can lead to performance deterioration in widely used RL algorithms with discrete action space; and 4) traditional RL struggles to adapt to new environments. To solve these problems, we propose a new meta-RL framework, Meta-RL with explicit task inference (Meta-ETI). Then, we apply this framework to efficiently train an energy-efficient AAV control policy for fair and effective coverage in 3-D dynamic environments. Meta-ETI is evaluated in both theoretical and application-related aspects and demonstrates superior performance compared to the baseline frameworks. The result shows that Meta-ETI demonstrates 2–3 times faster adaptation speed and a decent performance in sample efficiency. Furthermore, in the AAV-IoT coverage application, Meta-ETI shows 30%–50% better in energy efficiency and 40%–60% more served devices because of the fair coverage.

Federated learning (FL) has been popular recently as a framework for training machine learning (ML) models in a distributed and privacy-preserving manner. Traditional FL frameworks often struggle with model and statistical heterogeneity among participating clients, impacting learning performance and practicality. To overcome these fundamental limitations, we introduce Fed2KD+, a novel FL framework that leverages a set of tiny unified models and conditional variational auto-encoders (CVAEs) to enable FL training for heterogeneous models between network clients. Using forward and backward distillation processes, Fed2KD+ allows a seamless exchange of knowledge, mitigating data and heterogeneity problems of the model. Moreover, we propose a cosine similarity penalty in the loss function of CVAE+ to enhance the generalizability of CVAE for non-IID scenarios, improving the adaptability and efficiency of the framework. Furthermore, our framework design incorporates a co-design with radio access network (RAN) architecture, reducing the fronthaul traffic volume and improving scalability. Extensive evaluations of one image and two Internet of Things datasets demonstrate the superiority of Fed2KD+ in achieving higher accuracy and faster convergence compared to existing methods, including FedAvg, FedMD, and FedGen. Furthermore, we also performed hardware profiling on the Raspberry Pi and NVIDIA Jetson Nano to quantify the additional resources required to train the unified and CVAE+ models.

Advancements in Intelligent Transportation Systems (ITS) have led to innovative solutions for planning optimization, efficiency enhancement, and resource allocation in transportation networks, which are demonstrated in applications such as smart parking lot management and electric vehicle (EV) charging station allocation, where improved decision-making and system-wide optimization have been achieved. However, as these systems evolve, the demand for better adaptability and coordination continues to grow to maximize their overall effectiveness and efficiency. To achieve this, we propose the Multi-Personality Multi-Agent Meta-Reinforcement Learning (MPMA-MRL) framework. This approach incorporates multiple meta-trained, meta-tested explainable personality policies, which are deployed to each agent. A personality selector is trained and deployed on each agent to optimize the overall performance. MPMA-MRL is superior than traditional methods in terms of the adaptability and coordination in ITS by leveraging improved information from the environment, more practical coordination among agents, faster adaptation speed to intermediate tasks, and more appropriate allocation and planning. The proposed framework is evaluated in the applications of parking lot optimization and EV charging station allocation. Its broader impact on multi-agent smart systems is analyzed to demonstrate its generalizability. The results demonstrate that in parking lot optimization, MPMA-MRL significantly reduces the time required to direct all vehicles to available parking spots. In EV charging station allocation, MPMA-MRL effectively minimizes waiting times at charging stations. Moreover, in both applications, MPMA-MRL exhibits enhanced adaptability to previously unseen scenarios, improving its applicability.

Next Generation Radio Access Networks (NG-RANs) are a promising paradigm for meeting 6G and future application requirements. However, the practical implementation of NG-RAN systems faces significant challenges due to novel technologies, network densification, and more complex applications. Specifically, the limited capacity of front-haul links and privacy concerns have posed severe constraints that must be addressed. To overcome these obstacles, we present a novel approach, called FedBNG, which is a disaggregated and distributed Federated Learning (FL)-based algorithm for NG-RAN. This algorithm enables collaboration between User Equipment (UEs) and the NG-RAN infrastructure through a learning process and shared prediction models, ultimately improving privacy and alleviating the burden on the front-haul interface. Using a shared predictive model, our proposed approach facilitates cooperative learning between Radio Units (RUs) and Distributed Units (DUs). To accomplish this, we initially used the first-phase training models of RUs and DUs as input for local training. Subsequently, the suboptimal DU models are uploaded to the Central Unit (CU) for the next phase of global training. We present numerical results to evaluate the efficacy of our proposed approach in terms of accuracy, service latency, and traffic volume. Our algorithm’s convergence properties demonstrate that it outperforms the current state-of-the-art solution based on FedAvg.

We explore contextual biasing with Large Language Models (LLMs) to enhance Automatic Speech Recognition (ASR) in second-pass rescoring. Our approach introduces the utilization of prompts for LLMs during rescoring without the need for fine-tuning. These prompts incorporate a biasing list and a set of few-shot examples, serving as supplementary sources of information when evaluating the hypothesis score. Furthermore, we introduce multi-task training for LLMs to predict entity class and the subsequent token. To address sequence length constraints and improve the efficiency of contextual biasing, we propose dynamic prompting based on class tag predictions. Through dynamic prompting, we leverage the class tag predictions to identify the most probable entity class and subsequently utilize entities within this class as biasing context for the next token prediction. We evaluate the performance of proposed methods in terms of Word Error Rate (WER) on an internal entity-heavy and the SLUE-Voxpopuli datasets. Our results show significant improvements: biasing lists and few-shot examples achieved a relative improvement of 17.8% and 9.6%, while multitask training and dynamic prompting achieved 20.0% and 11.3% relative WER improvement, respectively.

We study the Traffic Steering (TS) problem in Open Radio Access Network (O-RAN), leveraging its RAN Intelligent Controller (RIC), in which RAN configuration parameters of cells can be jointly and dynamically optimized in near-real-time. To address the TS problem, we propose a novel Cascade Reinforcement Learning (CaRL) framework, where we propose state space factorization and policy decomposition to mitigate the need for large complex models and well-labeled datasets. For each sub-state space, an RL sub-policy is trained to optimize the Quality of Service (QoS). To apply CaRL to new network areas, we propose a knowledge transfer approach to initialize a new sub-policy based on knowledge learned by the trained policies. To evaluate CaRL, we build a data-driven and scalable RIC Digital Twin (DT) that is modeled using real-world data, including network setup, user geo-distribution, and traffic demand, among others, from a tier-1 RAN operator. We evaluated CaRL in two DT scenarios representing two different US cities and compared its performance with business-as-usual policy as a baseline and other competing optimization approaches (i.e., heuristic and Q-table algorithms). Furthermore, we have conducted a field trial with the RAN operator to evaluate the performance of CaRL in two areas in the Northeast US regions.

Unmanned Aerial Vehicles (UAVs) have become prevalent in Search-And-Rescue (SAR) missions. However, existing solutions to the control and coordination of UAV s are mostly limited to specific environments and are not robust to handle unreliable/unstable communications. To deal with these challenges, Hierarchical Multi-Agent Actor-Critic (HMAAC) framework is proposed where a high-level policy is placed on top of individual low-level actor-critic policies to relax the inter-dependency among the agents. The low-level policies are considered conditionally independent given the coordination action, which is generated by the high-level policy. A Central-ized Training Decentralized Execution (CTDE) would not work because it cannot be assumed that communication is always perfect during training and that the whole system can rely on stable communications during deployment. The proposed framework is evaluated in AirSim, a realistic multi-UAV simula-tor, and is compared against two existing algorithms, i.e., Multi- Agent Actor-Critic (MAAC) and decentralized REINFORCE, in two scenarios, (a) when packet drop is modeled as a Bernoulli process and (b) when shadow zones are created in the search space and communication will be lost if the agents are in these zones. Results show that HMAAC is scalable and robust to unreliable communication and outperforms the other algorithms in terms of exploration and coordination when the number of agents is large and communications are not stable.

Search and Rescue (SAR) is an important part of several applications of national and social interest. Existing solutions for search missions in both terrestrial and aerial domains are mostly limited to single agent and specific environments; however, search missions can significantly benefit from the use of multiple agents that can quickly adapt to new environments. In this paper, we propose a framework based on Multi-Agent Deep Reinforcement Learning (MADRL) that realizes the actor-critic framework in a distributed manner for coordinating multiple Unmanned Aerial Vehicles (UAVs) in the exploration of unknown regions. One of the original aspects of our work is that the actors represent simulated or actual UAVs exploring the environment in parallel instead of traditional computer threads. Also, we propose addition of Long Short Term Memory (LSTM) neural network layers to the actor and critic architectures to handle imperfect communication and partial observability scenarios. The proposed approach has been evaluated in a grid world and has been compared against other competing algorithms such as Multi-Agent Q-Learning, Multi-Agent Deep Q-Learning to show its advantages. More generally, our approach could be extended to image-based/continuous action space environments as well.