Advanced wireless networks must be highly dynamic and capable of managing heterogeneous service demands. A key feature of these networks is network slicing, which is supported in the next-generation radio access network (RAN) architecture, such as open RAN (O-RAN), by leveraging artificial intelligence (AI) and machine learning (ML) approaches within the ran intelligent controller (RIC) modules. Deep reinforcement learning (DRL) has much potential for managing dynamic networks but often needs to improve when dealing with unstructured, multi-modal data like RF signals and QoS metrics. This kind of data makes it challenging for DRL to grasp the high-level context fully. To tackle this, we're using large language models (LLMs) to give DRL a richer, more meaningful state representation. LLMs add semantic layers to raw data, helping the agent grasp deeper context and think more strategically over the long term. This way, the DRL agent can make smarter decisions even as the environment becomes more complex and changes over time. This paper proposes a task-related state representation that employs LLM to augment multi-agent DRL (MARL) approaches. Simulation results demonstrate that the proposed approach significantly outperforms related baselines.

In multimodal learning, dominant modalities often overshadow others, limiting generalization. We propose Modality-Aware Sharpness-Aware Minimization (M-SAM), a model-agnostic framework that applies to many modalities and supports early and late fusion scenarios. In every iteration, M-SAM in three steps optimizes learning. \textbf{First, it identifies the dominant modality} based on modalities' contribution in the accuracy using Shapley. \textbf{Second, it decomposes the loss landscape}, or in another language, it modulates the loss to prioritize the robustness of the model in favor of the dominant modality, and \textbf{third, M-SAM updates the weights} by backpropagation of modulated gradients. This ensures robust learning for the dominant modality while enhancing contributions from others, allowing the model to explore and exploit complementary features that strengthen overall performance. Extensive experiments on four diverse datasets show that M-SAM outperforms the latest state-of-the-art optimization and gradient manipulation methods and significantly balances and improves multimodal learning. The code will be released.

FlameFinder, a novel deep metric learning (DML) framework, accurately detects RGB-obscured flames using thermal images from firefighter drones during wildfire monitoring. In contrast to RGB, thermal cameras can capture smoke-obscured flame features but they lack absolute thermal reference points, detecting many nonflame hot spots as false positives. This issue suggests that extracting features from both modalities in unobscured cases can reduce the model’s bias to relative thermal gradients. Following this idea, our proposed model utilizes paired thermal-RGB images captured onboard drones for training, learning latent flame features from smoke-free samples. In testing, it identifies flames in smoky patches based on their equivalent thermal-domain distribution, improving performance with supervised and distance-based clustering metrics. The approach includes a flame segmentation method and a DML-aided detection framework with center loss (CL), triplet CL (TCL), and triplet cosine CL (TCCL), to find the optimal cluster representatives for classification. Evaluation of FLAME2 and FLAME3 datasets shows the method’s effectiveness in diverse fire and no-fire scenarios. However, the CL dominates the two other losses, resulting in the model missing features that are sensitive to them. To overcome this issue, an attention mechanism is proposed making nonuniform feature contribution possible and amplifying the critical role of cosine and triplet loss in the DML framework. Plus, the attentive DML shows improved interpretability, class discrimination, and decreased intraclass variance exploiting several other flame-related features. The proposed model surpasses the baseline with a binary classifier by 4.4% in FLAME2 and 7% in FLAME3 datasets for unobscured flame detection accuracy. It also demonstrates enhanced class separation in obscured scenarios compared to fine-tuned VGG19, ResNet18, and three other backbone models tailored for flame detection.