The rapid proliferation of end devices and their hosted applications has significantly increased the demand for data processing. However, lightweight edge endpoints often suffer from limited computational capabilities due to power and battery constraints. Consequently, computation-intensive tasks are typically offloaded to cloud servers located in data centers for processing. While existing studies often abstract the edge–cloud connection as a direct link, the actual backbone network features complex topologies and constrained bandwidth, which intensify as workload scales up. To address this challenge, this paper investigates routing optimization within the backbone network under task offloading scenarios, and proposes a workload-aware deep reinforcement learning (DRL)-based routing algorithm for joint minimization of transmission and processing delays. Specifically, we first develop a detailed system model using discrete event simulation. Then, we design an integrated DRL-based decision-making scheme that simultaneously determines optimal routing paths and target data centers. Extensive experiments on diverse real-world network topologies and heterogeneous task workloads demonstrate that the proposed algorithm significantly outperforms conventional baselines in reducing both transmission and processing delays.

The Internet today hosts a multitude of communication sessions from diverse vertical industries, each with distinct and increasingly stringent quality-of-service (QoS) requirements across multiple performance metrics. However, QoS-aware routing remains a significant challenge in traffic engineering, as existing solutions struggle to adapt to dynamic network conditions and meet these rigorous QoS demands. To address this issue, this paper proposes a multi-agent graph reinforcement learning-based routing algorithm that provides differentiated treatment for multiple services. First, we explore both nodebased and link-based graph reinforcement learning paradigms for performance comparison. Second, two key mechanisms, i.e., a packet sacrifice mechanism and a Tchebycheff-based reward function, are designed to realize adaptive sacrifice behavior patterns, aiming to optimize the lower bound of service satisfaction rates and enhance fairness across services. Furthermore, to ensure practical applicability, we devise a distributed computing architecture featuring neighborhood-restricted data acquisition and asynchronous historical information retrieval. Extensive simulation results demonstrate that our proposed algorithms significantly outperform benchmark methods regarding the minimum service satisfaction rate, even under unseen networks. Besides, the distributed computing architecture is proven to incur no performance penalty, which can be generalized to other resource-constrained applications.

Databases often store sensitive organizational data but may be exposed to the Internet through misconfiguration or vulnerabilities. However, such databases may be unintentionally exposed to the Internet, e.g., due to misconfiguration or be vulnerable. To study real-world attacks on public-facing database management systems (DBMS), we deployed 278 honeypots over 20 days in March–April 2024. Our 220 low-interaction honeypots emulate MySQL, MSSQL, PostgreSQL, and Redis, revealing that scanning activity is relatively low (?3,000 IPs), but brute-force attempts are persistent. We also deploy 58 medium/high-interaction honeypots, which reveal three distinct types of exploitation: (i) direct attacks on the database management system to manipulate the database, (ii) ransom-driven attacks that copy and delete the targeted data, and (iii) use the database as an attack vector to take over the underlying system. Our findings highlight that DBMS-targeted attacks are distinct from those on other Internet-facing systems and deserve focused attention.

The Third Generation Partnership Project has standardized new-radio vehicle-to-everything (NR-V2X) to facilitate advanced use cases for safety-critical message conveyance. However, there is a paucity of resource allocation research for high-performance NR-V2X communications. In this article, we investigate an adaptive optimization issue of energy efficiency (EE) and Quality of Service (QoS) in NR-V2X networks inspired by the Tchebycheff function. To address this issue, we first formulate the resource allocation task as a time-variant mixed-integer nonlinear programming (MINLP) problem. Then, we propose a fully decentralized multiagent deep reinforcement learning (MADRL)-based algorithm characterized by a multitask-actor shared-critic (MTA-SC) architecture and a localized training and distributed execution (LTDE) framework to promote efficient learning and minimize information exchange. Finally, we implement the proposed algorithm in a three-lane highway NR-V2X scenario. Numerical results demonstrate that the proposed algorithm comprehensively outperforms benchmarks in terms of convergence performance, scalability, and robustness.

As network services become more complex, efficient routing has become crucial for ensuring end-user satisfaction. To address this challenge, researchers are increasingly turning to routing algorithms that integrate Graph Neural Networks (GNNs) with Deep Reinforcement Learning (DRL), leveraging the natural graph structure of network topologies. However, a significant challenge with existing algorithms is their inability to generalize across different topologies without requiring retraining, a constraint that is impractical in real-world applications. To overcome this limitation, we propose a novel GNN-DRL-based routing algorithm, Link2Link, designed to decouple DRL-learned knowledge from specific network topologies by focusing on link-level features. Extensive experiments demonstrate that Link2Link achieves robust performance across diverse topologies, consistently outperforming OSPF without requiring retraining, making it a scalable and adaptable solution for modern network routing challenges.

Aspect-based sentiment analysis (ABSA) aims to conduct fine-grained sentiment analysis, necessitating the extraction of three key components: target entity, aspect category and sentiment polarity. These three components collectively form an integrated ABSA task known as TASD (Target-Aspect-Sentiment jointly Detection). Most of existing approaches on ABSA usually employ Recurrent neural networks(RNNs), Convolutional neural networks(CNNs) or pre-training models such as Bidirectional Encoder Representations from Transformers(BERT). However, they have some common weaknesses. First, most of the existing methods focus on one or two sub-tasks instead of triplet detection, thus they don’t establish an end-to-end (training a complex learning system represented by a single model) ABSA model and can not utilize the relevance of multiple ABSA sub-tasks during training. Second, they can not achieve accuracy and efficiency simultaneously due to the coupling of context and given aspects. Third, they are poor in recognizing implicit targets. To tackle these limitations, this paper proposes a novel method named the Twin Towers End to End model (TTEE) to solve TASD task. It transforms complex TASD task into a simple end-to-end multi-task framework, simultaneously conducting target and aspect-sentiment detection. It builds twin towers system based on BERT or its updated versions to decouple context and given aspects, which can reduce redundant calculation to improve computational efficiency significantly. It offers great advantage to identify implicit target entity and its associated aspect-sentiment in the context without introducing extra model architecture. Experiments on three real datasets on different domains demonstrate that our approach not only achieves better performance on various evaluation metrics, but also has high efficiency in training and inference phases, over a wide range of sample size and number of aspect categories.

Mobile edge computing (MEC) is an enabling technology for next-generation network architectures to deliver more diversified communication services and meet more demanding quality-of-service (QoS) requirements. However, owing to the growing scale and heterogeneity of the networks, energy-efficient resource allocation in heterogeneous MEC (Het-MEC) networks faces great challenges. As an emerging research area, multi-agent deep reinforcement learning (MADRL) is expected to realize autonomous resource allocation in Het-MEC networks by learning from trial and error. Nevertheless, existing MADRL-based solutions are usually not applicable to practical scenarios by the limitations of centralized control schemes or massive signaling overhead. To address the issue, we first formulate the energy-efficient resource allocation problem in Het-MEC networks as a time-variant mixed-integer nonlinear programming (MINLP) problem. Then, we propose a fully decentralized collaborative MADRL-based algorithm featuring the multi-actor shared-critic (MASC) architecture and the regional training distributed execution (RTDE) framework, which effectively stabilizes model training and reduces information exchange, respectively. Finally, we conduct extensive simulations to evaluate the performance of the proposed algorithm. Numerical results demonstrate that the proposed algorithm comprehensively outperforms several mainstream baseline methods in terms of convergence performance, scalability, and robustness.

This paper presents an empirical study on deep reinforcement learning (DRL) based probabilistic cognitive routing using the OMNeT++ framework and programming protocol-independent packet processors (P4). The proposed algorithm combines the power of DRL and cognitive routing to achieve efficient and adaptive probabilistic routing in software-defined networking (SDN) environments. To facilitate the research, we develop a dedicated network simulation environment using the OMNeT++ framework and a self-developed SDN platform based on P4. The empirical study highlights the importance of a comprehensive training and validation process in both simulation and real-world SDN environments. Through closed-loop training, the cognitive routing framework provides real-time feedback from the actual network environment to the simulation environment, allowing the agent to excel in real-world network environments. Meanwhile, the results demonstrate that solely testing the algorithm in either environment is inadequate for evaluating its performance accurately.

As a promising technology in the 5G era, the artificial intelligence (AI) enabled Internet of controllable things (IoCT) is expected to be an integral part of heterogeneous networks (HetNets) in the future. However, the realization of ultra-reliable low-latency communications (URLLC) in IoCT communications underlaid HetNet has stringent quality of service (QoS) requirements, resulting in unprecedented challenges for existing wireless resource allocation methods. In this paper, we first describe a cellular HetNet model with uplink IoCT communications, then formulate a dynamic mixed-integer nonlinear programming (MINLP) resource allocation problem for maximizing the long-term average energy efficiency under URLLC requirements including reliability, latency, and transmission rate. To solve the problem, we propose a decentralized MADRL-based resource allocation algorithm with a decentralized partially observable Markov decision process (dec-POMDP) and a mixed-centralized-decentralized (MCD) framework to address the partial observability and the scalability issues, respectively. In addition, we design a reward function featuring the objective decomposition, baseline-guided scaling, and QoS violation penalty so that the agents are coordinated. Extensive experiments demonstrate the convergence, scalability, and robustness of the proposed algorithm. Besides, the proposed algorithm substantially outperforms conventional resource allocation methods and different agent communication mechanisms in terms of maximizing energy efficiency.

Mobile edge computing (MEC) extends the computing power to the edge of communication networks, which has been considered as a promising technology to further improve the quality of communication services in the near future. Nevertheless, the issue of MEC-empowered energy efficient resource allocation has not been well studied. To maximize the longterm energy efficiency for green MEC-enabled heterogeneous networks (HetNets), we proposed a decentralized multi-agent deep reinforcement learning (MADRL) resource allocation algorithm. Based on the proximal policy optimization (PPO) framework, our proposed algorithm enables observation exchange to coordinate the policies of multiple agents. Simulation results show that our proposed algorithm significantly outperforms three baseline methods in terms of effectiveness, robustness, and scalability.

With the commercialization of the 5th generation mobile networks, vehicle-to-everything (V2X) communication has gained tremendous attention over the last decade. However, prevailing research has not sufficiently deliberated on the energy efficiency (EE) optimization issue. This paper proposes a decentralized multi-agent deep reinforcement learning (DRL) based resource allocation algorithm. Moreover, we leverage energy harvesting (EH) to achieve long-term EE maximization. Based on the proximal policy optimization (PPO) framework, we invoke power splitting (PS) to divide the harvested energy delicately. Numerical results demonstrate that our proposed algorithm outperforms traditional and straightforward DRL-based resource allocation approaches in effectiveness and robustness.

Advances in modern technologies have afforded end-users increased convenience in performing everyday activities. However, even seemingly trivial issues can cause great annoyance for the ordinary user who lacks domain expertise of the often complex systems that underpin these advances. A key challenge lies in assisting non-expert users to express their requirements of an obscure and complex system. This research proposes a semantic approach by using domain expert knowledge to enable real time semantic up-lift in supporting novice end-users to understand and control the complex dynamic systems they must manage. This presents a significant opportunity to increase user satisfaction and reduce associated support costs. This semantic approach has been designed and implemented in an early prototype of our Home Area Network Monitoring System (HANMS). This paper presents a detailed description of the current state of the research, an initial evaluation, and future work.