Multi-access Edge Computing (MEC) extends cloud computing to the network edge, supporting resource-intensive mobile applications. Service migration ensures seamless continuity and high-quality service (QoS) when users move between MEC servers. In the Internet of Vehicles (IoV), the high mobility of vehicles causes network instability, complicating the collection of system information. In addition, vehicles impose strict latency and green energy requirements, and the search for the Pareto front among conflicting objectives increases the complexity of migration. Existing service migration methods rely on centralized decision making using complete system information, which is not suitable for the user-centric IoV environment. Current multi-objective reinforcement learning approaches lack sufficient exploration randomness, leading to suboptimal performance. We propose the Adversarial Variational State Inference with Maximum Entropy Multi-Objective Policy Optimization (AVSIMEMPO) algorithm to address the partially observable and multi-objective optimization problem, optimizing migration node selection. The service migration problem is modeled as a partially observable Markov decision process (POMDP). To solve this, we design an encoding network, AVSI, integrating Long ShortTerm Memory (LSTM), Variational Autoencoders (VAE), and adversarial learning to extract hidden state. We also introduce the Maximum Entropy Multi-Objective Policy Optimization (MEMPO) algorithm, which enhances exploration randomness through maximum entropy and dynamic weight design. Extensive experiments based on real mobility trajectories show that our method outperforms baseline algorithms and achieves nearoptimal results in various MEC scenarios.

Existing joint user association and trajectory control (JUATC) methods provide remarkably high data rates for mobile users (MUs) in unmanned aerial vehicle (UAV)-assisted multi-access edge computing systems. Nevertheless, current methods give more attention to the uplink and downlink of MU, which must be associated with the same UAV or base station (BS), ignoring the network’s heterogeneity and considerably reducing the MU’s communication efficiency. Furthermore, UAVs typically provide communication services to MUs under partial observation, leading to challenges in achieving optimal service performance due to information loss. Moreover, although existing solutions can readily reach optimal, restriction-fulfilling strategies, they frequently breach restrictions during intermediate iterations. To address these issues, we present a fully decentralized JUATC algorithm based on the Communication and Proportional-Integral-Derivative (PID) Lagrangian-based Multi-Agent Actor-Critic (CPID-MAAC). First, to improve communication efficiency, we consider that each MU can be associated with a different UAV or BS in the uplink and downlink. Second, we establish a messaging mechanism between UAVs based on autoencoding UAV’s observations to handle the information loss. Finally, to alleviate constraint-violating behavior, we incorporate the PID Lagrangian algorithm. The experiments show that CPID-MAAC improves data rate by 7.88%~16.03% and drastically reduces the number of constraint violations during UAV agent training.

Existing joint trajectory control and task offloading (JTCTO) algorithms offer ultra-low latency services for smart devices (SDs) in unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC). However, these JTCTO algorithms typically require large training datasets to learn the optimal policies, leading to low learning efficiency. Additionally, most existing JTCTO algorithms are difficult to scale to environments with more than a few UAVs, as their complexity increases exponentially with the number of UAVs. In this paper, we propose a decentralized JTCTO algorithm based on the Policy Transfer and Mean Field-based Multi-Agent Actor-Critic (PTMF-MAAC). First, a novel policy transfer algorithm is proposed to determine which UAV's JTCTO strategy is helpful for each UAV and when to terminate the strategy to accelerate the learning efficiency of the UAV. Second, we propose a partially observable mean field algorithm that significantly reduces the model space by replacing the influence of all other UAVs on a particular UAV with an average value, thereby adapting to large-scale UAV scenarios. Experiments have shown that compared to the baseline, PTMF-MAAC reduces the system cost by 18.44%$\sim$28.57% and improves the model learning efficiency and adaptability to partially observable large-scale UAV-assisted MEC.

In mobile crowd sensing systems, existing flight control methods enable unmanned aerial vehicles (UAVs) to provide high-quality data collection services for various applications. However, due to limited communication range, UAVs typically collect data under partial observability, hindering optimal performance without global environmental information. Additionally, many methods fail to enforce critical safety constraints. This paper proposes a communication-assisted safe multi-agent actor-critic-based UAV flight control method (CSMAAC). First, we propose an independent prediction communication partner model to address the partial observability problem. Based on the UAV's local observation, causal inference is used to obtain prior communication information between UAVs through a feed-forward neural network to help UAVs determine potential communication partners. Second, we utilize a critic-network to predict and quantify inter-UAV influence and determine the necessity of communication. By exchanging necessary information inter-UAV, UAVs can perceive global information, thereby solving the UAV's partial observability problem and reducing communication overhead. Moreover, we propose a similarity enhancement mechanism to improve the learning efficiency of the model by enhancing the connection between UAV observations and the policies of other UAVs. Finally, we introduce a safety layer to Actor-Network to ensure safe UAV flight. The simulation results show that the proposed method outperforms the baselines.

Existing joint trajectory planning and task offloading (JTPTO) methods provide ultra-low latency services for mobile devices (MDs) in unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC). However, UAVs typically provide services to MDs under partial observation, leading to challenges in achieving optimal service performance due to information loss. Moreover, the JTPTO problem typically involves multi-objective optimization, which is challenging because the objectives may conflict with each other. In this paper, we present a decentralized JTPTO method based on Multi-Objective and Independently Predicted Communication Multi-Agent Actor-Critic (MOIPCMAAC). First, an IPC network is designed to facilitate UAV agents in learning a prior for communication between UAVs. UAV agents learn this prior through causal reasoning, which represents the mapping of UAV’s observation to the level of confidence in choosing communication partners. The effect of one UAV on another UAV is predicted through the critic-network in multi-agent reinforcement learning (MARL) and measured to indicate the necessity of UAV-UAV communication. Further, we regularize JTPTO policies to more effectively utilize exchanged messages. Second, a generalized variant of the Bellman optimality operator with multiple objectives is applied to address the JTPTO problem. We use it to learn a single parameterized expression that encompasses all the best JTPTO policies across the space of preferences. Experiments show that compared to existing solutions, MOIPC-MAAC reduces system costs by 14.23%~19.56% and the communication cost to approximately 11.23%. Moreover, compared to training from scratch, MOIPC-MAAC accelerates the adaptation of new JTPTO tasks with unknown preferences by 13.12%.

In multiaccess edge computing (MEC), when many mobile devices (MDs) offload their tasks to an edge server (ES), its resources might become constrained. These tasks may take a long time to complete or even be thrown away. Since the unknown information of both the ESs and other MDs, it is difficult for each MD to determine its offloading policy independently. Furthermore, most offloading methods have poor generalization to new environment since they focus on model architecture with a fixed quantity of MDs and ESs, preventing trained models from transferring to other environments. In this article, we provide a full decentralized offloading scheme based on the curriculum attention-weighted graph recurrent network-based multiagent actor–critic (CAGR-MAAC). First, we build MEC as a shared MD agents-ESs graph and an AGR-based message network is designed to enable each MD aggregate the information of ESs and other MDs and solve the partial observability of MD agents for MEC system. Second, a learnable differentiable encoder network is introduced to construct MD agent’s local information encoding. Subsequently, the MD agent converts overall the information regarding the MEC system into a fixed-size embedding via an AGR Network to handle different quantity of MDs and ESs. Finally, we introduce curriculum learning to address the huge complexity of the MEC system and the training difficulties induced by the large amounts of MDs and ESs. Experiments demonstrate that compared with existing algorithms, CAGR-MAAC boosts task completion rates and decreases system costs by 13.01%–15.03% and 16.45%–18.56%, and can quickly adapt to the new environment.

In multi-access edge computing systems, existing task offloading methods have provided ultra-short latency services for heterogeneous tasks on mobile devices (MDs). Nevertheless, the complexity of MEC systems grows exponentially with the number of MDs or edge servers (ES), so learning a good offloading policy is a huge challenge when the number of MDs or ESs is large. Moreover, MDs are often unable to find optimal ESs for offloading since the restricted ESs infrastructures and the spatiotemporally imbalanced task offloading requirements. To solve these problems, we propose a Curriculum Spatio-Temporal Multi-Agent Actor-Critic (CSTMAAC)-based task offloading method. Each ES is regarded as an agent and the problem is formulated as a multi-objective optimization task. To adapt to the large-scale MEC systems, we first introduce an evolutionary curriculum learning by gradually raising the number of trained ES agents in a phased way. Second, to facilitate the coordination of the offloading policies among geographically distributed ESs, we design an attention-based centralized critic-network. Besides, a delayed access mechanism is introduced that uses information about future task processing competition to capture the impact of potential future task processing contention and help ES agents obtain a better offloading strategy. Finally, critic-network is expanded to multi-critics and a dynamic weight mechanism is designed to adaptively optimize multi-objectives and obtain a good balance for multiple objectives. Real-world datasets used in experiments demonstrate that CSTMAAC raises task completion rates and total utility by 13.01% 15.21% and 16.89% 18.32% compared with the existing algorithms.

We investigate the joint trajectory control and task offloading (JTCTO) problem in multiunmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC). However, existing JTCTO solutions primarily focus on fixed UAV-enabled MEC scenario variations and necessitate extensive interaction to adapt to new scenarios. Moreover, we consider minimizing task latency and UAV’s energy consumption, and maximizing the quantity of tasks collected by the UAV as optimization goals. However, this optimization problem is characterized by multiple conflicting goals that should be adjusted according to their relative significance. In this article, we present a multiobjective actor-variations critic-based JTCTO solution (MO-AVC). First, a group of reinforcement learning strategies is utilized to collect experience on training scenarios, which are employed to learn embeddings of both strategies and scenarios. Further, these two embeddings are used as inputs to train the actor-variations critic (AVC), which explicitly estimates the total return in a space of JTCTO strategies and UAV-enabled MEC scenarios. When adapting to a new scenario, just a few steps of scenario interaction are enough to predict the scenario embedding, thus selecting strategies by maximizing the trained AVC. Second, we propose an actor-conditioned critic framework where the outputs are conditioned on the varying significance of goals, and present a weight dynamic memory-based experience replay to address the intrinsic instability of the dynamic weight context. Finally, simulation results show that MO-AVC can quickly adapt to new scenarios. Moreover, MO-AVC reduces the latency by 7.56%–10.57%, the energy consumption by 11.11%–17.27%, and increases the tasks number by 10.33%–15.54% compared to existing solutions.

Existing service migration approaches provide minimal service delay for mobile vehicles (MVs) in vehicular edge computing (VEC) systems. Nonetheless, existing approaches focus more on formulating migration strategies rely on global information of the system, which may incur high signaling overhead and poor scalability. Furthermore, existing approaches are difficult to reuse previous migration strategies and necessitate significant interaction to adapt to new VEC scenarios. In this paper, we present a decentralized service migration approache base on Variational Recurrent and Critic-Coached Strategy reuse Actor-Critic (VRCCS-AC). First, a variational recurrent model (VRM) is introduced to efficiently obtain information from MV's local state through modeling VEC scenarios. An actor-critic enhances migration strategies by accessing both VEC scenario and VRM. Second, CCS leverages the critic-network to assess and select source service migration strategy. In every state, CCS selects the source strategy that exhibits the most significant one-step enhancement compared to the current target strategy, and develops a coached strategy. Then, the target strategy is regularized to imitate the coached strategy to facilitate effective strategy search and efficient strategy transfer. Experiments on the real-world datasets demonstrate that compared to the baselines, VRCCS-AC reduces latency by 10.11%$\sim$18.57% and can quickly transfer to new VEC scenarios.

In vehicular fog computing, task offloading enables mobile vehicles (MVs) to offer ultralow latency services for computation-intensive tasks. Nevertheless, the edge server (ES) may have a high load when a large number of MVs offload their tasks to it, causing many tasks either experience long processing times or being dropped, particularly for latency-sensitive tasks. Moreover, most existing methods are largely limited to training a model from scratch for new environments. This is because they focus more on model structures with fixed input and output sizes, impeding the transfer of trained models across different environments. To solve these problems, we propose a decentralized task offloading method based on transformer and policy decoupling-based multiagent actor–critic (TPDMAAC). We first introduce a transformer-based long sequence forecasting network (TLSFN) for predicting the current and future queuing delay of ESs to solve uncertain load. Second, we redesign the actor-network using transformer-based temporal feature extraction network (TTFEN) and policy decoupling network (PDN). TTFEN can adapt to various input sizes through a transformer that accepts different tokens we build from the raw input. PDN provides a mapping between the transformer-based embedding features and offloading policies utilizing self-attention mechanism to address various output dimensions. Finally, the experiments on two real-world data sets show that TPDMAAC can quickly adapt to a new environment. And compared to existing algorithms, TPDMAAC reduces the system cost by 11.01%–12.03% as well as improves task completion rates by 10.45%–13.56%.

Recently, existing computation offloading methods have provided extremely low service latency for mobile users (MUs) in multi-access edge computing (MEC). However, this remains a challenge in large-scale mixed cooperative-competitive MUs heterogeneous MEC environments. Moreover, existing methods focus more on all offloaded tasks handled by static resource allocation MEC servers (ESs) within a time interval, ignoring on-demand requirements of heterogeneous tasks, resulting in many tasks being dropped or wasting resources, especially for latency-sensitive tasks. To address these issues, we present a decentralized computation offloading solution based on the Attention-weighted Recurrent Multi-Agent Actor-Critic (ARMAAC). First, we design a recurrent actor-critic framework to assist MU agents in remembering historical resource allocation information of ESs to better understand the future state of ESs, especially in dynamic resource allocation. Second, an attention mechanism is introduced to compress the joint observation space dimension of all MUs agent to adapt to large-scale MUs. Finally, the actor-critic framework with double centralized critics and Dueling network is redesigned considering the instability and convergence difficulties caused by the sensitive relationship between the actor and critic networks. The experiments show that ARMAAC improves task completion rates and reduces average system cost by 11.01%$\sim$14.03% and 10.45%$\sim$15.56% compared with baselines.

This paper proposes a method for predicting the performance interference of applications in the virtualized environment. In this method, we firstly analyze the relationship between the performance interference degree and the system-level workloads, and based on this we propose a linear regression algorithm to model relationship between the performance interference degree and the system-level workloads by using the historical data about performance interference degree as the training data set. For the applications without historical data about performance interference degree, we develop a method for predicting the performance interference by clustering the available models of performance interference and matchmaking between the workload pattern of the application and the workload patterns of the available models to generate the performance interference model for the application whose performance interference will to be predicted. By use of the available model, the performance interference of the application can be predicted without historical data about the performance interference among the applications co-located on the same physical host. The experiments show the effectiveness of the proposed measurement and prediction methods of the performance interference among the virtual machines.

This paper presents a composite service execution engine, which can support human tasks and improve their execution by an approach of human task scheduling. In the approach, the performance evaluation model is proposed, which can reflect the performance of the human service resources objectively and comprehensively. Based on this model, the initial scheduling as well as re-scheduling methods for solving the problem is proposed to find the human service resource with better performance.

Nowadays, Web services are usually aggregated into a composite one to satisfy customer’s more and more complex requirements. Generally, there may be several different candidate services to carry out one task in a composite service, so a choice needs to be made to help users select the most suitable one. Based on the quality of services, this paper generates a Weighted Multistage Graph for composite service, and transforms the problem of service selection into the one of how to get a longest path. Considering the problem of Interface Matching, this paper presents a 3-layer Web service organization model (WS3LM), which can help get an executable composite service. This paper describes and compares two types of selection approaches: one type of local optimal selection and the other type of global optimal selection using Exhaustive Search Algorithm, Dynamic Programming.

In order to realize a high-quality and good-performance service composition, based on current approach, we propose a new QoS-driven dynamic selection of composite Web services, which takes account of both the QoS properties and interface parameters matching degree. When doing the selection, we aware that the task is more or less a multistage decision-making process. Motivated by neural networks' high parallel performance and genetic algorithm's powerful computation ability, a genetic algorithm optimized neural network algorithm is proposed in this paper for such task. In order to make this algorithm more adaptable for multistage decision-making problem, we propose a new structured neural network to express the composed service instead of using the traditional neural networks, which minimizes the neurons involved and shows high performance than the earlier ones. Finally, through experimentation one can find that method proposed in this paper is more practical and effective than others

This paper first presents a 3-layer organization model and an evaluation model for services composition. Then it presents an approach for selecting global optimal execution plan of Web services composition, which is based on weighted multistage graph and fully considering interface-matching between Web services. Based on this approach, we can select the optimal execution plan dynamically by Dynamic Programming, Integer Programming, Genetic Algorithm or Immune Algorithm, which can solve the problem efficiently and make the selection more correct.

For the widely use of multistage decision-making problem in our normal life such as in the new research area of dynamic selection of composite web services, this paper exerts all its effort on proposing a new approach to solve such problem. Motivated by neural networks’ high parallel performance and Genetic Algorithm’s powerful computation, a novel Genetic Algorithm optimized neural network is proposed in this paper for this task. In order to make this algorithm more adaptable for multistage decision-making problem, a new neural network structure for implementing the algorithm is proposed which is a modification to the one used by Thomopoulos or Rauch and Winarske.