Multi-agent reinforcement learning provides promising prospect for task scheduling in cloud-edge computing environment in recent years. However, there remains a formidable challenge due to partial observation and the rigid coupling between action spaces and schedulable devices. These limit the ability of agent to perceive global communication patterns and adapt to dynamic environments, resulting in unsatisfactory scheduling decisions. To address these issues, this work proposes SPAD, a novel spatial perception and action decoupling empowered distributed multi-agent AI task scheduling framework. By constructing a global spatial feature distillation mechanism, SPAD can approximate the implicit heterogeneous connection patterns and communication dynamics between devices and tasks under constrained observability, enhancing its ability to make robust decisions in dynamic environments with limited observations. Additionally, SPAD employs a Lyapunov-based action decoupling module to alleviate scalability challenges from rigid action-device coupling, while a novel intrinsic penalty mechanism augments the agent’s advantage function with the instantaneous Lyapunov cost, thereby aligning the policy optimization process with the decoupling module’s underlying stability constraints. Through a comprehensive empirical evaluation spanning synthetic, bursty, and real-world trace-driven workloads, we show that SPAD consistently outperforms state-of-the-art benchmarks in reducing task completion latency and improving resource utilization, while maintaining remarkable resilience and scalability across diverse network topologies and under non-stationary load conditions.

Studies of resource provision in cloud computing have drawn extensive attention, since effective task scheduling solutions promise an energy-efficient way of utilizing resources while meeting diverse requirements of users. Deep reinforcement learning (DRL) has demonstrated its outstanding capability in tackling this issue with the ability of online self-learning, however, it is still prevented by the low sampling efficiency, poor sample validity, and slow convergence speed especially for deadline constrained applications. To address these challenges, an Imitation Learning Enabled Fast and Adaptive Task Scheduling (ILETS) framework based on DRL is proposed in this paper. First, we introduce behavior cloning to provide a well-behaved and robust model through Offline Initial Network Parameters Training (OINPT) so as to guarantee the initial decision-making quality of DRL. Next, we design a novel Online Asynchronous Imitation Learning (OAIL)-based method to assist the DRL agent to re-optimize its policy and to against the oscillations caused by the high dynamic of the cloud, which promises DRL agent moving towards the optimal policy with a fast and stable process. Extensive experiments on the real-world dataset have demonstrated that the proposed ILETS can consistently produce shorter response time , lower energy consumption and higher success rate than the baselines and other state-of-the-art methods at the accelerated convergence speed.

Managing scientific applications in the Cloud poses many challenges in terms of workflow scheduling, especially in handling multi-objective workflow scheduling under quality of service (QoS) constraints. However, most studies address the workflow scheduling problem on the premise of the unchanged environment, without considering the high dynamics of the Cloud. In this paper, we model the constrained workflow scheduling in a dynamic Cloud environment as a dynamic multi-objective optimization problem with preferences, and propose a transfer learning based multi-objective evolutionary algorithm (TL-MOEA) to tackle the workflow scheduling problem of dynamic nature. Specifically, an elite-led transfer learning strategy is proposed to explore effective parameter adaptation for the MOEA by transferring helpful knowledge from elite solutions in the past environment to accelerate the optimization process. In addition, a multi-space diversity learning strategy is developed to maintain the diversity of the population. To satisfy various QoS constraints of workflow scheduling, a preference-based selection strategy is further designed to enable promising solutions for each iteration. Extensive experiments on five well-known scientific workflows demonstrate that TL-MOEA can achieve highly competitive performance compared to several state-of-art algorithms, and can obtain triple win solutions with optimization objectives of minimizing makespan, cost and energy consumption for dynamic workflow scheduling with user-defined constraints.

With the proliferation of Web services, it is very difficult for inexperienced users to select the most appropriate service among numerous functionally identical or similar candidates, thus prediction of Quality of Service (QoS) becomes a growing concern in service discovery, selection and recommendation. However, a huge challenge is that in the reality of the existence of untrustworthy users, how the Web service recommendation system keeps the robustness of the QoS prediction while maintaining high accuracy. To address this problem, a Reputation Integrated Graph Convolution Network (RIGCN) is developed in this paper to realize robust and accurate QoS prediction. RIGCN has three main parts: Reputation Extraction, Multi-source Feature Extraction and GCN-based QoS Prediction. First, an Outlier and Pattern Measure (OPM) method is proposed to extract the real reputation of users based on both outliers and the distribution patterns of the historical QoS interaction records. Second, deep features of users and services are captured by multi-source feature extraction with an attention mechanism to make full use of the contextual information in service invocation. On this basis, a graph convolution network is specially designed to integrate multi-source features and user reputation in the message propagation process to complete final QoS prediction. Experimental results demonstrate that our RIGCN approach can not only extract and utilize implicit and explicit features of various multi-source data, but also can reduce the negative influence of untrustworthy users. Therefore, it is very robust and effective in improving the accuracy of QoS prediction with sparse and noisy data.

The existing works on human-object interaction (HOI) detection usually rely on expensive large-scale labeled image datasets. However, in real scenes, labeled data may be insufficient, and some rare HOI categories have few samples. This poses great challenges for deep-learning-based HOI detection models. Existing works tackle it by introducing compositional learning or word embedding but still need large-scale labeled data or extremely rely on the well-learned knowledge. In contrast, the freely available unlabeled videos contain rich motion-relevant information that can help infer rare HOIs. In this article, we creatively propose a multitask learning (MTL) perspective to assist in HOI detection with the aid of motion-relevant knowledge learning on unlabeled videos. Specifically, we design the appearance reconstruction loss (ARL) and sequential motion mining module in a self-supervised manner to learn more generalizable motion representations for promoting the detection of rare HOIs. Moreover, to better transfer motion-related knowledge from unlabeled videos to HOI images, a domain discriminator is introduced to decrease the domain gap between two domains. Extensive experiments on the HICO-DET dataset with rare categories and the V-COCO dataset with minimum supervision demonstrate the effectiveness of motion-aware knowledge implied in unlabeled videos for HOI detection.

With the explosive growth of Web services, the increasing sparse and dynamic QoS (quality of service) data pose a great challenge to QoS prediction in service recommendation. How to fully utilize the contextual information of service invocation and their potential relationships becomes the key to improving the accuracy of QoS prediction. In this paper, a spatial context-aware time series forecasting (SCATSF) framework is proposed for QoS prediction by considering both the temporal and the spatial context of users and services. SCATSF has three main parts: time-aware neighbors selection, spatial context-aware interaction learning and time series forecasting for QoS prediction. First, a novel time series similarity (TSS) is proposed to measure the similarity of users or services based on time-varying QoS fluctuation, in order to select appropriate neighbors for the target user or service. Furthermore, the contextual information of neighbors, as well as the contextual information of users and services, are both integrated to enrich the spatial context of service invocation, and a pairwise multi-layer deep network (PMLDN) is developed to consolidate multiple features and to learn feature interactions in pairs. With the help of spatial contextual information and time-aware QoS information, a spatial context-aware GRU model (SCA-GRU) is finally present to complete the time series forecasting for QoS prediction. Experimental results on the prediction of response time in a real-world dataset demonstrate that our SCATSF approach can effectively utilize spatio-temporal contextual information, so that it can achieve higher accuracy than many existing methods.

Intelligent task scheduling solutions are highly demanded in the operation of complex cloud data centers so that resources can be utilized in an energy-efficient way while still ensuring various requirements of users. However, the energy problem of task scheduling in cloud environment becomes more challenging with the ever-increasing number of users as well as the constant and unpredictable change of workloads. In this research, we propose an Adaptive Deep Reinforcement Learning-based (ADRL) task scheduling framework for energy-efficient cloud computing. We first present a Change Detection algorithm to detect whether the workload has changed greatly. On this basis, we built an Automatic Generation network to adjust the discount factor of Deep Reinforcement Learning (DRL) dynamically according to the changing workload, which enables faster and more accurate learning. We finally introduce the adaptive DRL to learn the optimal policy of dispatching arriving user requests with the reward aiming to minimize task response time and maximize resource utilization. Simulated experiments have confirmed that the proposed scheduling scheme performs well on accelerating learning convergence and promoting allocation accuracy, thus it is very effective in reducing the average response time of tasks and increasing the CPU utilization rate of resources, which eventually makes the cloud system more energy efficient.

The dramatic increasing of data and demands for computing capabilities may result in excessive use of resources in cloud data centers, which not only causes the raising of energy consumption, but also leads to the violation of Service Level Agreement (SLA). Dynamic consolidation of virtual machines (VMs) is proven to be an efficient way to tackle this issue. In this paper, we present an Adaptive Deep Reinforcement Learning (DRL)-based Virtual Machine Consolidation (ADVMC) framework for energy-efficient cloud data centers. ADVMC has two phases. In the first phase, Influence Coefficient is introduced to measure the impact of a VM on producing host overload, and a dynamic Influence Coefficient-based VM selection algorithm (ICVMS) is proposed to preferentially choose those VMs with the greatest impact for migration in order to remove the excessive workloads of the overloaded host quickly and accurately. In the second phase, a Prediction Aware DRL-based VM placement method (PADRL) is further proposed to automatically find suitable hosts for VMs to be migrated, in which a state prediction network is designed based on LSTM to provide DRL-based model more reasonable environment states so as to accelerate the convergence of DRL. Simulation experiments on the real-world workload provided by Google Cluster Trace have shown that our ADVMC approach can largely cut down system energy consumption and reduce SLA violation of users as compared to many other VM consolidation policies.

The ever-increasing diversity and dynamic of cloud environment pose many challenges on QoS prediction in service recommendation. One such challenge is how to extract and learn deep features of users/services from multi-source information to improve prediction accuracy. In this article, we propose a novel Joint Deep Networks based Multi-source Feature Learning(JDNMFL) framework for QoS prediction. JDNMFL has two parts: Multi-source Feature Extraction and Feature Interaction Learning. In the first part, a latent factor embedding method is first proposed to capture implicit features from QoS matrix, and then the multi-source features, combined by explicit features from WSDL(Web Services Description Language) document and contextual data as well as implicit features, are extracted based on the combination of matrix factorization and neural networks. In the second part, the CNN(Convolutional Neural Network)-based joint deep networks are built to learn both local and global high-order feature interactions, and to complete the final QoS prediction based on mixed features. Experimental results demonstrate that our JDNMFL approach can not only extract and integrate implicit and explicit features of various multi-source data, but also can learn feature sequence and feature interactions, so that it is very effective in improving the accuracy of QoS prediction with sparse data.

With the advent of the mobile network, the fusion of cloud computing and fog computing is becoming feasible to promise lower latency and short‐fat connection. However, there are a lot of redundant cloud‐aware services with identical functionalities but a different quality of service (QoS) in the fog cloud environment. In fact, since QoS information is stored in distributed fog servers rather than remote cloud, it is hard for individuals to make recommendation and selection with sparse QoS information. Collaborative filtering is an important method for the sparsity problems and has been widely adopted on the prediction of missing QoS values. Focusing on the fact that existing researchers often ignore the QoS fluctuation in a wide range in the fog cloud environment, a novel neighbor‐based QoS prediction method is proposed for service recommendation, in which a concept and calculation method is put forward to describe the stable status of services and users with quantifiable QoS values, and a NearestGraph algorithm is further designed to recognize stable or unstable candidate along with their popularity by a nearest neighbor graph structure which can help to make missing QoS values prediction in a certain order to improve final prediction accuracy. Experimental results confirm that the proposed method is effective in predicting unknown QoS values in terms of service recommendation accuracy and efficiency.