• Propose spatial and temporal encoders for container-level autoscaling decisions • Design a hierarchical action network adaptable to changing container numbers • Achieve higher QoS and cost savings than four state-of-the-art autoscaling methods • Validate effectiveness on real-world user request traces • Advance expert systems for complex and large-scale cloud environments Autoscaling is an important technique for cloud computing that dynamically adjusts resources allocated to cloud applications in response to fluctuating user requests to maintain Quality of Service (QoS) and adhere to a given budget. Recent advancements in Deep Reinforcement Learning (DRL) have shown promise in achieving effective autoscaling approaches. However, prior DRL-based approaches struggle to simultaneously consider the spatial dependencies within an application and the changing historical workload patterns, limiting their ability to make accurate scaling decisions. Moreover, existing approaches lack the fine-grained resource adjustment, leading to suboptimal autoscaling performance. To address these limitations, we propose a new DRL-based autoscaling approach with a novel spatial-temporal autoscaling policy, which jointly captures spatial and temporal features of cloud applications by Graph Neural Networks and Transformers. Meanwhile, this policy enables fine-grained resource adjustment. Extensive experiments on real-world user request traces show that the proposed approach significantly outperforms existing state-of-the-art methods, achieving up to a 78.23% reduction in mean response time without violating the cost budget.

Many IoT systems require deadline-constrained workflow scheduling, where missed deadlines can have serious consequences. Scheduling such IoT workflows in Fog–Cloud environments is challenging due to resource heterogeneity and the variability in workflow patterns and deadlines. Existing approaches, including heuristic and meta-heuristic algorithms, often fail to reliably satisfy deadline constraints while simultaneously minimizing the cost associated with the computational resources used for executing workflows. This paper introduces the Internet of Things Genetic Algorithm with Selective Repair under Combined Criteria (IoTGA-SRC 2 ) to effectively tackle these challenges. IoTGA-SRC 2 introduces a novel selection mechanism that prioritizes solutions based on deadline violations and execution costs. It also features an innovative repair method, which can systematically detect infeasible solutions, perform a root cause analysis to identify the key factors causing deadline violations, and reallocate critical tasks using a multi-criteria method. By properly managing delays caused by execution time, communication time, and waiting time, IoTGA-SRC 2 can consistently satisfy deadline constraints across a wide range of problem configurations. Extensive experiments demonstrate that IoTGA-SRC 2 consistently outperforms multiple state-of-the-art methods in reducing execution costs while adhering to stringent deadline constraints, making it a valuable choice for various real-world applications in heterogeneous IoT–Fog–Cloud computing environments.

The increasing complexity and variety of IoT systems require the integration of multiple services to meet a wide range of user needs. This paper addresses the challenge of multi-objective IoT service composition with replication problem by considering multiple Quality of Service (QoS) metrics such as response time and the number of selected service instances. We propose a new Memetic NSGA-II algorithm with Bottleneck-driven Local Search (MNSGA2-BLS) to effectively solve this difficult problem. By integrating genetic operations, clustering-based refinement, and a bottleneck-driven Estimation of Distribution Algorithm for local search, MNSGA2-BLS identifies and optimizes critical service instances causing QoS bottlenecks. This method leverages Pareto-optimal solutions to guide the local search refinement process, enhancing convergence and solution quality. Experimental results across various benchmark cases demonstrate that MNSGA2-BLS can outperform NSGA-II and several state-of-the-art algorithms, achieving superior results in both the hyper-volume and inverse generational distance metrics. This highlights the potential of MNSGA2-BLS to provide efficient and effective composite IoT services while addressing trade-offs between competing QoS objectives.

The Continuous Dynamic Flexible Job Shop Scheduling Problem (C-DFJSP) represents a critical challenge in smart manufacturing due to its continuous, dynamic nature, requiring real-time decision-making and high adaptability to diverse factory configurations. We provide a solution to the C-DFJSP problem, where the notion of a global optimum is inherently ill-defined. To solve the C-DFJSP, we propose a Multi-Agent Continuous Decision-Making (MACD) system. MACD leverages a novel state and observation design to extract novel factory-invariant features, enabling a learned scheduling policy to generalise across different factories, machine capabilities, and product libraries. MACD is an enhanced multi-agent continuous scheduling system that integrates Graph Neural Networks (GNNs) and Reinforcement Learning (RL) within a multi-agent framework. In addition, MACD employs an innovative negotiation strategy – a hybrid method combining Shortest Processing Time (SPT) and Least Loaded Machine (LLM) heuristics – supported by an empirical study to resolve rare conflicts among machine agents, enhancing adaptability and performance. Extensive experiments on public datasets validate MACD’s superior ability to generalise across diverse factory configurations, allowing a trained scheduling policy to be directly used in different factories with varying machine flexibilities, utilisation levels, and product libraries.

Community Detection (CD) in weighted social networks is a highly active research field, celebrated for its profound practical implications across a multitude of disciplines. Genetic algorithms (GAs) are frequently explored to tackle CD problems, leveraging their capability to navigate the extensive discrete search space effectively. Throughout the evolutionary process, genetic operators such as crossover and mutation assume pivotal roles in effectively exploring the vast solution space. Nonetheless, prevailing GA-based approaches often ignore crucial topology information, particularly information regarding edge weights, resulting in compromised algorithm performance. In light of this, this paper introduces Edge Information-based GA (EIGA) to effectively solve CD problems in weighted networks. This is achieved specifically through the innovative designs of edgeweight-aware crossover and mutation operators. These novel edge-weight-aware operators improve the extraction of meaningful community structures, advancing knowledge discovery from social networks. Empirical findings demonstrate the superior performance of EIGA over numerous state-of-the-art algorithms across various real-world and synthetic benchmark networks.

Containers have emerged as a prevalent mechanism for deploying software applications within cloud data centers, thereby abstracting many operational details from developers and transferring the responsibility of resource management to cloud service providers. These providers are continually motivated to reduce operational costs by optimizing both the placement of containers and the Virtual Machines (VMs) that host them, as efficient placement directly contributes to reduced energy expenditures. Furthermore, given that energy prices vary both temporally and geographically due to fluctuating power production, demand, and the increasing integration of renewable en-ergy sources, explicit consideration of dynamic, location-specific energy pricing is essential for informed resource allocation and container migration decisions. In this work, we propose a novel container-based cloud resource allocation model that integrates variable energy prices across multiple locations. To address this problem, we introduce an innovative Genetic Programming Hyper-Heuristic (GPHH) algorithm that concurrently evolves three heuristics for container allocation, VM placement, and container migration. A key technical novelty of our approach is the incorporation of newly designed terminals within the GPHH framework that are specifically engineered to capture and utilize the complex dynamic power pricing information. Experimental results demonstrate that our GPHH algorithm offers significant improvements over several state-of-the-art methods, thereby enhancing both cost efficiency and energy optimization in cloud environments.

Many machine learning models are susceptible to adversarial attacks, with decision-based black-box attacks representing the most critical threat in real-world applications. These attacks are extremely stealthy, generating adversarial examples using hard labels obtained from the target machine learning model. This is typically realized by optimizing perturbation directions, guided by decision boundaries identified through query-intensive exact search, significantly limiting the attack success rate. This paper introduces a novel approach using the Approximation Decision Boundary (ADB) to efficiently and accurately compare perturbation directions without precisely determining decision boundaries. The effectiveness of our ADB approach (ADBA) hinges on promptly identifying suitable ADB, ensuring reliable differentiation of all perturbation directions. For this purpose, we analyze the probability distribution of decision boundaries, confirming that using the distribution's median value as ADB can effectively distinguish different perturbation directions, giving rise to the development of the ADBA-md algorithm. ADBA-md only requires four queries on average to differentiate any pair of perturbation directions, which is highly query-efficient. Extensive experiments on six well-known image classifiers clearly demonstrate the superiority of ADBA and ADBA-md over multiple state-of-the-art black-box attacks.

Accurate assessment of biochemical compositions in fish products is essential for quality control in the seafood industry and nutritional research. While spectroscopic techniques enable non-destructive analysis, each method has limitations in prediction accuracy and reliability. Multi-modal data fusion offers a promising solution, but developing robust fusion strategies remains challenging due to complex relationships between spectral features and biochemical properties. This paper presents GP-Fusion, a genetic programming-based high-level fusion method that integrates multiple spectroscopic modalities. Unlike conventional approaches, GPFusion evolves interpretable fusion functions to optimize predictions from diverse spectroscopy work-flows. A key innovation is the replicate variance penalty, which enhances prediction consistency across replicate measurements by capturing within-sample variability and mitigating batch effects. Experimental evaluations on three biochemical targets, including Omega-3, Omega-6, and monounsaturated fatty acids, show that GPFusion improves the coefficient of determination by 6.9%, 8.0%, and 2.6%, respectively. Compared with other high-level fusion strategies, GPFusion delivers more stable predictions with lower variance while maintaining competitive accuracy. Additional empirical studies confirms the effectiveness of the replicate variance penalty and reveal critical trade-offs between tree depth and terminal flexibility for evolving compact and interpretable fusion functions.

A key challenge in black-box adversarial attacks is the high query complexity in hard-label settings, where only the top-1 predicted label from the target deep model is accessible. In this paper, we propose a novel normal-vector-based method called Two-third Bridge Attack (TtBA). A innovative bridge direction is introduced which is a weighted combination of the current unit perturbation direction and its unit normal vector, controlled by a weight parameter $k$. We further use binary search to identify $k=k_\text{bridge}$, which has identical decision boundary as the current direction. Notably, we observe that $k=2/3 k_\text{bridge}$ yields a near-optimal perturbation direction, ensuring the stealthiness of the attack. In addition, we investigate the critical importance of local optima during the perturbation direction optimization process and propose a simple and effective approach to detect and escape such local optima. Experimental results on MNIST, FASHION-MNIST, CIFAR10, CIFAR100, and ImageNet datasets demonstrate the strong performance and scalability of our approach. Compared to state-of-the-art non-targeted and targeted attack methods, TtBA consistently delivers superior performance across most experimented datasets and deep learning models. Code is available at https://anonymous.4open.science/r/TtBA-6ECF.

Cost-aware Dynamic Workflow Scheduling (CADWS) is a key challenge in cloud computing, focusing on devising an effective scheduling policy to efficiently schedule dynamically arriving workflow tasks, represented as Directed Acyclic Graphs (DAG), to suitable virtual machines (VMs). Deep reinforcement learning (DRL) has been widely employed for automated scheduling policy design. However, the performance of DRL is heavily influenced by the design of the problem-tailored policy network and is highly sensitive to hyperparameters and the design of reward feedback. Considering the above-mentioned issues, this study proposes a novel DRL method combining Graph Attention Networks-based policy network and Evolution Strategy, referred to as GATES. The contributions of GATES are summarized as follows: (1) GATES can capture the impact of current task scheduling on subsequent tasks by learning the topological relationships between tasks in a DAG. (2) GATES can assess the importance of each VM to the ready task, enabling it to adapt to dynamically changing VM resources. (3) Utilizing Evolution Strategy's robustness, exploratory nature, and tolerance for delayed rewards, GATES achieves stable policy learning in CADWS. Extensive experimental results demonstrate the superiority of the proposed GATES in CADWS, outperforming several state-of-the-art algorithms. The source code is available at: https://github.com/YaShen998/GATES.

Community detection, a vital technology for real-world applications, uncovers cohesive node groups (communities) by leveraging both topological and attribute similarities in social networks. However, existing Graph Convolutional Networks (GCNs) trained to maximize modularity often converge to suboptimal solutions. Additionally, directly using human-labeled communities for training can undermine topological cohesiveness by grouping disconnected nodes based solely on node attributes. We address these issues by proposing a novel Topological and Attributive Similarity-based Community detection (TAS-Com) method. TAS-Com introduces a novel loss function that exploits the highly effective and scalable Leiden algorithm to detect community structures with global optimal modularity. Leiden is further utilized to refine human-labeled communities to ensure connectivity within each community, enabling TAS-Com to detect community structures with desirable trade-offs between modularity and compliance with human labels. Experimental results on multiple benchmark networks confirm that TAS-Com can significantly outperform several state-of-the-art algorithms.

Integrated order batching and picker routing (IOBPR) is a complex combinatorial optimization problem in real-world intelligent manufacturing systems. Heuristics are often used for solving such complex scheduling problems. Manually designing scheduling heuristics suffer from two limitations: 1) few problem features can be taken into account and 2) the design process is time consuming. Genetic programming hyper heuristic (GPHH) approaches have been proposed on many scheduling problems to automatically evolve effective heuristics. However, existing GPHH approaches are often problem specific and requires careful design of problem specific terminal sets and evolution operators. The aim of this work is to develop a GPHH approach to evolve heuristics for the IOBPR problem. In particular, we propose a novel terminal set (NT) with three types of terminals, and a GPHH with elitist mutation (GPHH-EM) algorithm. Extensive experiments demonstrate that the heuristics evolved by GPHH-EM can significantly outperform other state-of-the-art competing algorithms designed by human experts. Further analysis indicates that the three types of terminals effectively complement to improve evolved heuristics for decision making. Furthermore, the newly developed elitist mutation operator expedites the evolutionary process for GPHH to find high-quality heuristics.

Dynamic workflow scheduling (DWS) is a challenging and important optimization problem in cloud computing, aiming to execute multiple heterogeneous workflows on dynamically leased virtual machine resources to satisfy user-defined Quality of Service requirements. For the popular deadline-constrained DWS in cloud problem, a virtual machine selection rule (VMSR) and a task selection rule (TSR) need to be designed simultaneously to minimize the rental fee and deadline violation penalty. For this purpose, Dual-Tree Genetic Programming (DTGP) has been previously developed to automatically evolve effective VMSRs and TSRs. However, existing DTGP approaches assume that VMSR and TSR, as well as terminals used by VMSRs and TSRs are equally important and evolve both VMSRs and TSRs in a black box manner, i.e., without using any knowledge about different impacts of trees and terminals. Several recent studies clearly indicate that different trees or terminals have varied performance impacts, making it critical to develop adaptive mutation mechanisms for effective DTGP. Driven by this motivation, this paper proposes two new levels of adaptive mutation mechanisms, contributing to the development of a new DTGP algorithm, which features the use of three new probability vectors for adaptive tree selection of VMSR and TSR at the first level and adaptive terminal selection at the second level while mutating any existing dual-tree individuals. Extensive experimental results demonstrate that the proposed two adaptive mechanisms can improve the effectiveness of DTGP compared to four baseline algorithms.

Effectively scheduling the execution of deadline-constrained IoT workflows in Fog-Cloud computing is an im-portant practical problem. The execution of IoT workflows must be carefully controlled, especially in applications like IoT health, where missed deadlines can be life-threatening. Existing approaches, including heuristic and meta-heuristic algorithms, have limited capabilities of handling deadline constraints while ensuring efficient execution of workflows. In this paper, we develop a Genetic Algorithm (GA) with a novel repair method to properly allocate workflow tasks to IoT devices and Fog/Cloud servers in order to meet deadlines while minimizing the resource costs. To effectively handle deadline constraints, our repair method first identifies infeasible solutions evolved by GA that can be potentially repaired. It then carries out root cause analysis to determine the primary reasons for deadline violation, and finally reallocate critical tasks to resolve violated deadlines. In this way, the risks of violating deadline constraints can be substantially reduced. On a wide range of problem instances commonly studied in literature, our experiments show that the new algorithm can clearly outperform multiple recently developed methods since it enjoys better chance of finding solutions that satisfy deadline constraints and can notably reduce the resource cost simultaneously.

In the domain of Cloud computing, Fog computing is integrated with the Cloud to offer a balanced approach that combines Cloud's scalability with Fog's low latency, enabling efficient software application deployment. However, many current studies overlook the unpredictability of future user requests, such as assuming all requests are known beforehand. User requests often arrive dynamically and may have different quality of service (QoS) preferences. Therefore we need effective methods to handle dynamic application deployment with multiple objectives. This paper tackles this gap by modeling a multi-objective application deployment problem that considers dynamically arriving users' requests on application deployment in a Cloud-Fog environment. We further introduce a multi-objective Genetic Programming Hyper-Heuristic based approach to automatically generate a set of deployment rules that can be chosen according to users' QoS preferences. These rules are generated with different trade-offs of two optimization objectives, i.e., minimizing cost and latency, which can be used for deploying applications dynamically. Our experimental evaluation using real-world data demonstrates that our GPHH approach can generate effective heuristics for deploying applications in an integrated Cloud-Fog environment.

Internet of Things Service Composition (SCIoT) aims to find the best composite IoT service to fulfil users' requirements. Given the NP-hard complexity of SCIoT, Evolutionary Computation methods, especially Estimation of Distribution Algorithms (EDAs), have received increasing attention to solve SCIoT problems. As one of the most popular EDA methods, Population-Based Incremental Learning (PBIL) has demonstrated its strong competency in optimising composed services in SCIoT. However, conventional PBIL does not explicitly utilize problem knowledge such as QoS and service replication, limiting its effectiveness for IoT service composition. In this paper, we propose a new PBIL based approach, named Population-Based Incremental Learning to Improve Service Composition (PBILISC), to solve the SCIoT problem. Different from traditional PBIL, PBILISC seamlessly integrates PBIL with QoS-aware local search (QLS) to effectively handle replicated services in the SCIoT problem. Specifically, PBILISC evolves a series of populations of solutions jointly through PBIL and QLS. PBIL leverages a probability distribution for population updates, while QLS focuses on improving the best evolved solution by searching promising neighboring solutions under the guidance of QoS. Experimental results show that PBILISC can outperform PBIL and several state-of-the-art methods on multiple benchmark SCIoT problems.

Long Short Term Memory (LSTM) network is an effective and popular deep learning model for processing complex sequential data. However, using a single LSTM network does not always achieve reliable performance with the increasing complexity of machine learning tasks. An important approach to improve the performance of LSTM networks is to learn an ensemble of these networks. This paper proposes a new multi-objective evolutionary algorithm to design ensembles of LSTMs. We use NSGA-II to evolve accurate and diverse LSTMs as the building blocks for creating effective ensembles. An integrated ensemble genetic algorithm further evolves the best-performing ensembles from the LSTMs created in each generation of NSGA-II in an efficient manner, allowing all LSTMs evolved by NSGA-II to be explicitly considered for building ensembles. The results show that the proposed approach significantly improves the efficiency and effectiveness over many state-of-the-art machine learning models on various classification and regression tasks.

The long short term memory (LSTM) network is a popular deep learning model with a wide range of applications. Skip connection is a promising and important architectural innovation that can noticeably improve the performance of LSTM networks on complex machine learning tasks with longterm temporal/spatial dependencies. However, it is difficult to manually design skip connections in deep LSTM networks. For example, iterative analysis of skip connections is impractical and has scalability challenges. This paper proposes a new approach based on genetic algorithm (GA) and differentiable architecture search to automatically design LSTM networks with suitable skip connections. To allow an LSTM network and its skip connections to be designed jointly, we relax the search space of skip connections to be continuous. Consequently, LSTM network architectures with appropriate skip connections can be optimized directly through gradient-based network training. The cost of designing skip connections can also be maintained at a low level. Experiment results obtained from various classification and regression tasks show that the proposed algorithm excels in evolving LSTM networks that can significantly outperform several state-of-the-art models.

Containers are a popular way of deploying software in cloud data centers. Containers are allocated to Virtual machines (VMs) which are allocated to Physical machines (PMs) within the data center. Since the resources required by containers often do not match those of VMs, where to allocate them must be decided. A poor solution can result in high energy costs. Many existing methods to solve this problem use heuristics which do not consider containers leaving the data center after being allocated. Some do consider migrating containers between VMs but few do for energy efficiency reasons. These overlooked aspects may lead to increased energy usage, particularly since studies have demonstrated that many containers run for only a brief duration. In this paper, we develop a model of the container-based cloud resource allocation problem that considers the energy impact of leaving and migrating containers. We then design a new Genetic Programming Hyper-Heuristic (GPHH) algorithm to jointly evolve three heuristics for container placement, VM placement and container migration control. We utilize newly designed terminals to ensure the effectiveness of our GPHH algorithm. Experiments have been conducted with results indicating that the heuristics evolved by our GPHH algorithm can achieve better performance compared to several state-of-the-art techniques.

Location-Aware Dynamic Online Application dePloyment (LADOAP) in clouds is an NP-hard combinatorial optimisation problem. Genetic Programming Hyper-Heuristic (GPHH) has emerged as a promising approach for addressing LADOAP demands by dynamically generating Virtual Machine (VM) selection heuristics online. However, the performance of GPHH is impeded by long simulation times and low sampling efficiency. In this paper, we propose a novel hyper-heuristic framework that integrates Genetic Programming Hyper-Heuristic (GPHH) and Reinforcement Learning (RL) approaches to evolve rules for efficiently selecting location-aware Virtual Machines (VMs) capable of hosting multiple containers. The RL policy's value function acts as a surrogate model, significantly expediting the evaluation of generated VM selection rules. By applying this hybrid framework to LADOAP problems, we achieve competitive performance with a notable reduction in the number of required simulations. This innovative approach not only enhances the efficiency of VM selection but also contributes to advancing the state-of-the-art in addressing complex LADOAP challenges.

Genetic Programming Hyper-heuristics (GPHH) have been successfully used to evolve scheduling rules for Dynamic Workflow Scheduling (DWS) as well as other challenging combinatorial optimization problems. The method of sampling training instances has a significant impact on the generalization ability of GPHH, yet they are rarely addressed in existing research. This article aims to fill this gap by proposing a GPHH algorithm with a sampling strategy to thoroughly investigate the impact of six instance sampling strategies on algorithmic generalization, including one rotation strategy, three mini-batch strategies, and two hybrid strategies. Experiments across four scenarios with varying settings reveal that: (1) mini-batch with random sampling can outperform rotation in generalizing to unseen workflow scheduling problems under the same computational cost; (2) employing a hybrid strategy that combines rotation and mini-batch further enhances the generalization ability of GPHH; and (3) mini-batch and hybrid strategies can effectively enable heuristics trained on small-scale training instances generalizing well to large-scale unseen ones. These findings highlight the potential of mini-batch strategies in GPHH, offering improved generalization performance while maintaining diversity and suggesting promising avenues for further exploration in GPHH domains.

Software-defined networking (SDN) allows flexible and centralized control in cloud data centers. An elastic set of distributed SDN controllers is often required to provide sufficient yet cost-effective processing capacity. However, this introduces a new challenge: Request Dispatching among the controllers by SDN switches. It is essential to design a dispatching policy for each switch to guide the request distribution. Existing policies are designed under certain assumptions, including a single centralized agent, global network knowledge, and a fixed number of controllers, which often cannot be satisfied in practice. This article proposes MADRina, Multiagent Deep Reinforcement Learning for request dispatching, to design policies with high dispatching adaptability and performance. First, we design a multiagent system to address the limitation of using a centralized agent with global network knowledge. Second, we propose a Deep Neural Network-based adaptive policy to enable request dispatching over an elastic set of controllers. Third, we develop a new algorithm to train the adaptive policies in a multiagent context. We prototype MADRina and build a simulation tool to evaluate its performance using real-world network data and topology. The results show that MADRina can significantly reduce response time by up to 30% compared to existing approaches.

Service composition in Internet of Things (SCIoT), as an emerging topic in service computing, aims to select optimal services to complete user requests according to various user requirements such as minimizing energy consumption and response time. To solve this NP-hard problem, numerous heuristic methods, e.g., local search and population-based algorithms, have been proposed, wherein Estimation of Distribution Algorithm (EDA) gains increasing attention because of its explicit global probabilistic nature. However, existing EDAs increase solution diversity by using fixed bias, yet interfere the stability of the learned distribution in the later stage of optimization. Therefore, this paper proposes an EDA with an adaptive bias strategy (EDA-AdaBias) to solve the service composition in IoT problem. The decreasing bias value is added onto the probability values for all choices of each solution variable over generations, which improves diversity of sampled solutions and avoids dramatic change of constructed distribution. Experiments indicate that EDA-AdaBias presents promising performance compared to other competitive methods on this problem.

Actor-critic deep reinforcement learning (DRL) algorithms have recently achieved prominent success in tackling various challenging reinforcement learning (RL) problems, particularly complex control tasks with high-dimensional continuous state and action spaces. Nevertheless, existing research showed that actor-critic DRL algorithms often failed to explore their learning environments effectively, resulting in limited learning stability and performance. To address this limitation, several ensemble DRL algorithms have been proposed lately to boost exploration and stabilize the learning process. However, most of existing ensemble algorithms do not explicitly train all base learners towards jointly optimizing the performance of the ensemble. In this paper, we propose a new technique to train an ensemble of base learners based on an innovative multi-step integration method. This training technique enables us to develop a new hierarchical learning algorithm for ensemble DRL that effectively promotes inter-learner collaboration through stable inter-learner parameter sharing. The design of our new algorithm is verified theoretically. The algorithm is also shown empirically to outperform several state-of-the-art DRL algorithms on multiple benchmark RL problems.

Policy gradient algorithms for reinforcement learning (RL) have successfully tackled a broad range of high-dimensional continuous RL problems, including many challenging robotic control problems. These algorithms can be largely divided into two categories, i.e., on-policy algorithms and off-policy algorithms. Off-policy deep RL (DRL) algorithms enjoy better sample efficiency than and often outperform on-policy algorithms. However, cutting-edge off-policy algorithms still suffer from the low-quality estimation of policy gradients, resulting in compromised learning performance and high sensitivity to hyper-parameter settings. To address this issue, we propose a new concept of robust policy gradient (RPG). Driven by RPG, this paper further develops a new policy ensemble gradient (PEG) algorithm for DRL, inspired by the recent success of several ensemble DRL algorithms. PEG efficiently and effectively estimates RPG by using multiple policy gradients obtained respectively from several off-policy base learners in an ensemble. The estimated RPG is then utilized for training all base learners simultaneously. Comprehensive experiments have been performed on six Mujoco benchmark problems. Compared to four state-of-the-art off-policy algorithms and four cutting-edge ensemble policy gradient algorithms, our new PEG algorithm achieved highly competitive stability, performance and sample efficiency. Further analysis shows that PEG is insensitive to varied hyper-parameter settings, confirming the positive role of RPG in building reliable and effective off-policy DRL algorithms.

Web service composition composes existing Web services to accommodate users’ requests for required functionalities with the best possible quality of services (QoS). Due to the computational complexity of this problem, evolutionary computation (EC) techniques have been employed to efficiently find composite services with near-optimal functional quality (i.e., quality of semantic matchmaking, QoSM for short) or nonfunctional quality (i.e., QoS) for each composition request individually. With a rapid increase in composition requests from a growing number of users, solving one composition request at a time can hardly meet the efficiency target anymore. Driven by the idea that the solutions obtained from solving one request can be highly useful for tackling other related requests, multitasking service composition approaches have been proposed to efficiently deal with multiple composition requests concurrently. However, existing attempts have not been effective in learning and sharing knowledge among solutions for multiple requests. In this article, we model the problem of collectively handling multiple service composition requests as a new multitasking service composition problem and propose a new permutation-based multifactorial evolutionary algorithm based on an estimation of distribution algorithm (EDA), named PMFEA-EDA, to effectively and efficiently solve this problem. In particular, we introduce a novel method for effective knowledge sharing across different service composition requests. For that, we develop a new sampling mechanism to increase the chance of identifying high-quality service compositions in both the single-tasking and multitasking contexts. Our experiment shows that our proposed approach, PMFEA-EDA, takes much less time than existing approaches that process each service request separately, and also outperforms them in terms of both QoSM and QoS.

As a lightweight and flexible infrastructure solution, containers have increasingly been used for application deployment on a global scale. By rapidly scaling containers at different locations, the deployed applications can handle dynamic workloads from the worldwide user community. Existing studies usually focus on the (dynamic) container scaling within a single data center or the (static) container deployment across geo-distributed data centers. This article studies an increasingly important container scaling problem for application deployment in geo-distributed clouds. Reinforcement learning (RL) has been widely used in container scaling due to its high adaptability and robustness. To handle high-dimensional state spaces in geo-distributed clouds, we propose a deep RL algorithm, namedDeepScale, to auto-scale containerized applications.DeepScaleinnovatively utilizes multi-step predicted future workloads to train a holistic scaling policy. It features several newly designed algorithmic components, including a domain-tailored state constructor and a heuristic-based action executor. These new algorithmic components are essential to meet the requirements of low deployment costs and achieve desirable application performance. We conduct extensive simulation studies using real-world datasets. The results show thatDeepScalecan significantly outperform an industry-leading scaling strategy and two state-of-the-art baselines in terms of both cost-effectiveness and constraint satisfaction.

Job Shop Scheduling (JSS) is an important real-world problem. However, the problem is challenging because of many conflicting objectives and the complexity of production flows. Genetic programming-based hyper-heuristic (GP-HH) is a useful approach for automatically evolving effective dispatching rules for many-objective JSS. However, the evolved Pareto-front is highly irregular, seriously affecting the effectiveness of GP-HH. Although the reference points method is one of the most prominent and efficient methods for diversity maintenance in many-objective problems, it usually uses a uniform distribution of reference points which is only appropriate for a regular Pareto-front. In fact, some reference points may never be linked to any Pareto-optimal solutions, rendering them useless. These useless reference points can significantly impact the performance of any reference-point-based many-objective optimization algorithms such as NSGA-III. This paper proposes a new reference point adaption process that explicitly constructs the distribution model using Gaussian process to effectively reduce the number of useless reference points to a low level, enabling a close match between reference points and the distribution of Pareto-optimal solutions. We incorporate this mechanism into NSGA-III to build a new algorithm called MARP-NSGA-III which is compared experimentally to several popular many-objective algorithms. Experiment results on a large collection of many-objective benchmark JSS instances clearly show that MARP-NSGA-III can significantly improve the performance by using our Gaussian Process-based reference point adaptation mechanism.

With the popularity of cloud computing, many organizations process their workflow tasks in cloud resources based on the Pay-As-Per-Use model. Dynamic Workflow Scheduling (DWS) aims to allocate dynamically arriving workflow tasks to cloud resources with optimal makespan, cost, load-balancing, etc. To timely allocate arriving tasks, heuristics have been used to solve the DWS problem in cloud environment. However, most of them are manually designed, considering a single objective, and use simple features to allocate resources to workflow tasks. In practice, multiple objectives should be considered to provide trade-off heuristics for users to choose from. In this paper, we propose a genetic programming hyper-heuristic (GPHH) approach to automatically generate multiple heuristics for multi-objective DWS. Our experimental evaluation using benchmark datasets demonstrates the effectiveness of our proposed GPHH approach.

Quality-of-service (QoS)-aware automated semantic Web service composition aims to find a composite service with optimized or near-optimized QoS and quality of semantic matchmaking within polynomial time. To cope with this NP-hard problem with high complexity, a variety of evolutionary computation (EC) techniques has been developed. To improve the effectiveness and efficiency of these techniques, in this article, we proposed a novel memetic estimation of the distribution algorithm-based approach, namely, MEEDA, to tackle this problem. In particular, MEEDA explores four different domain-dependent local search methods that search for effective composite services by utilizing several neighborhood structures. Apart from that, to significantly reduce the computational time of MEEDA, an efficient local search strategy is introduced by combining a uniform fitness distribution scheme for selecting suitable solutions and stochastic local search operators for effectively and efficiently exploiting neighbors. To better demonstrate MEEDA’s effectiveness and scalability, we create a more challenging, augmented version of the service composition benchmark dataset. Experimental results on this benchmark show that MEEDA with newly developed domain-dependent local search operator, i.e., layer-based constrained one-point swaps, significantly outperforms existing state-of-the-art algorithms in finding high-quality composite services.

Multi-cloud is becoming a popular cloud ecosystem because it allows enterprise users to share the workload across multiple cloud service providers to achieve high-quality services with lower operation cost and higher application resilience. In multi-cloud, cloud services are widely distributed at different locations with differentiated prices. Therefore, Web application providers face the challenge to select proper cloud services for application replication and deployment with the aim of minimizing the deployment cost. Meanwhile, the deployed application replicas must satisfy the constraint on request response time to maintain the quality of user experience. To meet the two major requirements, this article studies a new problem of Web application replication and deployment in multi-cloud (WARDMC) that jointly considers both the cost minimization and constraints on average response time, including particularly request processing time and network latency. To address the problem, we develop a new approach named MCApp. MCApp combines iterative mixed integer linear programming with domain-tailored large neighborhood search to optimize both application replicas deployment and user requests dispatching. Extensive experiments using the real-world datasets demonstrate that MCApp significantly outperforms several recently proposed approaches.

In distributed computing,Web Service Composition (WSC)leads to the effective reuse of existing services and produces added value. WSC must fulfil functional requirements and optimiseQuality of Service (QoS)attributes, simultaneously.Memetic Algorithms (MAs)are promising for automatically composing numerous Web services to satisfy the above requirements.Data-intensive Web servicesfocus on providing and updating data with a significant volume of data operation and exchange. However, current composition approaches have ignored the impact of data communication and the distribution of services, which significantly affect the performance when applied to the challengingDistributed Data-intensive Web Service Composition (DDWSC)problem. Although recent approaches have revealed the usefulness of local search, they have completely overlooked the question of preferring appropriate composition solutions for local search. To address this research issue, we propose a priority-based selection method for the local search that can be consistently integrated with any MA for DDWSC. This enables us to develop state-of-the-art algorithms for DDWSC by explicitly considering the problem-specific, population and solution-related information for choosing a solution. Extensive experimental evaluation using benchmark datasets shows that our proposed method significantly outperforms several recently proposed methods.

JSS (Job Shop Scheduling) is a significant and challenging combinatorial optimization issue. Dispatching rules have been successfully used to determine scheduling decisions in the JSS challenges. Genetic programming (GP) has been widely used to discover and develop dispatching rules for various scheduling problems. However, there has been relatively little research into feature selection in GP-HH for many-objective JSS. In many conflicting objective contexts, it's also vital to quantify the contribution of features. This work presents a new two-stage GP-HH methodology for many-objective JSS with feature selection for changing rules. The quality of the solutions (dispatching rules) after incorporating the many-objective algorithm with feature selection is investigated in this paper. On a four-objective JSS problem, the suggested algorithm (FS-GP-NSGA-III) is compared to the standard GP-NSGA-III. The experimental results show that using GP to pick relevant features improves the algorithm's performance. Furthermore, the proposed technique generates rules that are minimal in size and easy to understand.

Long Short Term Memory (LSTM) is a special kind of Recurrent Neural Networks popularly used in various applications. However, using a single LSTM is often not enough to attain reliable performance on complicated machine learning tasks. This is because LSTM is sensitive to the specifics of the training data. Ensemble learning is a promising approach to improve the performance of LSTMs on complicated tasks. However, it is difficult to design an ensemble of LSTMs. LSTMs that constitute the ensemble should be both accurate and diverse. This paper proposes a new two-phase evolutionary algorithm to design ensembles. The first phase is to evolve best performing LSTMs automatically. A connection weight inheritance approach is used in the first phase to improve the effectiveness and efficiency of the evolutionary process. The second phase is to design ensembles by choosing suitable LSTMs without fixing the ensemble size in advance. We use bagging to train the selected LSTMs to build the ensemble to achieve good diversity among the LSTMs. The proposed approach is evaluated on various classification tasks. The results show the effectiveness of the proposed approach and its significant improvement in performance over many state-of-the-art machine learning models. The results also show the efficiency of the proposed approach in comparison with the baseline algorithm.

Workflow with different patterns and sizes arrive at a cloud data center dynamically to be processed at virtual machines in the data center, with the aim to minimize overall cost and makespan while satisfying Service Level Agreement (SLA) requirement. To efficiently schedule workflows, manually designed heuristics are proposed in the literature. However, it is time consuming to manually design heuristics. The designed heuristics may not work effectively for heterogeneous workflow since only simple problem related factors are considered in the heuristics. Further, most of the existing approaches ignore the deadline constraints set in SLAs. Genetic Programming Hyper Heuristic (GPHH) can be used to automatically design heuristics for scheduling problems. In this paper, we propose a GPHH approach to automatically generate heuristics for the dynamic workflow scheduling problem, with the goal of minimizing the VM rental fees and SLA penalties. Experiments have been conducted to evaluate the performance of the proposed approach. Compared with several existing heuristics and conventional Genetic Programming (GP) approaches, the proposed Dynamic Workflow Scheduling Genetic Programming (DWSGP) has better performance and is highly adaptable to variations in cloud environment.

Traditional human-designed heuristics-based algorithms are commonly employed to address workflow scheduling problems. Such as Heterogeneous Earliest Finish Time (HEFT) and CriticalPath are two best-known list based heuristics. Generally, heuristics-based approaches can generate a single heuristic by making decisions based on the current status of the tasks and available resources and map the unscheduled tasks to the available resources. However, traditional heuristics can easily cause unbalancing load problem. For example, with the objective of minimizing the makespan, traditional heuristics prefer to use computation resources with high computation capacity. Usually, such resources are expensive, which will lead to a high cost as a result. Therefore it is hard for single traditional heuristics to do a trade-off; thus, traditional heuristics are hard to apply for multi-objective workflow scheduling. In this paper, we develop a novel algorithm for workflow scheduling by considering both minimizing cost and makespan. Experiments show that the MOSGP approach can effectively minimize makespan and cost simultaneously.

Genetic programming (GP) is considered the most popular method for automatically discovering and constructing dispatching rules for scheduling problems. Pareto Local Search (PLS) is a simple and effective local search method for tackling multi-objective combinatorial optimization problems. Researchers have studied the application of PLS to multiobjective evolutionary algorithms (MOEAs) with some success. In fact, by hybridizing global search with local search, the performance of many MOEAs can be noticeably improved. Despite its preliminary success, the practical use of PLS in GP is relatively limited. In this study, our aim is to enhance the quality of evolved dispatching rules for many-objective Job Shop Scheduling (JSS) through hybridizing GP with PLS techniques and designing an effective selection mechanism of initial solutions for PLS. In this paper, we propose a new GP-PLS algorithm that investigates whether the fitness-based selection mechanism for selecting initial solutions for PLS can increase the chance of discovering highly effective dispatching rules for many-objective JSS. To evaluate the effectiveness of our new algorithm, GPPLS is compared with the current state-of-the-art algorithms for many-objective JSS. The experimental results confirm that the proposed method can outperform the four recently proposed algorithms because of the proper use of local search techniques.

Deep Recurrent Neural Network (DRNN) is an effective deep learning method with a wide variety of applications. Manually designing the architecture of a DRNN for any specific task requires expert knowledge and the optimal DRNN architecture can vary substantially for different tasks. This paper focuses on developing an algorithm to automatically evolve task-specific DRNN architectures by using a Genetic Algorithm (GA). A variable-length encoding strategy is developed to represent DRNNs of different depths because it is not possible to determine the required depth of a DRNN in advance. Activation functions play an important role in the performance of DRNNs and must be carefully used in these networks. Driven by this understanding, knowledge-driven crossover and mutation operators will be proposed to carefully control the use of activation functions in GA in order for the algorithm to evolve best performing DRNNs. Our algorithm focuses particularly on evolving DRNN architectures that use Long Short Term Memory (LSTM) units. As a leading type of DRNN, LSTM-based DRNN can effectively handle long-term dependencies, achieving cutting-edge performance while processing various sequential data. Three different types of publicly available benchmark datasets for both classification and regression tasks have been considered in our experiments. The obtained results show that the proposed variable-length GA can evolve DRNN architectures that significantly outperform many state-of-the-art systems on most of the datasets.

To gain technical and economic benefits, enterprise application providers are increasingly moving their workloads to the cloud. With the increasing number of cloud resources from multiple cloud providers at different locations with differentiated prices, application providers face the challenge to select proper cloud resources to replicate and deploy applications to maintain low response time and high quality of user experience without running into the risk of over-spending. In this paper, we study the global-wide cloud application replication and deployment problem considering the application average response time, including particularly application execution time and network latency, subject to the budgetary control. To address the problem, we propose a GA-based approach with domain-tailored solution representation, fitness measurement, and population initialization. Extensive experiments using the real-world datasets demonstrate that our proposed GA-based approach significantly outperforms common application placement strategies, i.e., NearData and NearUsers, and our recently proposed hybrid GA approach.

Multiple users often share their Internet of Things (IoT) devices in a smart space. However, existing IoT systems do not support IoT sharing between multiple users or take into account the security risks associated with using shared devices. We address this problem by proposing a new multi-user IoT Secure Sharing (IoTSS) system supported by a newly designed sharing policy language. Our approach treats the policies as constraints in the context of an optimisation problem to fulfil user activities using the least vulnerable devices. We show how IoT sharing can be transformed into an equivalent Integer Linear Programming (ILP) problem, which can be solved efficiently and effectively by off-the-shelf Integer ILP solvers. To study the practical feasibility of IoTSS, we have implemented a proof-of-concept proxy-based prototype for the popularly used Mozilla WebThings Gateway. We found that the proxy service can achieve policies enforcement without incurring statistically significant time overhead.

One major research challenge for Software-Defined Networking is to properly deploy and efficiently utilize multiple controllers to improve resource utilization and maintain high network performance. While addressing this Controller Placement Problem (CPP), many existing studies overlooked the importance and influence of the Controller Scheduling Problem (CSP) with the central focus on proper distribution of requests from all switches among all controllers. In this paper, we define a new Controller Placement and Scheduling Problem (CPSP), emphasizing on the necessity and importance of tackling both CPP and CSP simultaneously in a coherent framework. To solve CPSP, we must seek a combination of solutions to both problems. Particularly, CSP is addressed based on a given solution to CPP and a Gradient-Descent-based (GD-based) scheduling algorithm is developed to optimize the probabilistic distribution of requests among all controllers. Built on the GD-based approach for controller scheduling, a Clustering-based Genetic Algorithm with Cooperative Clusters (CGA-CC) is further proposed to address CPP. In comparison to the majority of heuristic methods developed in the past, CGA-CC has two unique strengths. Specifically, it partitions a large network to substantially reduce the search space of the Genetic Algorithm (GA), resulting in fast identification of high-quality CPP solutions. Moreover, a greedy load re-distribution mechanism is developed to handle unexpected demand variations by dynamically forwarding bursting requests to neighboring sub-networks. Extensive simulations showed that our algorithms can significantly outperform several existing algorithms, including a recently proposed approach called Multi-controller Selection and Placement Algorithm (MSPA), in terms of both response time and controller utilization.

Enterprise application providers are increasingly moving their workloads to the cloud for technical and economic benefits. Multi-cloud environment makes it possible to orchestrate multiple cloud resources. With the increasing number of available cloud resources provided by multiple cloud providers at different locations with different prices, application providers face the challenge to select proper cloud resources to deploy their applications in the form of a workflow of component service units. Existing studies usually consider minimizing execution time or/and deployment cost. From the perspective of application providers, however, they also pay huge attention to application response time, including particularly network latency between deployed services and users. Meanwhile, application deployment is often subject to stringent budgetary control to ensure financial viability. This article studies a new type of composite application deployment problem that jointly considers both the performance optimization and budget control in multi-cloud at the global scale. To find solutions with minimal response time without running into the risk of over-spending, we propose a hybrid GA-based approach, featuring new design of domain-tailored service clustering, repair algorithm, solution representation, population initialization, and genetic operators. Extensive experiments using the real-world dataset demonstrate that our proposed hybrid GA approach outperforms some recently proposed approaches.

As smart environments become more common, IoT applications can automate more complex and dynamic activities. Users can define their activities as abstract workflows and suitable devices will be selected dynamically to execute them based on user quality of experience (QoE) requirements. However, many of such applications violate the principle of least privilege in terms of the allowed interactions between the IoT devices. We propose an Application-Centric Access Control (ACAC) framework to enable least privilege network access control for dynamic workflows while considering users' QoE. ACAC enables automatic derivation of an access control policy for an IoT application and allow this to be adjusted dynamically as new devices come and go in order to maintain user QoE.

Dynamic job shop scheduling is an important but difficult problem in manufacturing systems which becomes complex particularly in uncertain environments with varying shop scenarios. Genetic programming based hyper-heuristics (GPHH) have been a successful approach for dynamic job shop scheduling (DJSS) problems by enabling the automated design of dispatching rules for DJSS problems. GPHH is a computationally intensive and time consuming approach. Furthermore, when complex shop scenarios are considered, it requires a large number of training instances. When faced with multiple shop scenarios and a large number of problem instances, identifying good training instances to evolve dispatching rules which perform well over diverse scenarios is of vital importance though challenging. Essentially this requires the tackling of exploration versus exploitation trade-off. To address this challenge, we propose a new framework for GPHH which incorporates active sampling of good training instances during evolutionary process. We propose a sampling algorithm based on the ε-greedy method to evolve a set of dispatching rules. Through our experiments, we demonstrate the ability of our framework to efficiently identify useful training instances toward evolving dispatching rules which outperform the existing training methods.

Web Service Composition (WSC) is a particularly promising application of Web services, where multiple individual services with specific functionalities are composed to accomplish a more complex task, which must fulfil functional requirements and optimise Quality of Service (QoS) attributes, simultaneously. Additionally, large quantities of data, produced by technological advances, need to be exchanged between services. Data-intensive Web services, which manipulate and deal with those data, are of great interest to implement data-intensive processes, such as distributed Data-intensive Web Service Composition (DWSC). Researchers have proposed Evolutionary Computing (EC) fully-automated WSC techniques that meet all the above factors. Some of these works employed Memetic Algorithms (MAs) to enhance the performance of EC through increasing its exploitation ability of searching neighbourhood area of a solution. However, those works are not efficient or effective. This paper proposes an MA-based approach to solving the problem of distributed DWSC in an effective and efficient manner. In particular, we develop an MA that hybridises EC with a flexible local search technique incorporating distance of services. An evaluation using benchmark datasets is carried out, comparing existing state-of-the-art methods. Results show that our proposed method has the highest quality with an acceptable execution time.

To gain technical and economic benefits, various enterprises are increasingly moving their workloads to the cloud. Multi-cloud environment makes it possible to coordinate access and utilize multiple cloud resources. When business application developers host their business process in the cloud, they face the issue of choosing which cloud to deploy the instance-intensive business workflows. However, the existing studies rarely consider the optimization techniques for organizing cloud services with respect to various criteria, such as cost and performance. In this paper, we propose a seeding-based GA approach to address the multi-cloud workflow deployment problem, i.e. selecting and leasing virtual machines (VMs) to minimize deployment cost and response time. Experimental results show that the proposed GA approach with seeding strategy outperforms the existing approach proposed in the literature and standard GA algorithm.

Recurrent Neural Networks (RNNs) are a major class of artificial neural networks and one of the most effective deep learning models for human activity recognition (HAR). However designing the architecture of an RNN together with suitable hyper-parameters for any learning task can be very time-consuming and requires expert domain knowledge. This paper focuses on exploring a Genetic Algorithm (GA) based method to automatically design suitable architectures of Long Short Term Memory (LSTM)-based RNNs in a fully automated manner. An encoding strategy is proposed to encode the architectures and hyper-parameters of LSTM, for the sake of easy operation by GA. As a variant of RNN, LSTM is selected for this work due to its special feature to handle long-term dependencies. For verification and evaluation, three real-world benchmark datasets are used. We study three models of operation for the evolved deep LSTM-RNN, i.e., unidirectional, bidirectional and cascaded. The results of our experiments show that the RNN architectures automatically designed by our GA method can outperform state-of-the-art RNN systems and machine learning systems for HAR.

Web services are basic functions of a software system to support the concept of service-oriented architecture. They are often composed together to provide added values, known as web service composition. Researchers often employ Evolutionary Computation techniques to efficiently construct composite services with near-optimized functional quality (i.e., Quality of Semantic Matchmaking) or non-functional quality (i.e., Quality of Service) or both due to the complexity of this problem. With a significant increase in service composition requests, many composition requests have similar input and output requirements but may vary due to different preferences from different user segments. This problem is often treated as a multi-objective service composition so as to cope with different preferences from different user segments simultaneously. Without taking a multi-objective approach that gives rise to a solution selection challenge, we perceive multiple similar service composition requests as jointly forming an evolutionary multi-tasking problem in this work. We propose an effective permutation-based evolutionary multi-tasking approach that can simultaneously generate a set of solutions, with one for each service request. We also introduce a neighborhood structure over multiple tasks to allow newly evolved solutions to be evaluated on related tasks. Our proposed method can perform better at the cost of only a fraction of time, compared to one state-of-art single-tasking EC-based method. We also found that the use of the proper neighborhood structure can enhance the effectiveness of our approach.

Cloud computing enables enormous computational resources to be scheduled as parallel workflow applications. Most traditional heuristics can only solve one particular scheduling problem. For example, Heterogeneous Earliest Finish Time (HEFT) and Greedy algorithms allocate resources to given ordered list of tasks using a specific single heuristic, which only caters for a specific scheduling problem, e.g. the fixed number of tasks in a workflow and available resources. Many researchers considered the heterogeneous workflows and cloud resources in scheduling in order to minimize the cost and makespan, but the solutions provided are only for specific workflow pattern. In this paper, we demonstrate a workflow scheduling problem which considers the combination of heterogeneous workflows as well as heterogeneous computing resources. We proposed Flexible Scheduling using Genetic Programming (FSGP) approach to min-imize the total cost and makespan of heterogeneous workflows in the cloud. The performance of our proposed FSGP is regardless of the number of tasks in the workflow, available resources and workflow patterns. We evaluated our proposed approach using a benchmark dataset. Performance evaluation of some well-known algorithms such as HEFT and greedy algorithms exhibit that our FSGP approach perform better than other competing algorithms.

Recent research on Software-Defined Networking (SDN) strongly promotes the adoption of distributed controller architectures. To achieve high network performance, designing a scheduling function (SF) to properly dispatch requests from each switch to suitable controllers becomes critical. However, existing literature tends to design the SF targeted at specific network settings. In this paper, a reinforcement-learning-based (RL) approach is proposed with the aim to automatically learn a general, effective, and efficient SF. In particular, a new dispatching system is introduced in which the SF is represented as a neural network that determines the priority of each controller. Based on the priorities, a controller is selected using our proposed probability selection scheme to balance the tradeoff between exploration and exploitation during learning. In order to train a general SF, we first formulate the scheduling function design problem as an RL problem. Then a new training approach is developed based on a state-of-the-art deep RL algorithm. Our simulation results show that our RL approach can rapidly design (or learn) SFs with optimal performance. Apart from that, the trained SF can generalize well and outperforms commonly used scheduling heuristics under various network settings.

Insecure smart home IoT network is growing in number and size, and enforcing standard security solutions in IoT is a challenge due to its limited resources. The vulnerable smart home IoT poses huge security threats. It puts smart home network security at risk as it can be used as an entry point into the network, also it exposes users' privacy due to the amount of personal data it collects. Meanwhile, as IoT increases in popularity, it has a significant impact on the security of the rest of the Internet community (e.g. forming botnets). Previous research delegates IoT security to a third party (e.g. ISP) and ignores social and contextual factor. In this paper, we propose an SDN-based framework for enforcing network static and dynamic access control, where manufacturers, security providers, and users can cooperate to enhance the smart home IoT security. Proposed approach has three features: a) it allows the manufacturers to enforce the least privileged policy for IoT, and hence reduce the risk associated with exposing IoT to the Internet; b) it enables to enforce access policy as a feedback from security services; c) it enables users to customize IoT access based on social and contextual needs (e.g. only permits LAN access to the IoT through his/her mobile), which reduce the attack surface within the network. We also proposed IPv4 ARP server as an NFV security service to mitigate ARP spoofing attack by replying to ARP requests in the network. We implement a prototype to demonstrate the functionality of the framework against common attack scenarios (i.e. network scanning, ARP spoofing).

Automatic Web service composition has received much interest in the last decades. Data-intensive concepts have provided a promising computing paradigm for data-intensive Web service composition. Due to the complexity of the problem, metaheuristics in particular Evolutionary Computing (EC) techniques have been used for solving this composition problem. However, most of the current works neglected the distributed nature of data-intensive Web services. In this paper, we study the problem of distributed data-intensive service composition and propose a model which integrates attributes of constituent data-intensive Web services and attributes of the network. The core idea is to propose a communication cost and time model of a composed Web service considering communication delay and cost. We therefore propose a novel method based on Genetic Algorithm (GA) which uses a variation of K-means clustering algorithm.

In the last few years, the container-based Cloud computing paradigm has gradually emerged as a flexible approach for energy efficient resource utilization. Cloud providers aim to optimize resource utilization and energy consumption while performing container consolidation, which involves virtual machine (VM) selection and VM placement. Most of the related works focus either on VM selection or VM placement, neglecting the interplay of the two optimization problems. This paper proposes a Two-stage Multi-type Particle Swarm Optimization approach, named TMPSO, to energy-aware container consolidation in Cloud data centers. Our experimental evaluation based on benchmark datasets demonstrates that our proposed algorithm shows further energy saving comparing with some existing approaches.

NeuroEvolution of Augmenting Topology (NEAT) is one of the most successful algorithms for solving traditional reinforcement learning (RL) tasks such as pole-balancing. However, the algorithm faces serious challenges while tackling problems with large state spaces, particularly the Atari game playing tasks. This is due to the major flaw that NEAT aims at evolving a single neural network (NN) that must be able to simultaneously extract high-level state features and select action outputs. However such complicated NNs cannot be easily evolved directly through NEAT. To address this issue, we propose a new reinforcement learning scheme based on NEAT with two key technical advancements: (1) a new three-stage learning scheme is introduced to clearly separate feature learning and policy learning to allow effective knowledge sharing and learning across multiple agents; (2) various policy gradient search algorithms can be seamlessly integrated with NEAT for training policy networks with deep structures to achieve effective and sample efficient RL. Experiments on several Atari games confirm that our new learning scheme can be more effective and has higher sample efficiency than NEAT and three state-of-the-art algorithms from the most recent RL literature.

Recent advancement on reinforcement learning (RL) algorithms shows that effective learning of parametric action-selection policies can often be achieved through direct optimization of a performance lower bound subject to pre-defined policy behavioral constraints. Driven by this understanding, this paper seeks to develop new policy search techniques where RL is achieved through maximizing a performance lower bound obtained originally based on an Expectation-Maximization method. For reliable RL, our new learning techniques must also simultaneously guarantee constrained policy behavioral changes measured through KL divergence. Two separate approaches will be pursued to tackle our constrained policy optimization problems, resulting in two new RL algorithms. The first algorithm utilizes a conjugate gradient technique and a Bayesian learning method for approximate optimization. The second algorithm focuses on minimizing a loss function derived from solving the Lagrangian for constrained policy search. Both algorithms have been experimentally examined on several benchmark problems provided by OpenAI GYM. The experiment results clearly demonstrate that our algorithms can be highly effective in comparison to several well-known RL algorithms.

Dynamic vehicle routing problem with time windows is an important combinatorial optimisation problem in many real-world applications. The most challenging part of the problem is to make real-time decisions (i.e. whether to accept the newly arrived service requests or not) during the execution of the routes. It is hardly applicable to use the optimisation methods such as mathematical programming and evolutionary algorithms that are competitive for static problems, since they are usually time-consuming, and cannot give real-time responses. In this paper, we consider solving this problem using heuristics. A heuristic gradually builds a solution by adding the requests to the end of the route one by one. This way, it can take advantage of the latest information when making the next decision, and give immediate response. In this paper, we propose a meta-algorithm to generate a solution given any heuristic. The meta-algorithm maintains a set of routes throughout the scheduling horizon. Whenever a new request arrives, it tries to re-generate new routes to include the new request by the heuristic. It accepts the new request if successful, and reject otherwise. Then we manually designed several heuristics, and proposed a genetic programming-based hyper-heuristic to automatically evolve heuristics. The results showed that the heuristics evolved by genetic programming significantly outperformed the manually designed heuristics.

Dynamic Job shop scheduling (DJSS) is a complex and hard problem in real-world manufacturing systems. In practice, the parameters of a job shop like processing times, due dates, etc. are uncertain. But most of the current research on scheduling consider only deterministic scenarios. In a typical dynamic job shop, once the information about a job becomes available it is considered unchanged. In this work, we consider genetic programming based dispatching rules to generate schedules in an uncertain environment where the process time of an operation is not known exactly until it is finished. Our primary goal is to investigate methods to incorporate the uncertainty information into the dispatching rules. We develop two training approaches, namely ex-post and ex-ante to evolve the dispatching rules to generate good schedules under uncertainty. Both these methods consider different ways of incorporating the uncertainty parameters into the genetic programs during evolution. We test our methods under different scenarios and the results compare well against the existing approaches. We also test the generalization capability of our methods across different levels of uncertainty and observe that the proposed methods perform well. In particular, we observe that the proposed ex-ante training approach outperformed other methods.

Dynamic job shop scheduling (DJSS) is a complex problem which is an important aspect of manufacturing systems. Even though the manufacturing environment is uncertain, most of the existing research works consider deterministic scheduling problems where the time required for processing any job is known in advance and never changes. In this work, we consider DJSS problems with varied uncertainty configurations of machines in terms of processing times and the total flow time as scheduling objective. With the varying levels of uncertainty many machines become bottlenecks of the job shop. It is essential to identify these bottleneck machines and schedule the jobs to be performed by them carefully. Driven by this idea, we develop a new effective method to evolve pairs of dispatching rules each for a different bottleneck level of the machines. A clustering approach to classifying the bottleneck level of the machines arising in the system due to uncertain processing times is proposed. Then, a cooperative co-evolution technique to evolve pairs of dispatching rules which generalize well across different uncertainty configurations is presented. We perform empirical analysis to show its generalization characteristic over the different uncertainty configurations and show that the proposed method outperforms the current approaches.

Actor-Critic algorithms have been increasingly researched for tackling challenging reinforcement learning problems. These algorithms are usually composed of two distinct learning processes, namely actor (a.k.a, policy) learning and critic (a.k.a, value function) learning. Actor learning is heavily dependent on critic learning; particularly unreliable critic learning due to its divergence can significantly affect the effectiveness of actor-critic algorithms. To address this issue, many successful algorithms have been developed recently with the aim of improving the accuracy of value function approximation. However, these algorithms introduce extra complexities to the learning process and may actually increase the difficulty for effective learning. Thus, in this research, we consider a simpler approach to improving the critic learning reliability. This approach requires us to seamlessly integrate an adapted Sandpile Model with the critic learning process so as to achieve desirable self-organizing property for reliable critic learning. Following this approach, we propose a new actor-critic learning algorithm. Its effectiveness and learning reliability have been further evaluated experimentally. As strongly demonstrated in the experiment results, our new algorithm can perform much better than traditional actor-critic algorithms. Meanwhile, correlation analysis further suggests that a strong correlation exists in between learning reliability and effectiveness. This finding may be important for future development of powerful reinforcement learning algorithms.

Distributed controller architectures have been proposed for Software-Defined Networking (SDN) to ensure scalability and reliability. One major drawback of the existing architectures is the uneven load distribution among controllers stemming from the static binding between controllers and switches. To address this issue, several existing studies introduce dynamic binding by adopting some switch migration mechanisms that re-associate switches from overloaded controllers to underutilized controllers. However, the migration process adds a considerable amount of complexity to the system and may incur significant network latency. In this paper, we propose BLAC, a novel BindingLess Architecture for distributed Controllers (BLAC), in which load balance is achieved with the help of the proposed scheduling layer, which intercepts flow requests from switches and dispatches them to different controllers as determined by selected scheduling algorithms. The process is proceeded transparently with no extra modification required for off-the-shelf SDN switches. Besides, the scheduling layer can flexibly support various scheduling algorithms and causes neither disruption of service nor significant network delay. We build a prototype that can work with various distributed controller systems and conduct experiments to demonstrate its efficacy. The results show that our design outperforms the static-binding controller system in terms of both system throughput and response time without the complexity of the dynamic-binding controller system.

The real world is full of problems with multiple conflicting objectives. However, reinforcement learning traditionally deals with only a single learning objective. Recently, several multi-objective reinforcement learning algorithms have been proposed. Nevertheless, many of these algorithms rely on tabular representations of the value function which are only suitable for solving small-scaled problems. To address this limitation, various learning classifier systems have been developed to learn a scalable representation in the form of a population of classifiers. Among all learning classifier systems, XCS has been most popularly used for tackling single-objective reinforcement learning problems. Aimed at achieving multi-objective learning, a new algorithm has been developed in this paper based on XCS. Our algorithm is designed to learn a group of Pareto optimal solutions through a single learning process. For this purpose, four technical issues in XCS have been identified and addressed in this paper. Experimental studies on three bi-objective maze problems further demonstrate the effectiveness of our algorithm.

In Job Shop Scheduling (JSS) problems, there are usually many conflicting objectives to consider, such as the makespan, mean flowtime, maximal tardiness, number of tardy jobs, etc. Most studies considered these objectives separately or aggregated them into a single objective (fitness function) and treat the problem as a single-objective optimization. Very few studies attempted to solve the multi-objective JSS with two or three objectives, not to mention the many-objective JSS with more than three objectives. In this paper, we investigate the many-objective JSS, which takes all the objectives into account. On the other hand, dispatching rules have been widely used in JSS due to its flexibility, scalability and quick response in dynamic environment. In this paper, we focus on evolving a set of trade-off dispatching rules for many-objective JSS, which can generate non-dominated schedules given any unseen instance. To this end, a new hybridized algorithm that combines Genetic Programming (GP) and NSGA-III is proposed. The experimental results demonstrates the efficacy of the newly proposed algorithm on the tested job-shop benchmark instances.

Soft computing techniques including fuzzy logic have been successfully applied to Wireless Body Area Networks (WBANs). However, most of the existing research works rely on manual design of the Fuzzy Logic Controller (FLC). To address this issue, in this paper, we propose to use Evolutionary Computation (EC) techniques to automate the design of FLCs for cross layer medium access control in WBANs. With the goal of improving network reliability while keeping the communication delay at a low level, we have particularly experimented on three different coding schemes. The influence of fitness functions has also been examined carefully in order to achieve a good balance between reliability and performance. We have also evaluated the effectiveness of two widely used evolutionary algorithms. Particularly, Particle Swarm Optimisation (PSO) is shown to be more effective than Differential Evolution (DE) for our design problem. Moreover, the FLC designed by our approach is also shown to outperform some related algorithms as well as the IEEE 802.15.4 standard.

Evolutionary scheduling and combinatorial optimisation is an active research area and attracts the attentions of many researchers from computer science and operations research. Many advances have been made in this field and its scope in terms of techniques and applications has been continuously extended. In this position paper, we provide an overall picture of some key challenges in the field, discuss potential future research directions, and give our position in the field. We focus on three major issues that are encountered in practice, namely dynamic changes, multiple interdependent decisions, and multiple objectives. Our view is that the researchers should step out of our comfort zone to deal with messy and complicated issues in real-world applications.

Despite their proven effectiveness, many Michigan learning classifier systems (LCSs) cannot perform multistep reinforcement learning in continuous spaces. To meet this technical challenge, some LCSs have been designed to learn fuzzy logic rules. They can be largely classified into strength-based and accuracy-based systems. The latter is gaining more research attention in the last decade. However, existing accuracy-based learning systems either address primarily single-step learning problems or require the action space to be discrete. In this paper, a new accuracy-based learning fuzzy classifier system (LFCS) is developed to explicitly handle continuous state input and continuous action output during multistep reinforcement learning. Several technical improvements have been achieved while developing the new learning algorithm. Particularly, we have successfully extended Q-learning like credit assignment methods to continuous spaces. To enable direct learning of stochastic strategies for action selection, we have also proposed to use a new fuzzy logic system with stochastic action outputs. Moreover, fine-grained learning of fuzzy rules has been achieved effectively in our algorithm by using a natural gradient learning method. It is the first time that these techniques are utilized substantially in any accuracy-based LFCSs. Meanwhile, in comparison with several recently proposed learning algorithms, our algorithm is shown to perform highly competitively on four benchmark learning problems and a robotics problem. The practical usefulness of our algorithm is also demonstrated by improving the performance of a wireless body area network.

Over the past decade, advances in electronics, computer science, and wireless technologies have brought Wireless Body Area Network (WBAN) into many interesting applications. Particularly for the healthcare application, reliability is considered as a very important aspect for WBAN. Being the main focus of this paper, we aim at improving reliability by reducing the collision rate and increasing the packet delivery ratio. We also strive to enhance the performance in terms of throughput in the WBAN while maintaining message latency at a reasonable level. In an effort to achieve these goals, we introduce a new cross-layer fuzzy logic based backoff mechanism. Through this method, instead of relying merely on Medium Access Control (MAC), information from physical and application layers will be exploited as well. Moreover, since independent decision making is supported by each sensor without relying on any coordinating devices, communication in the WBAN becomes very flexible. Specifically, rather than determining Backoff Exponent (BE) in IEEE 802.15.4 through a blind try-and-error process, the proposed fuzzy logic system determines the BE by considering both the current channel condition and application requirements. This feature gives the proposed system a higher level of adaptability. The simulation results clearly show noticeable improvement in reliability and performance without significantly increasing the message latency.

To solve reinforcement learning problems, many learning classifier systems (LCSs) are designed to learn state-action value functions through a compact set of maximally general and accurate rules. Most of these systems focus primarily on learning deterministic policies by using a greedy action selection strategy. However, in practice, it may be more flexible and desirable to learn stochastic policies, which can be considered as direct extensions of their deterministic counterparts. In this paper, we aim to achieve this goal by extending each rule with a new policy parameter. Meanwhile, a new method for adaptive learning of stochastic action selection strategies based on a policy gradient framework has also been introduced. Using this method, we have developed two new learning systems, one based on a regular gradient learning technology and the other based on a new natural gradient learning method. Both learning systems have been evaluated on three different types of reinforcement learning problems. The promising performance of the two systems clearly shows that LCSs provide a suitable platform for efficient and reliable learning of stochastic policies.

Training data in real world is often presented in random chunks. Yet existing sequential Incremental IDR/QR LDA (s-QR/IncLDA) can only process data one sample after another. This paper proposes a constructive chunk Incremental IDR/QR LDA (c-QR/IncLDA) for multiple data samples incremental learning. Given a chunk of s samples for incremental learning, the proposed c-QR/IncLDA increments current discriminant model Ω, by implementing computation on the compressed the residue matrix Δ ϵ Rd×n, instead of the entire incoming data chunk X ϵ Rd×s, where η ≤ s holds. Meanwhile, we derive a more accurate reduced within-class scatter matrix W to minimize the discriminative information loss at every incremental learning cycle. It is noted that the computational complexity of c-QR/IncLDA can be more expensive than s-QR/IncLDA for single sample processing. However, for multiple samples processing, the computational efficiency of c-QR/IncLDA deterministically surpasses s-QR/IncLDA when the chunk size is large, i.e., s ≫ η holds. Moreover, experiments evaluation shows that the proposed c-QR/IncLDA can achieve an accuracy level that is competitive to batch QR/LDA and is consistently higher than s-QR/IncLDA.

In wireless multihop networks, cooperation is of utmost importance to ensure the success of communication. However, due to limited resources especially energy, nodes may be compelled to adopt selfish behaviour by not forwarding packets for other nodes. Selfishness is a very subjective element to be measured because it is hard to determine whether or not a particular node's behaviour is intentional or a consequence of the environment. Most, if not all, published work assumed that this behaviour can be assessed but do not explicitly describe how selfishness is measured or quantified. In this paper, we propose a method to quantify a node's behaviour in forwarding packets for other nodes from the perspective of a single observer node (i.e. first-hand observation) and provide quantifiable metrics to represent the node's actual effort. We show that by using the proposed method, we are able to classify several types of selfishness and fairness behaviour.

Different from traditional P2P systems, individuals nodes of a Federated Service Providing (FSP) system play a more active role by offering a variety of domain-specific services. The service provision control (SPC) problem is an important problem of the FSP system and will be tackled in this paper within a stochastic optimization framework through several steps. The first step focuses on using stochastic differential equations (SDEs) to model and analyze the dynamic evolution of the service demand. Driven by the SDE model, expected future performance of a FSP system is analytically evaluated in the second step. Step three utilizes the differential evolution (DE) algorithm to identify near-optimal service-providing policies for each node. The service subscription protocol is further proposed in step four to help every node adjust its local policy in accordance with the services provided by other nodes. The four steps together implement a complete solution of the SPC problem and will be called the SDE-based service-provision control (SSPC) mechanism in this paper. Experimental evaluation of the mechanism has been reported in the paper. The results show that our approach is effective in tackling the SPC problem and may be therefore suitable for many practical applications.

Linear Proximal Support Vector Machines (LPSVM), like decision trees, classic SVM, etc. are originally not equipped to handle drifting data streams that exhibit high and varying degrees of class imbalance. For online classification of data streams with imbalanced class distribution, we propose an incremental LPSVM termed DCIL-IncLPSVM that has robust learning performance under class imbalance. In doing so, we simplify a weighted LPSVM, which is computationally not renewable, as several core matrices multiplying two simple weight coefficients. When data addition and/or retirement occurs, the proposed DCIL-IncLPSVM accommodates current class imbalance by a simple matrix and coefficient updating, meanwhile ensures no discriminative information lost throughout the learning process. Experiments on benchmark datasets indicate that the proposed DCIL-IncLPSVM outperforms batch SVM and LPSVM in terms of F-measure, relative sensitivity and G-mean metrics. Moreover, our application to online face membership authentication shows that the proposed DCIL-IncLPSVM remains effective in the presence of highly dynamic class imbalance, which usually poses serious problems to classic incremental SVM (IncSVM) and incremental LPSVM (IncLPSVM).

P2P-like applications are quickly gaining popularity in the Internet. Such applications are commonly modeled as graphs with nodes and edges. Usually nodes represent running processes that exchange information with each other through communication channels as represented by the edges. They often need to autonomously determine their suitable working mode or local status for the purpose of improving performance, reducing operation cost, or achieving system-level design goals. In order to achieve this objective, the concept of status configuration is introduced in this article and a mathematical correspondence is further established between status configuration and an optimization index ( OI ), which serves as a unified abstraction of any system design goals. Guided by this correspondence and inspired by the cross-entropy algorithm, a cross-entropy-driven self-organization mechanism (CESM) is proposed in this article. CESM exhibits the self-organization property since desirable status configurations that lead to high OI values will quickly emerge from purely localized interactions. Both theoretical and experimental analysis have been performed. The results strongly indicate that CESM is a simple yet effective technique which is potentially suitable for many P2P-like applications.

Evolution programming (EP) is an important category of evolutionary algorithms. It relies primarily on mutation operators to search for solutions of function optimization problems (FOPs). Recently a series of new mutation operators have been proposed in order to improve the performance of EP. One prominent example is the fast EP (FEP) algorithm which employs a mutation operator based on the Cauchy distribution instead of the commonly used Gaussian distribution. In this paper, we seek to improve the performance of EP via exploring another important factor of EP, namely, the selection strategy. Three selection rules R1-R3 have been presented to encourage both fitness diversity and solution diversity. Meanwhile, two solution exchange rules R4 and R5 have been introduced to further exploit the preserved genetic diversity. Simple theoretical analysis suggests that through the proper use of R1-R5, EP is more likely to find high-fitness solutions quickly. Our claim has been examined on 25 benchmark functions. Empirical evidence shows that our solution selection and exchange rules can significantly enhance the performance of EP.

Globalization leads to an efficient new business paradigm generally known as Collaborative Virtual Enterprise (CVE) which demands flexible service orchestration and robust workflow execution. As two major service orchestration strategies, OWL-S and BPEL have their own strength and deficiencies. To meet the challenge in CVE, in this paper, we propose a Workflow Execution System (WES) that takes advantage of the complementary strengths of these two technologies. On the one hand, having semantic support, OWL-S is used in dynamic service discovery and composition at high level. On the other hand, at the concrete level, industry-based BPEL is exploited in service execution. The description of each Semantic Web service is enhanced with rule-based modeling of the essential business logic behind the service interface. In order to realize interoperability in OWL-S and BPEL without loss of semantic information, we further proposed an OWLS2BPEL Mapper to facilitate the workflow robustness and support rule evaluation to increase responsiveness to customers. A concrete scenario in PC manufacturing Collaborative Virtual Enterprise is analyzed to demonstrate the effectiveness of our workflow execution system.

In a typical manufacturing system, jobs are released from a production planning stage to a shop floor, where they are allocated to resources like machines. An optimal or near optimal schedule is generally found for those jobs. In reality, the execution of this schedule is often interrupted by dynamic events like unexpected incoming new jobs, machine breakdowns, etc. A rapid recovery or a self-organized feature of the interrupted schedule is desirable for manufacturing resource management. This paper proposes an ant-colony inspired approach to recover the interrupted schedule through a self-organization mechanism. The experiments show that this mechanism significantly improves the quality of the recovered schedules.

With an increasing complexity of business processes, self-healing capability is becoming an important issue in order to support robust service flow execution. In this paper, a dynamic self-healing mechanism is proposed, which can dynamically identify suitable alternatives and replace faulty services such that a service flow can be performed successfully despite of unexpected exceptions. This mechanism explicitly utilizes semantic Web services for service matching and selection of a composite service in business service flow, and Semantic Web services are equipped with rich business rules in a domain-dependent manner. We explore the self-healing mechanism for supporting self-healable service flow execution which is modeled in BPEL4WS. A demo system of self-healing capable service flow execution is built to validate its effectiveness by a concrete scenario, PC manufacturing application.

The ant colony optimization (ACO) algorithm is a metaheuristic inspired from the behavior of foraging ants. Instead of exploring its ability in finding optimal solutions, the current study investigates another unique property - self-healing mechanism for resource allocation in a service-oriented environment where unexpected resource breakdown can occur. A system architecture is first proposed to detect, diagnose and react to disturbances. Then the performance of the ACO self-healing mechanism is tested and compared based on a modified benchmark problem. The experimental results show that the self-healing mechanism can promptly recover an obsolete schedule with high quality solutions.

In recent years, inspired by the emerging Web services standard and peer-to-peer technology, a new federated service providing (FSP) system paradigm has attracted increasing research interests. Many existing systems have either explicitly or implicitly followed this paradigm. Instead of exchanging files, peers in FSP systems share their computation resources in order to offer domain-specific services. In this paper, we focused on the coordination problem of how to self-organize the service group structures in response to the varying service demand. We presented our solution in the form of a coordination mechanism, which includes a labor-market model, a recruiting protocol, and a policy-driven decision architecture. Peers make their service providing decisions based on their local policies, which can be added, removed, or modified by users. A general methodology is introduced in this paper to facilitate policy design. Specifically, a heuristic inspired by the extremal optimization technique is utilized to handle potential inconsistencies among policies. A stimulus-response mechanism was further applied to make the decision process adjustable. Experiments under five application scenarios verified our ideas and demonstrated the effectiveness of our coordination mechanism.

Peer-to-peer (P2P) networks establish loosely coupled application-level overlays on top of the Internet to facilitate efficient sharing of resources. They can be roughly classified as either structured or unstructured networks. Without stringent constraints over the network topology, unstructured P2P networks can be constructed very efficiently and are therefore considered suitable to the Internet environment. However, the random search strategies adopted by these networks usually perform poorly with a large network size. In this paper, we seek to enhance the search performance in unstructured P2P networks through exploiting users' common interest patterns captured within a probability-theoretic framework termed the user interest model (UIM). A search protocol and a routing table updating protocol are further proposed in order to expedite the search process through self organizing the P2P network into a small world. Both theoretical and experimental analyses are conducted and demonstrated the effectiveness and efficiency of our approach.

In this paper, we address agent coordination from a dynamic systems perspective and propose a dynamic coordination model, which is inspired by biological metabolic systems. A new coordination mechanism through dynamic local adjustment (CDLA) is presented, and coordination is achieved when every agent utilizes explicitly the global system dynamics and performs iteratively a dynamic local adjustment procedure. The CDLA mechanism is investigated in an example multiagent shop floor system. The results show that the example manufacturing process is well-coordinated and the coordination approach is practically applicable and effective

In recent years, multiagent systems have become a new attractive paradigm for developing Internet-based enterprise applications. In this paper, we investigate the agent coordination issue of multiagent systems. We explore the emerging JavaSpace technology for achieving coordination within a multi-level supply chain management environment. JavaSpace is a recent realization of the classic Linda model. The tuple space of the Linda model provides a convenient way for agent communication and coordination. The coordination protocol for the supply chain management system is presented. The protocol is designed based on JavaSpace, and is described using colored Petri nets. We argue that the emerging JavaSpaces technology provides a convenient, yet flexible approach to agent coordination in multiagent system environment.