In the mobile edge computing (MEC)-enabled air-terrestrial integrated network, unmanned aerial vehicles (UAVs) serve as air edge nodes with the purpose to collaboratively train the high-availability prediction model by federated learning (FL). Nonetheless, in view of the significance of data freshness for an accurate training model, UAVs suffer from the stochastic and intermittent nature in energy harvesting (EH). This paper formulates an inter-UAV cooperative federated computing (IUCFC) problem to jointly optimize prediction accuracy, age of information (AoI) in flight region, and overall energy consumption of EH-enabled UAVs for edge data processing. To address the intricate IUCFC problem, a deep reinforcement learning (DRL) based cooperative UAV intelligent decision (CUID) algorithm is proposed, which leverages a dual Actor-Critic architecture, in pursuit of the collective tuning of hybird actions. Further, the Ornstein-Uhlenbeck (OU) noise is engaged in continuous action spaces to prompt exploration, while a conditional iteration dropout (CID) scheme mitigates the infeasible actions caused by the noise introduced, thereby bolstering the exploration efficiency and quality of CUID algorithm. Considering the non-stationary environments originated from UAV mobility, priority experience replay (PER) is adopted to dynamically modify experience priority. Extensive experiments show that CUID attains superior performance over those advanced algorithms, upgrading system utility by 8.79%, while augmenting FL model accuracy by 3.68% in dynamic scenarios with heterogeneous data distributions.

The throughput issue has become an urgent problem for large-scale application of blockchain technology. Sharding technology can improve system throughput and scalability. Current research on the optimal blockchain sharding problem uses Model-Free Reinforcement Learning (MFRL) to maximize system throughput. However, these methods learn directly from interactions with the system, resulting in extremely low sampling efficiency. To bridge this gap, this paper presents a Model-based Policy Optimization Blockchain Sharding algorithm (MBPOBS), which utilizes the Gaussian Process Regression (GPR) to precisely predict the future states, based on which the best sharding strategy is learned through imitation learning from the Cross-Entropy Method (CEM) demonstrator. Simulation results show that compared to traditional MRFL algorithms, MBPOBS accelerates the learning by up to 1.8x.

With the continuous expansion of data centers, their carbon emission becomes a serious issue. A number of studies are committing to reduce the carbon emission of data centers. Carbon trading is a promising emission reduction technique which is, however, seldom applied to data centers. To bridge this gap, we propose a carbon-neutral architecture DC2for distributed data centers, where each data center consists of three subsystems, i.e., energy subsystem for energy supply, thermal subsystem for data center cooling and the carbon subsystem for carbon trading. Then, we formulate the energy management problem as a Decentralized Partially Observable Markov Decision Processe (Dec-POMDP) and develop a distributed solution framework using Multi-Agent Deep Deterministic Policy Gradient (MAD-DPG). Finally, simulations on real-world data shows that the DC2provides a cost saving of 8.16%, and applying MADDPG saves 10.04% of the overall cost compared to Independent MADDPG.

In this poster, the energy and carbon management problem in Data Center Microgrid (DCMG) is modeled as a Decentralized Partially Observable Markov Decision Process, and the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm is adopted to learn the optimal operating policy. In addition, the system uses blockchain and smart contract to ensure data security and auditability in the power trading process. Finally, simulation results show that the proposed system significantly reduces the overall cost of the DCMGs compared with traditional systems.

Mobile Edge Computing (MEC) reduces service latency and enhances Quality of Service (QoS) by offloading tasks to the wireless network edge. However, the rapid growth of task offloading and the associated data transmission security challenges deserve further investigation. This study proposes a blockchain-MEC hybrid solution where mobile devices process tasks and engage in block mining to boost system utility. The objective is to maximize the accumulated reward in the blockchain-MEC system by optimizing off loading decisions, channel selection, transmission power, computing resources, and block intervals. A Markov Decision Process is formulated to model the optimization problem which is solved via a multi-agent deep reinforcement learning (MADRL) algorithm. The results of the simulation demonstrate that our approach is more effective than the baseline method.

Although botnet had been at the top of the list of main threats to the cyber world for an extended period of time, its harmfulness has been constrained nowadays due to the development of kaleidoscopic network security enforcing tools and people’s increasing awareness. And the underlying technology of the botnet has been stagnant ascribing to many drawbacks such as inadequate protection of the identity of the Botmaster and weak resilience of the botnet’s infrastructure. In this article, we first introduce a new classification of the botnet based on botnets’ underlying network, then briefly analyze the main flaws of the traditional botnet and some looming Blockchain-based botnets, with pros and cons of leveraging Blockchain to construct botnets. Furthermore, we propose one IOTA of countless legions (OICL), a newfangled versatile botnet infrastructure that overcomes the bottlenecks that other contemporaries cannot eliminate. It leverages Blockchain, also known as distributed ledger technology (DLT), to be its premises and uses many advantages of it without paying too many tradeoffs. Also, we invent a whole set of communication protocols for OICL and a novel scheme called Proof of Honest (PoH) to identify the espionage infiltrated into the botnet to further promote the robustness. In addition, we discover and propose a mechanism called collateral damage binding (CDB), which proves that the botnet has it such as OICL is far more robust than those who do not. Performance evaluations show that OICL is effective, more cost-saving, and fast-responding compared with the Bitcoin-based botnets as baselines.

With the swift development of express delivery industry, the increasingly attention has been shifted to express delivery mechanism design. Generally, the revenue of the courier is the difference between the users' express fee and the courier's pickup cost. In order to improve the revenue of courier without increasing the user's express fee, this paper presents a low-cost package pickup covering system to find an optimal Hamiltonian pickup tour for the courier over a subset of packages, where packages who are not on the tour should be covered exactly by one package on the tour. A billing rule discounting the express fee to incentivize users to deliver their packages is also proposed. We formulateLow-cost Package Pickup Covering (LPPC)problem to maximize the revenue of the courier. Considering the complexity ofLPPC, we propose aLow-cost Package Pickup Covering Mechanism (LPPCM)to solve theLPPCproblem including problem transformation, hardness analyzing,Attention Model based on Encoder-Decoder Architecture (AMEDA)model design and model training.AMEDAis trained by a deep reinforcement learning algorithm in an unsupervised manner and it can directly output the solution based on the given instances. Through extensive simulations, we demonstrate that the average revenue of courier forAMEDAis at least 10.1% higher than the traditional heuristic local search and is 18.5% lower than the optimal solution on average.AMEDAprovides a desired trade-off between the execution time and solution quality, which is well suited for the large-scale tasks which require quick decisions.

As one of the major carbon producers, data centers produce around 1% of global carbon emissions per year. Researchers are making significant effort to reduce the data center carbon emissions. However, the current carbon-neutral data center solutions seldom take the carbon quotas into account, nor do they integrate new emission reduction technologies like Carbon Capture (CC) and Power-to-Gas (P2G), etc. To bridge this gap, in this paper a novel carbon-neutral data center architecture is proposed based on which a holistic cost minimizing problem is formulated. We use the Variational Mode Decomposition (VMD) and the Long Short-Term Memory (LSTM) neural network to construct highly accurate prediction models, and develop a Model Predictive Control (MPC) algorithm for energy and carbon management. Finally, simulations on real-world data demonstrate that our approach reduces up to 19.19% overall cost compared with traditional solutions.

The rapid advancement in blockchain technology has enabled its applications across wide spectrum of fields. The blockchain throughput, which is usually measured by Transactions Per Second (TPS), is one of the key metrics to reflect the performance of the blockchain systems. However, current blockchain systems have low TPS rates that makes them unsuitable for latency critical applications like Vehicle-to-vehicle (V2V) communication. To address the above issue, the sharding technology, which divides the network into multiple disjoint groups so that transactions can be processed in parallel, is applied to the blockchain systems as a promising solution to improve TPS. This paper considers the Optimal Blockchain Sharding (OBCS) problem which is formulated as a Markov Decision Process (MDP) where the decision variables are the number of shards, block size and block interval. Previous works solved the OBCS problem via Deep Reinforcement Learning (DRL) based methods where the action space has to be discretized such that it is not too large for tractability. However, the discretization degrades the solution quality since the optimal solution usually lies between discrete values. In this paper, we treat the block size and block interval as continuous decision variables and propose a sharding control algorithm based on Parametrized Deep Q-Networks (P-DQN) to efficiently handle the discrete-continuous hybrid action space without the scalability issue. Experimental results show that our Parametrized Deep Q-Networks Blockchain Sharding (P-DQNBS) method can effectively improve the TPS by up to 20%.

The growth in scale and power density of Data Centers (DC) poses serious challenges to the cooling management. Recently, there are many studies using machine learning to solve the cooling management problems. However, a comprehensive comparative study is still missing. In this work, we compare the performance of various Deep Reinforcement Learning (DRL) algorithms, including Deep-Q Networks (DQN), Deep Deterministic Policy Gradient (DDPG), and Branching Dueling Q-Network (BDQ), using the Active Ventilation Tiles (AVTs) control problem in raised-floor DC as an example. In particular, we design two multiagent algorithms based on DQN and three critic architectures for DDPG. Simulations based on real world workload show that DDPG provides the best performance over the considered algorithms.

Data centers worldwide consume a large amount of energy and pose serious threats to the environment. Combined cooling, heating, and power (CCHP) and waste heat reuse (WHR) systems are two effective approaches to improve the data center energy efficiency. However, whether and to what extent the combination of them can bring further performance improvement have not been explored yet. Moreover, it is unclear how to manage the energy flow in such a hybrid system. In this article, we try to answer these questions by proposing a data center energy architecture that includes both CCHP and WHR technologies. We formulate the energy management problem as two optimization problems with different objectives. By analytically solving the optimization problems, two energy management strategies, namely, Following the electrical load with waste heat reuse (FEL-WHR) and operating cost-aware energy management with waste heat reuse (OCM-WHR), are developed. We evaluate the performance of proposed solutions by extensive simulations using a real-world workload trace from a production data center. Numerical results suggest that the CCHP and WHR technologies generally reduce the operating cost and are more environmental friendly. If the power generation efficiency of CCHP system is high, the impact of WHR will be more conspicuous (up to 8 percent improvement) and FEL-WHR is recommended because it achieves better sustainability at an operating cost comparable to OCM-WHR. Otherwise, WHR only improves the sustainability trivially, and the energy management policy should be carefully chosen according to the data center operator's objective.

This paper investigates a critical access control issue in the Internet of Things (IoT). In particular, we propose a smart contract-based framework, which consists of multiple access control contracts (ACCs), one judge contract (JC), and one register contract (RC), to achieve distributed and trustworthy access control for IoT systems. Each ACC provides one access control method for a subject-object pair, and implements both static access right validation based on predefined policies and dynamic access right validation by checking the behavior of the subject. The JC implements a misbehavior-judging method to facilitate the dynamic validation of the ACCs by receiving misbehavior reports from the ACCs, judging the misbehavior and returning the corresponding penalty. The RC registers the information of the access control and misbehavior-judging methods as well as their smart contracts, and also provides functions (e.g., register, update, and delete) to manage these methods. To demonstrate the application of the framework, we provide a case study in an IoT system with one desktop computer, one laptop and two Raspberry Pi single-board computers, where the ACCs, JC, and RC are implemented based on the Ethereum smart contract platform to achieve the access control.

Traditional static pricing strategies are ineffective for cloud providers to maximize their profit since they cannot leverage the supply-and-demand relationship of computing resources hosted in data centers. In this paper, we consider a dynamic server pricing (DSP) problem where the data center operator determines the server price based on the resource demand. In addition, the operator should also take the renewable energy, spot power price, and battery level into account. To solve the DSP problem, we propose a reactive pricing (RP) algorithm which dynamically tunes the server price in response to state change. It is shown through theoretical analysis and real world traces driven simulations that RP achieves a close-to-optimal profit and is robust against exogenous environment variations.

In this paper, we consider the Request Routing (RR) strategy in the CDN live streaming application. We show that to find an optimal RR strategy is correspond to a static optimization problem if the total number of clients is known in advance. However, this static approach is ineffective due to the difficulty in precisely estimating the number of clients off-line. We then develop the MPS scheduling algorithm to compute the multiphase RR strategy. Experimental study shows that our algorithm can generate a close-to-optimal strategy with respect to a wide range of the number of clients.

This paper concentrates on the performance analysis of channel contention in wireless ad hoc networks by using Stochastic Game Nets (SGN). We refine the definition of SGN so that it can be used to precisely capture the underlying details of a given system. A SGN model is developed to evaluate the channel utilization in a two-node wireless system. We quantify the system performance under equilibrium strategies. The findings of this paper are instructive for the design and deployment of wireless ad hoc networks.

With the development of Ethernet technology, Full Duplex Switched Ethernet shows great potential in hard real-time applications since its low cost and high bandwidth. In general, traditional Ethernet cannot provide deterministic service required for hard real-time applications. In this paper, we address a framework, which uses traffic shaper and SCED scheduling policy, to provide deterministic service in Full Duplex Switched Ethernet. Our analytical study shows that this framework can satisfy end-to-end delay requirement of the data flows. We also show that this framework can effectively allocate MUX resource and make adjusting the path of data flow easier.