The recent progress in multi-agent deep reinforcement learning (MADRL) makes it more practical in real-world tasks, but its relatively poor scalability and the partially observable constraint raise more challenges for its performance and deployment. Based on our intuitive observation that human society could be regarded as a large-scale partially observable environment, where everyone has the functions of communicating with neighbors and remembering his/her own experience, we propose a novel network structure called the hierarchical graph recurrent network (HGRN) for multi-agent cooperation under partial observability. Specifically, we construct the multi-agent system as a graph, use a novel graph convolution structure to achieve communication between heterogeneous neighboring agents, and adopt a recurrent unit to enable agents to record historical information. To encourage exploration and improve robustness, we design a maximum-entropy learning method that can learn stochastic policies of a configurable target action entropy. Based on the above technologies, we propose a value-based MADRL algorithm called Soft-HGRN and its actor-critic variant called SAC-HGRN. Experimental results based on three homogeneous tasks and one heterogeneous environment not only show that our approach achieves clear improvements compared with four MADRL baselines, but also demonstrate the interpretability, scalability, and transferability of the proposed model.

Unfortunately the funding information was incorrect. It should be the National Key R&D Program of China (No. 2018AAA0102302).

In this paper, we aim to design a deep reinforcement learning (DRL) based control solution to navigating a swarm of unmanned aerial vehicles (UAVs) to fly around an unexplored target area under partial observation, which serves as Mobile Base Stations (MBSs) providing optimal communication coverage for the ground mobile users. To handle the information loss caused by the partial observability, we introduce a novel network architecture named Deep Recurrent Graph Network (DRGN), which could obtain extra spatial information through graph-convolution based inter-UAV communication, and utilize historical features with a recurrent unit. Based on DRGN and maximum-entropy learning, we propose a stochastic DRL policy named Soft Deep Recurrent Graph Network (SDRGN). In SDRGN, a heuristic reward function is elaborated, which is based on the local information of each UAV instead of the global information; thus, SDRGN reduces the training cost and enables distributed online learning. We conducted extensive experiments to design the structure of DRGN and examine the performance of SDRGN. The simulation results show that the proposed model outperforms four state-of-the-art DRL-based approaches and three heuristic baselines, and demonstrate the scalability, transferability, robustness, and interpretability of SDRGN.

In an unmanned aerial vehicle ad-hoc network (UANET), sparse and rapidly mobile unmanned aerial vehicles (UAVs)/nodes can dynamically change the UANET topology. This may lead to UANET service performance issues. In this study, for planning rapidly changing UAV swarms, we propose a dynamic value iteration network (DVIN) model trained using the episodic Q-learning method with the connection information of UANETs to generate a state value spread function, which enables UAVs/nodes to adapt to novel physical locations. We then evaluate the performance of the DVIN model and compare it with the non-dominated sorting genetic algorithm II and the exhaustive method. Simulation results demonstrate that the proposed model significantly reduces the decision-making time for UAV/node path planning with a high average success rate.

Optical flow estimation is a fundamental tool for computer vision applications. As a classical optical flow algorithm, Farneback version was a good blend of accuracy and runtime performance for a long time. This work presents a dataflow-based architecture of Farneback optical flow with high level synthesis (HLS) tools. Multi-dimension separable convolution, Block RAM array and deep pipeline optimization techniques are applied to reduce both space and time. The system is implemented on XC7K325T FPGA with an image size of $640 \times 480.$ It supports complete Farneback algorithm for any user-defined image pyramid levels and iteration times. It can process 31 fps, 8x faster than OpenCV version with 3 pyramid levels and 1 iteration time.

In dense traffic unmanned aerial vehicle (UAV) ad-hoc networks, traffic congestion can cause increased delay and packet loss, which limit the performance of the networks; therefore, a traffic balancing strategy is required to control the traffic. In this study, we propose TQNGPSR, a traffic-aware Q-network enhanced geographic routing protocol based on greedy perimeter stateless routing (GPSR), for UAV ad-hoc networks. The protocol enforces a traffic balancing strategy using the congestion information of neighbors, and evaluates the quality of a wireless link by the Q-network algorithm, which is a reinforcement learning algorithm. Based on the evaluation of each wireless link, the protocol makes routing decisions in multiple available choices to reduce delay and decrease packet loss. We simulate the performance of TQNGPSR and compare it with AODV, OLSR, GPSR, and QNGPSR. Simulation results show that TQNGPSR obtains higher packet delivery ratios and lower end-to-end delays than GPSR and QNGPSR. In high node density scenarios, it also outperforms AODV and OLSR in terms of the packet delivery ratio, end-to-end delay, and throughput.

Extensible Markup Language (XML) is widely used in web services. However, the task of XML parsing is always the bottleneck which consumes a lot of time and resources. In this work, we present a hybrid XML parser based on software and hardware co-design. We place hardware acceleration into a software-driven context. Our parser is based on document object model (DOM). It is capable of well-formed checking and tree construction at throughput of 1 cycle per byte (CPB). We implement the design on a Xilinx Kintex-7 FPGA with 0.8Gbps parsing throughput.

Energy consumed by Cloud datacenters has dramatically increased, driven by rapid uptake of applications and services globally provisioned through virtualization. By applying energy-aware virtual machine scheduling, Cloud providers are able to achieve enhanced energy efficiency and reduced operation cost. Energy consumption of datacenters consists of computing energy and cooling energy. However, due to the complexity of energy and thermal modeling of realistic Cloud datacenter operation, traditional approaches are unable to provide a comprehensive in-depth solution for virtual machine scheduling which encompasses both computing and cooling energy. This paper addresses this challenge by presenting an elaborate thermal model that analyzes the temperature distribution of airflow and server CPU. We propose GRANITE - a holistic virtual machine scheduling algorithm capable of minimizing total datacenter energy consumption. The algorithm is evaluated against other existing workload scheduling algorithms MaxUtil, TASA, IQR and Random using real Cloud workload characteristics extracted from Google datacenter tracelog. Results demonstrate that GRANITE consumes 4.3-43.6 percent less total energy in comparison to the state-of-the-art, and reduces the probability of critical temperature violation by 99.2 with 0.17 percent SLA violation rate as the performance penalty.

Temperature prediction can enhance datacenter thermal management towards minimizing cooling power draw. Traditional approaches achieve this through analyzing task-temperature profiles or resistor-capacitor circuit models to predict CPU temperature. However, they are unable to capture task resource heterogeneity within multi-tenant environments and make predictions under dynamic scenarios such as virtual machine migration, which is one of the main characteristics of Cloud computing. This paper proposes virtual machine level temperature prediction in Cloud datacenters. Experiments show that the mean squared error of stable CPU temperature prediction is within 1.10, and dynamic CPU temperature prediction can achieve 1.60 in most scenarios.

Improving energy efficiency of data centers has become increasingly important nowadays due to the significant amounts of power needed to operate these centers. An important method for achieving energy efficiency is server consolidation supported by virtualization. However, server consolidation may incur significant degradation to workload performance due to virtual machine (VM) co-location and migration. How to reduce such performance degradation becomes a critical issue to address. In this paper, we propose a profiling-based server consolidation framework which minimizes the number of physical machines (PMs) used in data centers while maintaining satisfactory performance of various workloads. Inside this framework, we first profile the performance losses of various workloads under two situations: running in co-location and experiencing migrations. We then design two modules: (1) consolidation planning module which, given a set of workloads, minimizes the number of PMs by an integer programming model, and (2) migration planning module which, given a source VM placement scenario and a target VM placement scenario, minimizes the number of VM migrations by a polynomial time algorithm. Also, based on the workload performance profiles, both modules can guarantee the performance losses of various workloads below configurable thresholds. Our experiments for workload profiling are conducted with real data center workloads and our experiments on our two modules validate the integer programming model and the polynomial time algorithm.

With the rapid development of big data and cloud computing, big data analytics as a service in the cloud is becoming increasingly popular. More and more individuals and organizations tend to rent virtual cluster to store and analyze data rather than building their own data centers. However, in virtualization environment, whether scaling out using a cluster with more nodes to process big data is better than scaling up by adding more resources to the original virtual machines (VMs) in cluster is not clear. In this paper, we study the scalability performance issues of hadoop virtual cluster with cost consideration. We first present the design and implementation of VirtualMR platform which can provide users with scalable hadoop virtual cluster services for the MapReduce based big data analytics. Then we run a series of hadoop benchmarks and real parallel machine learning algorithms to evaluate the scalability performance, including scale-up method and scale-out method. Finally, we integrate our platform with resource monitoring module and propose a system tuner. By analyzing the monitored data, we dynamically adjust the parameters of hadoop framework and virtual machine configuration to improve resource utilization and reduce rent cost. Experimental results show that the scale-up method outperforms the scale-out method for CPU-bound applications, and it is opposite for I/O-bound applications. The results also verify the efficiency of system tuner to increase resource utilization and reduce rent cost.

CloudSim is one of the most powerful simulation platforms for cloud computing. It supports the energy-conscious scheduling and network simulation in the latest version. However, it still faces several limitations: 1) Current CloudSim cannot support both the power model and the network model at the same time. 2) The network components in current CloudSim do not support power-aware simulation. 3) The simulation of migration does not take into account the network overheads. To overcome these limitations, we design and implement an enhanced cloud simulation platform called DartCSim+ that supports the energy-aware network simulation and network-aware live migration. Further, we also implement a resubmit mechanism for packets transmission to provide a more real network behavior to solve transmission failure which is caused by migration or network failure. Finally, three groups of experiments are performed to demonstrate the effectiveness of DartCSim+.

Virtualization technology is currently becoming increasingly popular and valuable in cloud computing environments due to the benefits of server consolidation, live migration, and resource isolation. Live migration of virtual machines can be used to implement energy saving and load balancing in cloud data center. However, to our knowledge, most of the previous work concentrated on the implementation of migration technology itself while didn't consider the impact of resource reservation strategy on migration efficiency. This paper focuses on the live migration strategy of multiple virtual machines with different resource reservation methods. We first describe the live migration framework of multiple virtual machines with resource reservation technology. Then we perform a series of experiments to investigate the impacts of different resource reservation methods on the performance of live migration in both source machine and target machine. Additionally, we analyze the efficiency of parallel migration strategy and workload-aware migration strategy. The metrics such as downtime, total migration time, and workload performance overheads are measured. Experiments reveal some new discovery of live migration of multiple virtual machines. Based on the observed results, we present corresponding optimization methods to improve the migration efficiency.

Cloud computing has recently received considerable attention. With the fast development of cloud computing, the data center is becoming larger in scale and consumes more energy. There is an emergency need to develop efficient energy-saving methods to reduce the huge energy consumption in the cloud data center. In this paper, we achieve this goal by dynamically allocating resources based on utilization analysis and prediction. We use ``Linear Predicting Method" (LPM) and ``Flat Period Reservation-Reduced Method" (FPRRM) to get useful information from the resource utilization log, and make M/M/1 queuing theory predicting method have better response time and less energy-consuming. Experimental evaluation performed on CloudSim cloud simulator shows that the proposed methods can effectively reduce the violation rate and energy-consuming in the cloud.

Virtualization technology is currently widely used due to its benefits on high resource utilization, flexible manageability and powerful system security. However, its use for high performance computing (HPC) is still not popular due to the unclearness of the virtualization overheads. It's worthy to evaluate the virtualization cost and to find the performance bottleneck when running HPC applications in virtual cluster. We first evaluate the basic performance overheads due to virtualization. Then we create a 16-node virtual cluster and perform a performance evaluation for both para-virtualization and full virtualization. After that, we evaluate the MPI (Message Passing Interface) scalability to investigate the impact of MPI and network communication between virtual machines. In addition to the macro assessment, we use the Oprofile/Xenoprof to investigate the architecture characterization like CPU cycle, L2 cache misses, DTLB misses and ITLB misses which is an auxiliary explanation to the performance bottleneck. Experimental results indicate that performance overheads of virtualization are acceptable for HPC, para-virtualization is very suitable for HPC due to the high virtualization efficiency and efficient inter-domain communication. Finally, we use the non-linear regression modeling technology to present a performance model for network latency and bandwidth to predict the performance in virtual cluster environment.

Virtualization brings many benefits such as improving system utilization and reducing cost through server consolidation. However, it also introduces isolation problem when running multiple virtual machine workloads in one physical platform. Additionally, with the advent of multi-core technology, more and more cores are built into one die in today's data center that will share and compete for the resource like cache. It's worthy to study the isolation of server consolidation in modern multi-core platform. However, to our knowledge there are few work done on the isolation property especially the fault isolation property when one of the virtual machine workloads is attacked in server consolidation. In this paper, we study the isolation property from performance perspective and provide two optimization methods to improve the isolation property. We first define the isolation property and quantify the performance isolation in consolidation and propose a VM-level optimization method. Then we study the fault isolation by introducing a misbehavior virtual machine in server consolidation scenario and propose a core-level cache-aware optimization method to improve the fault isolation. Experimental results show that our two optimization methods can effectively improve the performance isolation and fault isolation with 29.39% and 19.52% respectively. What's more, Oprofile/Xenoprof toolkits are used to find out the factors affecting isolation property from the hardware events level.

Probabilistic latent semantic analysis is a topic modeling technique to discover the hidden structure in binary and count data. As a mixture model, it performs a probabilistic mixture decomposition on the co-occurrence matrix, which produces two matrices assigned with probabilistic explanations. However, the factorized matrices may be rather smooth, which means we may obtain global feature and topic representations rather than expected local ones. To resolve this problem, one of the solutions is to revise the decomposition process with considerations of sparsity. In this paper, we present an approach that provides direct control over sparsity during the expectation maximization process. Furthermore, by using the log penalty function as sparsity measurement instead of the widely used L2 norm, we can approximate the re-estimation of parameters in linear time, as same as original PLSA does, while many other approaches require much more time. Experiments on face databases are reported to show visual representations on obtaining local features, and detailed improvements in clustering tasks compared with the original process.

Previous trust mechanisms are mainly focused on reputational models based on explicit trust ratings. However, the large amount of system-provided and user-generated content published on Web is often ignored. These information is very important for providing more accurate, fine-grained and efficient trust management. In this paper, we propose a content-based trust mechanism and introduce trust reasoning functionalities which can infer a user’s interests and expertise. We present a new method to calculate user similarity based on their feedbacks on reviews to reduce data sparseness of User Similarity matrix. We pinpoint several characteristics of trust which have been overlooked previously: trust is field-dependant, similarity-sensitive, personalized and etc. we differ various trust levels between pairs of users to support more accurate and efficient trust calculation. Although we choose Epinions as our targeted application, the mechanism proposed in this paper are also applicable to other e-commerce systems.

We conduct the first systematical adoption of the Semantic Web solution in the integration, management, and utilization of TCM information and knowledge resources. As the results, the largest TCM Semantic Web ontology is engineered as the uniform knowledge representation mechanism; the ontology-based query and search engine is deployed, mapping legacy and heterogeneous relational databases to the Semantic Web layer for query and search across database boundaries; the first global herb-drug interaction network is mapped through semantic integration, and the semantic graph mining methodology is implemented for discovering and interpreting interesting patterns from this network. The platform and underlying methodology are proved effective in TCM-related drug usage, discovery, and safety analysis.

There are two problems in the existing rendering platforms that support HLA-based simulation applications. One is the fact that the management of simulation entity objects and the management of rendering scene objects are separate, which leads to inefficiency in simulation and rendering. Another one is absence of rendering interface, especially for parallel rendering for users. In order to solve the above problems, we present a unified object model supporting HLA-based simulation and parallel rendering, which consists of heterogenous scene graph tree, action list and universal access interface. The Unified object model establishes an efficient data exchange bridge between HLA simulation and parallel rendering. Therefore massive complex scene and large quantity of simulation entities can be organized and managed in a unified form. An experimental demo is given at the end of this paper.

Distributed simulation and parallel rendering based on PC cluster have seen great success in recent years. To improve the overall performance, there is a trend to integrate modeling, simulation and visualization into a common distributed environment. In this paper, we propose a unified framework of building high performance distributed virtual reality (DVR) applications. The core components of this framework include the High Level Architecture (HLA), scene graphs and parallel rendering. The HLA supports interactive distributed simulation. Scene graphs are efficient to organize and manipulate scene data. And parallel rendering provides powerful rendering ability. This paper presents the in-depth architectural analysis of each components and derives a design that integrates them into a unified framework. Two DVR applications, including a remote navigation of massive virtual scenes and a multi-player video game, have been developed to evaluate the framework performance.

BACKGROUND: Recent advances in Web and information technologies with the increasing decentralization of organizational structures have resulted in massive amounts of information resources and domain-specific services in Traditional Chinese Medicine. The massive volume and diversity of information and services available have made it difficult to achieve seamless and interoperable e-Science for knowledge-intensive disciplines like TCM. Therefore, information integration and service coordination are two major challenges in e-Science for TCM. We still lack sophisticated approaches to integrate scientific data and services for TCM e-Science. RESULTS: We present a comprehensive approach to build dynamic and extendable e-Science applications for knowledge-intensive disciplines like TCM based on semantic and knowledge-based techniques. The semantic e-Science infrastructure for TCM supports large-scale database integration and service coordination in a virtual organization. We use domain ontologies to integrate TCM database resources and services in a semantic cyberspace and deliver a semantically superior experience including browsing, searching, querying and knowledge discovering to users. We have developed a collection of semantic-based toolkits to facilitate TCM scientists and researchers in information sharing and collaborative research. CONCLUSION: Semantic and knowledge-based techniques are suitable to knowledge-intensive disciplines like TCM. It's possible to build on-demand e-Science system for TCM based on existing semantic and knowledge-based techniques. The presented approach in the paper integrates heterogeneous distributed TCM databases and services, and provides scientists with semantically superior experience to support collaborative research in TCM discipline.