This paper aims to learn multi-agent cooperation where each agent performs its actions in a decentralized way. In this case, it is very challenging to learn decentralized policies when the rewards are global and sparse. Recently, learning from demonstrations (LfD) provides a promising way to handle this challenge. However, in many practical tasks, the available demonstrations are often sub-optimal. To learn better policies from these sub-optimal demonstrations, this paper follows a centralized learning and decentralized execution framework and proposes a novel hybrid learning method based on multi-agent actor-critic. At first, the expert trajectory returns generated from demonstration actions are used to pre-train the centralized critic network. Then, multi-agent decisions are made by best response dynamics based on the critic and used to train the decentralized actor networks. Finally, the demonstrations are updated by the actor networks, and the critic and actor networks are learned jointly by running the above two steps alliteratively. We evaluate the proposed approach on a real-time strategy combat game. Experimental results show that the approach outperforms many competing demonstration-based methods.

The task of real-time combat game is to coordinate multiple units to defeat their enemies controlled by the given opponent in a real-time combat scenario. It is difficult to design a high-level Artificial Intelligence (AI) program for such a task due to its extremely large state-action space and real-time requirements. This paper formulates this task as a collective decentralized partially observable Markov decision process, and designs a Deep Decentralized Policy Network (DDPN) to model the polices. To train DDPN effectively, a novel two-stage learning algorithm is proposed which combines imitation learning from opponent and reinforcement learning by no-regret dynamics. Extensive experimental results on various combat scenarios indicate that proposed method can defeat different opponent models and significantly outperforms many state-of-the-art approaches.

We study the problem of distributed synchronization for device-to-device(D2D) communication. For D2D out-of-network coverage (ONC) scenario, synchronization among multiple user equipment (UE) is a very important issue due to the fact that synchronization method does not have a common time/frequency standard, so synchronization in self-organized D2D network is adjusting their own time/frequency offset until a group consensus is reached. 3GPP agreed a synchronization method based on multi-hop relays. In this method, UE uses reference signal received power (RSRP) of D2D synchronization signals (D2DSS) as the only criterion for selecting the reference signal and synchronizing with it. However, this method doesn't consider actual timing error of D2DSS, which are accumulated through the previous multi-hop links. Therefore, in this paper, we propose a novel synchronization method where UE selects a reference timing with the smallest RMS error. We examine the problem from system-level perspective, and verify that the proposed method can efficiently make more UE's RMS timing error meet tolerance of SC-FDMA symbols, which means a UE can successfully communicate with its neighbors directly.

Future wireless networks will dynamically access spectrum to maximize its utilization. Conventional design of dynamic spectrum access focuses on maximizing spectrum utilization, but faces the problem of degraded reliability due to unregulated demands and access behaviors. Without providing proper reliability guarantee, dynamic spectrum access is unacceptable to many infrastructure networks and services. In this paper, we propose SPARTA, a new architecture for dynamic spectrum access that balances access reliability and spectrum utilization. SPARTA includes two complementary techniques: proactive admission control performed by a central entity to determine the set of wireless nodes to be supported with only statistical information of their spectrum demands, and online adaptation performed by admitted wireless nodes to adjust their instantaneous spectrum usage to time-varying demand. Using both theoretical analysis and simulation, we show that SPARTA fulfills the reliability requirements while dynamically multiplexing spectrum demands to improve utilization. Compared to conventional solutions, SPARTA improves spectrum utilization by 80%-200%. Finally, SPARTA also allows service providers to explore the tradeoff between utilization and reliability to make the best use of the spectrum. To our best knowledge, our work is the first to identify and address such a tradeoff.

Access to realistic, complex graph datasets is critical to research on social networking systems and applications. Simulations on graph data provide critical evaluation of new systems and applications ranging from community detection to spam filtering and social web search. Due to the high time and resource costs of gathering real graph datasets through direct measurements, researchers are anonymizing and sharing a small number of valuable datasets with the community. However, performing experiments using shared real datasets faces three key disadvantages: concerns that graphs can be de-anonymized to reveal private information, increasing costs of distributing large datasets, and that a small number of available social graphs limits the statistical confidence in the results.

This paper presents a method for automatically converting raw GPS traces from everyday vehicles into a routable road network. The method begins by smoothing raw GPS traces using a novel aggregation technique. This technique pulls together traces that belong on the same road in response to simulated potential energy wells created around each trace. After the traces are moved in response to the potential fields, they tend to coalesce into smooth paths. To help adjust the parameters of the constituent potential fields, we present a theoretical analysis of the behavior of our algorithm on a few different road configurations. With the resulting smooth traces, we apply a custom clustering algorithm to create a graph of nodes and edges representing the road network. We show how this network can be used to plan reasonable driving routes, much like consumer-oriented mapping Web sites. We demonstrate our algorithms using real GPS data collected on public roads, and we evaluate the effectiveness of our approach on public roads, and we evaluate the effectiveness of our approach by comparing the route planning results suggested by our generated graph to a commercial route planner.

Future wireless infrastructure networks will dynamically access spectrum for maximum utilization. However, the fundamental challenge is how to provide stable spectrum access required for most applications. Using dynamic spectrum access, each node's spectrum usage is inherently unpredictable and unstable. We propose to address this challenge by integrating interference-aware statistical admission control with stability- driven spectrum allocation. Specifically, we propose to proactively regulate nodes' spectrum demand to allow efficient statistical multiplexing while minimizing outages. Admitted nodes coordinate to adapt instantaneous spectrum allocation to match time- varying demand. While the optimization problem is NP-hard, we develop computational-efficient algorithms with strong analytical guarantees. Experimental results show that the proposed approach can provide stable spectrum usage while improving its utilization by 80-100% compared to conventional solutions.

Dynamic spectrum access is a promising technique to use spectrum efficiently. Without being restricted to any prefixed spectrum bands, nodes choose operating spectrum on-demand. Such flexibility, however, makes efficient and fair spectrum access in large-scale networks a great challenge. Prior work in this area focused on explicit coordination where nodes communicate with peers to modify local spectrum allocation, and may heavily stress the communication resource. In this paper, we introduce a distributed spectrum management architecture where nodes share spectrum resource fairly by making independent actions following spectrum rules. We present five spectrum rules to regulate node behavior and maximize system fairness and spectrum utilization, and analyze the associated complexity and overhead. We show analytically and experimentally that the proposed rule-based approach achieves similar performance with the explicit coordination approach, while significantly reducing communication cost.

In this paper, we address the following problem: given a set of wireless nodes on the plane, what is the number of channels needed to ensure a schedulable conflict-free communication? While the exact number of channels needed is very difficult to compute, we focus on an upper bound for this number, in terms of the node density D. In particular, we derive several upper bounds in terms of D based on different settings of network interference ratio r, and analyze the trend of the upper bound as r changes. We also examine the tightness of our upper bound, by studying some specific scenarios of transmission. Our result can be viewed as another theoretical investigation towards the relationship between the network performance and the number of available channels. In practical application, the result suggests an easy estimation on the number of channels needed in the configuration of a wireless network. On the other hand, when the number of channels in a wireless system is given, the result can be used to provide a suggestion on the node density in the node deployment.

Abstract — In this paper, we present an adaptive and dis-tributed approach to spectrum allocation in mobile ad-hoc networks. We propose a local bargaining approach where users affected by the mobility event self-organize into bargaining groups and adapt their spectrum assignment to approximate a new optimal assignment. The number of computations required to adapt to topology changes can be significantly reduced com-pared to that of the conventional topology-based optimizations that ignore the prior assignment. In particular, we propose a Fairness Bargaining with Feed Poverty to improve fairness in spectrum assignment and derive a theoretical lower bound on the minimum assignment each user can get from bargaining for certain network configurations. Such bound can be utilized to guide the bargaining process. We also show that the difference between the proposed bargaining approach and the true optimal approach is upper-bounded. Experimental results demonstrate that the proposed bargaining approach provides similar perfor-mance as the topology-based optimization but with more than 50 % of reduction in complexity. I.