Abstract The card game Hanabi is considered a strong medium for the testing and development of multi-agent reinforcement learning (MARL) algorithms, due to its cooperative nature, partial observability, limited communication and remarkable complexity. Previous research efforts have explored the capabilities of MARL algorithms within Hanabi, focusing largely on advanced architecture design and algorithmic manipulations to achieve state-of-the-art performance for various number of cooperators. However, this often leads to complex solution strategies with high computational cost and requiring large amounts of training data. For humans to solve the Hanabi game effectively, they require the use of conventions, which often allows for a means to implicitly convey ideas or knowledge based on a predefined, and mutually agreed upon, set of “rules” or principles. Multi-agent problems containing partial observability, especially when limited communication is present, can benefit greatly from the use of implicit knowledge sharing. In this paper, we propose a novel approach to augmenting an agent’s action space using conventions, which act as a sequence of special cooperative actions that span over and include multiple time steps and multiple agents, requiring agents to actively opt in for it to reach fruition. These conventions are based on existing human conventions, and result in a significant improvement on the performance of existing techniques for self-play and cross-play for various number of cooperators within Hanabi.

Map-based methods for autonomous racing estimate the vehicle’s location, which is used to follow a high-level plan. While map-based methods demonstrate high-performance results, they are limited by requiring a map of the environment. In contrast, mapless methods can operate in unmapped contexts since they directly process raw sensor data (often LiDAR) to calculate commands, but suffer from poor performance. In response, we propose the local map framework that uses easily extractable, low-level features to build local maps of the visible region that form the input to optimisation-based trajectory planners. Our local map generation extracts the visible racetrack boundaries and calculates a centre line and track widths used for planning. We evaluate our method for simulated F1Tenth autonomous racing using a trajectory optimisation and tracking strategy and a model predictive controller. Our method achieves lap times that are 8.8% faster than the Follow-The-Gap method and 3.22% faster than end-to-end neural networks due to the optimisation resulting in a faster speed profile. The local map planner is 3.28% slower than global methods that have access to an entire map of the track that can be used for planning. Critically, our approach enables high-speed autonomous racing on unmapped tracks, achieving performance similar to global methods without requiring a track map.

Publish/Subscribe (pub/sub) systems have been widely adopted in highly scalable environments. We see this especially with IoT/IIoT applications, an environment where low bandwidth and high latency is ideal. The projected growth of Iot/IIoT network nodes are in the billions in the next few years and as such, there is a need for network communication standards that can adapt to the evergrowing nature of this industry. While current pub/sub standards have produced positive results so far, they all adopt a "topic" based pub/sub approach. They do not leverage off modern devices having spatial information. Current open-source standards also focus heavily on centralized brokering of information. This makes the broker in this system a potential bottleneck as it means if that broker goes down, the entire network goes down. We have developed a new, unique and innovative open-source pub/sub standard called VAST that leverages spatial information of modern network devices to perform message communication. It uses a unique concept called Spatial Publish/Subscribe (SPS). It is built on a peer-to-peer network to enable high scalability. In addition to this, it provides a Voronoi Overlay to efficiently distribute the messages, ensuring that network brokers are not overloaded with requests and ensures the network self-organizes itself if one or more brokers break down. It also has a forwarding algorithm to eliminate redundancies in the network. We will demonstrate this concept with a simulator we developed. We will show how the simulator works and how to use it. We believe that with this simulator, we will help encourage researchers adopt this technology for their spatial applications. An example of such is Massively Multi-user Virtual Environments (MMVEs), where there is a need for a high number of spatial network nodes in virtual environments.

Abstract Robotic football has long been seen as a grand challenge in artificial intelligence. Despite recent success of learned policies over heuristics and handcrafted rules in general, current teams in the simulated RoboCup football leagues, where autonomous agents compete against each other, still rely on handcrafted strategies with only a few using reinforcement learning directly. This limits a learning agent’s ability to find stronger high-level strategies for the full game. In this paper, we show that it is possible for agents to learn competent football strategies on a full 22 player setting using limited computation resources (one GPU and one CPU), from tabula rasa through self-play. To do this, we build a 2D football simulator with faster simulation times than the RoboCup simulator. We propose various improvements to the standard single-agent PPO training algorithm which help it scale to our multi-agent setting. These improvements include (1) using a policy and critic network with an attention mechanism that scales linearly in the number of agents, (2) sharing networks between agents which allow for faster throughput using batching, and (3) using Polyak averaged opponents, league opponents and freezing the opponent team when necessary. We show through experimental results that stable training in the full 22 player setting is possible. Agents trained in the 22 player setting learn to defeat a variety of handcrafted strategies, and also achieve a higher win rate compared to agents trained in the 4 player setting and evaluated in the full game.

In zero-resource settings where transcribed speech audio is unavailable, unsupervised feature learning is essential for downstream speech processing tasks. Here we compare two recent methods for frame-level acoustic feature learning. For both methods, unsupervised term discovery is used to find pairs of word examples of the same unknown type. Dynamic programming is then used to align the feature frames between each word pair, serving as weak top-down supervision for the two models. For the correspondence autoencoder (CAE), matching frames are presented as input-output pairs. The Triamese network uses a contrastive loss to reduce the distance between frames of the same predicted word type while increasing the distance between negative examples. For the first time, these feature extractors are compared on the same discrimination tasks using the same weak supervision pairs. We find that, on the two datasets considered here, the CAE outperforms the Triamese network. However, we show that a new hybrid correspondence-Triamese approach (CTriamese), consistently outperforms both the CAE and Triamese models in terms of average precision and ABX error rates on both English and Xitsonga evaluation data.

Imagine a robot is shown new concepts visually together with spoken tags, e.g. "milk", "eggs", "butter". After seeing one paired audiovisual example per class, it is shown a new set of unseen instances of these objects, and asked to pick the "milk". Without receiving any hard labels, could it learn to match the new continuous speech input to the correct visual instance? Although unimodal one-shot learning has been studied, where one labelled example in a single modality is given per class, this example motivates multimodal one-shot learning. Our main contribution is to formally define this task, and to propose several baseline and advanced models. We use a dataset of paired spoken and visual digits to specifically investigate recent advances in Siamese convolutional neural networks. Our best Siamese model achieves twice the accuracy of a nearest neighbour model using pixel-distance over images and dynamic time warping over speech in 11-way cross-modal matching.

There has been significant research effort into peer-to-peer (P2P) massively multi-user virtual environments (MMVEs). A number of architectures have been proposed to implement the P2P approach; however, the development of fully distributed MMVEs has met with a number of challenges. In this work, we address one of the key remaining challenges of state consistency and persistency in P2P MMVEs. Having reviewed state management and persistency architectures currently receiving research attention, we have identified deficiencies such as lack of load balancing, responsiveness, and scalability. To address these deficiencies, we present Pithos—a reliable, responsive, secure, load-balanced, and scalable distributed storage system, suited to P2P MMVEs. Pithos is designed specifically for P2P MMVEs, and we show that it improves the reliability and responsiveness of storage architectures as compared to existing P2P state persistency architectures. Pithos is implemented as an OverSim simulation running on the OMNeT++ network simulation framework. It is evaluated using up to 10,400 peers, with realistic latency profiles, with up to 15.8 million storage and retrieval requests that are generated to store a total of 2.4 million objects. Each peer in Pithos uses a maximum of 1,950Bps bandwidth to achieve 99.98% storage reliability, while the most reliable overlay storage configuration tested only achieved 93.65% reliability, using 2,182Bps bandwidth. Pithos is also more responsive than overlay storage, with an average responsiveness of 0.192s, compared with the average overlay responsiveness of 1.4s when retrieving objects from storage.

This demonstration will showcase mixed reality technologies that we developed for a series of public art performances in Vienna in October 2015 in a collaboration of performance artists and researchers. The focus of the demonstration is on natural interaction techniques that can be used intuitively to control an avatar in a virtual 3D world. We combine virtual reality devices with optical location tracking, hand gesture recognition and smart devices. Conference attendees will be able to walk around in a Minecraft world by physically moving in the real world and to perform actions on virtual world items using hand gestures. They can also test our initial system for shared avatar control, in which a user in the real world cooperates with a user in the virtual world. Finally, attendees will have the opportunity to give us feedback about their experience with our system.

The emergence of cloud management systems, and the adoption of elastic cloud services enable dynamic adjustment of cloud hosted resources and provisioning. In order to effectively provision for dynamic workloads presented on cloud platforms, an accurate forecast of the load on the cloud resources is required. In this paper, we investigate various forecasting methods presented in recent research, identify and adapt evaluation metrics used in literature and compare forecasting methods on prediction performance. We investigate the performance gain of ensemble models when combining three of the best performing models into one model. We find that our 30th order Auto-regression model and Feed-Forward Neural Network method perform the best when evaluated on Google's Cluster dataset and using the provision specific metrics identified. We also show an improvement in forecasting accuracy when evaluating two ensemble models.

Experimental evaluation of MMVE research is mostly based on simulation. To convince industry of the advantages of alternate MMVE architectures requires experimental results obtained from using a commercially deployed MMVE, such as Minecraft. In this paper we describe the transformation of Minecraft into a flexible research platform, that allows researchers to modify Minecraft on different layers and functional areas. As proof-of-concept we replace Minecraft's client/server architecture with a distributed server cluster. The Minecraft virtual environment is distributed amongst the server cluster, allowing for migration of users between nodes of the cluster. This is achieved without modifying the Minecraft client and with minimal modification of the server.

Recently, there has been significant research focus on Peer-to-Peer (P2P) Massively Multiplayer Online Games (MMOGs). A number of architectures have been presented in the literature to implement the P2P approach. One aspect that has not received sufficient attention in these architectures is state persistency in P2P MMOGs. This survey presents an overview of the current challenges present in P2P MMOGs, followed by an overview of classic state consistency models used in C/S MMOGs. The survey then classifies the state persistency techniques currently used in P2P MMOGs into super peer storage, overlay storage, hybrid storage, and distance-based storage. Key characteristics, namely scalability, fairness, reliability, responsiveness, and security are then defined. Each state persistency technique is evaluated according to these characteristics and recommendations are then made of possible future areas of research into the different storage types.