Reinforcement Learning has been widely applied to play classic games where the agents learn the rules by playing the game by themselves. Recent works in general Reinforcement Learning use many improvements such as memory replay to boost the results and training time but we have not found research that focuses on the impact of memory replay in agents that play simple classic video games. In this research, we present an analysis of the impact of three different techniques of memory replay in the performance of a Deep Q-Learning model using different levels of difficulty of the Pacman video game. Also, we propose a multi-channel image - a novel way to create input tensors for training the model - inspired by one-hot encoding, and we show in the experiment section that the performance is improved by using this idea. We find that our model is able to learn faster than previous work and is even able to learn how to consistently win on the mediumClassic board after only 3,000 training episodes, previously thought to take much longer.

Interactive visualization has become a powerful insight-revealing medium. However, the close dependency of interactive visualization on its data inhibits its shareability. Users have to choose between the two extremes of (i) sharing non-interactive dataless formats such as images and videos, or (ii) giving access to their data and software to others with no control over how the data will be used. In this work, we fill the gap between the two extremes and present a new system, called Loom. Loom captures interactive visualizations as standalone dataless objects. Users can interact with Loom objects as if they still have the original software and data that created those visualizations. Yet, Loom objects are completely independent and can therefore be shared online without requiring the data or the visualization software. Loom objects are efficient to store and use, and provide privacy preserving mechanisms. We demonstrate Loom's efficacy with examples of scientific visualization using Paraview, information visualization using Tableau, and journalistic visualization from New York Times.

In this work, we propose a framework to simplify the creation of Augmented Reality (AR) extensions for web applications, without modifying the original web applications. We implemented the framework in an open source package called Alpaca. AR extensions developed using Alpaca appear as a web-browser extension, and automatically bridge the Document Object Model (DOM) of the web with the SceneGraph model of AR. To transform the web application into a multi-device, mixed-space web application, we designed a restrictive and minimized interface for cross-device event handling. We demonstrate our approach to develop mixed-space applications using three examples. These applications are, respectively, for exploring Google Books, exploring biodiversity distribution hosted by the National Park Service of the United States, and exploring YouTube’s recommendation engine. The first two cases show how a 3rd-party developer can create AR extensions without making any modifications to the original web applications. The last case serves as an example of how to create AR extensions when a developer creates a web application from scratch. Alpaca works on the iPhone X, the Google Pixel, and the Microsoft HoloLens.