Understanding the time-varying magnetic field within a fusion device is critical for the successful design and construction of clean-burning fusion power plants. Poincaré analysis provides a powerful method for the analysis and visualization of magnetic field lines in fusion devices. Current state-of-the-art relies on manually and iteratively generating Poincaré plots from simulation data. Using Poincaré plots in deep analysis is very time consuming because Poincaré plots can be very computationally expensive, especially for a time-varying simulation with thousands of time steps. Further, the visualization results are so complex that only expert users know how to explore, interpret, and control. In addition, collaboration is hampered due to the high barrier to entry. To this end, we contribute Fugent, a reinforcement learning-based agent capable of recommending and evaluating the importance of exploration regions based on training data captured from historic expert user usage. Using Fugent, we show that important regions can be identified and recommended for further exploration. Fugent is open source.

Many scientific communities today have community datasets that are continuously created, curated, and maintained for community use. Such datasets are often hosted and shared through cloud-based data repositories. In this work, we propose a lightweight and affordable visualization cloud service that can be deployed as a companion service of a community dataset. Our target visualization use case is parallel flow visualization, which is crucial for understanding planet-scale phenomena such as the Earth’s atmosphere and ocean. As a core research topic of scientific visualization, parallel flow visualization typically uses HPC computing platforms. It is complex to implement with scalability, deploy with efficiency, and is often considered an advanced form of scientific visualization. Because of the heterogeneous nature of cloud platforms, in this work, we use a swarm-based parallel design to replace traditional HPC designs that assume homogeneity and rely upon conventional methods such as Message Passing Interface (MPI). This design enables interactive visualization of large flow fields in a way that is lightweight, efficient and easily deployable as a cloud service. We demonstrate our proposed system using NOAA’s NCEP ensemble data, which captures turbulent planet-scale atmospheric flows in observed forms, as well as in forecast forms for varying time scales. We evaluate the performance and efficacies of our system on Amazon Web Services (AWS) for three use cases, where remote users can use their laptops to (i) interactively explore global atmospheric flow patterns in general, (ii) to specifically compare how a forecast is different from the observation, and (iii) to explore flow patterns in a typical information visualization dashboard.

Data parallel techniques have been widely adopted both in academia and industry as a tool to enable scalable training of deep learning models. At scale, DL training jobs can fail due to software or hardware bugs, may need to be preempted or terminated due to unexpected events, or may perform suboptimally because they were misconfigured. Under such circumstances, there is a need to recover and/or reconfigure data-parallel DL training jobs on-the-fly, while minimizing the impact on the accuracy of the DNN model and the runtime overhead. In this regard, state-of-art techniques adopted by the HPC community mostly rely on checkpoint-restart, which inevitably leads to loss of progress, thus increasing the runtime overhead. In this paper we explore alternative techniques that exploit the properties of modern deep learning frameworks (overlapping of gradient averaging and weight updates with local gradient computations through pipeline parallelism) to reduce the overhead of resilience/elasticity. To this end we introduce a failure simulation framework and two resilience strategies (immediate mini-batch rollback and lossy forward recovery), which we study compared with checkpoint-restart approaches in a variety of settings in order to understand the trade-offs between the accuracy loss of the DNN model and the runtime overhead.

Scientific computation increasingly consists of a workflow of interrelated tasks. Containerization can make workflow systems more manageable, reproducible, and portable, but containers can impede communication due to their focus on encapsulation. In some circumstances, shared-memory regions are an effective way to improve performance of workflows; however sharing memory between containerized workflow tasks is difficult. In this work, we have created a software library called Dhmem that manages shared memory between workflow tasks in separate containers, with minimal code change and performance overhead. Instead of all code being in the same container, Dhmem allows a separate container for each workflow task to be constructed completely independently. Dhmem enables additional functionality: easy integration in existing workflow systems, communication configuration at runtime based on the environment, and scalable performance.

Interactive visualization has become a powerful insight-revealing medium. However, the close dependency of interactive visualization on its data inhibits its shareability. Users have to choose between the two extremes of (i) sharing non-interactive dataless formats such as images and videos, or (ii) giving access to their data and software to others with no control over how the data will be used. In this work, we fill the gap between the two extremes and present a new system, called Loom. Loom captures interactive visualizations as standalone dataless objects. Users can interact with Loom objects as if they still have the original software and data that created those visualizations. Yet, Loom objects are completely independent and can therefore be shared online without requiring the data or the visualization software. Loom objects are efficient to store and use, and provide privacy preserving mechanisms. We demonstrate Loom's efficacy with examples of scientific visualization using Paraview, information visualization using Tableau, and journalistic visualization from New York Times.

In this work, we propose a framework to simplify the creation of Augmented Reality (AR) extensions for web applications, without modifying the original web applications. We implemented the framework in an open source package called Alpaca. AR extensions developed using Alpaca appear as a web-browser extension, and automatically bridge the Document Object Model (DOM) of the web with the SceneGraph model of AR. To transform the web application into a multi-device, mixed-space web application, we designed a restrictive and minimized interface for cross-device event handling. We demonstrate our approach to develop mixed-space applications using three examples. These applications are, respectively, for exploring Google Books, exploring biodiversity distribution hosted by the National Park Service of the United States, and exploring YouTube’s recommendation engine. The first two cases show how a 3rd-party developer can create AR extensions without making any modifications to the original web applications. The last case serves as an example of how to create AR extensions when a developer creates a web application from scratch. Alpaca works on the iPhone X, the Google Pixel, and the Microsoft HoloLens.

In this paper, we propose using a decoupled architecture to create a microservice that can deliver scientific visualization remotely with efficiency, scalability, and superior availability, affordability and accessibility. Through our effort, we have created an open source platform, Tapestry, which can be deployed on Amazon AWS as a production use microservice. The applications we use to demonstrate the efficacy of the Tapestry microservice in this work are: (1) embedding interactive visualizations into lightweight web pages, (2) creating scientific visualization movies that are fully controllable by the viewers, (3) serving as a rendering engine for high-end displays such as power-walls, and (4) embedding data-intensive visualizations into augmented reality devices efficiently. In addition, we show results of an extensive performance study, and suggest how applications can make optimal use of microservices such as Tapestry.