Modern supercomputer applications are demanding high-performance storage resources in addition to fast computing resources. However, these storage resources, especially parallel shared filesystems, have become the Achilles' heel of many powerful supercomputers. Due to the lack of mechanism of IO resource provisioning on the file server side, a single user's IO-intensive work running on a small number of nodes can overload the metadata server and result in global filesystem performance degradation and even unresponsiveness. To tackle this issue, we developed an innovative tool, Optimal Overloaded IO Protection System (OOOPS). This tool is designed to control the IO workload from applications side. Supercomputer administrators can easily assign the maximum number of function calls of open() and stat() allowed per second. OOOPS can automatically detect and throttle intensive IO workload to protect parallel shared filesystems. It also allows supercomputer administrators to dynamically adjust how much metadata throughput one job can utilize during the job runs without interruption.

While monocular roadside cameras have been widely deployed and used to monitor traffic conditions across the United States, the analysis of those video data are commonly implemented either manually or through commercial applications tailor-made for specific tasks. The goal of this project is to develop an efficient system that can meet dynamic content based video analysis needs and scale to large scale traffic camera video data. The proposed system utilizes deep learning methods to recognize objects in the video data. That information can then be processed and analyzed through an analysis layer implemented using Spark and Hive. The analysis layer supports HiveQL, which enables end users to conduct sophisticated analysis with customized queries. In this paper, we present the implementation of this prototype application in details. The application can utilize both GPU and multiple CPUs to accelerate its computation. We evaluated its performance and scalability with different hardware and parameter settings, including Intel Knights Landing, Intel Skylake, Nvidia K40 GPU, and Nvidia P100 GPU, for object recognition. To demonstrate its versatile, we show two practical use case examples: counting moving vehicles and identifying scenes including pedestrians and vehicles. We show the accuracy of the system by comparing vehicular counts produced by the analysis with manually annotated results. The comparison shows our methods can achieve over eighty percent accuracy comparing to manual results.

The Lustre file system provides high aggregated I/O bandwidth and is in widespread use throughout the HPC community. Here we report on work (1) developing a model for understanding collective parallel MPI write operations on Lustre, and (2) producing a library that optimizes parallel write performance in a user-friendly way. We note that a system's default stripe count is rarely a good choice for parallel I/O, and that performance depends on a delicate balance between the number of stripes and the actual (not requested) number of collective writers. Unfortunate combinations of these parameters may degrade performance considerably. For the programmer, however, it's all about the stripe count: an informed choice of this single parameter allows MPI to assign writers in a way that achieves near-optimal performance. We offer recommendations for those who wish to tune performance manually and describe the easy-to-use T3PIO library that manages the tuning automatically.