HPC systems use monitoring and operational data analytics to ensure efficiency, performance, and orderly operations. Application-specific insights are crucial for analyzing the increasing complexity and diversity of HPC workloads, particularly through the identification of unknown software and recognition of repeated executions, which facilitate system optimization and security improvements. However, traditional identification methods using job or file names are unreliable for arbitrary user-provided names. Fuzzy hashing the content of executables detects similarities despite different code versions or compilation approaches while preserving privacy and file integrity, overcoming these limitations. We introduce SIREN, a process-level data collection framework for software identification and recognition. SIREN improves observability in HPC job execution by enabling analysis of process metadata, environment information, and executable fuzzy hashes. Findings from an opt-in deployment campaign on LUMI show SIREN’s ability to provide insights into software usage, recognition of repeated executions of known applications, and similarity-based identification of unknown applications.

High Performance Computing (HPC) systems are ever-evolving, increasing in complexity and heterogeneity, while providing high computing power to scientific and data analysis applications. They are attracting a broad range of users that execute applications from computational chemistry, physics, digital humanities, life sciences, and machine learning, to name a few. Not all users possess the knowledge and expertise to harness the full performance capabilities of complex HPC systems. These users tend to submit inaccurate resource requests for their jobs, especially regarding time limits. This leads to inefficient scheduling of jobs, increased wait times for other users, overall inefficient system utilization, and wasted computing resources and energy, ultimately slowing down scientific discovery. This situation motivates the analysis of the accuracy of resource requests and its reporting to users to raise awareness about job efficiency, improve future job submissions, and reduce job wait times. Existing analyses of job wait times often neglect the connection between the causes of job wait times that can range from user-given and Quality of Service (QoS) limits, to unavailable resources, and approaches of reducing wait times that can be reported to users in an understandable way. In this work, we analyze almost 350’000 jobs collected over 2 months on a local university HPC cluster. Our analysis placed an emphasis on time limit accuracy, QoS characteristics, and reasons for long wait times, as well as how to engage users to improve resource requests. This work shows the importance of analyzing reasons for wait times, approaches to reduce wait times, and motivates further research into supporting users to improve resource requests and system utilization, and to reduce avoidable resource waste.

As we increasingly rely on digital technologies, cyber security is of paramount importance. While computing systems offer numerous advantages, they also introduce unwanted security risks. In High Performance Computing, security risks have largely been ignored in the name of high performance [1]. Nevertheless, ensuring security and privacy of computations and data is essential [2]. Linux operating systems, running on all Top500 HPC systems, use kernel and user modes to implement security, which prevents unauthorized access to critical kernel functions. System calls connect the two modes. Therefore, they were frequently attacked, as reported in over 100 Common Vulnerabilities and Exposures (CVEs) in the last 7 years [3]. Combining static and dynamic syscalls analysis [4] [5] has recently been shown to be imperative for creating a syscalls whitelist, to minimize the potential attack surface they introduce. This work introduces a novel dynamic analysis technique, Hot-n-Cold, to detect anomalies in the Linux commands’ behavior by monitoring the CPU temperature. We use Hot-n-Cold to map the syscall attack surface on a local HPC system. Hot-n-Cold can be extended and applied to detect, in real-time, an anomaly that may facilitate creation of an attack. The results on two Linux frequently used commands (ls & chmod) show a positive correlation of up to 80% between the original Linux command and a version augmented with syscalls from CVEs. This work shows the importance of security in HPC, and motivates further research into studying and designing security mechanisms that preserve high performance.

Applications running on HPC systems waste time and energy if they: (a) use resources inefficiently, (b) deviate from allocation purpose (e.g. cryptocurrency mining), or (c) encounter errors and failures. It is important to know which applications are running on the system, how they use the system, and whether they have been executed before. To recognize known applications during execution on a noisy system, we draw inspiration from the way Shazam recognizes known songs playing in a crowded bar. Our contribution is an Execution Fingerprint Dictionary (EFD) that stores execution fingerprints of system metrics (keys) linked to application and input size information (values) as key-value pairs for application recognition. Related work often relies on extensive system monitoring (many system metrics collected over large time windows) and employs machine learning methods to identify applications. Our solution only uses the first 2 minutes and a single system metric to achieve F-scores above 95 percent, providing comparable results to related work but with a fraction of the necessary data and a straightforward mechanism of recognition.