We introduce Ensemble, a runtime framework and associated tools for building application performance models on-the-fly. These dynamic performance models can be used to support complex, highly dimensional resource allocation, and/or what-if performance inquiry in modern heterogeneous environments, such as data centers and Clouds. Ensemble combines simple, partially specified, and lower-dimensionality models to provide good initial approximations for higher dimensionality application performance models. We evaluated Ensemble on industry-standard and scientific applications. The results show that Ensemble provides accurate, fast, and flexible performance models even in the presence of significant environment variability.

We introduce Stage-aware Anomaly Detection (SAAD), a low-overhead real-time solution for detecting runtime anomalies in storage systems. Modern storage server architectures are multi-threaded and structured as a set of modules, which we call stages. SAAD leverages this to collect stage-level log summaries at runtime and to perform statistical analysis across stage instances. Stages that generate rare execution flows and/or register unusually high duration for regular flows at run-time indicate anomalies. SAAD makes two key contributions: i) limits the search space for root causes, by pinpointing specific anomalous code stages, and ii) reduces compute and storage requirements for log analysis, while preserving accuracy, through a novel technique based on log summarization. We evaluate SAAD on three distributed storage systems: HBase, Hadoop Distributed File System (HDFS), and Cassandra. We show that, with practically zero overhead, we uncover various anomalies in real-time.

As modern multi-tier systems are becoming increasingly large and complex, it becomes more difficult for system analysts to understand the overall behavior of the system, and diagnose performance problems. To assist analysts inspect performance behavior, we introduce SelfTalk, a novel declarative language that allows analysts to query and understand the status of a large scale system. SelfTalk is sufficiently expressive to encode an analyst's high-level hypotheses about system invariants, normal correlations between system metrics, or other a priori derived performance models, such as, "I expect that the throughputs of interconnected system components are linearly correlated". Given a hypothesis, Dena, our runtime support system, instantiates and validates it using actual monitoring data within specific system configurations. We evaluate SelfTalk/Dena by posing several hypotheses about system behavior and querying Dena to validate system behavior in a multi-tier dynamic content server. We find that Dena automatically validates the system performance based on the pre-existing hypotheses and helps to diagnose system misbehavior.