Feature engineering is a crucial but tedious task that requires up to 80% of the total time in data science projects. A significant challenge is when data consists of tables from different data sources, thus data scientists need to wisely aggregate and join tables while performing feature engineering task. In this work, we demonstrate a novel system called OneBM (One Button Machine), that enables data scientists to increase their efficiency with automated feature engineering for relational data. OneBM takes as input a relational dataset with multiple tables and its entity relation diagram (ERD) which can be declared with a novel, easy-to-use drag-and-drop graphical user interface. The system then automatically identifies and executes relevant joins and aggregates in the data, and generates new features with a rich set of transformations for various types of data including but not limited to time-series, sequences, number sets and itemsets, etc. The generated features then can be used by automated model selection and hyper-parameter optimization algorithms to complete a fully end-to-end automated data science (or AutoDS) workflow. A follow-up user evaluation illustrated how data scientists can perform multi-table feature engineering tasks in minutes using our system, compared to repeatedly coding SQL-like queries to transform and aggregate relational data requiring weeks of manual labor for comparable performance. In the live demos we plan to show two use cases with real-world datasets (video demos are available at the links in the footnote): sale prediction1and call center user experience2. Pre-registered partcipants can play with these use-cases and the given datasets via Watson Studio on the cloud.

Workload modeling and performance evaluation play crucial roles in the study of scheduling algorithms on large-scale parallel and distributed systems. An effective design of a scheduling algorithm for these systems requires experiments with hundreds of simulations to evaluate its performance. Since each simulation needs one workload as input, only real workloads with usually a limited availability are not sufficient, and so representative workload models are needed. Several studies have shown that realistic workload characteristics such as burstiness, bag-of-tasks, etc., cause significant performance impacts on scheduling. Therefore, we argue that realistic workload models should contain as many characteristics of real workloads as possible. In practice, researchers use unrealistic workloads in their scheduling evaluations because they lack models that can help generate realistic workloads. In this article, we analyze real parallel workloads to show the presence of important characteristics including long range dependence, periodicity and temporal burstiness of job arrivals, bag-of-tasks behavior, and correlation of runtime and number of processors. Then, we present a systematic approach to create a complete model that contains all of these characteristics. Validation of our model with real world data shows that it does not only capture the above characteristics, but also can fit marginal distributions well.

Parallel workloads in practice are far from being randomly distributed, instead they are highly repetitive because users tend to run the same applications over and over again. We refer to this phenomenon as temporal locality. In addition, the workloads exhibit a correlation between runtime and parallelism (i.e., number of processors) as is analysed in this paper. According to our best knowledge, there are very few studies on the impacts of these features on the performance of parallel systems. Since these impacts are not well known, researchers often evaluate scheduling algorithms with random workloads, which neglect the phenomenon of temporal locality and the correlation. This can result in an inaccurate scheduling evaluation for parallel systems, because our study shows that these two features can significantly affect scheduling performance. In our simulation-based experiments, an increase of the correlation can quickly degrade the parallel system performance and can change the result of comparing different scheduling policies. With respect to temporal locality, we indicate that this feature does not always seriously affect schedulers of parallel systems. Instead in particular situations, it can help to improve scheduling performance. Furthermore, we also discuss in this paper the necessity of using workloads with these features in scheduling evaluation as well as how to utilize the features to enhance the performance of schedulers.

The Bag-of-Tasks (BoT) behaviour has recently drawn the attention of scheduling researchers [2, 36, 37] and seems to be very common in workloads of parallel systems (up to 70% of jobs [27]) and grids (up to 96% of the total CPU time is consumed by BoTs [11]). To enable a reliable evaluation of BoT-oriented scheduling algorithms, researchers require realistic workload models that take BoTs into account. Regrettably, very few such models are available in the liturature. To our best knowledge, there are only two studies on modeling that incorporate BoTs into their models to generate synthetic workloads for parallel systems [27] and grids [12]. However, these models only focus on fitting the marginal distributions and neglect several other statistical properties of BoTs such as periodicity, autocorrelation and cross-correlation among BoT attributes. We believe that these crucial characteristics deserve to be taken into account in modeling research. Therefore in this paper, we will focus on characterising the BoT behaviour to further improve researchers' understanding of this well-known behaviour in parallel system workloads. In addition, we also study how BoTs affect parallel system performance. Our experimental results indicate that the presence of BoTs leads to a considerable performance degradation, but it is interesting that a realistic association between job arrivals and job runtimes helps BoTs to improve the performance of parallel systems. Moreover, we also show the necessity of using workloads with BoTs in scheduling evaluation to obtain reliable results.

Performance evaluation is a significant step in the study of scheduling algorithms in large-scale parallel systems ranging from supercomputers to clusters and grids. One of the key factors that have a strong effect on the evaluation results is the workloads (or traces) used in experiments. In practice, several researchers use unrealistic synthetic workloads in their scheduling evaluations because they lack models that can help generate realistic synthetic workloads. In this paper we propose a full model to capture the following characteristics of real parallel system workloads: 1) long range dependence in the job arrival process, 2) temporal and spatial burstiness, 3) bag-oftasks behaviour, and 4) correlation between the runtime and the number of processors. Validation of our model with real traces shows that our model not only captures the above characteristics but also fits the marginal distributions well. In addition, we also present an approach to quantify burstiness in a job arrival process (temporal) as well as burstiness in the load of a trace (spatial).

We present in this paper a novel method to predict application runtimes on backfilling parallel systems. The method is based on mining historical data to obtain important parameters. These parameters are then applied to predict the runtime of future applications. It has been shown in previous works that both underestimate and inaccuracy in prediction have adverse impacts on scheduling performance of backfilling systems. In our study, we try to reduce the number of jobs that are underestimated and reduce the prediction error as much as possible. Comparing with other predictors, experimental results show that our predictor is up to 25% better with respect to the problem of underestimate. Moreover, using the metric proposed in for the accuracy, our predictor improves up to 32%.

Workload modeling plays a significant role in performance evaluation of large-scale parallel systems such as clusters and grids. It helps to generate synthetic workloads which capture some dominant characteristics of traces (real workloads). Modeling job arrival process is an essential part of workload modeling. Although a job arrival process has many important characteristics such as long range dependence (LRD) and burstiness, most researchers, for simplicity, assume it as a Poisson process in their evaluation work. Furthermore, there is currently almost no research focusing on both LRD and burstiness at the same time according to our investigation. With respect to this research trend, the multifractal wavelet model (MWM) recently has been introduced as a good choice to yield LRD for a job arrival process. Though LRD is well controlled, we observe that a job arrival process produced by MWM does not keep burstiness. In this paper, we present our study on modifying MWM so that not only LRD but also burstiness are kept in the job arrival process. In addition, our modification also fits the marginal distribution better than MWM.