In scientific computing and data science disciplines, it is often necessary to share application workflows and repeat results. Current tools containerize application workflows, and share the resulting container for repeating results. These tools, due to containerization, do improve sharing of results. However, they do not improve the efficiency of replay. In this paper, we present the multiversion replay problem, which arises when multiple versions of an application are containerized, and each version must be replayed to repeat results. To avoid executing each version separately, we develop CHEX , which checkpoints program state and determines when it is permissible to reuse program state across versions. It does so using system call-based execution lineage. Our capability to identify common computations across versions enables us to consider optimizing replay using an in-memory cache, based on a checkpoint-restore-switch system. We show the multiversion replay problem is NP-hard, and propose efficient heuristics for it. CHEX reduces overall replay time by sharing common computations but avoids storing a large number of checkpoints. We demonstrate that CHEX maintains lightweight package sharing, and improves the total time of multiversion replay by 50% on average.

Data-intensive applications involving the analysis of large datasets often require large amounts of compute and storage resources, for which locality can be crucial to high throughput and performance. We propose a data approach that acquires compute and storage resources dynamically, replicates in response to demand, and schedules computations close to data. As demand increases, more resources are acquired, thus allowing faster response to subsequent requests that refer to the same data; when demand drops, resources are released. This approach can provide the benefits of dedicated hardware without the associated high costs, depending on workload and resource characteristics. To explore the feasibility of diffusion, we offer both a theoretical and an empirical analysis. We define an abstract model for diffusion, introduce new scheduling policies with heuristics to optimize real-world performance, and develop a competitive online cache eviction policy. We also offer many empirical experiments to explore the benefits of dynamically expanding and contracting resources based on load, to improve system responsiveness while keeping wasted resources small. We show performance improvements of one to two orders of magnitude across three diverse workloads when compared to the performance of parallel file systems with throughputs approaching 80 Gb/s on a modest cluster of 200 processors. We also compare diffusion with a best model for active storage, contrasting the difference between a pull-model found in diffusion and a push-model found in active storage.

The multi-period newsvendor problem describes the dilemma of a newspaper salesman—how many papers should he purchase each day to resell, when he doesn't know the demand? We develop approaches for this well known problem based on two machine learning algorithms: Weighted Majority of Warmuth and Littlestone, and Follow the Perturbed Leader of Kalai and Vempala. With some modified analysis, it isn't hard to show theoretical bounds for our modified versions of these algorithms. More importantly, we test the algorithms in a variety of simulated conditions, and compare the results to those given by traditional stochastic approaches which assume more information about the demands than is typically known. Our tests indicate that such online learning algorithms can perform well in comparison to stochastic approaches, even when the stochastic approaches are given perfect information.

In this paper we consider the k edge‐disjoint paths problem (k‐EDP), a generalization of the well‐known edge‐disjoint paths problem. Given a graph $G=(V,E)$ and a set of terminal pairs (or requests) T, the problem is to find a maximum subset of the pairs in T for which it is possible to select paths such that each pair is connected by k edge‐disjoint paths and the paths for different pairs are mutually disjoint. To the best of our knowledge, no nontrivial result is known for this problem for $k>1$. To measure the performance of our algorithms we use the recently introduced flow number F of a graph. This parameter is known to fulfill $F=O(\Delta \alpha^{-1} \log n)$, where $\Delta$ is the maximum degree, $\alpha$ is the edge expansion of G, and n is the number of vertices in G. We show that a simple greedy online algorithm achieves a competitive ratio of $O(k^3 F)$ which naturally extends the best known bound of $O(F)$ for $k=1$ to higher k. To achieve this competitive ratio, we introduce a new method of converting a system of k disjoint paths into a system of k length‐bounded disjoint paths. We also show that any deterministic online algorithm has a competitive ratio of $\Omega(k F)$. In addition, we study the k disjoint flows problem (k‐DFP), which is a generalization of the previously studied unsplittable flow problem. The difference between the k‐DFP and the k‐EDP is that now we consider a graph with edge capacities and our requests are allowed to have arbitrary demands $d_i$. The aim is to find a subset of requests of maximum total demand for which it is possible to select flow paths such that all the capacity constraints are maintained and each selected request with demand $d_i$ is connected by k disjoint paths, each of flow value $d_i/k$. The k‐EDP and k‐DFP problems have important applications in fault‐tolerant (virtual) circuit switching, which plays a key role in optical networks.

We present memory-efficient deterministic algorithms for constructing ϵ-nets and ϵ-approximations of streams of geometric data. Unlike probabilistic approaches, these deterministic samples provide guaranteed bounds on their approximation factors. We show how our deterministic samples can be used to answer approximate online iceberg geometric queries on data streams. We use these techniques to approximate several robust statistics of geometric data streams, including Tukey depth, simplicial depth, regression depth, the Thiel-Sen estimator, and the least median of squares. Our algorithms use only a polylogarithmic amount of memory, provided the desired approximation factors are at least inverse-polylogarithmic. We also include a lower bound for noniceberg geometric queries.

When exploring computing elements made from technologies other than complementary metal-oxide-semiconductor, it is imperative to investigate circuits and systems assuming realistic physical implementation constraints. This paper looks at molecular quantum-dot cellular automata (QCA) devices within this context. With molecular QCA, physical coplanar wire crossings may be very difficult to fabricate in the near to midterm. Here, we consider how this will affect interconnect. We introduce a novel technique to remove wire crossings in a given design in order to facilitate the self-assembly of real circuits - thus, providing meaningful and functional design targets for both physical and computer scientists. The proposed methodology eliminates all wire crossings with minimal logic gate/node duplications. Simulation results based on existing QCA circuits and other benchmarks are presented, and suggest that further investigation is needed.

Traditional discretization techniques for machine learning, from examples with continuous feature spaces, are not efficient when the data is in the form of a stream from an unknown, possibly changing, distribution. We present a time-and-memory-efficient discretization technique based on computing ε -approximate exponential frequency quantiles, and prove bounds on the worst-case error introduced in computing information entropy in data streams compared to an offline algorithm that has no efficiency constraints. We compare the empirical performance of the technique, using it for feature selection, with (streaming adaptations of) two popular methods of discretization, equal width binning and equal frequency binning, under a variety of streaming scenarios for real and artificial datasets. Our experiments show that ε -approximate exponential frequency quantiles are remarkably consistent in their performance, in contrast to the simple and efficient equal width binning that perform quite well when the streams are from stationary distributions, and quite poorly otherwise. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Many database applications that need to disseminate dynamic information from a server to various clients can suffer from heavy communication costs. Data caching at a client can help mitigate these costs, particularly when individual PUSH-PULL decisions are made for the different semantic regions in the data space. The server is responsible for notifying the client about updates in the PUSH regions. The client needs to contact the server for queries that ask for data in the PULL regions. We call the idea of partitioning the data space into PUSH-PULL regions to minimize communication cost data gerrymandering. In this paper, we present solutions to technical challenges in adopting this simple but powerful idea. We give a provably optimal-cost dynamic programming algorithm for gerrymandering on a single query attribute. We propose a family of efficient heuristics for gerrymandering on multiple query attributes. We handle the dynamic case in which the workloads of queries and updates evolve over time. We validate our methods through extensive experiments on real and synthetic data sets

When exploring computing elements made from technologies other than CMOS, it is imperative to investigate the effects of physical implementation constraints. This paper focuses on molecular quantum-dot cellular automata circuits. For these circuits, it is very difficult for chemists to fabricate wire crossings (at least in the near future). A novel technique is introduced to remove wire crossings in a given circuit to facilitate the self assembly of real circuits - thus providing meaningful and functional design targets for both physical and computer scientists. The technique eliminates all wire crossings with minimal logic gate/node duplications. Experimental results based on existing QCA circuits and other benchmarks are quite encouraging, and suggest that further investigation is needed.

Scientific database federations are geographically distributed and network bound. Thus, they could benefit from proxy caching. However, existing caching techniques are not suitable for their workloads, which compare and join large data sets. Existing techniques reduce parallelism by conducting distributed queries in a single cache and lose the data reduction benefits of performing selections at each database. We develop the bypass-yield formulation of caching, which reduces network traffic in wide-area database federations, while preserving parallelism and data reduction. Bypass-yield caching is altruistic; caches minimize the overall network traffic generated by the federation, rather than focusing on local performance. We present an adaptive, workload-driven algorithm for managing a bypass-yield cache. We also develop on-line algorithms that make no assumptions about workload: a k-competitive deterministic algorithm and a randomized algorithm with minimal space complexity. We verify the efficacy of bypass-yield caching by running workload traces collected from the Sloan Digital Sky Survey through a prototype implementation.

We present memory-efficient deterministic algorithms for constructing ∈-nets and ∈-approximations of streams of geometric data. Unlike probabilistic approaches, these deterministic samples provide guaranteed bounds on their approximation factors. We show how our deterministic samples can be used to answer approximate online iceberg geometric queries on data streams. We use these techniques to approximate several robust statistics of geometric data streams, including Tukey depth, simplicial depth, regression depth, the Thiel-Sen estimator, and the least median of squares. Our algorithms use only a polylogarithmic amount of memory, provided the desired approximation factors are inverse-polylogarithmic. We also include a lower bound for non-iceberg geometric queries.

In this paper we study the problem of how resilient networks are to node faults. Specifically, we investigate the question of how many faults a network can sustain so that it still contains a large (i.e. linear-sized) connected component that still has approximately the same expansion as the original fault-free network. For this we apply a pruning technique which culls away parts of the faulty network which have poor expansion. This technique can be applied to both adversarial faults and to random faults. For adversarial faults we prove that for every network with expansion α, a large connected component with basically the same expansion as the original network exists for up to a constant times α • n faults. This result is tight in the sense that every graph G of size n and uniform expansion α (•),i.e. G has an expansion of α (n) and every subgraph G' of size m of G has an expansion of O (α (m)), can be broken into sublinear components with w(α (n) • n) faults.For random faults we observe that the situation is significantly different. In this case the expansion of a graph only gives a very weak bound on its resilience to random faults. Specifically, there are networks of uniform expansion O(≾n) that are resilient against a constant fault probability but there are also networks of uniform expansion Ω(1/log n) that are not resilient against a O(1/log n) fault probability. Thus, a different parameter is needed. For this we introduce the span of a graph which allows us to determine the maximum fault probability in a much better way than the expansion can. We use the span to show the first known results for the effect of random faults on the expansion of d-dimensional meshes.

We deal with a generalization of the Minimum path colouring problem, k-Edge disjoint path systems colouring: given a graph G and a set of pairs of vertices of G, the task is to connect each pair by a system of k-edge disjoint paths (a k-system) and to colour the k-systems by minimal number of colours in such way that any two edge-intersecting k-systems have different colours. Multiple connecting paths between the same pair of vertices are motivated by a need for fault tolerant connections. We propose an O(k2 F) approximation algorithm for this problem where F is the flow number of the graph. As a byproduct of our analysis we also show that any two k-connected vertices in G are connected by k edge disjoint paths of average length O(k F) which improves previously known bounds for many classes of graphs.

In this paper we consider the k edge-disjoint paths problem (k-EDP), a generalization of the well-known edge-disjoint paths problem. Given a graph G=(V,E) and a set of terminal pairs (or requests) T, the problem is to find a maximum subset of the pairs in T for which it is possible to select paths such that each pair is connected by k edge-disjoint paths and the paths for different pairs are mutually disjoint. To the best of our knowledge, no nontrivial result is known for this problem for k>1. To measure the performance of our algorithms we will use the recently introduced flow number F of a graph. This parameter is known to satisfy F=O(\Delta \alpha^-1 \log n), where \Delta is the maximum degree and \alpha is the edge expansion of G. We show that a simple, greedy online algorithm achieves a competitive ratio of O(k^3 \cdot F) which naturally extends the best known bound of O(F) for k=1 to higher $k$. To get this bound, we introduce a new method of converting a system of k disjoint paths into a system of k length-bounded disjoint paths. Also, an almost matching deterministic online lower bound \Omega(k \cdot F) is given.In addition, we study the k disjoint flows problem (k-DFP), which is a generalization of the well-known unsplittable flow problem (UFP). The k-DFP is similar to the k-EDP with the difference that we now consider a graph with edge capacities and the requests can have arbitrary demands d_i. The aim is to find a subset of requests of maximum total demand for which it is possible to select flow paths such that all the capacity constraints are maintained and each selected request with demand d_i is connected by k disjoint paths, each of flow value d_i/k.The k-EDP and k-DFP problems have important applications in fault-tolerant (virtual) circuit switching which plays a key role in optical networks.