Abstract Blockchain infrastructures have emerged as a disruptive technology and have led to the realization of cryptocurrencies (peer‐to‐peer payment systems) and smart contracts. They can have a wide range of application areas in e‐Science due to their open, public nature and global accessability in a trustless manner. We propose and implement a smart contract called eBlocBroker, which is an autonomous blockchain‐based middleware system for volunteer computing and providing data resources for e‐Science. The eBlocBroker infrastructure connects requesters who need to combine applications (jobs) with datasets and run them via an Ethereum‐based private blockchain network (Bloxberg) on providers that utilize computational and data resources on clouds or home servers. It uses cloud storage, such as B2DROP, IPFS, or Google Drive, to store and transfer data between requesters and providers. Each provider utilizes the Slurm workload manager to execute jobs submitted through eBlocBroker. In this paper, we demonstrate how an autonomous organization programmed as a smart contract can be used to deploy a marketplace that supports data and computation‐intensive research projects. We propose a cost model implemented as a function in the smart contract which calculates and records computation and dataset usage costs. We develop a Python‐based system to communicate with eBlocBroker and orchestrate jobs' execution on the provider's end. We present eBlocBroker's features, infrastructure, implementation, algorithms and experimental results.

Abstract In this work, we contribute a parallel implementation of the network simplex algorithm that is used for the solution of minimum cost flow problem. In the network simplex algorithm, finding an entering arc requires searching through many arcs to decide which one should be included in the spanning tree solution on the next iteration. We propose finding the entering arc in parallel as it often takes the majority of the execution time. A usual strategy is to pick the arc violating the optimality the most out of all possible candidates. Scanning all arcs can take quite some time, so it is common to consider only a fixed number of arcs which is referred as the block search pivoting rule. Arc scans can easily be done in parallel to find the best candidate as the calculations are independent of each other. We used shared memory parallelism using OpenMP along with vectorization using AVX instructions. We also tried adjusting block sizes to increase the parallel portion of the algorithm. Our dataset consists of various natural and synthetic graphs with sizes up to a billion arc. Our experiments show speedups up to four are possible, though they are typically lower.

The amount of data generated by social media, social networks and distributed platforms such as blockchain, have reached quite high levels. Various data analysis methods could be applied this big data. One of these methods is to classify geo-tagged social network data in order to report geographical area associated with the data. We propose an efficient parallel classification approach and implement a classifier tool which is capable of processing huge amount of data. To test our approach, we collect Twitter data over five densest areas of Turkey. There are important factors affecting the classification performance such as the spatial indexing and the parallelization strategies. Hierarchical Triangular Mesh (HTM) and R-Tree spatial indexes are used for indexing regions. For parallel processing data streams classifier tool is implemented based on Apache Spark and Kafka platforms in order to obtain high scalability. To show effectiveness of our method, we perform tests on Amazon Web Services (AWS) Cloud environment and compare our method against a method which implements HTM on a Microsoft SQL Server. Results show that 1.6 - 4.5 fold speed-up is obtained and Twitter data that is collected over a month can be processed effectively in three hours.

The maximum flow problem is one of the most common network flow problems. This problem involves finding the maximum possible amount of flow between two designated nodes on a network with arcs having flow capacities. The push-relabel algorithm is one of the fastest algorithms to solve this problem. We present a shared memory parallel push-relabel algorithm. Graph coloring is used to avoid collisions between threads for concurrent push and relabel operations. In addition, excess values of target nodes are updated using atomic instructions to prevent race conditions. The experiments show that our algorithm is competitive for wide graphs with low diameters. Results from three different data sets are included, computer vision problems, DIMACS challenge problems, and KaHIP partitioning problems. These are compared with existing push-relabel and pseudoflow implementations. We show that high speedup rates are possible using our coloring based parallelization technique on sparse networks. However, we also observe that the pseudoflow algorithm runs faster than the push-relabel algorithm on dense and long networks.

The emerging blockchain technologies have enabled development of crypto-currencies and autonomous smart contracts that can operate in decentralized and trustless settings. Distributed autonomous organizations can be implemented using smart contracts available on the Ethereum blockchain. In this paper, we propose a distributed autonomous software organization model and its Ethereum smart contract implementation called AutonomousSoftwareOrg for providing a continuously operating virtual organization for software development communities and users. AutonomousSoftwareOrg facilitates a funding mechanism based on crypto-currencies, a decision making mechanism based on voting and record keeping for software usage citations and executions. AutonomousSoftwareOrg is deployed and tested on our local Ethereum based blockchain system (http://ebloc.cmpe.boun.edu.tr). Its Solidity language source code is available at https://github.com/ebloc/AutonomousSoftwareOrg.

Summary Slurm resource management system is used on many TOP500 supercomputers. We present a new auction‐based heterogeneous cluster scheduler plug‐in called AUCSCHED2. AUCSCHED2 contributes two major enhancements: the first is the extension of Slurm to support generic resource moldability by specification of resource ranges. The generic resources include accelerators like graphics processing unit or Xeon Phi. The current version of Slurm supports specification of node ranges but not of generic resource ranges. Such a feature can be very useful to run‐time auto‐tuning applications and systems that can make use of variable number of generic resources. The second enhancement involves the implementation of a new integer programming formulation in AUCSCHED2 that drastically reduces the number of variables. This allows faster solution and larger number of bids to be generated. Slurm emulation results are presented for the heterogeneous 1408 node Tsubame supercomputer, which has 12 cores and three graphics processing units on each of its nodes. AUCSCHED2 is available at https://github.com/aucsched/aucsched2 . Copyright © 2015 John Wiley & Sons, Ltd.

This paper presents Fast Instruction SyntHesis (FISH), a system that supports automatic generation of custom instruction processors from high-level application descriptions to enable fast design space exploration. FISH is based on novel methods for automatically adapting the instruction set to match an application in a high-level language such as C or C++. FISH identifies custom instruction candidates using two approaches: 1) by enumerating maximal convex subgraphs of application data flow graphs and 2) by integer linear programming (ILP). The experiments, involving ten multimedia and cryptography benchmarks, show that our contributed algorithms are the fastest among the state-of-the-art techniques. In most cases, enumeration takes only milliseconds to execute. The longest enumeration run-time observed is less than six seconds. ILP is usually slower than enumeration, but provides us with a complementary solution technique. Both enumeration and ILP allow the use of multiple different merit functions in the evaluation of data-flow subgraphs. The experiments demonstrate that, using only modest additional hardware resources, up to 30-fold performance improvement can be obtained with respect to a single-issue base processor.

Automatic generation of custom instruction processors from high-level application descriptions enables fast design space exploration, while offering very favorable performance and silicon area combinations. This work introduces a novel method for adapting the instruction set to match an application captured in a high-level language. A simplified model is used to find the optimal instructions via enumeration of maximal convex subgraphs of application data flow graphs (DFGs). Our experiments involving a set of multimedia and cryptography benchmarks show that an order of magnitude performance improvement can be achieved using only a limited amount of hardware resources. In most cases, our algorithm takes less than a second to execute.

This paper describes an integer-linear-programming (ILP)-based system called custom hardware instruction processor synthesis (CHIPS) that identifies custom instructions for critical code segments, given the available data bandwidth and transfer latencies between custom logic and a baseline processor with architecturally visible state registers. Our approach enables designers to optionally constrain the number of input and output operands for custom instructions. We describe a design flow to identify promising area, performance, and code-size tradeoffs. We study the effect of input/output constraints, register-file ports, and compiler transformations such as if-conversion. Our experiments show that, in most cases, the solutions with the highest performance are identified when the input/output constraints are removed. However, input/output constraints help our algorithms identify frequently used code segments, reducing the overall area overhead. Results for 11 benchmarks covering cryptography and multimedia are shown, with speed-ups between 1.7 and 6.6 times, code-size reductions between 6% and 72%, and area costs ranging between 12 and 256 adders for maximum speed-up. Our ILP-based approach scales well: benchmarks with basic blocks consisting of more than 1000 instructions can be optimally solved, most of the time within a few seconds.

The authors present a methodology for generating optimized architectures for data bandwidth constrained extensible processors. The authors describe a scalable integer linear programming (ILP) formulation, that extracts the most profitable set of instruction-set extensions given the available data bandwidth and transfer latency. Unlike previous approaches, the authors differentiate between number of inputs and outputs for instruction-set extensions and the number of register file ports. This differentiation makes the approach applicable to architectures that include architecturally visible state registers and dedicated data transfer channels. The authors support a comprehensive design space exploration to characterize the area/performance trade-offs for various applications. The authors evaluate our approach using actual ASIC implementations to demonstrate that our automatically customized processors meet timing within the target silicon area. For an embedded processor with only two register read ports and one register write port, the authors obtain up to 4.3times speed-up with extensions incurring only a 35% area overhead