SUMMARY: The classification of biological samples by means of their respective molecular profiles is a topic of great interest for its potential diagnostic, prognostic and investigational applications. rScudo is an R package for the classification of molecular profiles based on a radically new approach consisting in the analysis of the similarity of rank-based sample-specific signatures. The validity of rScudo unconventional approach has been validated through direct comparison with current methods in the international SBV IMPROVER Diagnostic Signature Challenge. Due to its novelty, there is ample room for conceptual improvements and for exploring additional applications. The rScudo package has been specifically designed to facilitate experimenting with the rank-based signature approach, to test its application to different types of molecular profiles and to simplify direct comparison with existing methods. AVAILABILITY AND IMPLEMENTATION: The package is available as part of the Bioconductor suite at https://bioconductor.org/packages/rScudo.

MOTIVATION: After more than a decade since microarrays were used to predict phenotype of biological samples, real-life applications for disease screening and identification of patients who would best benefit from treatment are still emerging. The interest of the scientific community in identifying best approaches to develop such prediction models was reaffirmed in a competition style international collaboration called IMPROVER Diagnostic Signature Challenge whose results we describe herein. RESULTS: Fifty-four teams used public data to develop prediction models in four disease areas including multiple sclerosis, lung cancer, psoriasis and chronic obstructive pulmonary disease, and made predictions on blinded new data that we generated. Teams were scored using three metrics that captured various aspects of the quality of predictions, and best performers were awarded. This article presents the challenge results and introduces to the community the approaches of the best overall three performers, as well as an R package that implements the approach of the best overall team. The analyses of model performance data submitted in the challenge as well as additional simulations that we have performed revealed that (i) the quality of predictions depends more on the disease endpoint than on the particular approaches used in the challenge; (ii) the most important modeling factor (e.g. data preprocessing, feature selection and classifier type) is problem dependent; and (iii) for optimal results datasets and methods have to be carefully matched. Biomedical factors such as the disease severity and confidence in diagnostic were found to be associated with the misclassification rates across the different teams. AVAILABILITY: The lung cancer dataset is available from Gene Expression Omnibus (accession, GSE43580). The maPredictDSC R package implementing the approach of the best overall team is available at www.bioconductor.org or http://bioinformaticsprb.med.wayne.edu/.

MOTIVATION: Sequence alignment is one of the most popular tools of modern biology. NCBI's PSI-BLAST utilizes iterative model building in order to better detect distant homologs with greater sensitivity than non-iterative BLAST. However, PSI-BLAST's performance is limited by the fact that it relies on deterministic alignments. Using a semi-probabilistic alignment scheme such as Hybrid alignment should allow for better informed model building and improved identification of homologous sequences, particularly remote homologs. RESULTS: We have built a new version of the tool in which the Smith-Waterman alignment algorithm core is replaced by the hybrid alignment algorithm. The favorable statistical properties of the hybrid algorithm allow the introduction of position-specific gap penalties in Hybrid PSI-BLAST. This improves the position-specific modeling of protein families and results in an overall improvement of performance. AVAILABILITY: Source code is freely available for download at http://bioserv.mps.ohio-state.edu/HybridPSI, implemented in C and supported on linux.

In previous works we have proposed to use of self-organization based on emergent design as a model for the programming of very large aggregates of heterogeneous computing resources. In our approach, a large scale computation is divided into small independent units of computation, each provided with its own uniform, autonomous behavior; only local information is used by each unit of computation to take all the decisions needed to carry out the computation. One of the challenges of this novel approach is to provide some theoretical foundation that can assist in the rational design of new systems. In this paper is to demonstrate the use of combinatorial techniques for obtaining quantitative analytical models of the organization pattern emerging from a specific type of self-organizing computation. Specifically, in a previous experiment we have demonstrated a computation in which mobile agents organize themselves around an overlay tree that constantly restructures itself in response to changing node availability and performance levels. In this paper we derive an analytical expression describing how nodes distribute themselves over the tree based on their performance, in a simplified version of the above problem. This result represents an instance of a theoretical tool that can be used to predict global patterns emerging as a result of a self-organizing design, and to establish a direct connection between global features and local behavior parameters.

An increasing number of scientific applications need efficient access to large datasets held at remote storage facilities. However, despite the availability of high-speed Internet backbones, the performance penalty of remote I/O is still relatively high compared to local I/O. In this paper we describe different ways in which asynchronous primitives can be used to improve the performance of remote I/O in the grid environment. We have implemented and evaluated three optimization techniques using asynchronous primitives. These primitives have been integrated into SEMPLAR, a high-performance, remote I/O library based on the SDSC storage resource broker. Based on measurements of representative high-performance applications running on three different clusters, we show that different optimization techniques work best for each specific combination of application and platform characteristics. We achieved over 90% overlap between the computation and I/O phase of two applications by using an asynchronous version of remote I/O primitives. We were able to increase the average read and write bandwidth of the ROMIO perf benchmark by 96% and 43% respectively by moving data concurrently over multiple remote connections. Finally, we experienced an improvement of up to 84% in the average write bandwidth when using asynchronous, on-the-fly data compression

PSI-BLAST remains one of the popular tools for searching remote homologs in sequence databases. We recently demonstrated that hybrid alignment can function as the alignment core for PSI-BLAST without loss of sensitivity. Here, we start to exploit the benefits of hybrid alignment. We show that incorporating information about the suboptimal alignments, otherwise ignored in PSI-BLAST, already improves the sensitivity of our enhanced version of PSI-BLAST. More interestingly, we find a set of sequences on which our tool disagrees with the classification given by SCOP. Careful examination points to a possible misclassification in SCOP. Cross-referencing with two other methods of protein structure classification, CATH and DALI, supports this view, indicating that the enriched information from suboptimal alignments is valuable for detecting more weakly homologous sequences.

One of the challenges in high-performance computing is to provide users with reliable, remote data access in a distributed, heterogeneous environment. The increasing popularity of high-speed wide area networks and centralized data repositories lead to the possibility of direct high-speed access to remote data sets from within a parallel application. In this paper, we describe SEMPLAR, a library for remote, parallel I/O that combines the standard programming interface of MPI-10 with the remote storage functionality of the SDSC storage resource broker (SRB). SEMPLAR relies on parallel TCP streams to maximize the remote data throughput in a design that preserves the parallelism of the access all the way from the storage to the application. We have provided I/O performance results for a high-performance computing workload on three different clusters. On the NCSA TeraGrid cluster, the ROMIO perf benchmark attained an aggregate read bandwidth of 291 Mbps with 18 processors. The NAS btio benchmark achieved an aggregate write bandwidth of 74 Mbps with 16 processors. The benchmark results are encouraging and show that SEMPLAR provides applications with scalable, high-bandwidth I/O across wide area networks.

Desktop grids have been used to perform some of the largest computations in the world and have the potential to grow by several more orders of magnitude. However, current approaches to utilizing desktop resources require either centralized servers or extensive knowledge of the underlying system, limiting their scalability. We propose a new design for desktop grids that relies on a self-organizing, fully decentralized approach to the organization of the computation. Our approach, called the organic grid, is a radical departure from current approaches and is modeled after the way complex biological systems organize themselves. Similar to current desktop grids, a large computational task is broken down into sufficiently small subtasks. Each subtask is encapsulated into a mobile agent, which is then released on the grid and discovers computational resources using autonomous behavior. In the process of "colonization" of available resources, the judicious design of the agent behavior produces the emergence of crucial properties of the computation that can be tailored to specific classes of applications. We demonstrate this concept with a reduced-scale proof-of-concept implementation that executes a data-intensive independent-task application on a set of heterogeneous, geographically distributed machines. We present a detailed exploration of the design space of our system and a performance evaluation of our implementation using metrics appropriate for assessing self-organizing desktop grids.

Summary form only given. We propose a biologically inspired and fully-decentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. Our approach achieves the following design objectives: near-zero knowledge of network topology, zero knowledge of system status, autonomous scheduling, distributed computation, lack of specialized nodes. Every node is equally responsible for scheduling and computation, both of which are performed with practically no information about the system. We believe that this model is ideally suited for large-scale unstructured grids such as desktop grids. This model avoids the extensive system knowledge requirements of traditional grid scheduling approaches. Contrary to the popular master/worker organization of current desktop grids, our approach does not rely on specialized super-servers or on application-specific clients. By encapsulating computation and scheduling behavior into mobile agents, we decouple both application code and scheduling functionality from the underlying infrastructure. The resulting system is one where every node can start a large grid job, and where the computation naturally organizes itself around available resources. Through the careful design of agent behavior, the resulting global organization of the computation can be customized for different classes of applications. In a previous paper, we described a proof-of-concept prototype for an independent task application. We generalize the scheduling framework and demonstrate that our approach is applicable to a computation with a highly synchronous communication pattern, namely Cannon's matrix multiplication.

Striping data across multiple nodes has been recognized as an effective technique for delivering high-bandwidth I/O to applications running on clusters. However the technique is vulnerable to disk failure. We present an I/O architecture for clusters called reliable array of autonomous controllers (RAAC) that builds on the technique of RAID style data redundancy. The RAAC architecture uses a two-tier layout that enables the system to scale in terms of storage capacity and transfer bandwidth while avoiding the synchronization overhead incurred in a distributed RAID system. We describe our implementation of RAAC in PVFS, and compare the performance of parity-based redundancy in RAAC and in a conventional distributed RAID architecture.

Abstract Progressive sequence model refinement by means of iterative searches is an effective technique for high sensitivity database searches and is currently employed in popular tools such as PSI‐BLAST and SAM. Recently, a novel alignment algorithm has been proposed that offers features expected to improve the sensitivity of such iterative approaches, specifically a well‐characterized theory of its statistics even in the presence of position‐specific gap costs. Here, we demonstrate that the new hybrid alignment algorithm is ready to be used as the alignment core of PSI‐BLAST. In addition, we evaluate the accuracy of two proposed approaches to edge effect correction in short sequence alignment statistics that turns out to be one of the crucial issues in developing a hybrid‐alignment based version of PSI‐BLAST. Copyright © 2004 John Wiley & Sons, Ltd.

An increasing number of online applications operate on data from disparate, and often wide-spread, data sources. This paper studies the design of a system for the automated monitoring of on-line data sources. In this system a number of ad-hoc data warehouses, which maintain client-specified views, are interposed between clients and data sources. We present a model of coherence, referred to here as slacker coherence, to address the freshness problem in the context of pull-based protocols. We experimentally examine various techniques for estimating update rates and polling adaptively. We also look at the impact on the coherence model performance of the request scheduling algorithm at the source.

A known issue in the design of striped file systems is their vulnerability to disk failures. In this paper we address the challenges of augmenting an existing striped file system with traditional RAID redundancy, and proposed a novel redundancy scheme designed to maximize disk throughput seen by applications. We implement our new scheme in CSAR, a proof-of-concept redundant file system based on PVFS, along with two other well known schemes. Our tests using both microbenchmarks and representative scientific applications show that our scheme consistently performs as well as the best of the other two schemes. The application-dependent, potentially larger storage occupation of our scheme is justified by current technological trends that put I/O bandwidth at a premium over disk space.

As data and computational grids grow in size and complexity, the crucial task of identifying, monitoring and utilizing available resources in an efficient manner is becoming increasingly difficult. The design of monitoring systems that are scalable both in the number of sources being monitored and in the number of clients served is a challenging issue. In this paper we investigate the trade-offs of different polling strategies that can be used to monitor resource availability on machines in a distributed environment. We show how adaptive polling protocols can substantially increase scalability with a less than proportional loss of precision, and how these protocols can be personalized for different types of resource usage patterns.

Striped file systems such as the parallel virtual file system (PVFS) deliver high-bandwidth I/O to applications running on clusters. An open problem in the design of striped file systems is how to reduce their vulnerability to disk failures with the minimum performance penalty. We describe a novel data redundancy scheme designed specifically to address the performance issue. We demonstrate the new scheme within CSAR, a proof-of-concept implementation based on PVFS. By dynamically switching between RAID1 and RAID5 redundancy based on write size, CSAR consistently achieves the best of two worlds & RAID1 performance on small writes, and RAID5 efficiency on large writes. Using the popular parallel I/O benchmark BTIO, our scheme achieves 82% of the write bandwidth of the unmodified PVFS. We describe the issues in implementing our new scheme in a popular striped file system such as PVFS on a Linux cluster

An essential building block for any data grid infrastructure is the storage server. In this paper we describe a high-performance cluster storage server built around the SDSC Storage Resource Broker (SRB) and commodity workstations. A number of performance critical design issues and our solutions to them are described. We incorporate pipeline optimizations into SRB to enable the full overlapping of communication and disk I/O. With these optimizations we were able to deliver to the application more than 95% of the disk throughput achievable through a remote connection. Then we show how our approach to network-striped transport is effective in achieving aggregate cluster-to-cluster throughput which scales with the number of connections. Finally, we present a federated SRB service over MPI that allows fast TCP connections to stripe data across multiple server disks reaching 97% of the combined write capacity of multiple nodes.

The authors describe their experience designing, implementing, and evaluating two generations of the high performance communication library, Fast Messages (FM) for Myrinet. In FM 1.x, they designed a simple interface and provided guarantees of reliable and in-order delivery, and flow control. While this was a significant improvement over previous systems, it was not enough. Layering MPI atop FM 1.x showed that only about 20% of the FM 1.x bandwidth could be delivered to higher level communication APIs. The second generation communication layer, FM 2.0, addresses the identified problems, providing gather-scatter, interlayer scheduling, receiver flow control, as well as some convenient API features which simplify programming. FM 2.x can deliver 70-90% to higher level APIs such as MPI. This is especially impressive as the absolute bandwidths delivered have increased nearly fourfold to 70 MB/s. They describe general issues encountered in matching two communication layers, and the solutions as embodied in FM 2.x.