The MapReduce programming model, introduced by Google, has become popular over the past few years as a mechanism for processing large amounts of data, using shared-nothing parallelism. In this paper, we investigate the use of MapReduce technology for a local gridding algorithm for the generation of Digital Elevation Models (DEM). The local gridding algorithm utilizes the elevation information from LIDAR (Light, Detection, and Ranging) measurements contained within a circular search area to compute the elevation of each grid cell. The method is data parallel, lending itself to implementation using the MapReduce model. Here, we compare our initial C++ implementation of the gridding algorithm to a MapReduce-based implementation, and present observations on the performance (in particular, price/performance) and the implementation complexity. We also discuss the applicability of MapReduce technologies for related applications.

Screening for tuberculosis (TB) in low- and middle-income countries is centered on the microscope. We present methods for the automated identification of Mycobacterium tuberculosis in images of Ziehl-Neelsen (ZN) stained sputum smears obtained using a bright-field microscope. We segment candidate bacillus objects using a combination of two-class pixel classifiers. The algorithm produces results that agree well with manual segmentations, as judged by the Hausdorff distance and the modified Williams index. The extraction of geometric-transformation-invariant features and optimization of the feature set by feature subset selection and Fisher transformation follow. Finally, different two-class object classifiers are compared. The sensitivity and specificity of all tested classifiers is above 95% for the identification of bacillus objects represented by Fisher-transformed features. Our results may be used to reduce technician involvement in screening for TB, and would be particularly useful in laboratories in countries with a high burden of TB, where, typically, ZN rather than auramine staining of sputum smears is the method of choice.

The avian flu grid is a virtual organization dedicated to the discovery of novel inhibitors for the pandemic avian flu threat, leveraging grid technologies and computational resources provided by PRAGMA and its partners. In this context it is essential to adopt tools which increase the productivity of the computational scientists without advanced training on grid technologies. To reduce the learning curve, we have augmented tools that most domain scientists are already familiar with, and hidden the complexity of the underlaying infrastructure, through automatic user interface generation and workflow support, beyond the standard command line based approach. Here we describe the current state of the infrastructure deployed and how it has facilitated the training of new researchers in the drug discovery area. We provide details on Vision, a visual programming environment used to define scientific workflows, and Opal, an automatic Web service wrappers for scientific applications on Grid resources, and how they are integrated with other established back-end technologies.

Privacy-preserving data mining (PPDM) is an emergent research area that addresses the incorporation of privacy preserving concerns to data mining techniques. In this paper we propose a privacy-preserving (PP) Cox model for survival analysis, and consider a real clinical setting where the data is horizontally distributed among different institutions. The proposed model is based on linearly projecting the data to a lower dimensional space through an optimal mapping obtained by solving a linear programming problem. Our approach differs from the commonly used random projection approach since it instead finds a projection that is optimal at preserving the properties of the data that are important for the specific problem at hand. Since our proposed approach produces an sparse mapping, it also generates a PP mapping that not only projects the data to a lower dimensional space but it also depends on a smaller subset of the original features (it provides explicit feature selection). Real data from several European healthcare institutions are used to test our model for survival prediction of non-small-cell lung cancer patients. These results are also confirmed using publicly available benchmark datasets. Our experimental results show that we are able to achieve a near-optimal performance without directly sharing the data across different data sources. This model makes it possible to conduct large-scale multi-centric survival analysis without violating privacy-preserving requirements.

Over the past several years, with the advent of the open grid services architecture (OGSA) and the Web services resource framework (WSRF), service-oriented architectures (SOA) and Web service technologies have been embraced in the field of scientific and grid computing. These new principles promise to help make scientific infrastructures simpler to use, more cost effective to implement, and easier to maintain. However, understanding how to leverage these developments to actually design and build a system remains more of an art than a science. In this paper, we present some positions learned through experience that provide guidance in leveraging SOA technologies to build scientific infrastructures. In addition, we present the technical challenges that need to be addressed in building an SOA, and as a case study, we present the SOA that we have designed for the national biomedical computation resource (NBCR) community. We discuss how we have addressed these technical challenges, and present the overall architecture, the individual software toolkits developed, the client interfaces, and the usage scenarios. We hope that our experiences prove to be useful in building similar infrastructures for other scientific applications.

In biology and medical sciences, highly parallel biological assays spurred a revolution leading to the emergence of the '-omics' era. Dimensionality reduction techniques are necessary to be able to analyze, interpret, validate and take advantage of the tremendous wealth of highly dimensional data they provide. This paper is based on a DNA microarray study providing gene signatures for hypoxia. These gene signatures were tested on a large breast cancer data set for assessing their prognostic power by means of Kaplan-Meier survival, univariate, and multivariate analyses. We explore the use of several mathematical programming-based techniques that aim to reduce the gene signature sizes as much as possible while maintaining the key characteristics of the original signature, more precisely: the signature prognostic and diagnostic significance. The proposed signature reduction techniques have very interesting potential uses. Indeed, by downsizing the relevant data to a manageable size, one can then patent the core set of biomarkers and also create a dedicated assay (e.g.: on a customized array) for routine applications (e.g.: in the clinical set up) leading to individualized medicine capabilities. Our experiments show that the reduced hypoxia signatures reproduced qualitatively and quantitatively in a similar way that of the original ones.

The grid-based infrastructure enables large-scale scientific applications to be run on distributed resources and coupled in innovative ways. However, in practice, grid resources are not very easy to use for the end-users who have to learn how to generate security credentials, stage inputs and outputs, access grid-based schedulers, and install complex client software. There is an imminent need to provide transparent access to these resources so that the end-users are shielded from the complicated details, and free to concentrate on their domain science. Scientific applications wrapped as Web services alleviate some of these problems by hiding the complexities of the back-end security and computational infrastructure, only exposing a simple SOAP API that can be accessed programmatically by application-specific user interfaces. However, writing the application services that access grid resources can be quite complicated, especially if it has to be replicated for every application. In this paper, we present Opal which is a toolkit for wrapping scientific applications as Web services in a matter of hours, providing features such as scheduling, standards-based grid security and data management in an easy-to-use and configurable manner

Coronary Heart Disease is the single leading cause of death world-wide, with lack of early diagnosis being a key contributory factor. This disease can be diagnosed by measuring and scoring regional motion of the heart wall in echocardiography images of the left ventricle (LV) of the heart. We describe a completely automated and robust technique that detects diseased hearts based on automatic detection and tracking of the endocardium and epicardium of the LV. We describe a novel feature selection technique based on mathematical programming that results in a robust hyperplane-based classifier. The classifier depends only on a small subset of numerical feature extracted from dualcontours tracked through time. We verify the robustness of our system on echocardiograms collected in routine clinical practice at one hospital, both with the standard crossvalidation analysis, and then on a held-out set of completely unseen echocardiography images.

This work describes an approach to building Grid applications based on the premise that users who wish to access and run these applications prefer to do so without becoming experts on Grid technology. We describe an application architecture based on wrapping user applications and application workflows as Web services and Web service resources. These services are visible to the users and to resource providers through a family of Grid portal components that can be used to configure, launch, and monitor complex applications in the scientific language of the end user. The applications in this model are instantiated by an application factory service. The layered design of the architecture makes it possible for an expert to configure an application factory service with a custom user interface client that may be dynamically loaded into the portal.

The benefits of component technologies are well known: they enable encapsulation, modular construction of applications and software reuse. The DOE sponsored common component architecture (CCA) as stated in R. Armstrong et al. (1999) project adopts a component-based approach for building large scale scientific applications. On the other hand, the Web services-based open grid service architecture (OGSA) and infrastructure (OGSI) based on I. Foster et al. (2002) come close to defining a component architecture for the Grid. Using an approach where a CCA component is modeled as a set of Grid services, the XCAT3 framework allows for CCA components to be compatible with the OGSI specification. This enables CCA components to be accessible via standard Grid clients, especially the ones that are portal-based. For CCA compatibility, XCAT3 uses interfaces generated by the Babel toolkit based CASC (2003), and for OGSI compatibility, it uses the Extreme GSX toolkit based on Indiana University Extreme Computing Lab. Grid Service Extensions (2003). In this paper, we describe our experience in implementing the XCAT3 system, and how it can be used to compose complex distributed applications on the Grid in a modular fashion.

Ultrasound is a main noninvasive modality for the assessment of the heart function. Wall tracking from ultrasound data is, however, inherently difficult due to weak echoes, clutter, poor signal-to-noise ratio, and signal dropouts. To cope with these artifacts, pretrained shape models can be applied to constrain the tracking. However, existing methods for incorporating subspace shape constraints in myocardial border tracking use only partial information from the model distribution, and do not exploit spatially varying uncertainties from feature tracking. In this paper, we propose a complete fusion formulation in the information space for robust shape tracking, optimally resolving uncertainties from the system dynamics, heteroscedastic measurement noise, and subspace shape model. We also exploit information from the ground truth initialization where this is available. The new framework is applied for tracking of myocardial borders in very noisy echocardiography sequences. Numerous myocardium tracking experiments validate the theory and show the potential of very accurate wall motion measurements. The proposed framework outperforms the traditional shape-space-constrained tracking algorithm by a significant margin. Due to the optimal fusion of different sources of uncertainties, robust performance is observed even for the most challenging cases.

The most important recent development in Grid systems is the adoption of the Web Services model as its basic architecture. The result is called the Open Grid Services Architecture (OGSA). This paper describes a component framework for distributed Grid applications that is consistent with that model. The framework, called XCAT, is based on the U.S. Department of Energy Common Component Architecture (CCA) but with an implementation based on the standard Web Services stack. Using this framework, an application programmer can compose an application from a set of distributed components. The result is a set of Web Services that collectively represent the executing application instance. This paper describes the basic architecture of XCAT and the design issues to be considered for a component to serve as both a CCA and Open Grid Service Infrastructure (OGSI) service.

The design and prototype implementation of the XCAT Grid Science Portal is described in this paper. The portal lets grid application programmers easily script complex distributed computations and package these applications with simple interfaces for others to use. Each application is packaged as a "notebook" which consists of web pages and editable parameterized scripts. The portal is a workstation-based specialized "personal" web server, capable of executing the application scripts and launching remote grid applications for the user. The portal server can receive event streams published by the application and grid resource information published by Network Weather Service (NWS) [32] or Autopilot [15] sensors. Notebooks can be "published" and stored in web based archives for others to retrieve and modify. The XCAT Grid Science Portal has been tested with various applications, including the distributed simulation of chemical processes in semiconductor manufacturing and collaboratory support for X-ray crystallographers.