The information stored in biological collections can help us model biodiversity, develop new medicines, control agricultural pests, and understand or avoid epidemics, among many other benefits. The digitization of this information, including the transcription of the Darwin Core (DC) terms from the text in photographs, is currently performed through human-oriented processes that rely on input from experts or consensus from crowdsourced work. Previous studies utilized heuristics, only applicable to a couple of DC terms, for their semi-automated extraction. In this paper, we propose a human-in-the-loop workflow for the general extraction of DC terms. The workflow utilizes crowdsourced data to iteratively train a named-entity recognition model for the extraction of the DC terms. Because the source text can contain errors from the optical character recognition process, the extracted values are validated by a confidence estimation structure of frequency lists. These lists are created from millions of previously digitized records, of other collections, and data already extracted from the collection being processed. The proposed human-in-the-loop approach accelerates the transcription of DC terms by partially automating the extraction of values with a near-human quality. The named-entity recognition model was tested in the extraction of a heterogeneous group of 10 terms, obtaining a recall of 0.859, precision of 0.931, and similarity to the ground-truth data of 0.941, on average. The number of required crowdsourcing sessions was reduced by 84%.

In the last decade, institutions from around the world have implemented initiatives for digitizing biological collections (biocollections) and sharing their information online. The transcription of the metadata from photographs of specimens' labels is performed through human-centered approaches (e.g., crowdsourcing) because fully automated Information Extraction (IE) methods still generate a significant number of errors. The integration of human and machine tasks has been proposed to accelerate the IE from the billions of specimens waiting to be digitized. Nevertheless, in order to conduct research and trying new techniques, IE practitioners need to prepare sets of images, crowdsourcing experiments, recruit volunteers, process the transcriptions, generate ground truth values, program automated methods, etc. These research resources and processes require time and effort to be developed and architected into a functional system. In this paper, we present a simulator intended to accelerate the ability to experiment with workflows for extracting Darwin Core (DC) terms from images of specimens. The so-called HuMaIN Simulator includes the engine, the human-machine IE workflows for three DC terms, the code of the automated IE methods, crowdsourced and ground truth transcriptions of the DC terms of three biocollections, and several experiments that exemplify its potential use. The simulator adds Human-in-the-loop capabilities, for iterative IE and research on optimal methods. Its practical design permits the quick definition, customization, and implementation of experimental IE scenarios.

Information Extraction (IE) from imaged text is affected by the output quality of the text-recognition process. Misspelled or missing text may propagate errors or even preclude IE. Low confidence in automated methods is the reason why some IE projects rely exclusively on human work (crowdsourcing). That is the case of biological collections (biocollections), where the metadata (Darwin-core Terms) found in digitized labels are transcribed by citizen scientists. In this paper, we present an approach to reduce the number of crowdsourcing tasks required to obtain the transcription of the text found in biocollections' images. By using an ensemble of Optical Character Recognition (OCR) engines - OCRopus, Tesseract, and the Google Cloud OCR - our approach identifies the lines and characters that have a high probability of being correct. This reduces the need for crowdsourced transcription to be done for only low confidence fragments of text. The number of lines to transcribe is also reduced through hybrid human-machine crowdsourcing where the output of the ensemble of OCRs is used as the first "human" transcription of the redundant crowdsourcing process. Our approach was tested in six biocollections (2,966 images), reducing the number of crowdsourcing tasks by 76% (58% due to lines accepted by the ensemble of OCRs and about 18% due to accelerated convergence when using hybrid crowdsourcing). The automatically extracted text presented a character error rate of 0.001 (0.1%).

Biological collections store information with broad societal and environmental impact. In the last 15 years, after worldwide investments and crowdsourcing efforts, 25% of the collected specimens have been digitized; a process that includes the imaging of text attached to specimens and subsequent extraction of information from the resulting image. This information extraction (IE) process is complex, thus slow and typically involving human tasks. We propose a hybrid (Human-Machine) information extraction model that efficiently uses resources of different cost (machines, volunteers and/or experts) and speeds up the biocollections' digitization process, while striving to maintain the same quality as human-only IE processes. In the proposed model, called SELFIE, self-aware IE processes determine whether their output quality is satisfactory. If the quality is unsatisfactory, additional or alternative processes that yield higher quality output at higher cost are triggered. The effectiveness of this model is demonstrated by three SELFIE workflows for the extraction of Darwin-core terms from specimens' images. Compared to the traditional human-driven IE approach, SELFIE workflows showed, on average, a reduction of 27% in the information-capture time and a decrease of 32% in the required number of humans and their associated cost, while the quality of the results was negligibly reduced by 0.27%.

Historical data sources, like medical records or biological collections, consist of unstructured heterogeneous content: handwritten text, different sizes and types of fonts, and text overlapped with lines, images, stamps, and sketches. The information these documents can provide is important, from a historical perspective and mainly because we can learn from it. The automatic digitization of these historical documents is a complex machine learning process that usually produces poor results, requiring costly interventions by experts, who have to transcribe and interpret the content. This paper describes hybrid (Human- and Machine-Intelligent) workflows for scientific data extraction, combining machine-learning and crowdsourcing software elements. Our results demonstrate that the mix of human and machine processes has advantages in data extraction time and quality, when compared to a machine-only workflow. More specifically, we show how OCRopus and Tesseract, two widely used open source Optical Character Recognition (OCR) tools, can improve their accuracy by more than 42%, when text areas are cropped by humans prior to OCR, while the total time can increase or decrease depending on the OCR selection. The digitization of 400 images, with Entomology, Bryophyte, and Lichen specimens, is evaluated following four different approaches: processing the whole specimen image (machine-only), processing crowd cropped labels (hybrid), processing crowd cropped fields (hybrid), and cleaning the machine-only output. As a secondary result, our experiments reveal differences in speed and quality between Tesseract and OCRopus.

With the constant increase in the number and variety of small molecule chemical compounds, drug discovery is becoming a very resource intensive endeavor. Performing molecular simulations of ligand-protein binding by virtual screening has become an integral part of the discovery process. Cloud computing is an efficient choice to execute these large-scale screenings, given that large compute allocations are not accessible to many researchers. This research focused on developing a multi-site cloud environment that combines small allocations of virtual machines in multiple locations connected through a virtual networking system (ViNe), and compared two parallelization approaches: Message Passing Interface (MPI) and MapReduce using Hadoop. Virtual screenings were conducted using DOCK, a protein-ligand molecular interaction simulation program. Multiple DOCK test simulations through MPI and Hadoop were run to assess the performance and flexibility of the environment. These tests indicated that MPI and MapReduce offer comparable scalability performance, and that network latency has a significant influence on low accuracy simulations. Furthermore, differences in performance at individual cloud resource sites were reduced on average because of the larger combined pool of resources. This project prototyped and assessed a fully functional multi-site cloud environment for virtual screenings, which can be used to guide small laboratories in deploying their own cloud-based screenings.

Crowdsourcing can be a cost-effective method for tackling the problem of digitizing historical bio collections data, and a number of crowd sourcing platforms have been developed to facilitate interaction with the public and to design simple "Human Intelligence Tasks". However, the problem of reaching consensus on the response of the crowd is still challenging for tasks for which a simple majority vote is inadequate. This paper (a) describes the challenges faced when trying to reach consensus on data transcribed by different workers, (b) offers consensus algorithms for textual data and a consensus-based controller to assign a dynamic number of workers per task, and (c) proposes further enhancements of future crowd sourcing tasks in order to minimize the need for complex consensus algorithms. Experiments using the proposed algorithms show up to a 45-fold increase in ability to reach consensus when compared to majority voting using exact string matching. In addition, the controller is able to decrease the crowd sourcing cost by 55% when compared to a strategy that uses a fixed number of workers.

A core mission of the Integrated Digitized Biocollections (iDigBio) project is the building and deployment of a cloud computing environment customized to support the digitization workflow and integration of data from all U.S. non-federal biocollections. iDigBio chose to use cloud computing technologies to deliver a cyber infrastructure that is flexible, agile, resilient, and scalable to meet the needs of the biodiversity community. In this context, this paper describes the integration of open source cloud middleware, applications, and third party services using standard formats, protocols, and services. In addition, this paper demonstrates the value of the digitized information from collections in a broader scenario involving multiple disciplines.

Most data centers, clouds and grids consist of multiple generations of computing systems, each with different performance profiles, posing a challenge to job schedulers in achieving the best usage of the infrastructure. A useful piece of information for scheduling jobs, typically not available, is the extent to which applications will use available resources once they are executed. This paper comparatively assesses the suitability of several machine learning techniques for predicting spatio temporal utilization of resources by applications. Modern machine learning techniques able to handle large number of attributes are used, taking into account application- and system-specific attributes (e.g., CPU micro architecture, size and speed of memory and storage, input data characteristics and input parameters). The work also extends an existing classification tree algorithm, called Predicting Query Runtime (PQR), to the regression problem by allowing the leaves of the tree to select the best regression method for each collection of data on leaves. The new method (PQR2) yields the best average percentage error, predicting execution time, memory and disk consumption for two bioinformatics applications, BLAST and RAxML, deployed on scenarios that differ in system and usage. In specific scenarios where usage is a non-linear function of system and application attributes, certain configurations of two other machine learning algorithms, Support Vector Machine and k-nearest neighbors, also yield competitive results. In addition, experiments show that the inclusion of system performance and application-specific attributes also improves the performance of machine learning algorithms investigated.

This paper describes the design and implementation of an improved version (2.0) of a computational cyberinfrastructure for neuroscience research, called CyberWorkstation (CW). CW can provide to neurophysiology laboratories the following: (1) data storage for large volumes of neural signals, experime

With the emergence of multiple cloud providers of Infrastructure-as-a-Service, it becomes possible to envision a near-future when high-performance computing users could combine services from different clouds to access huge numbers of resources. However, as more administrative privileges are exposed to end users, providers are required to deploy network security measures that present challenges to the network virtualization technologies that are needed to enable inter-cloud communication. This paper studies these challenges and proposes techniques to enable unmodified applications on resources across distinct clouds. The techniques are implemented in TinyViNe, an extension to ViNe, a virtual networking technology for distributed resources in different administrative domains. The results of evaluating TinyViNe on a WAN-based testbed across three sites are reported for a bioinformatics application (BLAST) and MPI benchmarks. The results confirm that TinyViNe enables cross-cloud computing while having little impact on application performance. TinyViNe also has auto-configuration and ¿download-and-run¿ capabilities for easy deployment by users who are not knowledgeable about networking.

This paper proposes and evaluates an approach to the parallelization, deployment and management of bioinformatics applications that integrates several emerging technologies for distributed computing. The proposed approach uses the MapReduce paradigm to parallelize tools and manage their execution, machine virtualization to encapsulate their execution environments and commonly used data sets into flexibly deployable virtual machines, and network virtualization to connect resources behind firewalls/NATs while preserving the necessary performance and the communication environment. An implementation of this approach is described and used to demonstrate and evaluate the proposed approach. The implementation integrates Hadoop, Virtual Workspaces, and ViNe as the MapReduce, virtual machine and virtual network technologies, respectively, to deploy the commonly used bioinformatics tool NCBI BLAST on a WAN-based test bed consisting of clusters at two distinct locations, the University of Florida and the University of Chicago. This WAN-based implementation, called CloudBLAST, was evaluated against both non-virtualized and LAN-based implementations in order to assess the overheads of machine and network virtualization, which were shown to be insignificant. To compare the proposed approach against an MPI-based solution, CloudBLAST performance was experimentally contrasted against the publicly available mpiBLAST on the same WAN-based test bed. Both versions demonstrated performance gains as the number of available processors increased, with CloudBLAST delivering speedups of 57 against 52.4 of MPI version, when 64 processors on 2 sites were used. The results encourage the use of the proposed approach for the execution of large-scale bioinformatics applications on emerging distributed environments that provide access to computing resources as a service.

Abstract Science gateways require the easy enabling of legacy scientific applications on computing Grids and the generation of user‐friendly interfaces that hide the complexity of the Grid from the user. This paper presents the In‐VIGO approach to the creation and management of science gateways. First, we discuss the virtualization of machines, networks and data to facilitate the dynamic creation of secure execution environments that meet application requirements. Then we discuss the virtualization of applications, i.e. the execution on shared resources of multiple isolated application instances with customized behavior, in the context of In‐VIGO. A Virtual Application Service (VAS) architecture for automatically generating, customizing, deploying, and using virtual applications as Grid services is then described. Starting with a grammar‐based description of the command‐line syntax, the automated process generates the VAS description and the VAS implementation (code for application encapsulation and data binding) that is deployed and made available through a Web interface. A VAS can be customized on a per‐user basis by restricting the capabilities of the original application or by adding to it features such as parameter sweeping. This is a scalable approach to the integration of scientific applications as services into Grids and can be applied to any tool with an arbitrarily complex command‐line syntax. Copyright © 2006 John Wiley & Sons, Ltd.

To best exploit the potential of the grid, it is necessary to "grid-enable" legacy applications that were not originally developed to run on a grid. In this paper, a generic framework to grid-enable legacy applications is presented. In particular, this paper focuses on a general approach to model and represent an application in a way that is supportive of the key properties of the grid-enabling framework: generality, automatic generation and integration of grid applications, plug-and-play deployment, and interoperability. Based on the model, a configuration language is developed to describe a wide range of command-line applications and their execution requirements. It is easy to use and does not require the application enabler to know the details of the grid middleware. A case study is presented to illustrate the automated grid-enabling of a legacy application in In-VIGO (in-virtual information grid organizations), a grid computing infrastructure that makes extensive use of visualization technology

This poster briefly introduces two resource-virtualization techniques needed for the creation of virtual(ized) grids: virtual networks and virtual application services. The former provides bidirectional network connectivity even in the presence of firewalls, network address translation gateways and proxies by creating virtual routers and virtual IP space. The later allows automated creation and deployment of legacy applications into grids by generating a virtual application service that allows the execution in shared resources of multiple isolated application instances with customized behavior.

This paper describes a scalable approach to the enabling of legacy scientific applications on computing grids using a service-oriented architecture. In the context of this paper grid-enabling means turning an existing application, installed on a grid resource, into a service and generating the application-specific user interfaces to use that application through a Web portal. Scalability is achieved by providing a common abstraction for a category of applications and providing a "generic" application service to wrap those applications as services. The focus of this paper's approach is on grid-enabling "command-oriented" scientific applications. The novel aspect of the approach is that the entire process -from turning an application into a service to the user-interface generation for that application - is done automatically, without requiring coding or grid-system downtime. Portlet technology is used to dynamically generate application-specific interfaces. Further, the approach makes it possible to customize the applications for different user groups by way of simplifying, restricting or composing the functionalities of applications. The approach is useful for building grid portals on which a large number of applications need to be dynamically enabled.

Summary form only given. Single sign-on (SSO) is an essential desired feature of computational grids. Its implementation is challenging because resources cross administrative domains and are managed by heterogeneous access schemes. We present an approach for single sign-on in a deployed functioning grid called In-VIGO. The approach relies on decoupling grid user accounts from local user accounts and making use of role-based access control lists. Role-based accesses via delegation mechanisms using short-lived user identities enable In-VIGO to handle interactive applications and application-specific authentication mechanisms. This capability is not present in existing grid architectures. SSO implementations for usage scenarios in In-VIGO are described to highlight the applicability of the proposed approach. In particular, access to interactive applications with their own security mechanisms, such as VNC, and access to remote data can be achieved using proxies that delegate In-VIGO user access via short-lived user identities.

One of the factors that can influence the performance of a DSM system is the efficiency of multipoint access on the interconnection network. This work presents an evaluation of the Brazos system, a software implemented distributed shared memory (DSM) system designed for x86 SMP nodes running Windows NT and that takes advantage of multicast communication, with regard to different configurations of the communication network: broadcast fast Ethernet, switched fast Ethernet and LANE-ATM. In this evaluation, although the bandwidth of the ATM is larger than the fast Ethernet, low performance using LANE-ATM was observed, compared to the Brazos system performance using broadcast or switched Fast Ethernet. It shows the importance of the communication subsystem and the necessity of improvements on aspects like multipoint access, which has been not much explored on networks like ATM.