The Hausdorff distance (HD) is a mathematical measure of the similarity (or dissimilarity) between two sets of points, typically used to compare geometric shapes, images, or spatial distributions. HD computation has a broad range of applications in large-scale data analysis across domains like medical imaging, geospatial information systems, and graphics. As the volume of spatial data continues to grow rapidly, HD computation has become increasingly time-consuming, demanding effective and scalable hardware acceleration. Recent research on utilizing Ray Tracing (RT) cores to accelerate k-nearest neighbor (k-NN) search shed light on this problem, as k-NN is the building block of the HD algorithm. However, naively using an RT-accelerated k-NN library yields poor performance due to the lack of domain-specific optimizations, leading to poor utilization of hardware resources. In this paper, we propose X-HD, a general-purpose HD algorithm with RT acceleration for large-scale datasets. X-HD has three core optimization techniques: (1) We use a grid to organize spatially proximal points, which reduces the traversal intensity of the Bounding Volume Hierarchy (BVH) tree. (2) We use HD estimators to prune non-contributing points for faster processing. (3) Introducing the grid also results in severe load imbalance in RT shaders. Our solution addresses this by selectively offloading the distance computation from RT shaders to a dedicated CUDA kernel to improve load balance. X-HD outperforms the state-of-the-art industrial software ITK by 5.3 × on average. Compared to a GPU-optimized HD solution, X-HD achieves a speedup of up to 6.4 ×.

3D spatial data management is increasingly vital across various application scenarios, such as GIS, digital twins, human atlases, and tissue imaging. However, the inherent complexity of 3D spatial data, primarily represented by 3D geometries in real-world applications, hinders the efficient evaluation of spatial relationships through resource-intensive geometric computations. Geometric simplification algorithms have been developed to reduce the complexity of 3D representations, albeit at the cost of querying accuracy. Previous work has aimed to address precision loss by leveraging the spatial relationship between the simplified and original 3D object representations. However, this approach relied on specialized geometric simplification algorithms tailored to regions with specific criteria. In this paper, we introduce a novel approach to achieve highly efficient and accurate 3D spatial queries, incorporating geometric computation and simplification. We present a generalized progressive refinement methodology applicable to general geometric simplification algorithms, involving accurate querying of 3D geometry data using low-resolution representations and simplification extents quantified using Hausdorff distances at the facet level. Additionally, we propose techniques for calculating and storing Hausdorff distances efficiently. Extensive experimental evaluations validate the effectiveness of the proposed method which outperforms state-of-the-art systems by a factor of 4 while minimizing computational and storage overhead.

Current medical prediction models struggle with three key challenges: (1) lackluster performance on real-world health records, (2) reliance on non-routine tests (e.g., ECGs or blood tests), and (3) model uninterpretability, which prevents adoption. A key challenge is that increasing transparency often decreases the model’s performance. We present FRISTS (Feature-Ranked Interpretable Sequential Time Series), a novel medical prediction approach that uses patients’ previous medical history records to predict future diagnoses. FRISTS combines time series-based recurrent neural networks, e.g. LSTMS, AI-guided feature selection, and a Shapley-inspired permutation method that enables interpretability. We applied our model to 18 million health records in the Cerner Health Facts database to predict heart failure, which yielded a six-fold increase over decision trees and a significant increase over LSTM while capturing more condition-specific features. Since FRISTS is extendable to any prediction task on electronic health records, it accelerates the adoption of machine learning methods in AI-assisted healthcare that achieve both high real-world performance and interpretability.

Spatial join has become a frequently used yet resource-intensive operation in geospatial applications, driven by the increasing volume and complexity of geospatial data. With Hadoop and Spark becoming the de facto standard platforms for distributed computing, scalable spatial data processing is primarily achieved by partitioning the input space to form parallel units on these platforms. Effective spatial data partitioning is critical for task parallelization and load balancing, but it faces significant challenges due to data skew and the geometric and topological complexity of spatial objects, particularly in supporting spatial joins. This paper examines the interplay among query performance, spatial data partitioning, query types, data, and system characteristics. We qualitatively and quantitatively analyze the features of representative partitioning algorithms that impact overall query performance. Along with these analyses, we propose a data sampling-based approach for selecting optimized partitioning strategies. Extensive experiments on large and complex datasets using MapReduce frameworks are conducted to validate the correctness of our analysis and the effectiveness of our optimization approach.

Electric bus transport, a popular mode of public transportation, offers punctual, safe, and comfortable services to passengers through the efficient and effective use of designated road space. The performance of electric bus transport systems depends largely on the design of proper locations of bus stops, with the consideration of passenger demands, waiting time, and traveling time. Optimal electric bus route planning can attract an increasing number of passengers and increase public transit services. Aiming to provide guidance for the electric bus route planning of developing cities, this study proposed an intelligent route planning method to minimize the waiting time and traveling time of passengers, in order to achieve the best comfortable level. In addition, a self‐learning anomaly detection method based on reinforcement learning (RL) was proposed to eliminate abnormal data caused by traffic accidents or emergencies. With a large spatiotemporal dataset collected over 3 years from a real electric bus project in Yantai, China, we developed a prototype system and conducted extensive experiments to evaluate the proposed intelligent route planning method. The results showed that the proposed method can reduce the passengers’ waiting time and attract more passengers traveling by electric bus. In addition, the proposed method has achieved optimal route planning recommendation (RPR) subject to 1,872,391 passenger demands on electric bus services; more than 86% of them were accurately predicted, and more than 97% were satisfied with recommendation results.

We released open-source software Hadoop-GIS in 2011, and presented and published the work in VLDB 2013. This work initiated the development of a new spatial data analytical ecosystem characterized by its large-scale capacity in both computing and data storage, high scalability, compatibility with low-cost commodity processors in clusters and open-source software. After more than a decade of research and development, this ecosystem has matured and is now serving many applications across various fields. In this paper, we provide the background on why we started this project and give an overview of the original Hadoop-GIS software architecture, along with its unique technical contributions and legacy. We present the evolution of the ecosystem and its current state-of-the-art, which has been influenced by the Hadoop-GIS project. We also describe the ongoing efforts to further enhance this ecosystem with hardware accelerations to meet the increasing demands for low latency and high throughput in various spatial data analysis tasks. Finally, we will summarize the insights gained and lessons learned over more than a decade in pursuing high-performance spatial data analytics.

Writing text with eye gaze only is an appealing hands-free text entry method. However, existing gaze-based text entry methods introduce eye fatigue and are slow in typing speed because they often require users to dwell on letters of a word, or mark the starting and ending positions of a gaze path with extra operations for entering a word. In this paper, we propose GlanceWriter, a text entry method that allows users to enter text by glancing over keys one by one without any need to dwell on any keys or specify the starting and ending positions of a gaze path when typing a word. To achieve so, GlanceWriter probabilistically determines the letters to be typed based on the dynamics of gaze movements and gaze locations. Our user studies demonstrate that GlanceWriter significantly improves the text entry performance over EyeSwipe, a dwell-free input method using “reverse crossing” to identify the starting and ending keys. GlanceWriter also outperforms the dwell-free gaze input method of Tobii’s Communicator 5, a commercial eye gaze-based communication system. Overall, GlanceWriter achieves dwell-free and crossing-free text entry by probabilistically decoding gaze paths, offering a promising gaze-based text entry method.

MOTIVATION: Morphological analyses with flatmount fluorescent images are essential to retinal pigment epithelial (RPE) aging studies and thus require accurate RPE cell segmentation. Although rapid technology advances in deep learning semantic segmentation have achieved great success in many biomedical research, the performance of these supervised learning methods for RPE cell segmentation is still limited by inadequate training data with high-quality annotations. RESULTS: To address this problem, we develop a Self-Supervised Semantic Segmentation (S4) method that utilizes a self-supervised learning strategy to train a semantic segmentation network with an encoder-decoder architecture. We employ a reconstruction and a pairwise representation loss to make the encoder extract structural information, while we create a morphology loss to produce the segmentation map. In addition, we develop a novel image augmentation algorithm (AugCut) to produce multiple views for self-supervised learning and enhance the network training performance. To validate the efficacy of our method, we applied our developed S4 method for RPE cell segmentation to a large set of flatmount fluorescent microscopy images, we compare our developed method for RPE cell segmentation with other state-of-the-art deep learning approaches. Compared with other state-of-the-art deep learning approaches, our method demonstrates better performance in both qualitative and quantitative evaluations, suggesting its promising potential to support large-scale cell morphological analyses in RPE aging investigations. AVAILABILITY AND IMPLEMENTATION: The codes and the documentation are available at: https://github.com/jkonglab/S4_RPE.

Despite the abundance of subphenotype clustering studies on sepsis and acute kidney injury (AKI), few models consider the real-time information of clinical features. The lack of supervision may lead to patient subgroups being derived as clusters without the stratification of patients based on the outcome of interests. The sensitivity of the dimension in clustering methods is generally ignored, so clusters lack robustness. In this study, we propose an ensembled outcome-driven bidirectional long short-term memory autoencoder (BiLSTM-AE) architecture with high robustness and transferability to identify subphenotypes. BiLSTM-AE learns the advanced representation of the time-series clinical features by co-training the encoder and a weak predictor to achieve the risk-stratified clustering of patients. Clusters of a variety of dimensions are ensembled to combine global and local information. Four different datasets from three public datasets, MIMIC-III-AKI, MIMIC-IV-sepsis, eICU-AKI, and eICU-sepsis, were used to assess the method’s effectiveness in clustering and prediction. Compared to baseline approaches including latent class analysis (LCA), subgroups generated by BiLSTMAE exhibited the highest mortality risk ratios between subgroups: the mortality for subphenotypes 1, 2, and 3 of BiLSTM and LCA was 6.91%, 17.53%, and 75.56% vs. 13.2%, 14.4%, and 19.7% for MIMIC-III-AKI. The prediction metric area under the receiver operating characteristic curve was 0.86 for MIMIC-IIIAKI, 0.91 for eICU-AKI, 0. SS for MIMIC-IV-sepsis, and 0. S9 for eICU-sepsis. Additionally, clinical evaluation of BiLSTM-AE generated subgroups revealed more meaningful distributions of member characteristics across subgroups. Thus, the method is an effective means to consider the real-time information of clinical features.

Suggesting multiple target candidates based on touch input is a possible option for high-accuracy target selection on small touchscreen devices. But it can become overwhelming if suggestions are triggered too often. To address this, we propose SATS, a Suggestion-based Accurate Target Selection method, where target selection is formulated as a sequential decision problem. The objective is to maximize the utility: the negative time cost for the entire target selection procedure. The SATS decision process is dictated by a policy generated using reinforcement learning. It automatically decides when to provide suggestions and when to directly select the target. Our user studies show that SATS reduced error rate and selection time over Shift [51], a magnification-based method, and MUCS, a suggestion-based alternative that optimizes the utility for the current selection. SATS also significantly reduced error rate over BayesianCommand [58], which directly selects targets based on posteriors, with only a minor increase in selection time.

, a system that supports efficient spatial queries for complex 3D objects. 3DPro uses progressive compression of 3D objects preserving multiple levels of details, which significantly reduces the size of the objects and has the data fit into memory. Through a novel Filter-Progressive-Refine paradigm, 3DPro can have query results returned early whenever possible to minimize decompression and geometric computations of 3D objects in higher resolution representations. Our experiments demonstrate that 3DPro out-performs the state-of-the-art 3D data processing techniques by up to an order of magnitude for typical spatial queries.

With the widespread use of mobile and sensing devices, there has been an explosion of high velocity, transient data having spatial and temporal characteristics. Interactive analysis at such scale and speed require support for highly efficient query processing backend frameworks coupled with lightweight yet powerful frontend interfaces. While existing in-memory distributed stream processing frameworks are perfect candidates for scalable big data processing, spatio-temporal systems in this domain are mostly dominated by specify-once-apply-continuously query model. Any modification in query state requires query restart limiting system responsiveness and producing outdated or in the worst case erroneous results. Furthermore, most of the contemporary spatio-temporal big data systems are designed to operate in a single execution environment limiting their applicability to users accustomed to other similar frameworks with different APIs. In this paper, we demon-strate SPEAR-Board; an interactive web-based interface integrated with cross-platform stream processing engine; SPEAR, capable of seamlessly handling spatio-temporal query state changes in real-time. We demonstrate working of SPEAR-Board with respect to spatio-temporal Range and Nearest Neighbor queries backed by Apache Spark and Apache Flink deployed over cloud resources.

Text correction on mobile devices usually requires precise and repetitive manual control. In this paper, we present EyeSayCorrect, an eye gaze and voice based hands-free text correction method for mobile devices. To correct text with EyeSayCorrect, the user first utilizes the gaze location on the screen to select a word, then speaks the new phrase. EyeSayCorrect would then infer the user’s correction intention based on the inputs and the text context. We used a Bayesian approach for determining the selected word given an eye-gaze trajectory. Given each sampling point in an eye-gaze trajectory, the posterior probability of selecting a word is calculated and accumulated. The target word would be selected when its accumulated interest is larger than a threshold. The misspelt words have higher priors. Our user studies showed that using priors for misspelt words reduced the task completion time up to 23.79% and the text selection time up to 40.35%, and EyeSayCorrect is a feasible hands-free text correction method on mobile devices.

Artificial intelligence (AI) has been transforming the practice of drug discovery in the past decade. Various AI techniques have been used in many drug discovery applications, such as virtual screening and drug design. In this survey, we first give an overview on drug discovery and discuss related applications, which can be reduced to two major tasks, i.e. molecular property prediction and molecule generation. We then present common data resources, molecule representations and benchmark platforms. As a major part of the survey, AI techniques are dissected into model architectures and learning paradigms. To reflect the technical development of AI in drug discovery over the years, the surveyed works are organized chronologically. We expect that this survey provides a comprehensive review on AI in drug discovery. We also provide a GitHub repository with a collection of papers (and codes, if applicable) as a learning resource, which is regularly updated.

MOTIVATION: Predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in triple-negative breast cancer (TNBC) patients accurately is direly needed for clinical decision making. pCR is also regarded as a strong predictor of overall survival. In this work, we propose a deep learning system to predict pCR to NAC based on serial pathology images stained with hematoxylin and eosin and two immunohistochemical biomarkers (Ki67 and PHH3). To support human prior domain knowledge-based guidance and enhance interpretability of the deep learning system, we introduce a human knowledge-derived spatial attention mechanism to inform deep learning models of informative tissue areas of interest. For each patient, three serial breast tumor tissue sections from biopsy blocks were sectioned, stained in three different stains and integrated. The resulting comprehensive attention information from the image triplets is used to guide our prediction system for prognostic tissue regions. RESULTS: The experimental dataset consists of 26 419 pathology image patches of 1000×1000 pixels from 73 TNBC patients treated with NAC. Image patches from randomly selected 43 patients are used as a training dataset and images patches from the rest 30 are used as a testing dataset. By the maximum voting from patch-level results, our proposed model achieves a 93% patient-level accuracy, outperforming baselines and other state-of-the-art systems, suggesting its high potential for clinical decision making. AVAILABILITY AND IMPLEMENTATION: The codes, the documentation and example data are available on an open source at: https://github.com/jkonglab/PCR_Prediction_Serial_WSIs_biomarkers. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Cancer impacts many people across New York State, with breast cancer being one type of cancer that is especially prevalent. Breast cancer can be detected through screening procedures, and if caught early in its development, has a higher likelihood of treatability and a less severe outcome. In order to better understand existing patterns of breast cancer throughout New York State, we conducted a geospatial and demographic analysis in which we sought to identify disparities in screening, diagnosis, and mortality rates across regions and demographic groups. Statistical analyses were completed using patient-level data from the New York Statewide Planning and Research Cooperative System (SPARCS) from 2005 to 2019, where demographic data including age, race, and ethnicity, as well as location-based data such as addresses were collected. Geospatial analysis results revealed clustering for screening and diagnosis rates in certain regions of the state, as well as hot spots. Results from the demographic analysis indicated temporal trends in cancer rates for various age, race, and ethnic groups, as well as a disparity in the mortality rates of breast cancer across race and ethnic groups.

Contact tracing is gaining its importance in controlling the spread of COVID-19. However, the enormous volume of the frequently sampled tracing data brings major challenges for real-time processing. In this paper, we propose a GPU-based real-time contact tracing system based on spatial proximity queries with temporal constraints using location data. We provide dynamic indexing of moving objects using an adaptive partitioning schema on GPU with extremely low overhead. Our system optimizes the retrieval of contacted pairs to match both the requirements of contact tracing scenarios and GPU centered parallelism. We propose an efficient contacts evaluation mechanism to keep only the spatially and temporally valid contacts. Our experiments demonstrate that the system can achieve sub-second level response for large-scale contact tracing of tens of millions of people, with two magnitudes of performance boost over CPU based approach.

Selecting targets accurately and quickly with eye-gaze input remains an open research question. In this paper, we introduce BayesGaze, a Bayesian approach of determining the selected target given an eye-gaze trajectory. This approach views each sampling point in an eye-gaze trajectory as a signal for selecting a target. It then uses the Bayes' theorem to calculate the posterior probability of selecting a target given a sampling point, and accumulates the posterior probabilities weighted by sampling interval to determine the selected target. The selection results are fed back to update the prior distribution of targets, which is modeled by a categorical distribution. Our investigation shows that BayesGaze improves target selection accuracy and speed over a dwell-based selection method, and the Center of Gravity Mapping (CM) method. Our research shows that both accumulating posterior and incorporating the prior are effective in improving the performance of eye-gaze based target selection.

With the advent of IoT and emerging 5G technology, real-time streaming data are being generated at unprecedented speed and volume, and coming with both temporal and spatial dimensions. Effective analysis at such scale and speed requires support for dynamically adjusting querying capabilities in real-time. In spatio-temporal domain, this warrants for data as well as query optimization strategies especially for objects with changing motion states. Contemporary spatio-temporal data stream management systems in distributed domain are mostly dominated by specified-once-applied-continuously query model. Any modification in query state requires query restart limiting system responsiveness and producing outdated or in worst case erroneous results. In this paper, we propose adaptations of principles from streaming databases, spatial data management and distributed computing to support dynamic spatio-temporal query processing over high velocity big data streams. We first formulate a set of spatio-temporal data types and functions to seamlessly handle changes in distributed query states. We develop a comprehensive set of streaming spatio-temporal querying methods, and propose geohash based dynamic spatial partitioning for effective parallel processing. We implement a prototype on top of Apache Flink, where the in-memory stream processing fits nicely with our spatio-temporal models. Comparative evaluation of our prototype demonstrates the effectiveness our strategy by maintaining high consistent processing rates for both stationary as well as moving queries over high velocity spatio-temporal big data streams.

Geometric computation can be heavy duty for spatial queries, in particular for complex geometries such as polygons with many edges based on a vector-based representation. While many techniques have been provided for spatial partitioning and indexing, they are mainly built on minimal bounding boxes or other approximation methods, which will not mitigate the high cost of geometric computation. In this paper, we propose a novel vector-raster hybrid approach through rasterization, where pixel-centric rich information is preserved to help not only filtering out more candidates but also reducing geometry computation load. Based on the hybrid model, we develop an efficient rasterization based ray casting method for point-in-polygon queries and a circle buffering method for point-to-polygon distance calculation, which is a common operation for distance based queries. Our experiments demonstrate that the hybrid model can boost the performance of spatial queries on complex polygons by up to one order of magnitude.

MOTIVATION: In most tissue-based biomedical research, the lack of sufficient pathology training images with well-annotated ground truth inevitably limits the performance of deep learning systems. In this study, we propose a convolutional neural network with foveal blur enriching datasets with multiple local nuclei regions of interest derived from original pathology images. We further propose a human-knowledge boosted deep learning system by inclusion to the convolutional neural network new loss function terms capturing shape prior knowledge and imposing smoothness constraints on the predicted probability maps. RESULTS: Our proposed system outperforms all state-of-the-art deep learning and non-deep learning methods by Jaccard coefficient, Dice coefficient, Accuracy and Panoptic Quality in three independent datasets. The high segmentation accuracy and execution speed suggest its promising potential for automating histopathology nuclei segmentation in biomedical research and clinical settings. AVAILABILITY AND IMPLEMENTATION: The codes, the documentation and example data are available on an open source at: https://github.com/HongyiDuanmu26/FovealBoosted. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

OBJECTIVE: The United States is experiencing an opioid epidemic. In recent years, there were more than 10 million opioid misusers aged 12 years or older annually. Identifying patients at high risk of opioid use disorder (OUD) can help to make early clinical interventions to reduce the risk of OUD. Our goal is to develop and evaluate models to predict OUD for patients on opioid medications using electronic health records and deep learning methods. The resulting models help us to better understand OUD, providing new insights on the opioid epidemic. Further, these models provide a foundation for clinical tools to predict OUD before it occurs, permitting early interventions. METHODS: Electronic health records of patients who have been prescribed with medications containing active opioid ingredients were extracted from Cerner's Health Facts database for encounters between January 1, 2008, and December 31, 2017. Long short-term memory models were applied to predict OUD risk based on five recent prior encounters before the target encounter and compared with logistic regression, random forest, decision tree, and dense neural network. Prediction performance was assessed using F1 score, precision, recall, and area under the receiver-operating characteristic curve. RESULTS: The long short-term memory (LSTM) model provided promising prediction results which outperformed other methods, with an F1 score of 0.8023 (about 0.016 higher than dense neural network (DNN)) and an area under the receiver-operating characteristic curve (AUROC) of 0.9369 (about 0.145 higher than DNN). CONCLUSIONS: LSTM-based sequential deep learning models can accurately predict OUD using a patient's history of electronic health records, with minimal prior domain knowledge. This tool has the potential to improve clinical decision support for early intervention and prevention to combat the opioid epidemic.

Spatial cross-matching operation over geospatial polygonal datasets is important to a variety of GIS applications. However, it involves extensive computation cost associated with intersection and union of a geospatial polygon pair from large scale datasets. This mandates for exploration of parallel computing capabilities such as GPU to increase the efficiency of such operations. In this paper, we present a CPU-GPU hybrid platform to accelerate the cross-matching operation of geospatial datasets. The computing tasks are dynamically scheduled to be executed either on CPU or GPU. To accommodate geospatial datasets processing on GPU using pixelization approach, we convert the floating point-valued vertices into integer-valued vertices with an adaptive scaling factor as a function of area of minimum bounding box. We test our framework over Natural Earth Dataset and achieve 10x speedup on NVIDIA GeForce GTX750 GPU and 14x speedup on Tesla K80 GPU over 280,000 polygon pairs in one tile and 400 tiles in total. We also investigate the effects of input data size to the IO / computation ratio and note that the sufficiently large input data size is required to better utilize the computing power of GPU. Finally, with comparison between two GPUs, our results demonstrate that the efficient cross-matching comparison can be achieved with a cost-effective GPU.

Increased availability of large amounts of address data provides opportunities for data driven studies to improve decision making in business applications and support precision public health with high resolution geolocations. Geocoding large number of addresses is challenging due to high cost and often disclosure of sensitive data to vendors over the Web. Most geocoders take advantage of Web APIs which require sending private addresses over the Internet, which may not be an option for many applications with sensitive data including public health and geo-medicine. Meanwhile, the cost for geocoding massive number of addresses could be high and becomes a major hurdle for many users. To overcome these challenges, we developed an open source on-premise geocoding software EaserGeocoder, which uses a novel integrative geocoding model to achieve high accuracy through integrating multiple open data sources. EaserGeocoder takes advantage of machine learning based approaches to determine best answers from multiple data sources. EaserGeocoder can also be easily parallelized to achieve high scalability through parallelized search and distributed computing. EaserGeocoder is on a par with commercial geocoding systems, outperforms open source systems, and is available for free.

The recent technological advancement in digital pathology has enabled 3D tissue-based investigation of human diseases at extremely high resolutions. Discovering and verifying spatial patterns among massive 3D micro-anatomic biological objects such as blood vessels and cells derived from 3D pathology image volumes plays a pivotal role in understanding diseases. However, the exponential increase of available 3D data and the complex structures of biological objects make it extremely difficult to support spatial queries due to high I/O, communication and computational cost for 3D spatial queries. In this demonstration, we present our scalable in-memory based spatial query system iSPEED for large-scale 3D data with complex structures. Low latency is managed by storing in memory with progressive compression including successive levels of detail on object level. On the other hand, low computational cost is achieved by pre-generation of global spatial indexes in memory and additional on-demand generation of indexing at run-time. Furthermore, iSPEED applies structural indexing on complex structured objects in multiple query types to gain performance advantage. During query processing, the memory footprint of iSPEED is minimal due to its indexing structure and progressive decompression on-demand. We demonstrate iSPEED query capability with three representative queries: 3D spatial joins, nearest neighbor and spatial proximity estimation on multiple datasets using a web based RESTful interface. Users can furthermore explore the input data structure, manage and adjust query pipeline parameters on the interface.

Much effort has been devoted to support high performance spatial queries on large volumes of spatial data in distributed spatial computing systems, especially in the MapReduce paradigm. Recent works have focused on extending spatial MapReduce frameworks to leverage high performance in-memory distributed processing capabilities of systems such as Spark. However, the performance advantage comes with the requirement of having enough memory and comprehensive configuration. Failing to fulfill this falls back to disk IO, defeating the purpose of such systems or in worst case gets out of memory and fails the job. The problem is aggravated further for spatial processing since the underlying in-memory systems are oblivious of spatial data features and characteristics. In this paper we present SparkGIS - an in-memory oriented spatial data querying system for high throughput and low latency spatial query handling by adapting Apache Spark's distributed processing capabilities. It supports basic spatial queries including containment, spatial join and k-nearest neighbor and allows extending these to complex query pipelines. SparkGIS mitigates skew in distributed processing by supporting several dynamic partitioning algorithms suitable for a rich set of contemporary application scenarios. Multilevel global and local, pre-generated and on-demand in-memory indexes, allow SparkGIS to prune input data and apply compute intensive operations on a subset of relevant spatial objects only. Finally, SparkGIS employs dynamic query rewriting to gracefully manage large spatial query workflows that exceed available distributed resources. Our comparative evaluation has shown that the performance of SparkGIS is on par with contemporary Spark based platforms for relatively smaller queries and outperforms them for larger data and memory intensive workflows by dynamic query rewriting and efficient spatial data management.

Recent advances in digital pathology make it possible to support 3D tissue-based investigation of human diseases at extremely high resolutions. Exploring spatial relationships and patterns among massive 3D micro-anatomic biological objects such as blood vessels and cells derived from 3D pathology image volumes plays a critical role in studying human diseases. In this paper, we present our work on building an effective and scalable in-memory based spatial query system iSPEED for large-scale 3D data with complex structures. To achieve low latency, iSPEED stores data in memory with effective progressive compression for each 3D object with successive levels of detail. To minimize search space and computation cost, iSPEED pre-generates global spatial indexes in memory and employs on-demand indexing at run-time. In particular, iSPEED exploits structural indexing for complex structured objects in distance based queries. iSPEED provides a 3D spatial query engine that can be invoked on-demand to run many instances in parallel implemented with, but not limited to, MapReduce. iSPEED builds in-memory indexes and decompresses data on-demand, which has minimal memory footprint. We evaluate iSPEED with two representative queries: 3D spatial joins and 3D spatial proximity estimation. Our experiments demonstrate that iSPEED significantly improves the performance over traditional non-memory based spatial query systems.

With increased accessibility of large scale open data, public health studies are able to take advantage of integrative spatial big data to increase the spatial resolution to community or neighborhood level. One critical information for such studies is the large number of addresses of patients, which is private and highly sensitive. Geocoding such massive private addresses poses major challenges for public health researchers. Many geocoders provide only Web APIs which require sending private addresses over the Internet, which is not feasible. Commercial geocoders require high licensing fee and often have limitations on daily usage, which becomes a major hurdle for researchers. Scalability is another major challenge for large scale address dataset. In this paper, we present EaserGeocoder, a novel open source geocoder for effectively geocoding massive address datasets. EaserGeocoder takes an integrative approach by using multiple references based on open address data sources contributed by governments or communities. It takes a machine learning approach to automatically find the best answer from candidates produced by multiple references. The system provides high scalability through parallel processing. Our comparative studies demonstrate Easer-Geocoder outperforms open source geocoders and is comparable to commercial ones in terms of both accuracy and error. It provides a cost-effective and feasible solution for large scale public health studies.

3D analytical pathology imaging examines high resolution 3D image volumes of human tissues to facilitate biomedical research and provide potential effective diagnostic assistance. Such approach - quantitative analysis of large- scale 3D pathology image volumes - generates tremendous amounts of spatially derived 3D micro-anatomic objects, such as 3D blood vessels and nuclei. Spatial exploration of such massive 3D spatial data requires effective and efficient querying methods. In this paper, we present a scalable and efficient 3D spatial query system for querying massive 3D spatial data based on MapReduce. The system provides an on-demand spatial querying engine which can be executed with as many instances as needed on MapReduce at runtime. Our system supports multiple types of spatial queries on MapReduce through 3D spatial data partitioning, customizable 3D spatial query engine, and implicit parallel spatial query execution. We utilize multi-level spatial indexing to achieve efficient query processing, including global partition indexing for data retrieval and on-demand local spatial indexing for spatial query processing. We evaluate our system with two representative queries: 3D spatial joins and 3D k-nearest neighbor query. Our experiments demonstrate that our system scales to large number of computing nodes, and efficiently handles data-intensive 3D spatial queries that are challenging in analytical pathology imaging.

BACKGROUND: We describe a suite of tools and methods that form a core set of capabilities for researchers and clinical investigators to evaluate multiple analytical pipelines and quantify sensitivity and variability of the results while conducting large-scale studies in investigative pathology and oncology. The overarching objective of the current investigation is to address the challenges of large data sizes and high computational demands. RESULTS: The proposed tools and methods take advantage of state-of-the-art parallel machines and efficient content-based image searching strategies. The content based image retrieval (CBIR) algorithms can quickly detect and retrieve image patches similar to a query patch using a hierarchical analysis approach. The analysis component based on high performance computing can carry out consensus clustering on 500,000 data points using a large shared memory system. CONCLUSIONS: Our work demonstrates efficient CBIR algorithms and high performance computing can be leveraged for efficient analysis of large microscopy images to meet the challenges of clinically salient applications in pathology. These technologies enable researchers and clinical investigators to make more effective use of the rich informational content contained within digitized microscopy specimens.

Scalable spatial query processing relies on effective spatial data partitioning for query parallelization, data pruning, and load balancing. These are often challenged by the intrinsic characteristics of spatial data, such as high skew in data distribution and high complexity of irregular multi-dimensional objects. In this demo, we present SATO, a spatial data partitioning framework that can quickly analyze and partition spatial data with an optimal spatial partitioning strategy for scalable query processing. SATO works in following steps: 1) Sample, which samples a small fraction of input data for analysis, 2) Analyze, which quickly analyzes sampled data to find an optimal partition strategy, 3) Tear, which provides data skew aware partitioning and supports MapReduce based scalable partitioning, and 4) Optimize, which collects succinct partition statistics for potential query optimization. SATO also provides multiple level partitioning, which can be used to significantly improve window based queries in cloud based spatial query processing systems. SATO comes with a visualization component that provides heat maps and histograms for qualitative evaluation. SATO has been implemented within the Hadoop-GIS, a high performance spatial data warehousing system over MapReduce. SATO is also released as an independent software package to support various scalable spatial query processing systems. Our experiments have demonstrated that SATO can generate much balanced partitioning that can significantly improve spatial query performance with MapReduce comparing to traditional spatial partitioning approaches.

In this paper, we present a novel framework for microscopic image analysis of nuclei, data management, and high performance computation to support translational research involving nuclear morphometry features, molecular data, and clinical outcomes. Our image analysis pipeline consists of nuclei segmentation and feature computation facilitated by high performance computing with coordinated execution in multi-core CPUs and Graphical Processor Units (GPUs). All data derived from image analysis are managed in a spatial relational database supporting highly efficient scientific queries. We applied our image analysis workflow to 159 glioblastomas (GBM) from The Cancer Genome Atlas dataset. With integrative studies, we found statistics of four specific nuclear features were significantly associated with patient survival. Additionally, we correlated nuclear features with molecular data and found interesting results that support pathologic domain knowledge. We found that Proneural subtype GBMs had the smallest mean of nuclear Eccentricity and the largest mean of nuclear Extent, and MinorAxisLength. We also found gene expressions of stem cell marker MYC and cell proliferation maker MKI67 were correlated with nuclear features. To complement and inform pathologists of relevant diagnostic features, we queried the most representative nuclear instances from each patient population based on genetic and transcriptional classes. Our results demonstrate that specific nuclear features carry prognostic significance and associations with transcriptional and genetic classes, highlighting the potential of high throughput pathology image analysis as a complementary approach to human-based review and translational research.

The fifth ACM international workshop on cloud data management is held in San Francisco, California, USA on October 28, 2013 and co-located with the ACM 22nd Conference on Information and Knowledge Management (CIKM). The main objective of the workshop is to address the challenges of large scale data management based on the cloud computing infrastructure. The workshop brings together researchers and practitioners from cloud computing, distributed storage, query processing, parallel algorithms, data mining, and system analysis, all attendees share common research interests in maximizing performance, reducing cost of cloud data management and enlarging the scale of their endeavors. We have constructed an exciting program of four refereed papers and an invited keynote talk that will give participants a full dose of emerging research.

- a scalable and high performance spatial query system over MapReduce. Hadoop-GIS provides an efficient spatial query engine to process spatial queries, data and space based partitioning, and query pipelines that parallelize queries implicitly on MapReduce. Hadoop-GIS also provides an expressive, SQL-like spatial query language for workload specification. We will demonstrate how spatial queries are expressed in spatially extended SQL queries, and submitted through a command line/web interface for execution. Parallel to our system demonstration, we explain the system architecture and details on how queries are translated to MapReduce operators, optimized, and executed on Hadoop. In addition, we will showcase how the system can be used to support two representative real world use cases: large scale pathology analytical imaging, and geo-spatial data warehousing.

BACKGROUND: The integration and visualization of multimodal datasets is a common challenge in biomedical informatics. Several recent studies of The Cancer Genome Atlas (TCGA) data have illustrated important relationships between morphology observed in whole-slide images, outcome, and genetic events. The pairing of genomics and rich clinical descriptions with whole-slide imaging provided by TCGA presents a unique opportunity to perform these correlative studies. However, better tools are needed to integrate the vast and disparate data types. OBJECTIVE: To build an integrated web-based platform supporting whole-slide pathology image visualization and data integration. MATERIALS AND METHODS: All images and genomic data were directly obtained from the TCGA and National Cancer Institute (NCI) websites. RESULTS: The Cancer Digital Slide Archive (CDSA) produced is accessible to the public (http://cancer.digitalslidearchive.net) and currently hosts more than 20,000 whole-slide images from 22 cancer types. DISCUSSION: The capabilities of CDSA are demonstrated using TCGA datasets to integrate pathology imaging with associated clinical, genomic and MRI measurements in glioblastomas and can be extended to other tumor types. CDSA also allows URL-based sharing of whole-slide images, and has preliminary support for directly sharing regions of interest and other annotations. Images can also be selected on the basis of other metadata, such as mutational profile, patient age, and other relevant characteristics. CONCLUSIONS: With the increasing availability of whole-slide scanners, analysis of digitized pathology images will become increasingly important in linking morphologic observations with genomic and clinical endpoints.

Support of high performance queries on large volumes of spatial data becomes increasingly important in many application domains, including geospatial problems in numerous fields, location based services, and emerging scientific applications that are increasingly data- and compute-intensive. The emergence of massive scale spatial data is due to the proliferation of cost effective and ubiquitous positioning technologies, development of high resolution imaging technologies, and contribution from a large number of community users. There are two major challenges for managing and querying massive spatial data to support spatial queries: the explosion of spatial data, and the high computational complexity of spatial queries. In this paper, we present Hadoop-GIS - a scalable and high performance spatial data warehousing system for running large scale spatial queries on Hadoop. Hadoop-GIS supports multiple types of spatial queries on MapReduce through spatial partitioning, customizable spatial query engine RESQUE, implicit parallel spatial query execution on MapReduce, and effective methods for amending query results through handling boundary objects. Hadoop-GIS utilizes global partition indexing and customizable on demand local spatial indexing to achieve efficient query processing. Hadoop-GIS is integrated into Hive to support declarative spatial queries with an integrated architecture. Our experiments have demonstrated the high efficiency of Hadoop-GIS on query response and high scalability to run on commodity clusters. Our comparative experiments have showed that performance of Hadoop-GIS is on par with parallel SDBMS and outperforms SDBMS for compute-intensive queries. Hadoop-GIS is available as a set of library for processing spatial queries, and as an integrated software package in Hive.

The fourth ACM international workshop on cloud data management is held in Maui, Hawaii, USA on October 29, 2012 and co-located with the ACM 21th Conference on Information and Knowledge Management (CIKM). The main objective of the workshop is to address the challenges of large scale data management based on the cloud computing infrastructure. The workshop brings together researchers and practitioners from cloud computing, distributed storage, query processing, parallel algorithms, data mining, and system analysis, all attendees share common research interests in maximizing performance, reducing cost of cloud data management and enlarging the scale of their endeavors. We have constructed an exciting program of seven refereed papers and four invited keynote talks that will give participants a full dose of emerging research.

Cloud-based data management systems are emerging as scalable, fault-tolerant, and efficient solutions to manage large volumes of data with cost effective infrastructures, and more and more data analysis applications are migrated to the cloud. As an attractive solution to provide a quick sketch of massive data before a long wait of the final accurate query result, online processing of aggregate queries in the cloud is of paramount importance. This problem is challenging to solve because of the large block based data organization and distributed processing mode in the cloud. In this paper, we present COLA, a system for Cloud Online Aggregation to provide progressive approximate answers for both single tables and joined multiple tables. We develop an online query processing algorithm for MapReduce to support incremental and continuous computing of aggregations on joins which minimizes the waiting time before an acceptable estimate is achieved. We formulate a statistical foundation that supports block-level sampling for single-table online aggregations and effective estimation of approximate results and confidence intervals of statistical significance. We also develop a two-phase stratified sampling method to support multi-table aggregations to improve the approximate query answers and speed up the convergence of confidence intervals. We implement COLA in Hadoop, and our experiments demonstrate that COLA can deliver reasonable precise online estimates within a time period two orders of magnitude shorter than that used to produce exact answers.

While current biomedical ontology repositories offer primitive query capabilities, it is difficult or cumbersome to support ontology based semantic queries directly in semantically annotated biomedical databases. The problem may be largely attributed to the mismatch between the models of the ontologies and the databases, and the mismatch between the query interfaces of the two systems. To fully realize semantic query capabilities based on ontologies, we develop a system DBOntoLink to provide unified semantic query interfaces by extending database query languages. With DBOntoLink, semantic queries can be directly and naturally specified as extended functions of the database query languages without any programming needed. DBOntoLink is adaptable to different ontologies through customizations and supports major biomedical ontologies hosted at the NCBO BioPortal. We demonstrate the use of DBOntoLink in a real world biomedical database with semantically annotated medical image annotations.

Support of high performance queries on large volumes of scientific spatial data is becoming increasingly important in many applications. This growth is driven by not only geospatial problems in numerous fields, but also emerging scientific applications that are increasingly data- and compute-intensive. For example, digital pathology imaging has become an emerging field during the past decade, where examination of high resolution images of human tissue specimens enables more effective diagnosis, prediction and treatment of diseases. Systematic analysis of large-scale pathology images generates tremendous amounts of spatially derived quantifications of micro-anatomic objects, such as nuclei, blood vessels, and tissue regions. Analytical pathology imaging provides high potential to support image based computer aided diagnosis. One major requirement for this is effective querying of such enormous amount of data with fast response, which is faced with two major challenges: the "big data" challenge and the high computation complexity. In this paper, we present our work towards building a high performance spatial query system for querying massive spatial data on MapReduce. Our framework takes an on demand index building approach for processing spatial queries and a partition-merge approach for building parallel spatial query pipelines, which fits nicely with the computing model of MapReduce. We demonstrate our framework on supporting multi-way spatial joins for algorithm evaluation and nearest neighbor queries for microanatomic objects. To reduce query response time, we propose cost based query optimization to mitigate the effect of data skew. Our experiments show that the framework can efficiently support complex analytical spatial queries on MapReduce.

BACKGROUND AND OBJECTIVE: Morphologic variations of disease are often linked to underlying molecular events and patient outcome, suggesting that quantitative morphometric analysis may provide further insight into disease mechanisms. In this paper a methodology for the subclassification of disease is developed using image analysis techniques. Morphologic signatures that represent patient-specific tumor morphology are derived from the analysis of hundreds of millions of cells in digitized whole slide images. Clustering these signatures aggregates tumors into groups with cohesive morphologic characteristics. This methodology is demonstrated with an analysis of glioblastoma, using data from The Cancer Genome Atlas to identify a prognostically significant morphology-driven subclassification, in which clusters are correlated with transcriptional, genetic, and epigenetic events. MATERIALS AND METHODS: Methodology was applied to 162 glioblastomas from The Cancer Genome Atlas to identify morphology-driven clusters and their clinical and molecular correlates. Signatures of patient-specific tumor morphology were generated from analysis of 200 million cells in 462 whole slide images. Morphology-driven clusters were interrogated for associations with patient outcome, response to therapy, molecular classifications, and genetic alterations. An additional layer of deep, genome-wide analysis identified characteristic transcriptional, epigenetic, and copy number variation events. RESULTS AND DISCUSSION: Analysis of glioblastoma identified three prognostically significant patient clusters (median survival 15.3, 10.7, and 13.0 months, log rank p=1.4e-3). Clustering results were validated in a separate dataset. Clusters were characterized by molecular events in nuclear compartment signaling including developmental and cell cycle checkpoint pathways. This analysis demonstrates the potential of high-throughput morphometrics for the subclassification of disease, establishing an approach that complements genomics.

Pathology is a medical subspecialty that practices the diagnosis of disease. Microscopic examination of tissue reveals information enabling the pathologist to render accurate diagnoses and to guide therapy. The basic process by which anatomic pathologists render diagnoses has remained relatively unchanged over the last century, yet advances in information technology now offer significant opportunities in image-based diagnostic and research applications. Pathology has lagged behind other healthcare practices such as radiology where digital adoption is widespread. As devices that generate whole slide images become more practical and affordable, practices will increasingly adopt this technology and eventually produce an explosion of data that will quickly eclipse the already vast quantities of radiology imaging data. These advances are accompanied by significant challenges for data management and storage, but they also introduce new opportunities to improve patient care by streamlining and standardizing diagnostic approaches and uncovering disease mechanisms. Computer-based image analysis is already available in commercial diagnostic systems, but further advances in image analysis algorithms are warranted in order to fully realize the benefits of digital pathology in medical discovery and patient care. In coming decades, pathology image analysis will extend beyond the streamlining of diagnostic workflows and minimizing interobserver variability and will begin to provide diagnostic assistance, identify therapeutic targets, and predict patient outcomes and therapeutic responses.

As an important application of spatial databases in pathology imaging analysis, cross-comparing the spatial boundaries of a huge amount of segmented micro-anatomic objects demands extremely data- and compute-intensive operations, requiring high throughput at an affordable cost. However, the performance of spatial database systems has not been satisfactory since their implementations of spatial operations cannot fully utilize the power of modern parallel hardware. In this paper, we provide a customized software solution that exploits GPUs and multi-core CPUs to accelerate spatial cross-comparison in a cost-effective way. Our solution consists of an efficient GPU algorithm and a pipelined system framework with task migration support. Extensive experiments with real-world data sets demonstrate the effectiveness of our solution, which improves the performance of spatial cross-comparison by over 18 times compared with a parallelized spatial database approach.

MapReduce has become an effective approach to big data analytics in large cluster systems, where SQL-like queries play important roles to interface between users and systems. However, based on our Facebook daily operation results, certain types of queries are executed at an unacceptable low speed by Hive (a production SQL-to-MapReduce translator). In this paper, we demonstrate that existing SQL-to-MapReduce translators that operate in a one-operation-to-one-job mode and do not consider query correlations cannot generate high-performance MapReduce programs for certain queries, due to the mismatch between complex SQL structures and simple MapReduce framework. We propose and develop a system called Y Smart, a correlation aware SQL-to-MapReduce translator. Y Smart applies a set of rules to use the minimal number of MapReduce jobs to execute multiple correlated operations in a complex query. Y Smart can significantly reduce redundant computations, I/O operations and network transfers compared to existing translators. We have implemented Y Smart with intensive evaluation for complex queries on two Amazon EC2 clusters and one Facebook production cluster. The results show that Y Smart can outperform Hive and Pig, two widely used SQL-to-MapReduce translators, by more than four times for query execution.

OBJECTIVE AND DESIGN: The design and implementation of ImageMiner, a software platform for performing comparative analysis of expression patterns in imaged microscopy specimens such as tissue microarrays (TMAs), is described. ImageMiner is a federated system of services that provides a reliable set of analytical and data management capabilities for investigative research applications in pathology. It provides a library of image processing methods, including automated registration, segmentation, feature extraction, and classification, all of which have been tailored, in these studies, to support TMA analysis. The system is designed to leverage high-performance computing machines so that investigators can rapidly analyze large ensembles of imaged TMA specimens. To support deployment in collaborative, multi-institutional projects, ImageMiner features grid-enabled, service-based components so that multiple instances of ImageMiner can be accessed remotely and federated. RESULTS: The experimental evaluation shows that: (1) ImageMiner is able to support reliable detection and feature extraction of tumor regions within imaged tissues; (2) images and analysis results managed in ImageMiner can be searched for and retrieved on the basis of image-based features, classification information, and any correlated clinical data, including any metadata that have been generated to describe the specified tissue and TMA; and (3) the system is able to reduce computation time of analyses by exploiting computing clusters, which facilitates analysis of larger sets of tissue samples.

Image annotation becomes increasingly important for clinical applications and medical research. In particular, collaborative image annotations can harness the collective intelligence from distributed experts. There are several challenges to support collaborative medical image annotations: i) Medical image annotation includes not only metadata annotation but also multimedia annotation, such as graphical annotation. The latter often requires a heavy-duty tool, which can be difficult to realize in a distributed environment; ii) Annotations need to be well modeled for easy exchange and support of queries, i.e., there is a gap between image annotation and content retrieval; and iii) An annotation platform is needed to provide authoring tools and the collaborative infrastructure. Meanwhile, the Web is evolving quickly on supporting interaction, participation and collaboration enabled by Web 2.0 technologies. Among them, scalable vector graphics (SVG) now becomes a standard language for vector graphics on the Web natively supported by latest Web browsers. In our work, we develop a collaborative image annotation platform, which provides: i) a flexible data model to support both metadata and multimedia annotations on 2-D medical images; ii) SVG based implementation of the data model that can support complex textual, spatial, and collaborative queries on annotations with XQuery; iii) a lightweight native Web browser enabled annotation authoring tool without any plugin needed; and iv) an architecture that provides authoring, storing, querying, and exchanging of annotations, and supports Web-based collaboration.

Image annotation is becoming increasingly important for clinical applications and medical research. Especially, collaborative image annotation can harness the collective intelligence from distributed users. There are several challenges to model medical image annotations: i) Medical image annotation includes not only textual annotation but also graphical annotation, which can consist of multiple areas or layers; ii) To support collaborative annotation from multiple users, collaboration has to be supported and represented in a proper way; Hi) Both textual and graphical annotations should be searchable, e.g., support of spatial queries; iv) annotation history needs to be tracked and searchable in applications such as clinical trials; and v) To enable convenient annotation, and save cost, lightweight system should be provided. Most current medical image annotation applications are workstation based applications, and use proprietary methods to represent annotations, which make it difficult to search, share, and exchange annotations. Meanwhile, the Web is evolving quickly on supporting strong collaboration through several new technologies, including Ajax (Asynchronous JavaScript and XML) and SVG (Scalable Vector Graphics). Especially, SVG now becomes a standard language for vector graphics on the Web, which, indeed, is recently natively supported by latest Web browsers. This brings a new opportunity to support collaborative image annotations on the Web. In our work, we develop a comprehensive data model to support both textual and graphical annotations and their history on 2-D medical images, and implement the data model with the standard language SVG. This data model can support powerful queries on annotations including complex textual, spatial, collaboration, group, and history queries based on standard XML Query language XQuery. Besides, the approach can be used to support collaborative image annotations in a lightweight Web browser environment.

RFID technology provides significant advantages over traditional object-tracking technologies and is increasingly adopted and deployed in real applications. RFID applications generate large volume of streaming data, which have to be automatically filtered, processed, and transformed into semantic data, and integrated into business applications. Indeed, RFID data are highly temporal, and RFID observations form complex temporal event patterns which can be very different for various RFID applications. Thus, it is desirable to have a general RFID data processing framework with a powerful language, for the end users to express a variety of queries on RFID data streams, as well as detecting complex events patterns. While data stream management systems (DSMSs) are emerging for optimized stream data processing, they usually lack the language construct support for temporal event detection. In this paper, we discuss a stream query language to provide comprehensive temporal event detection, through temporal operators and extension of sliding-window constructs. With the integration of temporal event detection, a DSMS has the capability to serve as a powerful system for RFID data processing.

Scientific research becomes increasingly reliant on collaborative effort among multiple institutions and interdisciplinary consortia, through sharing scientific experiments and data and collaboration on analysis of data and results. Besides information sharing, there is a growing need to create a framework to enable collaborative, cross-disciplinary research, which can facilitate a research community into a default mode of collaboration, including: i) enabling scientific researchers to dynamically interact with others, collaboratively author, annotate, review, comment on others' data, and discuss their research; ii) providing a dynamic collaboration environment by harnessing collective contributions from a group of researchers; and iii) facilitating broader participation of people into research projects. While traditional data management and integration systems put much focus on passively integrating existing data, the collaboration among data providers and users is quite limited. In this paper, we present a Web-based collaborative platform to dynamically integrate information for scientific research. Besides data integration, the system provides essential collaboration capabilities to boost user participation and collaboration. The system will not only enable users as active information contributors to bring significant new values to the data, but also provide an environment for scientific researchers to do collaborative research in a large research community.

RFID can be used for identifying, locating, tracking and monitoring physical objects without line of sight, and thus can be used in a wide range of pervasive computing applications. To achieve these goals, RFID data have to be collected, transformed and expressively modeled as their counterparts in the virtual world. To this end, in this paper, we propose a general framework for modeling RFID data. We first explore the fundamental characteristics of RFID applications and classify applications into a set of basic scenarios based on these characteristics. We then develop constructs for modeling each basic scenario, which can be integrated to model complex RFID applications in real world as demonstrated in our case study.

No abstract available.

Scientific research in many fields is increasingly a collaborative effort across multiple institutions and disciplines. Scientific researchers need not only an effective system to manage their data, results, and the experiments that generate the results, but also a platform to integrate, share and search these across multiple institutions. Therefore, researchers are able to reuse experiments, pool expertise and validate approaches. In this paper, we present Sci- Port, a system of experiment management and integration for collaborative scientific research. SciPort’s architecture uses i) a general transformation-based data model to represent and link experiment processes; ii) hierarchical data classification across multiple institutions according to research programs’ goals and organization; iii) metadatacentric representation that concisely captures the context of experiments; and iv) virtual data integration through centralized metadata integration. The system is built for open source, and the metadata-based representation and integration provides a unified framework and tool set to manage and share experiments for scientific research communities.

In this paper, we present the ArchIS system that achieves full-functionality transaction-time databases without requiring temporal extensions in XML or database standards. ArchIS’ architecture uses (a) XML to support temporally grouped (virtual) representations of the database history, (b) XQuery to express powerful temporal queries on such views, (c) temporal clustering and indexing techniques for managing the actual historical data in a relational database, and (d) SQL/XML for executing the queries on the XML views as equivalent queries on the relational database. The performance studies presented in the paper show that ArchIS is quite effective at storing and retrieving under complex query conditions the transaction-time history of relational databases.

While the introduction of temporal extensions into database standards has proven difficult to achieve, the newly introduced SQL:2003 and XML/XQuery standards have actually enhanced our ability to support temporal applications in commercial database systems. We illustrate this point by discussing three approaches that use temporally grouped representations. We first compare the approaches at the logical level using a common set of queries; then we turn to the physical level and discuss our ArchIS system that supports the three different approaches efficiently in one unified physical implementation. We conclude that the approaches of managing transaction-time information using XML and SQL can be integrated and supported efficiently within the current standards, and claim that the proposed approach can be extended to valid-time and bitemporal databases.

Historical information can be effectively preserved using XML and searched through powerful historical queries written in XQuery. Indeed, by storing the successive versions of a document in an incremental fashion, XML repositories and data warehouses can achieve (i) the efficient preservation of critical information, (ii) its representation using a temporally grouped data model and (iii) the ability of supporting historical queries on the evolution of documents and their contents using XQuery. The proposed approach can be applied uniformly to (a) XML document archives and (b) transaction time databases published in XML and queried in XQuery. Our case studies include the UCLA course catalog, W3C Xlink standards, and the CIA WorldFact Book, besides relational databases. The experience described here and in the work of Wang et al. (2005 and 2003) suggests that current standards provide reasonable support for temporal applications at the logical level, whereas many challenges remain at the physical level.

By storing the successive versions of a document in an incremental fashion, XML repositories and data warehouses achieve: (i) the efficient preservation of critical information; and (ii) the ability of supporting historical queries on the evolution of documents and their contents. In this paper, we present efficient techniques for managing multi-version document histories and supporting powerful temporal queries on such documents. Our approach consists in: (i) concisely representing the successive versions of a document as an XML document that implements a temporally grouped data model; and (ii) using XML query languages, such as XQuery, to express complex queries on the content of a particular version, and on the temporal evolution of the document elements and their contents. We show that the data definition & manipulation framework of XML & XQuery can support temporal models and historical queries significantly better than the traditional framework of Relational Tables & SQL. To demonstrate this point, we investigate how to express complex queries on the history of relational tables published as XML data.

There is much current interest in publishing and viewing databases as XML documents. The general benefits of this approach follow from the popularity of XML and the tool set available for visualizing and processing information encoded in this universal standard. In this paper, we explore the additional and unique benefits achieved by this approach on temporal database applications. We show that XML views combined with XQuery can provide surprisingly effective solutions to the problem of supporting historical queries on past content of database relations and their evolution. Indeed, using XML, the histories of database relations can be naturally represented by temporally grouped data models. Thus, we identify mappings from relations to XML that are most conducive to modeling and querying database histories, and show that temporal queries that would be very difficult to express in SQL can be easily expressed in standard XQuery. Then, we turn to the problem of supporting efficiently the storage and the querying of relational table histories. We present an experimental study of the pros and cons of using native XML databases, versus using traditional databases, where the XML-represented histories are supported as views on the historical tables.