Gagan Agrawal

dblp:a/GAgrawal · DBLP profile ↗
← Back
45ranked-venue papers in the field
0as first author
5since 2021 · last 2024
—ORCID · conflict

Domains — venue-derived; a paper can count in several

Database Systems & Data Management · 23Data Mining & Knowledge Discovery · 16Big Data, Cloud & Distributed Data Systems · 5Information Retrieval & Web Search · 1
YearPublicationVenuePosition
2024 Federated Contrastive Learning of Graph-Level Representations
abstract
Graph-level representations (and clustering/classification based on these representations) are required in a variety of applications. Examples include identifying malicious network traffic, prediction of protein properties, and many others. Often, data has to stay in isolated local systems due to a variety of considerations like privacy concerns, lack of trust between the parties, regulations, or simply because the data is too large to be shared sufficiently quickly. This points to the need for federated learning for graph-level representations, a topic that has not been explored much, especially in an unsupervised setting.Addressing this problem, this paper presents a new framework we refer to as Federated Contrastive Learning of Graph-level Representations (FCLG). Our approach builds on contrastive learning. However, what is unique is that we apply contrastive learning at two levels. The first application is for local unsupervised learning of graph representations. The second level is to address the challenge associated with data distribution variation (i.e. the "Non-IID issue") when combining local models. Through extensive experiments on the downstream task of graph-level clustering, we demonstrate FCLG outperforms baselines with significant margins.
Gagan Agrawal, Rajiv Ramnath, Ruoming Jin
IEEE Big Data2
2022 Deep Graph Clustering with Random-walk based Scalable Learning
abstract
Interactions between (social) entities can be frequently represented by an attributed graph, and node clustering in such graphs has received much attention lately. Multiple efforts have successfully applied Graph Convolutional Networks (GCN), though with some limits on accuracy as GCNs have been shown to suffer from over-smoothing issues. Though other methods (particularly those based on Laplacian Smoothing) have reported better accuracy, a fundamental limitation of all the work is a lack of scalability. This paper addresses this open problem by relating the Laplacian smoothing to the Generalized PageRank, and applying a random-walk based algorithm as a scalable graph filter. This forms the basis for our scalable deep clustering algorithm, RwSL. Using 6 real-world datasets and 6 clustering metrics, we show that RwSL achieved improved results over several recent baselines. Most notably, by demonstrating execution of RwSL on a graph with 1.8 billion edges using only a single GPU. We show that RwSL can continue to scale, unlike other existing deep clustering frameworks.
Dong Li 0047, Ruoming Jin, Rajiv Ramnath, Gagan Agrawal
ASONAM5
2022 Securing Pseudonym Schemes for Vehicular Privacy
abstract
This poster reports on a project to improve privacy in the presence of vehicle-to-everything (V2X) communication. Cooperative Intelligent Transport Systems (C-ITS) enable communication and information sharing between a vehicle and other entities using V2X. While there are many (emerging) applications for such messages, they also compromise privacy. Though pseudonym change strategies have been developed, privacy attacks have been demonstrated. This paper reports on a study that shows that by adding a small random noise to the length and widths of cars in V2X messages, these attacks can be defeated without negatively impacting the applications of these messages.
Md Hasan, Hoda Maleki, Gagan Agrawal
IEEE Big Data3
2021 Constraint-embedded paraphrase generation for commercial tweets
abstract
Automated generation of commercial tweets has become a useful and important tool in the use of social media for marketing and advertising. In this context, paraphrase generation has emerged as an important problem. This type of paraphrase generation has the unique requirement of requiring certain elements to be kept in the result, such as the product name or the promotion details. To address this need, we propose a Constraint-Embedded Language Modeling (CELM) framework, in which hard constraints are embedded in the text content and learned through a language model. This embedding helps the model learn not only paraphrase generation but also constraints in the content of the paraphrase specific to commercial tweets. In addition, we apply knowledge learned from a general domain to the generation task of commercial tweets. Our model is shown to outperform general paraphrase generation models as well as the state-of-the-art CopyNet model, in terms of paraphrase similarity, diversity, and the ability to conform to hard constraints.
Renhao Cui, Gagan Agrawal, Rajiv Ramnath
ASONAM2
2021 Privacy-Preserving Framework to Facilitate Shared Data Access for Wearable Devices
abstract
Wearable devices are emerging as effective modalities for the collection of individuals’ data. While this data can be leveraged for use in several areas ranging from health-care to crime investigation, storing and securely accessing such information while preserving privacy and detecting any tampering attempts are significant challenges. This paper describes a decentralized system that ensures an individual’s privacy, maintains an immutable log of any data access, and provides decentralized access control management. Our proposed framework uses a custom permissioned blockchain protocol to securely log data transactions from wearable devices in the blockchain ledger. We have implemented a proof-of-concept for our framework, and our preliminary evaluation is summarized to demonstrate our proposed framework’s capabilities. We have also discussed various application scenarios of our privacy-preserving model using blockchain and proof-of-authority. Our research aims to detect data tampering attempts in data sharing scenarios using a thorough transaction log model.
Dane Troyer, Justin Henry, Hoda Maleki, Gokila Dorai, Bethany Sumner, Gagan Agrawal, Jon Ingram
IEEE BigData6
2020 DistriPlan: an optimized join execution framework for geo-distributed scientific data
Roee Ebenstein, Gagan Agrawal
Distributed Parallel Databases2
2019 Tweets can tell: activity recognition using hybrid long short-term memory model
abstract
This paper presents techniques to detect offline activities of a person when she is tweeting in order to create a dynamic profile of the user, for uses such as better targeting of advertisements. To this end, we propose a hybrid LSTM model for rich contextual learning, along with studies on the effects of applying and combining multiple LSTM based methods with different contextual features. The hybrid model outperforms a set of baselines as well as state-of-the-art methods.
Renhao Cui, Gagan Agrawal, Rajiv Ramnath
ASONAM2
2019 Accelerating array joining with integrated value-index
abstract
Large-scale multidimensional array processing is becoming an increasingly important problem with the rise of big data, scientific data processing, and machine learning workloads. One of the prevalent query types, array join, compares the cells of two arrays and finds matching cell pairs and is useful in finding patterns and differences across multiple arrays. These queries of ten have two distinct characteristics: first, the queries usually have a value-based filter predicate, because one may only be interested in a small subset of the entire array, and second, because many multidimensional data are numerical and has inaccuracies in nature, the comparison between cells are of ten required to be approximate. These two characteristics motivate the need for value similarity array joins. While prior works exist in both array join and similarity join, they primarily target dimension-based comparisons and do not address this problem.
Haoyuan Xing, Gagan Agrawal
SSDBM2
2018 COMPASS: compact array storage with value index
abstract
Efficient array storage is the backbone of scientific data processing. With an explosion of data, rapidly answering queries on array data is becoming increasingly important. Although most of the array storages today support subsetting of an array based on dimensions efficiently, they fall back to full scan while executing value-based filter operations. This has lead to an interest in approximate query processing, but such methods can have substantial inaccuracies.
Haoyuan Xing, Gagan Agrawal
SSDBM2
2017 DistriPlan: An Optimized Join Execution Framework for Geo-Distributed Scientific Data
abstract
Scientific data is frequently stored across geographically distributed data repositories. Although there have been recent efforts to query scientific datasets using structured query operators, they have not yet supported joins across distributed data repositories. This paper describes a framework that supports join-like operations over multi-dimensional array datasets that are spread across multiple sites.
Roee Ebenstein, Gagan Agrawal
SSDBM2
2015 Towards methods for systematic research on big data
abstract
Big Data is characterized by the five V's - of Volume, Velocity, Variety, Veracity and Value. Research on Big Data, that is, the practice of gaining insights from it, challenges the intellectual, process, and computational limits of an enterprise. Leveraging the correct and appropriate toolset requires careful consideration of a large software ecosystem. Powerful algorithms exist, but the exploratory and often ad-hoc nature of analytic demands and a distinct lack of established processes and methodologies make it difficult for Big Data teams to set expectations or even create valid project plans. The exponential growth of data generated exceeds the capacity of humans to process it, and compels us to develop automated computing methods that require significant and expensive computing power in order to scale effectively. In this paper, we characterize data-driven practice and research and explore how we might design effective methods for systematizing such practice and research [19, 22]. Brief case studies are presented in order to ground our conclusions and insights.
Manirupa Das, Renhao Cui, David R. Campbell, Gagan Agrawal, Rajiv Ramnath
IEEE BigData4
2015 DSDQuery DSI - Querying scientific data repositories with structured operators
abstract
Scientific data is often distributed through repositories that host a large number of files in formats such as NetCDF or HDF5. With recent and anticipated increases in the size of observational and simulation data, it is important to transport just the data that are of interest from a large distributed dataset. Unfortunately, existing portals provide limited querying interfaces - typically a set of predefined hard coded subsettings, limiting user's querying flexibility. This paper describes a system that addresses this gap. The relational algebra is adapted for scientific array querying allowing us to adapt a subset of SQL for this domain, which enables nuanced subsetting conditions to be applied on a set of dataset files within a repository. A query processing algorithm extracts and collects data from relevant datasets, based on metadata that was earlier extracted using an automatic metadata extraction engine. Finally, the system stitches a new structured, NetCDF, file to be returned as a resultset, allowing the returned data to be used and analyzed by existing tools. The system has been extensively evaluated to show its ability to handle increasing data and/or number of files.
Roee Ebenstein, Gagan Agrawal
IEEE BigData2
2015 A novel approach for approximate aggregations over arrays
abstract
Approximate aggregation has been a popular approach for interactive data analysis and decision making, especially on large-scale datasets. While there is clearly a need to apply this approach for scientific datasets comprising massive arrays, existing algorithms have largely been developed for relational data, and cannot handle both dimension-based and value-based predicates efficiently while maintaining accuracy. In this paper, we present a novel approach for approximate aggregations over array data, using bitmap indices or bitvectors as the summary structure, as they preserve both spatial and value distribution of the data. We develop approximate aggregation algorithms using only the bitvectors and certain additional pre-aggregation statistics (equivalent to a 1-dimensional histogram) that we require. Another key development is choosing a binning strategy that can improve aggregation accuracy -- we introduce a v-optimized binning strategy and its weighted extension, and present a bitmap construction algorithm with such binning. We compare our method with other existing methods including sampling and multi-dimensional histograms, as well as the use of other binning strategies with bitmaps. We demonstrate both high accuracy and efficiency of our approach. Specifically, we show that in most cases, our method is more accurate than other methods by at least one order of magnitude. Despite achieving much higher accuracy, our method can require significantly less storage than multi-dimensional histograms.
Yu Su 0011, Gagan Agrawal
SSDBM3
2015 SciCSM: novel contrast set mining over scientific datasets using bitmap indices
abstract
Contrast set mining is a broadly applicable exploratory technique, which identifies interesting differences across contrast groups. The existing algorithms primarily target relational datasets with categorical attributes. There is clearly a need to apply this method to discover interesting patterns across scientific datasets, which feature arrays with numeric values. In this paper, we present a novel algorithm, SciCSM, for efficient contrast set mining over array-based datasets. We define how "interesting" contrast sets can be characterized for numeric and array data -- handling the fact that subsets can involve both value-based and/or dimension-based attributes. We extensively use bitmap indices to reduce computational complexity and enable processing of larger-scale data. We demonstrate both high efficiency and effectiveness of our algorithm by using multiple real-life datasets.
Gangyi Zhu, Gagan Agrawal
SSDBM3
2014 SAGA: array storage as a DB with support for structural aggregations
abstract
In recent years, many Array DBMSs, including SciDB and RasDaMan have emerged to meet the needs of data management applications where the natural structures are the arrays. These systems, like their relational counterparts, involve an expensive data ingestion phase. The paradigm of using native storage as a DB and providing database-like support (e.g., the NoDB approach) has recently been shown to be an effective approach for dealing with infrequently queried data, where data ingestion costs cannot be justified, though only in context of relational data.
Arnab Nandi 0001, Gagan Agrawal
SSDBM3
2012 Stratified k-means clustering over a deep web data source
abstract
This paper focuses on the problem of clustering data from a {\em hidden} or a deep web data source. A key characteristic of deep web data sources is that data can only be accessed through the limited query interface they support. Because the underlying data set cannot be accessed directly, data mining must be performed based on sampling of the datasets. The samples, in turn, can only be obtained by querying the deep web databases with specific inputs.
Tantan Liu, Gagan Agrawal
KDD2
2012 Stratification Based Hierarchical Clustering Over a Deep Web Data Source
abstract
This paper focuses on the problem of clustering data from a hidden or a deep web data source. A key characteristics of deep web data sources is that data can only be accessed through the limited query interface they support. Because the underlying data set cannot be accessed directly, data mining must be performed based on sampling of the datasets. The samples, in turn, can only be obtained by querying the deep web databases with specific inputs. Unlike existing sampling based methods, sampling costs, and not the computation or memory costs, are the dominant consideration in designing the technique for sampling. We have developed a new methodology for addressing the clustering problem on the deep web. Our work includes three new ideas, which are a method for stratifying a deep web data source, an algorithm for hierarchical clustering based on stratified sampling, and a two phase technique for sampling, which includes a representative sampling in the first phase, and sampling focusing on the boundary points between the clusters in the second phase. We have evaluated our approach using two synthetic and one real data set. Our experiments show that each of the three ideas we have introduced leads to significant improvements in accuracy and efficiency of clustering a hidden data source. Specifically, we improve the accuracy of the clusters obtained (measured by average distance to centers) by up to 20% over the existing approach. Compared in another way, our method can achieve the same accuracy with up to 25% fewer samples, thus reducing the sampling cost.
Tantan Liu, Gagan Agrawal
SDM2
2011 Effective stratification for low selectivity queries on deep web data sources
abstract
We study the problem of estimating the result of an aggregation query with low selectivity when a data source only supports limited data accesses. Existing stratified sampling techniques cannot be applied to such a problem since either it is very hard, if not impossible, to gather certain critical statistics from such a data source, or more importantly, the selective attribute of the query may not be queriable on the data source. In such cases, we need an effective mechanism to stratify the data and form homogeneous strata with respect to the selective attribute of the query, despite not being able to query the data source with the selective attribute.
Fan Wang 0004, Gagan Agrawal
CIKM2
2011 Effective and efficient sampling methods for deep web aggregation queries
abstract
A large part of the data on the World Wide Web resides in the deep web. Executing structured, high-level queries on deep web data sources involves a number of challenges, several of which arise because query execution engines have a very limited access to data. In this paper, we consider the problem of executing aggregation queries involving data enumeration on these data sources, which requires sampling. The existing work in this area (HDSampler and its variants) is based on simple random sampling. We observe that this approach cannot obtain good estimates when the data is skewed. While there has been a lot of work on sampling skewed data, the existing methods are based on prior knowledge of data, and are therefore not applicable to hidden databases.
Fan Wang 0004, Gagan Agrawal
EDBT2
2011 Active learning based frequent itemset mining over the deep web
abstract
In recent years, one mode of data dissemination has become extremely popular, which is the deep web. A key characteristics of deep web data sources is that data can only be accessed through the limited query interface they support. This paper develops a methodology for mining the deep web. Because these data sources cannot be accessed directly, thus, data mining must be performed based on sampling of the datasets. The samples, in turn, can only be obtained by querying the deep web databases with specific inputs. Unlike existing sampling based methods, which are typically applied on relational databases or streaming data, sampling costs, and not the computation or memory costs, are the dominant consideration in designing the algorithm. In this paper, we specifically target the frequent itemset mining problem, and develop a method based on the theory of active learning. We focus on effectively obtaining a sample that can achieve a good estimation for the support values of 1-itemsets comprising an output attribute. In our method, a Bayesian network is utilized to describe the relationship between the input and the output attributes. We have evaluated our method using one synthetic and two real datasets. Our comparison shows significant gains in estimation accuracy from both the novel aspects of our work, i.e., the use of active learning and modeling a deep web source with a Bayesian network. On all three datasets, by sampling less than 10% of all data records, we could achieve more than 95% accuracy in estimating the support of frequent itemsets.
Tantan Liu, Gagan Agrawal
ICDE2
2011 An approach towards automatic workflow composition through information retrieval
abstract
Understanding how to design, manage, and execute scientific workflows has become increasingly esoteric. Yet, despite the development of scientific workflow management systems, which have simplified workflow planning to some extent, a means to reduce the complexity of user interaction without forfeiting some robustness has been elusive. We believe that a keyword interface may be highly beneficial to common users in need of information which requires workflow planning and execution. In this paper, we describe a system that can automatically compose a set of relevant workflows, which may or may not have been previously defined by other users, given only a keyword query. We present a way to index data sets and Web services (utilized to compose workflows in our system) on their ontological attributes. This ontology allows us to facilitate an IR-based workflow retrieval model. We conducted a case study in geoinformatics with a set of real geospatial Web services, data, and their metadata annotations. our system was capable of answering six keyword queries with fast search times (2.16ms on average) and relatively high Top-N precision values: 78%, 77.3%, and 76.2% for the Top 3, 5, and 10 retrieved workflows respectively.
David Chiu 0001, Travis Hall, Farhana Kabir, Gagan Agrawal
IDEAS4
2011 Answering complex structured queries over the deep web
abstract
A large part of the data on the World Wide Web resides in the deep web. Most deep web data sources only support simple text interfaces for querying them, which are easy to use but have limited expressive power. Therefore, processing complex structured queries over the deep web currently involves a large amount of manual work. Our work focuses on addressing the existing gap between users' need of expressing and executing complex structured queries over the deep web, and the simple and limited input interfaces of the existing deep web data sources.
Fan Wang 0004, Gagan Agrawal
IDEAS2
2011 Keyword Search Support for Automating Scientific Workflow Composition
David Chiu 0001, Travis Hall, Farhana Kabir, Gagan Agrawal
SSDBM4
2010 Query Reuse Based Query Planning for Searches over the Deep Web
Fan Wang 0004, Gagan Agrawal
DEXA (2)2
2010 Stratified Sampling for Data Mining on the Deep Web
abstract
In recent years, one mode of data dissemination has become extremely popular, which is the deep web. Like any other data source, data mining on the deep web can produce important insights or summary of results. However, data mining on the deep web is challenging because the databases cannot be accessed directly, and therefore, data mining must be performed based on sampling of the datasets. The samples, in turn, can only be obtained by querying the deep web databases with specific inputs. In this paper, we target two related data mining problems, which are association mining and differential rule mining. We develop stratified sampling methods to perform these mining tasks on a deep web source. Our contributions include a novel greedy stratification approach, which processes the query space of a deep web data source recursively, and considers both the estimation error and the sampling costs. We have also developed an optimized sample allocation method that integrates estimation error and sampling costs. Our experiment results show that our algorithms effectively and consistently reduce sampling costs, compared with a stratified sampling method that only considers estimation error. In addition, compared with simple random sampling, our algorithm has higher sampling accuracy and lower sampling costs.
Tantan Liu, Fan Wang 0004, Gagan Agrawal
ICDM3
2009 Extracting Output Metadata from Scientific Deep Web Data Sources
abstract
Increasingly, many data sources appear as online databases, hidden behind query forms, thus forming the deep Web. The popularity of this new medium for data dissemination is leading to new problems in data integration. Particularly, to enable data integration from multiple deep Web data sources, one needs to obtain the metadata for each of the data sources. Obtaining the metadata, particularly, the output schema, can be very challenging. This is because, given an input query, many deep web data sources only return a subset of the output schema attributes, i.e, the ones that have a non-NULL value for the corresponding input. In this paper, we propose two approaches, which are the sampling model approach and the mixture model approach, respectively, to efficiently obtain an approximately complete set of output schema attributes from a deep Web data source. Our experiments show while each of the above two approaches has limitations, a hybrid strategy, where we combine the two approaches, achieves high recall with good precision for most data sources.
Fan Wang 0004, Gagan Agrawal
ICDM2
2009 Enabling Ad Hoc Queries over Low-Level Scientific Data Sets
David Chiu 0001, Gagan Agrawal
SSDBM2
2009 SEEDEEP: A System for Exploring and Querying Scientific Deep Web Data Sources
Fan Wang 0004, Gagan Agrawal
SSDBM2
2008 Composing geoinformatics workflows with user preferences
abstract
With the advent of the data grid came a novel distributed scientific computing paradigm known as service-oriented sci-ence. Among the plethora of systems included under this framework are scientific workflow management systems, which enable large-scale process scheduling and execution. To en-sure quality of service, these systems typically seek to min-imize workflow execution time as well as costs for slices of data grid access. The geospatial domain, among other sci-ences, involves yet another optimization factor, the accu-racy of results. The relationship between execution time and workflow accuracy can often be exploited to offer more flexibility in handling user preferences. We present a system which meets user constraints through a dynamic adjustment of the accuracy of workflow results.
David Chiu 0001, Sagar Deshpande, Gagan Agrawal, Rongxing Li
GIS3
2008 Query Planning for Searching Inter-dependent Deep-Web Databases
Fan Wang 0004, Gagan Agrawal, Ruoming Jin
SSDBM2
2006 Systematic Approach for Optimizing Complex Mining Tasks on Multiple Databases
abstract
Many real world applications involve not just a single dataset, but a view of multiple datasets. These datasets may be collected from different sources and/or at different time instances. In such scenarios, comparing patterns or features from different datasets and understanding their relationships can be an extremely important part of the KDD process. This paper considers the problem of optimizing a mining task over multiple datasets, when it has been expressed using a highlevel interface. Specifically, we make the following contributions: 1) We present an SQL-based mechanism for querying frequent patterns across multiple datasets, and establish an algebra for these queries. 2) We develop a systematic method for enumerating query plans and present several algorithms for finding optimized query plan which reduce execution costs. 3) We evaluate our algorithms on real and synthetic datasets, and show up to an order of magnitude performance improvement
Ruoming Jin, Gagan Agrawal
ICDE2
2006 New Sampling-Based Estimators for OLAP Queries
abstract
One important way in which sampling for approximate query processing in a database environment differs from traditional applications of sampling is that in a database, it is feasible to collect accurate summary statistics from the data in addition to the sample. This paper describes a set of sampling-based estimators for approximate query processing that make use of simple summary statistics to to greatly increase the accuracy of sampling-based estimators. Our estimators are able to give tight probabilistic guarantees on estimation accuracy. They are suitable for low or high dimensional data, and work with categorical or numerical attributes. Furthermore, the information used by our estimators can easily be gathered in a single pass, making them suitable for use in a streaming environment.
Ruoming Jin, Leonid Glimcher, Chris Jermaine, Gagan Agrawal
ICDE4
2006 Parallelizing XQuery In a Cluster Environment
abstract
In this paper, we report on a parallel implementation of XQuery. As XQuery is being used for processing large datasets, and/or for compute-intensive applications, efficiency of XQuery implementations is becoming an important issue. Our work has specifically focused on scientific data processing and data mining applications. Parallelization of this class of XQuery queries involves a number of challenges, which include data distribution, parallelization of generalized reductions, and translation to an imperative language like C/C++, so as to invoke efficient parallel communication libraries. In this paper, we report our solutions towards the above problems. By implementing the techniques in a compiler and generating code based on a C++ SAX parser and the message passing interface (MPI), we are able to achieve efficient parallel execution on a cluster of machines
Xiaogang Li 0001, Gagan Agrawal
IDEAS2
2006 Fast and exact out-of-core and distributed k-means clustering
Ruoming Jin, Anjan Goswami, Gagan Agrawal
Knowl. Inf. Syst.3
2005 An Algorithm for In-Core Frequent Itemset Mining on Streaming Data
abstract
Frequent item set mining is a core data mining operation and has been extensively studied over the last decade. This paper takes a new approach for this problem and makes two major contributions. First, we present a one pass algorithm for frequent item set mining, which has deterministic bounds on the accuracy, and does not require any out-of-core summary structure. Second, because our one pass algorithm does not produce any false negatives, it can be easily extended to a two pass accurate algorithm. Our two pass algorithm is very memory efficient, and allows mining of datasets with large number of distinct items and/or very low support levels. Our detailed experimental evaluation on synthetic and real datasets shows the following. First, our one pass algorithm is very accurate in practice. Second, our algorithm requires significantly lower memory than Manku and Motwani's one pass algorithm and the multi-pass Apriori algorithm. Our two pass algorithm outperforms Apriori and FP-tree when the number of distinct items is large and/or support levels are very low. In other cases, it is quite competitive, with possible exception of cases where the average length of frequent item sets is quite high.
Ruoming Jin, Gagan Agrawal
ICDM2
2005 Simultaneous optimization of complex mining tasks with a knowledgeable cache
abstract
With an increasing use of data mining tools and techniques, we envision that a Knowledge Discovery and Data Mining System (KDDMS) will have to support and optimize for the following scenarios: 1) Sequence of Queries: A user may analyze one or more datasets by issuing a sequence of related complex mining queries, and 2) Multiple Simultaneous Queries: Several users may be analyzing a set of datasets concurrently, and may issue related complex queries.This paper presents a systematic mechanism to optimize for the above cases, targeting the class of mining queries involving frequent pattern mining on one or multiple datasets. We present a system architecture and propose new algorithms to simultaneously optimize multiple such queries and use a knowledgeable cache to store and utilize the past query results. We have implemented and evaluated our system with both real and synthetic datasets. Our experimental results show that our techniques can achieve a speedup of up to a factor of 9, compared with the systems which do not support caching or optimize for multiple queries.
Ruoming Jin, Kaushik Sinha, Gagan Agrawal
KDD3
2005 Discovering frequent topological structures from graph datasets
abstract
The problem of finding frequent patterns from graph-based datasets is an important one that finds applications in drug discovery, protein structure analysis, XML querying, and social network analysis among others. In this paper we propose a framework to mine frequent large-scale structures, formally defined as frequent topological structures, from graph datasets. Key elements of our framework include, fast algorithms for discovering frequent topological patterns based on the well known notion of a topological minor, algorithms for specifying and pushing constraints deep into the mining process for discovering constrained topological patterns, and mechanisms for specifying approximate matches when discovering frequent topological patterns in noisy datasets. We demonstrate the viability and scalability of the proposed algorithms on real and synthetic datasets and also discuss the use of the framework to discover meaningful topological structures from protein structure data.
Ruoming Jin, Chao Wang 0050, Dmitrii Polshakov, Srinivasan Parthasarathy 0001, Gagan Agrawal
KDD5
2005 Efficient Evaluation of XQuery over Streaming Data
Xiaogang Li 0001, Gagan Agrawal
VLDB2
2005 Shared Memory Parallelization of Data Mining Algorithms: Techniques, Programming Interface, and Performance
abstract
With recent technological advances, shared memory parallel machines have become more scalable, and offer large main memories and high bus bandwidths. They are emerging as good platforms for data warehousing and data mining. In This work, we focus on shared memory parallelization of data mining algorithms. We have developed a series of techniques for parallelization of data mining algorithms, including full replication, full locking, fixed locking, optimized full locking, and cache-sensitive locking. Unlike previous work on shared memory parallelization of specific data mining algorithms, all of our techniques apply to a large number of popular data mining algorithms. In addition, we propose a reduction-object-based interface for specifying a data mining algorithm. We show how our runtime system can apply any of the techniques we have developed starting from a common specification of the algorithm. We have carried out a detailed evaluation of the parallelization techniques and the programming interface. We have experimented with apriori and fp-tree-based association mining, k-means clustering, k-nearest neighbor classifier, and decision tree construction. The main results from our experiments are as follows: 1) Among full replication, optimized full locking, and cache-sensitive locking, there is no clear winner. Each of these three techniques can outperform others depending upon machine and dataset parameters. These three techniques perform significantly better than the other two techniques. 2) Good parallel efficiency is achieved for each of the four algorithms we experimented with, using our techniques and runtime system. 3) The overhead of the interface is within 10 percent in almost all cases. 4) In the case of decision tree construction, combining different techniques turned out to be crucial for achieving high performance.
Ruoming Jin, Ge Yang 0001, Gagan Agrawal
IEEE Trans. Knowl. Data Eng.3
2004 Fast and Exact Out-of-Core K-Means Clustering
abstract
Clustering has been one of the most widely studied topics in data mining and k-means clustering has been one of the popular clustering algorithms. K-means requires several passes on the entire dataset, which can make it very expensive for large disk-resident datasets. In view of this, a lot of work has been done on various approximate versions of k-means, which require only one or a small number of passes on the entire dataset. In this paper, we present a new algorithm which typically requires only one or a small number of passes on the entire dataset, and provably produces the same cluster centers as reported by the original k-means algorithm. The algorithm uses sampling to create initial cluster centers, and then takes one or more passes over the entire dataset to adjust these cluster centers. We provide theoretical analysis to show that the cluster centers thus reported are the same as the ones computed by the original k-means algorithm. Experimental results from a number of real and synthetic datasets show speedup between a factor of 2 and 4.5, as compared to k-means.
Anjan Goswami, Ruoming Jin, Gagan Agrawal
ICDM3
2003 Efficient decision tree construction on streaming data
abstract
Decision tree construction is a well studied problem in data mining. Recently, there has been much interest in mining streaming data. Domingos and Hulten have presented a one-pass algorithm for decision tree construction. Their work uses Hoeffding inequality to achieve a probabilistic bound on the accuracy of the tree constructed.In this paper, we revisit this problem. We make the following two contributions: 1) We present a numerical interval pruning (NIP) approach for efficiently processing numerical attributes. Our results show an average of 39% reduction in execution times. 2) We exploit the properties of the gain function entropy (and gini) to reduce the sample size required for obtaining a given bound on the accuracy. Our experimental results show a 37% reduction in the number of data instances required.
Ruoming Jin, Gagan Agrawal
KDD2
2003 Communication and Memory Efficient Parallel Decision Tree Construction
abstract
Decision tree construction is an important data mining problem. In this paper, we revisit this problem, with a new goal, i.e. Can we develop an efficient parallel algorithm for decision tree construction that can be parallelized in the same way as algorithms for other major mining tasks ?. We report a new approach to decision tree construction, which we refer to as SPIES (Statistical Pruning of Intervals for Enhanced Scalability). This approach combines RainForest based AVC groups with sampling to achieve memory efficient processing of numerical attributes. Overall, this algorithm has the following properties: 1) no preprocessing or sorting of input data is required, 2) the size of the data-structure required in the main memory is very small, 3) the only disk-traffic required is one pass for splitting nodes for each level of the tree, and no writing-back of data, 4) very low communication volume when this algorithm is parallelized, and 5) the same level of accuracy as an algorithm that does not use sampling or pruning. We show that this algorithm can be efficiently parallelized using the same high-level interface and runtime support that was previously used to parallelize association mining and clustering algorithms. This, we believe, is an important step towards offering high-level interfaces for parallel data mining. Moreover, we have efficiently parallelized this algorithm on a cluster of SMPs, i.e. combining shared memory and distributed memory parallelism, and over disk-resident datasets.
Ruoming Jin, Gagan Agrawal
SDM2
2002 Shared Memory Paraellization of Data Mining Algorithms: Techniques, Programming Interface, and Performance
abstract
1 Introduction With the availability of large datasets in application areas like bioinformatics, medical informatics, scientific data analysis, financial analysis, telecommunications, retailing, and marketing, it is becoming increasingly important to execute data mining tasks in parallel. At the same time, technological advances have made shared memory parallel machines commonly available to organizations and individuals. Vendors of these machines are targeting data warehousing and data mining as the major markets.
Ruoming Jin, Gagan Agrawal
SDM2
2001 A Middleware for Developing Parallel Data Mining Applications
abstract
1 Introduction Data mining is an interdisciplinary field, having applications in diverse areas like bioinformatics, medical informatics, scientific data analysis, financial analysis, consumer profiling, etc. In each of these application domains, the amount of data available for analysis has exploded in recent years, making the scalability of data mining implementations a critical factor. To this end, parallel versions of most of the well-known data mining techniques have been developed in recent years. However, the expertise and effort currently required in implementing, maintaining, and performance tuning a parallel data mining application is a severe impediment in the wide use of parallel computers for scalable data mining.
Ruoming Jin, Gagan Agrawal
SDM2
1992 Using Coding to Support Data Resiliency in Distributed Systems
abstract
A scheme for maintaining replicated files is suggested. The authors describe how the coding scheme suggested by M.O. Rabin (1987, 1989) can be used to store replicated data and how the voting algorithm and the quorum requirements change to manage this replication. It is shown that the disk storage space required to achieve a given availability is significantly lower than that for the conventional scheme with full file replication. Since coding is used, this scheme also provides a high degree of data security.>
Pankaj Jalote, Gagan Agrawal
ICDE2