Members from disadvantaged or minority groups are often more likely to have missing values in their record. Imputation is a common approach to deal with missing values before the data is being analyzed. Several studies have found interplay of imputation methods and classification methods with respect to accuracy and fairness: different combinations of imputation and classification methods will lead to different accuracy and fairness results. However, it is unclear how to choose the combination of imputation method and classification method to optimize the tradeoff between accuracy and fairness. An exhaustive search approach will be too expensive because it needs to check all combinations and measure both accuracy and fairness for every combination. This paper proposes a beam-search based method to select the optimal combination of imputation methods and classification methods. An empirical study was also conducted to compare the performance of the proposed method to exhaustive search. The proposed solution achieves the same result as the exhaustive search method but with much lower search cost.

To mitigate climate change impacts, carbon capture technologies have been implemented at significant CO2 emission points, such as industrial sites and electric power generation facilities. Solvent-based carbon capture solutions are pivotal in reducing atmospheric CO2 levels and enhancing air quality by capturing harmful pollutants. Amine-based solvents, favored for their efficiency in post-combustion CO2 capture, are susceptible to thermal and oxidative degradation, leading to complex emissions profiles that demand comprehensive management strategies. We develop a Machine Learning model designed to predict future amine emissions in real-time, thereby assisting in the formulation of mitigation strategies required for the operation of capture plants. We conducted an experiment using data from test campaigns run at the Technology Centre Mongstad (TCM). We employed a Long Short-Term Memory (LSTM) autoencoder model with dual-stage attention mechanisms to predict amine emissions using historical data. The results were quite promising: we achieved a mean absolute percentage error ranging from 5.8% to 6.8% percent for the real-time prediction of amine emissions. The results are better than existing approaches using simpler machine learning models as well as the standard LSTM autoencoder model.

Federated RDF systems allow users to retrieve data from multiple independent sources without needing to have all the data in the same triple store. The performance of these systems can be poor for large and geographically distributed RDF data where network transfer costs are high. This article introduces CBTP-OL and CBTP-Nhop, two novel join algorithms that take advantage of network topology to decrease the cost of processing Basic Graph Pattern (BGP) SPARQL queries in a geographically distributed environment. Federation members are grouped in clusters, based on the network communication cost between the members, and the bulk of the join processing is pushed to the clusters. Our CBTP-OL and CBTL-Nhop algorithms use anoverlap listand, respectively, anN-hop overlap list, to efficiently compute join results from triples in different clusters. We implement our algorithms in the OpenRDF Sesame federated framework and use Apache Rya triple store instances as federation members. Experimental evaluation results show the advantages of our approach over existing techniques.

Machine learning models have been widely used in security applications. However, it is well-known that adversaries can adapt their attacks to evade detection. There has been some work on making machine learning models more robust to such attacks. However, one simple but promising approach calledrandomizationis under-explored. In addition, most existing works focus on models with differentiable error functions while tree-based models do not have such error functions but are quite popular because they are easy to interpret. This paper proposes a novel randomization-based approach to improve robustness of tree-based models against evasion attacks. The proposed approach incorporates randomization into both model training time and model application time (meaning when the model is used to detect attacks). We also apply this approach to random forest, an existing ML method which already has incorporated randomness at training time but still often fails to generate robust models. We proposed a novel weighted-random-forest method to generate more robust models and a clustering method to add randomness at model application time. We also proposed a theoretical framework to provide a lower bound for adversaries’ effort. Experiments on intrusion detection and spam filtering data show that our approach further improves robustness of random-forest method.

Intrusion Detection Systems are commonly used by organizations to monitor network traffic and detect attacks or suspicious behaviours. However, many attacks occur across organizations and are often difficult to detect using any single IDS. Collaborative Intrusion Detection Systems could lead to more accurate prediction and detection of cyber threats as well as a reduction of security administrators’ workload as similar threats from different places can be merged. However, most organizations are unwilling to disclose sensitive information about their internal network topology and traffic, lending these systems unusable. Existing solutions using homomorphic encryption and secure multi-party computation are often expensive. In this paper, we propose efficient and privacy preserving techniques to correlate alerts generated at different organizations. We propose skPrototypes, a distributed clustering algorithm for horizontally partitioned mixed data using additive secret sharing. This algorithm can be used to create a privacy preserving, collaborative intrusion detection system. We also propose dpkPrototypes which uses differential privacy on categorical attributes and is more efficient than skPrototypes for categorical attributes with many distinct values. Theoretical and experimental results validate the effectiveness of our algorithms.

Users often need to integrate large amounts of RDF data from multiple sources. Although there has been a lot of work on data cleaning and integration for structured data, relatively little work has been done for RDF data. We consider two different approaches to data integration: 1) a traditional data warehouse approach where relevant RDF data is extracted from different sources and then integrated in a data warehouse; 2) a virtual integration approach where RDF data still resides at each source and data integration happens when the data is queried, through a mediator that coordinates with wrappers at each source. It is often unclear how to choose the appropriate approach given an application scenario. This paper proposes RDFINT, a benchmark to compare these two approaches for integrating RDF data. We describe typical data integration operations, metrics that can be used to compare these two approaches, and factors that affect these metrics. We also report preliminary results of an implementation of these two approaches using the Apache Jena Fuseki framework.

With advances of big data technologies, there are many possible ways to program for each big data application. A challenge is to know the differences of the program approaches and decide which programming approach is the best for a particular big data application. In this paper, we use vehicle travel companion as a case study to explore four different programming approaches, including Spark RDD (with GroupBy or Join), Spark SQL with Hive and Hive on Hadoop, and tune the programmed big data applications. Our experiments show that the execution time of one programming approach could be more than 100-fold longer than that of another for the same application logic, which verifies that programming approach decision is important. We also explain the reasons for the differences. The findings could be applied to the selection of programming approach for other big data applications.

Modern scientific databases and web databases maintain large and heterogeneous data. These real-world databases contain hundreds or even thousands of relations and attributes. Traditional predefined query forms are not able to satisfy various ad-hoc queries from users on those databases. This paper proposes DQF, a novel database query form interface, which is able to dynamically generate query forms. The essence of DQF is to capture a user's preference and rank query form components, assisting him/her in making decisions. The generation of a query form is an iterative process and is guided by the user. At each iteration, the system automatically generates ranking lists of form components and the user then adds the desired form components into the query form. The ranking of form components is based on the captured user preference. A user can also fill the query form and submit queries to view the query result at each iteration. In this way, a query form could be dynamically refined until the user is satisfied with the query results. We utilize the expected F-measure for measuring the goodness of a query form. A probabilistic model is developed for estimating the goodness of a query form in DQF. Our experimental evaluation and user study demonstrate the effectiveness and efficiency of the system.

Users often find that their queries against a database return too many answers, many of them irrelevant. A common solution is to rank the query results. The effectiveness of a ranking function depends on how well it captures users' preferences. However, database systems often do not have the complete information about users' preferences and users' preferences are often heterogeneous (i.e., some preferences are static and common to all users while some are dynamic and diverse). Existing solutions do not address these two issues. In this paper, we propose a novel approach to address these shortcomings: 1) it addresses the heterogeneous issue by using skyline to capture users' static and common preferences and using users' current navigational behavior to capture users' dynamic and diverse preferences; 2) it addresses the incompleteness issue by using a machine learning technique to learn a ranking function based on training examples constructed from the above two types of information. Experimental results demonstrate the benefits of our approach.

Helpdesk databases are used to store past interactions between customers and companies to improve customer service quality. One common scenario of using helpdesk database is to find whether recommendations exist given a new problem from a customer. However, customers often provide incomplete or even inaccurate information. Manually preparing a list of clarification questions does not work for large databases. This paper investigates the problem of automatic generation of a minimal number of questions to reach an appropriate recommendation. This paper proposes a novel dynamic active probing method. Compared to other alternatives such as decision tree and case-based reasoning, this method has two distinctive features. First, it actively probe the customer to get useful information to reach the recommendation, and the information provided by customer will be immediately used by the method to dynamically generate the next questions to probe. This feature ensures that all available information from the customer is used. Second, this method is based on a probabilistic model, and uses a data augmentation method which avoids overfitting when estimating the probabilities in the model. This feature ensures that the method is robust to databases that are incomplete or contain errors. Experimental results verify the effectiveness of our approach.

Database queries are often exploratory and users often find their queries return too many answers, many of them irrelevant. Existing work either categorizes or ranks the results to help users locate interesting results. The success of both approaches depends on the utilization of user preferences. However, most existing work assumes that all users have the same user preferences, but in real life different users often have different preferences. This paper proposes a two-step solution to address the diversity issue of user preferences for the categorization approach. The proposed solution does not require explicit user involvement. The first step analyzes query history of all users in the system offline and generates a set of clusters over the data, each corresponding to one type of user preferences. When user asks a query, the second step presents to the user a navigational tree over clusters generated in the first step such that the user can easily select the subset of clusters matching his needs. The user then can browse, rank, or categorize the results in selected clusters. The navigational tree is automatically constructed using a cost-based algorithm which considers the cost of visiting both intermediate nodes and leaf nodes in the tree. An empirical study demonstrates the benefits of our approach. Categories and Subject Descriptors

Environmental research and knowledge discovery both require extensive use of data stored in various sources and created in different ways for diverse purposes. We describe a new metadata approach to elicit semantic information from environmental data and implement semantic-based techniques to assist users in integrating, navigating, and mining multiple environmental data sources. Our system contains specifications of various environmental data sources and the relationships that are formed among them. User requests are augmented with semantically related data sources and automatically presented as a visual semantic network. In addition, we present a methodology for data navigation and pattern discovery using multi-resolution browsing and data mining. The data semantics are captured and utilized in terms of their patterns and trends at multiple levels of resolution. We present the efficacy of our methodology through experimental results.

Much of business XML data has accompanying XSD specifications. In many scenarios "shredding" such XML data into a relational storage is a popular paradigm. Optimizing evaluation of XPath queries overmuch XML data requires paying careful attention to both the logical and physical designs of the relational database where XML data is shredded. None of the existing solutions has taken into account physical design of the generated relational database. In this paper, we study the interplay of logical and physical design and conclude that 1) solving them independently leads to suboptimal performance and 2) there is substantial overlap between logical and physical designs: some well-known logical design transformations generate the same mappings as physical design. Furthermore, existing search algorithms are inefficient to search the extremely large space of logical and physical design combinations. We propose a search algorithm that carefully avoids searching duplicated mappings and utilizes the workload information to further prune the search space. Experimental results confirm the effectiveness of our approach.

In this paper, we examine the interplay of logical and physical design, and experimentally demonstrate that: (1) solving the logical mapping and the physical design problem independently leads to a suboptimal solution; (2) taking into account the physical design space impacts the space of logical mapping. Specifically, well-known outlining and inlining mapping options are rendered unnecessary because they are functionally subsumed by two physical design options: indexes and vertical partitioning. We propose a search algorithm that judiciously explores the extreme large combined space of logical and physical design. The algorithm only searches the XSD-specific logical design options and uses heuristics to further prune the search space. We experimentally compare the quality (in terms of the time to execute the query workload on resulting design) and efficiency (in terms of the search time) of our algorithm with known algorithms as well as a default XSD based mapping and an Edge-Table Mapping that does not use XSD on both real and synthetic data.

Describes efficient algorithms for accurately estimating the number of matches of a small node-labeled tree, i.e. a twig, in a large node-labeled tree, using a summary data structure. This problem is of interest for queries on XML and other hierarchical data, to provide query feedback and for cost-based query optimization. Our summary data structure scalably represents approximate frequency information about twiglets (i.e. small twigs) in the data tree. Given a twig query, the number of matches is estimated by creating a set of query twiglets, and combining two complementary approaches: set hashing, used to estimate the number of matches of each query twiglet, and maximal overlap, used to combine the query twiglet estimates into an estimate for the twig query. We propose several estimation algorithms that apply these approaches on query twiglets formed using variations on different twiglet decomposition techniques. We present an extensive experimental evaluation using several real XML data sets, with a variety of twig queries. Our results demonstrate that accurate and robust estimates can be achieved, even with limited space.

Over the last decades, improvements in CPU speed have outpaced improvements in main memory and disk access rates by orders of magnitude, enabling the use of data compression techniques to improve the performance of database systems. Previous work describes the benefits of compression for numerical attributes, where data is stored in compressed format on disk. Despite the abundance of string-valued attributes in relational schemas there is little work on compression for string attributes in a database context. Moreover, none of the previous work suitably addresses the role of the query optimizer: During query execution, data is either eagerly decompressed when it is read into main memory, or data lazily stays compressed in main memory and is decompressed on demand only

Decision-support applications in emerging environments require that SQL query results or intermediate results be shipped to clients for further analysis and presentation. These clients may use low bandwidth connections or have severe storage restrictions. Consequently, there is a need to compress the results of a query for efficient transfer and client-side access. This paper explores a variety of techniques that address this issue. Instead of using a fixed method, we choose a combination of compression methods that use statistical and semantic information of the query results to enhance the effect of compression. To represent such a combination, we present a framework of "compression plans" formed by composing primitive compression operators. We also present optimization algorithms that enumerate valid compression plans and choose an optimal plan. Our experiments show that our techniques achieve significant performance improvement over standard compression tools like WinZip.

In a variety of applications ranging from optimizing queries on alphanumeric attributes to providing approximate counts of documents containing several query terms, there is an increasing need to quickly and reliably estimate the number of strings (tuples, documents, etc.) matching a Boolean query. Boolean queries in this context consist of substring predicates composed using Boolean operators. While there has been some work in estimating the selectivity of substring queries, the more general problem of estimating the selectivity of Boolean queries over substring predicates has not been studied.

The Cornell Jaguar Project is exploring a variety of issues related to mobility and query processing. One broad theme is to break down the traditional client and server boundaries, leading to ubiquitous query processing. Another theme is to extend database and query processing techniques to small-scale and mobile devices. The project builds on and extends the Cornell PREDATOR database engine.