We design and implement an experimental analysis comparing two relational systems (PostgreSQL and MariaDB) and two document-based NoSQL systems (MongoDB and CouchBase). We compare their performance on a single server, Decision Support (DSS) scenario. We argue that DSS is becoming an important case study for NoSQL. We experiment with several database designs and several query translations in order to investigate the effect of physical design and query optimization in document-based stores. Our results show that design is very important for MongoDB's performance, and that query optimization over documents is much less sophisticated on document-based stores than in relational data bases and needs to improve. Our results also offer some ideas to guide further development in this area.

Database technology, while quite powerful, is not well suited to Intelligence analysis. One of its drawbacks is the fact that queries are answered as they are posed to the system, and then forgotten. But analysts that are asked to write a report and query the database in order to gather information usually have a certain amount of time to work on such report. If the database is updated and changes are not reported, the analyst may miss relevant, up-to-date information. In this paper, we propose an approach to solve this problem. The approach uses a Broker architecture to develop a push-pull system that allows users to either push documents into the database or pull documents out of the database as needed. Our approach is conceptually simple and can be implemented on top of off-the-shelf relational technology. Also, it can be extended to support more sophisticated functionality.

Codd’s relational model describes just one possible world. To better cope with incomplete information, extended database models allow several possible worlds. Vague tables are one such convenient extended model where attributes accept sets of possible values (e.g., the manager is either Jill or Bob). However, conceptual database design in such cases remains an open problem. In particular, there is no canonical definition of functional dependencies (FDs) over possible worlds (e.g., each employee has just one manager). We identify several desirable properties that the semantics of such FDs should meet including Armstrong’s axioms, the independence from irrelevant attributes, seamless satisfaction and implied by strong satisfaction. We show that we can define FDs such that they have all our desirable properties over vague tables. However, we also show that no notion of FD can satisfy all our desirable properties over a more general model (disjunctive tables). Our work formalizes a trade-off between having a general model and having well-behaved FDs.

Functional dependencies (FDs) are an integral part of database design. However, they are only defined when we exclude null markers. However, we commonly use null markers in practice. To bridge this gap between theory and practice, researchers have proposed definitions of FDs over relations with null markers. Though sound, these definitions lack some qualities that we find desirable. For example, some fail to satisfy Armstrong's axioms—while these axioms are part of the foundation of common database methodologies. We propose a set of properties that any extension of FDs over relations with null markers should possess. We then propose two new extensions having these properties. These extensions attempt to allow null markers where they make sense to practitioners. They both support Armstrong's axioms and provide realizablenull markers: at any time, some or all of the null markers can be replaced by actual values without causing an anomaly. Our proposals may improve database designs.

I analyze the text of an article that appeared in this journal in 2007 that published the results of a questionnaire in which a number of experts were asked to define the concepts of data, information, and knowledge. I apply standard information retrieval techniques to build a list of the most frequent terms in each set of definitions. I then apply information extraction techniques to analyze how the top terms are used in the definitions. As a result, I draw data‐driven conclusions about the aggregate opinion of the experts. I contrast this with the original analysis of the data to provide readers with an alternative viewpoint on what the data tell us.

We propose a logical framework to analyze complex predicates (those involving a subquery) in SQL. We propose a new operator in the relational algebra for handling such predicates, and study its properties and how it combines with traditional relational operator. We focus on predicates of the form att θ MOD S, where att is an attribute, θ a comparison operator, MOD is one of SOME or ALL, and S is a (correlated or non-correlated) subquery. We provide a formal characterization of these predicate, as well as an implementation and optimization strategies for it. We show that our approach is extendible, so we can support the expression and optimization of other, similar predicates. Finally, we describe experimental evidence that the proposed approach is more efficient than the traditional approach across a variety of conditions.

We present research aimed at improving our understanding of the use and implementation of quantification in relational query languages in general and SQL in particular. In order to make our results as general as possible, we use the framework of Generalized Quantification. Generalized Quantifiers (GQs) are high-level, declarative logical operators that in the past have been studied from a theoretical perspective. In this paper we focus on their practical use, showing how to incorporate a dynamic set of GQs in relational query languages, how to implement them efficiently and use them in the context of SQL. We present experimental evidence of the performance of the approach, showing that it improves over traditional (relational) approaches.

One of the main challenges in intelligence work is to assess the trustworthiness of data sources. In an adversarial setting, in which the subjects under study actively try to disturb the data gathering process, trustworthiness is one of the most important properties of a source. The recent increase in usage of open source data has exacerbated the problem, due to the proliferation of sources. In this paper we propose computerized methods to help analysts evaluate the truthfulness of data sources (open or not). We apply methods developed in database and Semantic Web research to determine data quality (which includes truthfulness but also other related aspects like accuracy, timeliness, etc.). Research on data quality has made frequent use of provenance metadata. This is metadata related to the origin of the data: where it comes from, how and when it was obtained, and any relevant conditions that might help determine how it came to be in its current form. We study the application of similar methods to the particular situation of the Intelligence analyst, focusing on trust. This paper describes ongoing research; what is explained here is a first attempt at tackling this complex but very important problem. Due to lack of space, relevant work in the research literature is not discussed, and several technical considerations are omitted; finally, further research directions are only sketched.

We analyze the restrictions to information sharing imposed by MultiLevel Secure Databases. We offer several measures to quantify the information loss due to security levels, and propose the problem of minimizing such loss without allowing users to access information above their security clearance. We give a partial solution to the problem, and discuss some of its shortcomings.

The use of tags to annotate content creates an opportunity to explore alternatives to automate the process of extracting semantics from data sources. Semantic information is needed for many complex tasks like concept extraction and information integration. In order to establish the value of user-gen

We propose a solution to the problem of exploring large, complex data sets in a (relational) database by a human user. In a nutshell, our proposed solution is to develop the tools to support data exploration and browsing in an organized manner. We introduce: (a) an organization of data around the idea of defining distances on data sets, to reflect the intuitive notion of data that is (closely) related to other data; and (b) a new set of operators to mediate user-data interaction that exploits the defined distances to let the user ldquomove aroundrdquo the data. Our operators are simple and intuitive, yet they can be combined in a flexible manner to support complex browsing interactions. The operators are combined in a novel interface that enables intelligent data exploration and analysis.

In this paper, we present a complete framework and findings in mining Web usage patterns from Web log files of a real Web site that has all the challenging aspects of real-life Web usage mining, including evolving user profiles and external data describing an ontology of the Web content. Even though the Web site under study is part of a nonprofit organization that does not "sell" any products, it was crucial to understand "who" the users were, "what" they looked at, and "how their interests changed with time," all of which are important questions in Customer Relationship Management (CRM). Hence, we present an approach for discovering and tracking evolving user profiles. We also describe how the discovered user profiles can be enriched with explicit information need that is inferred from search queries extracted from Web log data. Profiles are also enriched with other domain-specific information facets that give a panoramic view of the discovered mass usage modes. An objective validation strategy is also used to assess the quality of the mined profiles, in particular their adaptability in the face of evolving user behavior.

Natural language analysis tools are very important for Intelligence tasks, since a considerable amount of information is available in documents of various types. The recent increase on use of open source (OSINT) has made documents even more abundant. Intelligence analysts require tools to help inspect, classify and analyze all this raw data. Situating documents (that is, finding their temporal and spatial coordinates) is vital to put events in the proper geo-strategical context; this in turn is an important part of the complex task of interpreting such events. Such information can help analysts relate events. In our project, we analyze documents at the sentence level. Each sentence is translated to a recursive structure anchored by a triple subject-action-object.

In first order logic there are two main extensions to quantification: generalized quantifiers and non-linear prefixes. While generalized quantifiers have been explored from a database perspective, non-linear prefixes have not -- most likely because of complexity concerns. In this paper we first illustrate the usefulness of non-linear prefixes in query languages by means of example queries. We then introduce the subject formally, \ndistinguishing between two forms of non-linearity: branching and cumulation. To escape complexity concerns, we focus on \nmonadic quantifiers. In this context, we show that branching does not extend the expressive power of first order logic when it is interpreted over finite models, while cumulation does not extend the expressive power when it is interpreted over bounded models. Branching and cumulation do, however, allow us to formulate some queries in a succinct and elegant manner. When branching and cumulation are interpreted over infinite models, we show that the resulting language can be embedded in an infinitary logic proposed by Libkin. We also discuss non-linear prefixes from an algorithmic point of view.

Most research work on optimization of nested queries focuses on aggregate subqueries. In this article, we show that existing approaches are not adequate for nonaggregate subqueries, especially for those having multiple subqueries and certain comparison operators. We then propose a new efficient approach, the nested relational approach, based on the nested relational algebra. The nested relational approach treats all subqueries in a uniform manner, being able to deal with nested queries of any type and any level. We report on experimental work that confirms that existing approaches have difficulties dealing with nonaggregate subqueries, and that the nested relational approach offers better performance. We also discuss algebraic optimization rules for further optimizing the nested relational approach and the issue of integrating it into relational database systems.

No abstract available.

One of the most powerful features of SQL is the use of nested queries. Most research work on the optimization of nested queries focuses on aggregate subqueries. However, the solutions proposed for non-aggregate subqueries are still limited, especially for queries having multiple subqueries and null values. In this paper, we show that existing approaches to queries containing non-aggregate subqueries proposed in the literature (including rewrites) are not adequate. We then propose a new efficient approach, the nested relational approach, based on the nested relational algebra. Our approach directly unnests non-aggregate subqueries using hash joins, and treats all subqueries in a uniform manner, being able to deal with nested queries of any type and any level. We report on experimental work that confirms that existing approaches have difficulties dealing with non-aggregate subqueries, and that our approach offers better performance. We also discuss some possibilities for algebraic optimization and the issue of integrating our approach in a relational database system.

MySQL is an OLTP system without the ability to handle SQL subqueries. As part of our project in query optimization, we add subquery processing to MySQL through rewriting. First, we incorporate the ability to handle subqueries in the FROM clause; then rewrite queries with subqueries in the WHERE clause by moving the subquery to the FROM clause and introducing suitable conditions to link the subquery and the main query tables. The result is a complete and correct unnesting procedure for SQL queries. The idiosyncrasies of SQL semantics make some rewritings tricky. While optimization issues are not directly addressed, the rewriting opens up the possibility of more efficient query processing in several cases.

Relational database systems do not effectively support complex queries containing quantifiers (quantified queries) that are increasingly becoming important in decision support applications. Generalized quantifiers provide an effective way of expressing such queries naturally. In this paper, we consider the problem of processing quantified queries within the generalized quantifier framework. We demonstrate that current relational systems are ill-equipped, both at the language and at the query processing level, to deal with such queries. We also provide insights into the intrinsic difficulties associated with processing such queries. We then describe the implementation of a quantified query processor, Q2P, that is based on multidimensional and boolean matrix structures. We provide results of performance experiments run on Q2P that demonstrate superior performance on quantified queries. Our results indicate that it is feasible to augment relational systems with query subsystems like Q2P for significant performance benefits for quantified queries in decision support applications.