In recent years, we have witnessed an increasing demand to process big data in numerous applications. It is observed that there often exist substantial amounts of repetitive data in different portions of a big data repository/dataset for applications such as genome sequence analyses. In this paper, we present a novel method, called the VA-Store, to reduce the large space requirement for repetitive data in prevailing genome sequence analysis tasks using k-mers (i.e., subsequences of length k) with multiple k values. The VA-Store maintains a physical store for one portion of the input dataset (i.e., k0-mers) and supports multiple virtual stores for other portions of the dataset (i.e., k-mers with k ≠ k0). Utilizing important relationships among repetitive data, the VA-Store transforms a given query on a virtual store into one or more queries on the physical store for execution. Both precise and approximate transformations are considered. Accuracy estimation models for approximate solutions are derived. Query optimization strategies are suggested to improve query performance. Our experiments using real and synthetic datasets demonstrate that the VA-Store is quite promising in providing effective storage and efficient query processing for solving a kernel database problem on repetitive big data for genome sequence analysis applications.

Many existing clustering methods usually compute clusters from the reduced data sets obtained by summarizing the original very large data sets. BIRCH is a popular summary-based clustering method that first builds a CF tree, and then performs a global clustering using the leaf entries of the tree. However, to the best of our knowledge, no prior studies have proposed a global clustering method that uses the structure of a CF tree. Therefore, we propose a novel global clustering method ERC (effective multiple range queries-based clustering), which takes advantage of the structure of a CF tree. We further propose a CF+tree, which optimizes the node split scheme used in the CF tree. As a result, the CF+-ERC (CF+tree-based ERC) method effectively computes clusters over large data sets. Furthermore, it does not require a predefined number of clusters to compute the clusters. We present in-depth theoretical and experimental analyses of our method. Experimental results on very large synthetic data sets demonstrate that the proposed approach is effective in terms of cluster quality and robustness and is significantly faster than existing clustering methods. In addition, we apply our clustering method to real data sets and achieve promising results.

Effective mining of huge amount of DNA and RNA fragments generated by next generation sequencing (NGS) technologies is facilitated by developing efficient tools to partition these sequence fragments (reads) based on their level of similarities using edit distance. However, edit distance calculation for all pairwise sequence fragments to cluster these huge data sets is a significant performance bottleneck. In this paper we propose a predicted Edit distance based clustering to significantly lower clustering time. Existing clustering methods for sequence fragments, such as, k-mer based VSEARCH and Locality Sensitive Hash based LSH-Div achieve much reduced clustering time but at the cost of significantly lower cluster quality. We show, through extensive performance analysis, clustering based on this predicted Edit distance provides more than 99% accurate clusters while providing an order of magnitude faster clustering time than actual Edit distance based clustering.

Box queries on a dataset in a multidimensional data space are a type of query which specifies a set of allowed values for each dimension. Indexing a dataset in a multidimensional Non-ordered Discrete Data Space (NDDS) for supporting efficient box queries is becoming increasingly important in many application domains such as genome sequence analysis. The BoND-tree was recently introduced as an index structure specifically designed for box queries in an NDDS. Earlier work focused on developing strategies for building an effective BoND-tree to achieve high query performance. Developing efficient and effective techniques for deleting indexed vectors from the BoND-tree remains an open issue. In this paper, we present three deletion algorithms based on different underflow handling strategies in an NDDS. Our study shows that incorporating a new BoND-tree inspired heuristic can provide improved performance compared to the traditional underflow handling heuristics in NDDSs.

In the past, genome sequence databases had used main memory indexing, such as the suffix tree, for fast sequence searches. With next generation sequencing technologies, the amount of sequence data being generated is huge and main memory indexing is limited by the amount of memory available. K-mer based techniques are being more used for various genome sequence database applications such as local alignment. K-mer can also provide an excellent basis for creating efficient disk based indexing. In this paper, we have proposed a k-mer based database searching and local alignment tool using box queries on BoND-SD-tree indexing. BoND-tree is quite efficient for indexing and searching in Non-Ordered Discrete Data Space (NDDS). We have conducted experiments on searching DNA sequence databases using back translated protein query sequences and have compared with existing methods. We have also implemented local alignment of back translated protein query sequences with large DNA sequence databases using this index based k-mer search. Performances of this local alignment approach has been compared with that of Tblastn of NCBI. The results are quite promising and justify significance of the proposed approach.

This paper presents a novel approach that transforms the feature space into a new feature space such that a range query in the original space is mapped into an equivalent box query in the transformed space. Since box queries are axis aligned, there are several implementational advantages that can be exploited to speed up the retrieval of query results using R-Tree [9] like indexing schemes. For two dimensional data, the transformation is precise. For larger than two dimensions, we propose a space transformation scheme based on disjoint planer rotation and a new type of query, pruning box query, to get the precise results. Experimental results with large synthetic databases and some real databases show the effectiveness of the proposed transformation scheme. These experimental results have been corroborated with suitable mathematical models. In disjoint planer rotation, additional computation time is required to remove the false positives produced due to the bounding box not being precise. A second topological transformation scheme is presented based on optimized bounding box, which reduces the amount of false positives. The amount of this reduction is more with increasing dimensions. Optimized bounding box for higher dimensions is computed based on a novel approach of simultaneous local optimal projections.

Box queries (or window queries) are a type of query which specifies a set of allowed values in each dimension. Indexing feature vectors in the multidimensional Nonordered Discrete Data Spaces (NDDS) for efficient box queries are becoming increasingly important in many application domains such as genome sequence databases. Most of the existing work in this field targets the similarity queries (range queries and k-NN queries). Box queries, however, are fundamentally different from similarity queries. Hence, the same indexing schemes designed for similarity queries may not be efficient for box queries. In this paper, we present a new indexing structure specifically designed for box queries in the NDDS. Unique characteristics of the NDDS are exploited to develop new node splitting heuristics. For the BoND-tree, we also provide theoretical analysis to show the optimality of the proposed heuristics. Extensive experiments with synthetic data demonstrate that the proposed scheme is significantly more efficient than the existing ones when applied to support box queries in NDDSs. We also show effectiveness of the proposed scheme in a real-world application of primer design for genome sequence databases.

With the advent of next-generation sequencing and culture-independent methods, we now are accumulating an enormous amount of metagenomic data from microbial communities. These data sets are large, hard to assemble, and might encode rare or novel proteins, posing new computational challenges for protein homology search. This paper presents a novel protein homology search algorithm that combines the salient features of pairwise sequence alignment programs such as Blast and protein family based tools such as Hmmer. It is optimized for protein annotation in metagenomic data sets because: 1) it is fast, 2) it can classify short protein fragments encoded by individual sequence reads, 3) it can find homologs to novel or rare protein families when there is not enough member sequences to build a probabilistic model. Our algorithm builds a new indexing data structure called BlastTree, which can index a large sequence family database because of our effective compression techniques. In addition, BlastTree fully exploits sequence family membership information to improve homology search sensitivity. When the BlastTree Search algorithm is incorporated into Hmmer, it runs in a fraction of the time with comparable quality.

Range queries based on L 1 distance are a common type of queries in multimedia databases containing feature vectors. We propose a novel approach that transforms the feature space into a new feature space such that range queries in the original space are mapped into equivalent box queries in the transformed space. Since box queries are axes aligned, there are several implementational advantages that can be exploited to speed up the retrieval of query results. For two dimensional data the transformation is precise. For greater than two dimensions we propose a space transformation scheme based on disjoint planer rotation, and along with pruning query box the results are precise. Experimental results with large synthetic databases and some real databases show the effectiveness of the proposed transformation scheme. These experimental results have been corroborated with appropriate mathematical models.

Numerous techniques have been proposed in the past for supporting efficient k -nearest neighbor ( k -NN) queries in continuous data spaces. Limited work has been reported in the literature for k -NN queries in a nonordered discrete data space (NDDS). Performing k -NN queries in an NDDS raises new challenges. The Hamming distance is usually used to measure the distance between two vectors (objects) in an NDDS. Due to the coarse granularity of the Hamming distance, a k -NN query in an NDDS may lead to a high degree of nondeterminism for the query result. We propose a new distance measure, called Granularity-Enhanced Hamming (GEH) distance, which effectively reduces the number of candidate solutions for a query. We have also implemented k -NN queries using multidimensional database indexing in NDDSs. Further, we use the properties of our multidimensional NDDS index to derive the probability of encountering valid neighbors within specific regions of the index. This probability is used to develop a new search ordering heuristic. Our experiments on synthetic and genomic data sets demonstrate that our index-based k -NN algorithm is efficient in finding k -NNs in both uniform and nonuniform data sets in NDDSs and that our heuristics are effective in improving the performance of such queries.

Contemporary database applications often perform queries in hybrid data spaces (HDS) where vectors can have a mix of continuous valued and non-ordered discrete valued dimensions. To support efficient query processing for an HDS, a robust indexing method is required. Existing indexing techniques to process queries efficiently either apply to continuous data spaces (e.g., the R-tree) or non-ordered discrete data spaces (e.g., the ND-tree). No techniques directly indexing vectors in HDSs have been reported in the literature. In this paper, we propose a new multidimensional indexing technique, called the C-ND tree, to directly index vectors in an HDS. To build such an index, we first introduce some essential geometric concepts (e.g., hybrid bounding rectangle) in HDSs. The C-ND tree structure and the relevant tree building and query processing algorithms based on these geometric concepts in HDSs are then presented. Strategies have been suggested to make the values in continuous dimensions and non-ordered discrete dimensions comparable and controllable. Novel node splitting heuristics which exploit characteristics of both continuous and discrete dimensions are proposed. Performance of the C-ND tree is compared with that of linear scan, R*-tree and ND-tree using range queries on hybrid data. Experimental results demonstrate that the C-ND tree is quite promising in supporting range queries in HDSs.

A k-nearest neighbor (k-NN) query retrieves k objects from a database that are considered to be the closest to a given query point. Numerous techniques have been proposed in the past for supporting efficient k-NN searches in continuous data spaces. No such work has been reported in the literature for k-NN searches in a non-ordered discrete data space (NDDS). Performing k-NN searches in an NDDS raises new challenges. The Hamming distance is usually used to measure the distance between two vectors (objects) in an NDDS. Due to the coarse granularity of the Hamming distance, a k-NN query in an NDDS may lead to a large set of candidate solutions, creating a high degree of non-determinism for the query result. We propose a new distance measure, called granularity-enhanced Hamming (GEH) distance, that effectively reduces the number of candidate solutions for a query. We have also considered using multidimensional database indexing for implementing k-NN searches in NDDSs. Our experiments on synthetic and genomic data sets demonstrate that our index-based k-NN algorithm is effective and efficient in finding k-NNs in NDDSs.

The profile hidden Markov model (PHMM) has received increasing attention in the field of protein homology detection, since profile-based methods are much more sensitive in detecting distant homologous relationships than pairwise methods. Pure dynamic-programming-based systems are often used for PHMM searches. However, these dynamic-programming- based systems are very time consuming for a large database. For instance, it may take approximately 15 minutes to search a short model of length 12 in the GenBank protein sequence database. Instead of searching the database sequentially, we search the database based on a tree-structured database indexing, called the HD-tree. The HD-tree is able to reduce the PHMM search time significantly without reducing the quality of search results. Performance of search using the HD-tree is compared with that of HMMER, a popular implementation of PHMM for protein sequence analysis. It is shown that the HD-tree approach is orders of magnitude faster than HMMER for short queries

Similarity searches in multidimensional Non-ordered Discrete Data Spaces (NDDS) are becoming increasingly important for application areas such as bioinformatics, biometrics, data mining and E-commerce. Efficient similarity searches require robust indexing techniques. Unfortunately, existing indexing methods developed for multidimensional (ordered) Continuous Data Spaces (CDS) such as the R-tree cannot be directly applied to an NDDS. This is because some essential geometric concepts/properties such as the minimum bounding region and the area of a region in a CDS are no longer valid in an NDDS. Other indexing methods based on metric spaces such as the M-tree and the Slim-trees are too general to effectively utilize the special characteristics of NDDSs, resulting in nonoptimized performance. In this article, we propose a new dynamic data-partitioning-based indexing technique, called the ND-tree, to support efficient similarity searches in an NDDS. The key idea is to extend the relevant geometric concepts as well as some indexing strategies used in CDSs to NDDSs. Efficient algorithms for ND-tree construction and techniques to solve relevant issues such as handling dimensions with different alphabets in an NDDS are presented. Our experimental results on synthetic data and real genome sequence data demonstrate that the ND-tree outperforms the linear scan, the M-tree and the Slim-trees for similarity searches in multidimensional NDDSs. A theoretical model is also developed to predict the performance of the ND-tree for random data.

There is an increasing demand for similarity searches in a multidimensional non-ordered discrete data space (NDDS) from application areas such as bioinformatics and data mining. The non-ordered and discrete nature of an NDDS raises new challenges for developing efficient indexing methods for similarity searches. In this article, we propose a new indexing technique, called the NSP-tree , to support efficient similarity searches in an NDDS. As we know, overlap causes a performance degradation for indexing methods (e.g., the R-tree) for a continuous data space. In an NDDS, this problem is even worse due to the limited number of elements available on each dimension of an NDDS. The key idea of the NSP-tree is to use a novel discrete space-partitioning (SP) scheme to ensure no overlap at each level in the tree. A number of heuristics and strategies are incorporated into the tree construction algorithms to deal with the challenges for developing an SP-based index tree for an NDDS. Our experiments demonstrate that the NSP-tree is quite promising in supporting efficient similarity searches in NDDSs. We have compared the NSP-tree with the ND-tree, a data-partitioning-based indexing technique for NDDSs that was proposed recently, and the linear scan using different NDDSs. It was found that the search performance of the NSP-tree was better than those of both methods.

Broadcast has often been used to disseminate frequently requested data efficiently to a large volume of mobile units over single or multiple channels. Since mobile units have limited battery power, the minimization of the access and tuning times for the broadcast data is an important problem. There have been many research efforts that focus on minimizing access and tuning times by providing indexes on the broadcast data. We have studied an efficient index allocation method for broadcast data with skewed access frequencies over multiple physical channels which cannot be coalesced into a single high bandwidth channel. Previously proposed index allocation techniques have one of two problems. The first problem is that they require equal size for both index and data. The second problem is that their performance degrades when the number of given physical channels is not enough. These two problems result in an increased average access time for the broadcast data. To cope with these problems, we propose a tree-structured index allocation method. Our method minimizes the average access time by broadcasting the hot data and their indices more frequently than the less hot data and their indexes over the dedicated index and data channels. We present an in-depth experimental and theoretical analysis of our method by comparing it with other similar techniques. Our performance analysis shows that it significantly decreases the average access and tuning times for the broadcast data over existing methods.

The Euclidean distance measure has been used in comparing feature vectors of images, while the cosine angle distance measure is used in document retrieval. We theoretically analyze these two distance measures based on feature vectors normalized by image size and experiment with them in the context of a color image database. We find that the cosine angle distance, in general, works equally well for image databases. We show, for a given query vector, the characteristics of feature vectors that will be favored by one measure but not by the other. We compute k-nearest neighbors for query images using both Euclidean and cosine angle distance for a small image database. The experimental data corroborate our theoretical results.

We have analyzed the properties of the HSV (hue, saturation and value) color space with emphasis on the visual perception of the variation in hue, saturation and intensity values of an image pixel. We extract pixel features by either choosing the hue or the intensity as the dominant property based on the saturation value of a pixel. The feature extraction method has been applied for both image segmentation as well as histogram generation applications - two distinct approaches to content based image retrieval (CBIR). Segmentation using this method shows better identification of objects in an image. The histogram retains a uniform color transition that enables us to do a window-based smoothing during retrieval. The results have been compared with those generated using the RGB color space.

In this paper, we have developed a HiTi (Hierarchical MulTi) graph model for structuring large topographical road maps to speed up the minimum cost route computation. The HiTi graph model provides a novel approach to abstracting and structuring a topographical road map in a hierarchical fashion. We propose a new shortest path algorithm named SPAH, which utilizes HiTi graph model of a topographical road map for its computation. We give the proof for the optimality of SPAH. Our performance analysis of SPAH on grid graphs showed that it significantly reduces the search space over existing methods. We also present an in-depth experimental analysis of HiTi graph method by comparing it with other similar works on grid graphs. Within the HiTi graph framework, we also propose a parallel shortest path algorithm named ISPAH. Experimental results show that inter query shortest path problem provides more opportunity for scalable parallelism than the intra query shortest path problem.

We present an efficient indexing method called clustered SS-tree for high dimensional database for approximate nearest neighbor queries. The main idea of this approach is to cluster data before creating the index and then apply filtering of index nodes based on angle property in high dimension for efficient searching for clustered SS-tree provides several orders of magnitude better performance than exact nearest neighbor search of SS-tree and VA file.

We present a fast approximate nearest neighbor (NN) search index structure called the AB-tree, which uses heuristics to decide whether or not to access a node in the index tree based on the intersecting angle and the weight of the node. The goal of the NN search algorithm presented is to decrease unnecessary node accesses in the search due to overlap among bounding regions in existing index structures. We have observed the following three properties of bounding hyperspheres that motivated the creation of the proposed heuristics for NN search: distance property; radius property; and angle property.

MOTIVATION: Numerous database management systems have been developed for processing various taxonomic data bases on biological classification or phylogenetic information. In this paper, we present an integrated system to deal with interacting classifications and phylogenies concerning particular taxonomic groups. RESULTS: An information-theoretic view (taxon view) has been applied to capture taxonomic concepts as taxonomic data entities. A data model which is suitable for supporting semantically interacting dynamic views of hierarchic classifications and a query method for interacting classifications have been developed. The concept of taxonomic view and the data model can also be expanded to carry phylogenetic information in phylogenetic trees. We have designed a prototype taxonomic database system called HICLAS (HIerarchical CLAssification System) based on the concept of taxon view, and the data models and query methods have been designed and implemented. This system can be effectively used in the taxonomic revisionary process, especially when databases are being constructed by specialists in particular groups, and the system can be used to compare classifications and phylogenetic trees. AVAILABILITY: Freely available at the WWW URL: http://aims.cps.msu.edu/hiclas/ CONTACT: [email protected]; [email protected]

A number of hybrid systems have been proposed to combine the advantages of shared nothing and shared everything concepts for computing relational join operations. Most of these proposed systems, however, presented a few analytical results and have produced limited or no implementations on actual multiprocessors. In this paper, we present a parallel join algorithm with load-balancing for a hybrid system that combines both shared-nothing and shared-everything architectures. We derive an analytical model for the join algorithm on this architecture and validate it using both hardware/software simulations and actual experimentations. We study the performance of the join on the hybrid system for a wide range of system parameter values. We conclude that the hybrid system outperforms both shared-nothing and shared-everything architectures.

In navigation systems, a primary task is to compute the minimum cost route from the current location to the destination. One of the major problems for navigation systems is that a significant amount of computation time is required to find a minimum cost path when the topographical road map is large. Since navigation systems are real time systems, it is critical that the path be computed while satisfying a time constraint. We propose a new graph model named HiTi (hierarchical multi graph model), for efficiently computing an optimal minimum cost path. Based on HiTi graph model, we propose a new single pair minimum cost path algorithm. We empirically show that our proposed algorithm performs far better than the traditional A* algorithm. Further, we empirically analyze our algorithm by varying both edge cost distribution and hierarchical level number of HiTi graphs.

Multiple sequence alignment has been a useful technique for identifying RNA secondary structures. In this paper, an algorithm for aligning multiple RNA sequences to identify possible secondary structure is presented. In this algorithm, dot matrices generated from intra-sequence comparisons are used to obtain possible common secondary structures. A hit probability for dot matrices is calculated and a score function based on this hit probability is defined. Simulated annealing is applied to optimize the score function. The solution set of multiple sequence alignment is introduced, and the effects on the solution set of increasing the number of alignment gaps and the alignment length are analyzed. Several additional strategies to reduce simulated annealing time are applied. A method is applied to reduce the computation time based on the solution set. Also, an optimized transition rule, double shuffle, which moves two positions in a sequence with each iteration, is applied to increase the rate of convergence. This algorithm was used to find possible common secondary structures in RNA sequences.

In a distributed environment, it is advantageous to fragment a relation and store the fragments at various sites. Based on the concept of lattice structures, we develop a framework to study the fragmentation problems of distributed recursive relations. Two of the fragmentation problems are how to describe and identify fragments. Description and identification methods previously suggested are more suitable in parallel environments than in distributed databases. We propose a method to describe and identify fragments based on lattice structures. Finding lattice descriptions of fragments is shown to be an NP complete problem. We analyze the performance of the lattice approach both theoretically and experimentally. This is done by creating a database of recursive relations. The empirical analysis shows that our proposed algorithms give near optimal solutions.

In the domain of biological classification, classifications are performed hierarchically. There are no standard classifications which are unanimously accepted by the community of each domain; many different interacting views of classification exist about the same data, and the discovery of new data results in changes to the existing classification. Even a single individual may change his or her own classification of a particular group. Since multiple classification views interact, they are semantically related. It is difficult to model this kind of dynamically evolving and semantically interacting classification system using traditional data models, which lack the structural flexibility necessary to support dynamic views of hierarchic classifications, and cannot properly capture the history of these complex interactions. We have developed a new data model which is suitable for supporting semantically interacting dynamic views of hierarchic biological classifications. On the basis of our new data model we have developed a prototype database system called HICLAS (HIerarchical CLAssification System); its domain is plant taxonomy. HICLAS is available through the Internet and an X-window interface has been implemented to support queries to classification data.

We present a new semantics for extended logic programs with rule prioritization (EPP). The semantics, called reliable semantics (RS), generalizes the well-founded semantics which is defined for normal programs. It also generalizes the extended well-founded semantics which is defined for non-contradictory extended programs. Because of the classical negation in the head of the rules, the well-founded model of a program can be contradictory. RS is contradiction-free and defined for all EPPs. To avoid contradictions, only reliable rules and reliable closed world assumptions are used in the fixpoint computation of RS. A closed world assumption (CW A) is considered reliable if it is not ‘suspect’ for any constraint violation. A rule r is considered reliable if it is not ‘suspect’ for any constraint violation caused by rules with priority no lower than r. We define the stable r-models of an EPP and we show that the RS of a program coincides with its least stable r-model. The RS of an EPP, P, represents the skeptical ‘meaning’ of P whereas stable r-models of P represent the possible ‘meanings’ of P.

Parallel joins have been widely studied during the past decade and a number of efficient algorithms were presented. While it is known that the performance of these algorithms may suffer greatly in the presence of skewed input data, the work on load balancing schemes for parallel join has been limited. The main contribution of this paper is the development and analysis of a new distributed data structure and an effective load balancing scheme for parallel main memory hash join on NUMA architecture. Multiprocessors based on this architecture are scalable in both size of main memory and number of processors, and provide very high memory bandwidth. The load balancing scheme is based on random probing to avoid the hot spot problems caused by probing sequentially. We have modeled this load balancing scheme both analytically and experimentally. The experiments were run on a BBN TC2000 multiprocessor system.>

Multiple sequence alignment is a useful technique for studying molecular evolution and analyzing structure-sequence relationships. Dynamic programming of multiple sequence alignment has been widely used to find an optimal alignment. However, dynamic programming does not allow for certain types of gap costs, and it limits the number of sequences that can be aligned due to its high computational complexity. The focus of this paper is to use simulated annealing as the basis for developing an efficient multiple sequence alignment algorithm. An algorithm called Multiple Sequence Alignment using Simulated Annealing (MSASA) has been developed. The computational complexity of MSASA is significantly reduced by replacing the high-temperature phase of the annealing process by a fast heuristic algorithm. This heuristic algorithm facilitates in minimizing the solution set of the low-temperature phase of the annealing process. Compared to the dynamic programming approach, MSASA can (i) use natural gap costs which can generate better solution, (ii) align more sequences and (iii) take less computation time.

In data parallel applications, a major source of load trnbalance is in the uneven didribtitton of data between the nodes. The majar contribution of this paper is the analysis o j a new distributed Ioad balancing scheme based on random probing. Sequential probing was shown to create hot spot problems. Random probing, on the other hand, avoids these hot spots problems. We hove modeled this load balancing scheme bath anaiytically and experimentally. Distributed Hash Joan was used as an example application. The experimenls were done on a BEN TC2OOO rnulttprocessov- system.

The objective of this paper is to develop and analyze high performance hash based search methods for main memory databases. We define optimal search in main memory databases as the search that requires at most one key comparison to locate a record. Existing hashing techniques become impractical when they are adapted to yield optimal search in main memory databases because of their large directory size. Multi-directory hashing techniques can provide significantly improved directory utilization over single-directory hashing techniques. A multi-directory hashing scheme, called fast search multi-directory hashing, and its generalization, called controlled search multi-directory hashing, are presented. Both methods achieve linearly increasing expected directory size with the number of records. Their performance is compared to existing alternatives.

A multikey search query has many qualified records because it specifies some of the attributes and the rest of them are unspecified. Thus, data distribution among the parallel devices is important to enhance access concurrency for multikey search queries. Though there are some heuristics on data distribution for multikey accesses, range specification in a query has not been considered in those methods. In this paper we investigate optimal data distribution for multiattribute range queries in parallel processing file systems. We show for various-types of multiattribute range queries that there are inherent limitations to achieve optimal distribution. The results show that optimal distribution does not exist in many cases even for files with two fields. We give sufficient conditions for the nonexistence of perfect optimal distribution for certain types of multiattribute range queries. We have also developed data distribution methods for several useful multiattribute range queries. Sufficient conditions for optimal distribution by these proposed data distribution methods are given. It will be shown that the proposed distribution methods are perfect optimal for certain types of multiattribute range queries, and strict optimal for a large class of multiattribute range queries.

The performance improvement afforded by parallel processing of large-node B-trees is investigated. A node partitioning scheme to enhance concurrency for B-trees is presented. In the method, the tree height is reduced by increasing the node size. However, it is shown that increasing node size alone in conventional B-trees on parallel disks may not improve the response time, and may even degrade the performance. The proposed method achieves better performance while avoiding this anomaly.>

Two graph models are developed to determine the minimum required buffer size for achieving the theoretical lower bound on the number of disk accesses for performing relational joins. Here, the lower bound implies only one disk access per joining block or page. The first graph model is based on the block connectivity of the joining relations. Using this model, the problem of determining an ordered list of joining blocks that requires the smallest buffer is considered. It is shown that this problem as well as the problem of computing the least upper bound on the buffer size is NP-hard. The second graph model represents the page connectivity of the joining relations. It is shown that the problem of computing the least upper bound on the buffer size for the page connectivity model is also NP-hard. Heuristic procedures are presented for the page connectivity model and it is shown that the sequence obtained using the heuristics requires a near-optimal buffer size The authors also show the performance improvement of the proposed heuristics over the hybrid-has join algorithm for a wide range of join factors.>

In this paper we present data distribution methods for parallel processing environment. The primary objective is to process partial match retrieval type queries for parallel devices.

A reduced cover set of the set of full reducer semijoin programs for an acyclic query graph for a distributed database system is given. An algorithm is presented that determines the minimum cost full reducer program. The computational complexity of finding the optimal full reducer for a single relation is of the same order as that of finding the optimal full reducer for all relations. The optimization algorithm is able to handle query graphs where more than one attribute is common between the relations. A method for determining the optimum profitable semijoin program is presented. A low-cost algorithm which determines a near-optimal profitable semijoin program is outlined. This is done by converting a semijoin program into a partial order graph. This graph also allows one to maximize the concurrent processing of the semijoins. It is shown that the minimum response time is given by the largest cost path of the partial order graph. This reducibility is used as a post optimizer for the SSD-1 query optimization algorithm. It is shown that the least upper bound on the length of any profitable semijoin program is N(N-1) for a query graph of N nodes.>

Implementation of relational database queries by means of an index database machine is presented. The indexes are represented by a two-level structure. Performance analysis of this two-level index structure for reducing m-way join complexity is given. We also give a performance analysis of an algorithm for implementing m-way join on a proposed hardware architecture.

Several hardware algorithms to search for a large number of keys in a database are presented. These algorithms allow some false matches but guarantee hits for the desired search keys. This imperfectness allows us to design a variety of simple hardware searchers by using only RAM's and shift registers. Analytic models are developed to compare the performances of these hardware devices. Applications of these devices for database systems are also discussed.

A hardware pattern matcher is presented, which searches for patterns on a data flow, such as characters read from a disk. The backing up on the data flow, for a general pattern matching, is avoided by means of a set of cells running in parallel. Each cell can search for a pattern independently, but requires only one one-character comparator. The interesting feature of this search hardware is the use of simple priority line which can dynamically allocate these cells. Further, the number of cells required can be arbitrarily reduced by means of a marking technique which is also accomplished by the priority line. Finally, an information retrieval system, based on this basic pattern matching hardware is presented. Here the content as well as the context search are done by usign the same marking technique.

A graph model is presented to analyze the performance of a relational join. The amount of page reaccesses, the page access sequence, and the amount of buffer needed are represented in terms of graph parameters. By using the graph model formed from the index on the join attributes, we determine the relationships between these parameters. Two types of buffer allocation strategies are studied, and the upper bound on the buffer size with no page reaccess is given. This bound is shown to be the maximum cut value of a graph. Hence, the problem of computing this upper bound is NP-hard. We also give algorithms to determine a page access sequence requiring a near optimal buffer size with no page reaccess. The optimal page access sequence for a fixed buffer size has also been considered.

Several hardware database-searchers for a large number of patterns or keys are presented. These searchers can be implemented by a random access memory and are suitable for VLSI implementation. Application of these searchers as database filters is described; a filter detects all the matched records in the database as well as a few others which don't match. The percentage of these unmatched records that manage to pass through can be reduced to any arbitrary minimum value; this is done, for example, by using several filters together, or passing the output records repeatedly through the same filters. Several filters can be used together by cascading them in series, parallel, and series-parallel fashion. The performance of the filters using iterative approach depends very much on the regrouping algorithms of the patterns/keys. Several such algorithms are presented and their performances compared. A single pass is required if they are pipelined. Hardware organization for different pipelined approaches are also studied. Experiments are performed for all the different hardware organizations mentioned above on an employee-name database. The results are presented in graph forms. They support the usefulness of these hardware organizations as filters.