Most research in statistical databases has concentrated on retrieval, sampling, and aggregation type statistical queries. Data management issues associated with computational statistical operations have been ignored. As a first step towards integrating database management support of statistical operations, we have analyzed the performance of X'X, the QR decomposition, and the Singular Value Factorization. Alternative implementation strategies with respect to the relational and transposed storage organizations are developed. Implementation strategies corresponding to vector building block, vector-matrix, and direct algorithms with explicit buffer management are compared in terms of efficiency in performance.

This paper is intended to introduce those familiar with database management issues to the problems of managing large statistical databases. We begin with a characterization of statistical databases based on the structure and use of the data in the database. Several data management problems are then described. In particular, we discuss the problem of repetitive computations on large segments of the database during the lifetime of a statistical analysis. The organization of a data management system which avoids this problem by caching previously computed results and automatically maintaining their integrity is presented. We conclude with a list of problems that this organization raises and a discussion of related work.