Modern data estates are spread across data located on premises, on the edge and in one or more public clouds, spread across various sources like multiple relational databases, file and storage systems, and no-SQL systems, both operational and analytic; this phenomenon is referred to as data sprawl. Data administrators who wish to enforce compliance across the entire organization have to inventory their data, identify what parts of it are sensitive, and govern the sensitive data appropriately --- across the entirety of their sprawling data estate. Today, governance of data is completely siloed; each of the data subsystems has its own (and varied) governance features. Policies applied to sensitive data are applied piece-meal by iterating over all the data sources in a custom language specific to each source. This makes data governance cumbersome, error-prone (because a given policy must be manually enforced across different subsystems, inconsistencies can easily arise), and expensive. This paper presents Microsoft Purview , a service for unified governance of the entire data estate of an organization from a single central pane of glass. The Purview service consists of three parts: (1) a Data Map or metadata catalog that is populated by automated scanning of data sources in the organization, (2) a system to store and manage sensitivity classification of data, and (3) a policy system that enables data security officers to author and implement policies that span the entire organization, e.g., a policy that says, "Non-full-time employees should be denied access to data classified as PII (Personally Identifiable Information.") Purview transforms data governance across a complex data estate by offering the ability to govern centrally and automating data discovery, classification and policy enforcement. While other commercial catalog systems also build a global catalog, Purview is unique in its support for policies. It is also distinguished by covering both structured and unstructured data, thanks to its deep integration with Office 365 and its governance framework; indeed, "Microsoft Purview" represents a new unified offering that combines Office 365 governance and what was formerly a service for governing structured data called "Azure Purview". By integrating with Office 365's Rights Management Service, Purview offers central governance over structured data stored in databases and stores, reports in systems such as Power BI, as well as document data stored in Office 365. The Purview vision is to make the metadata in the Data Map increasingly richer through further automation and curation support and to use this 360 degree view of the data estate to support a wide range of governance policies, ranging from access control to lifecycle management (e.g., retention, deletion, restricting data movement). This paper covers the design and implementation challenges in building the Purview service for Attribute-Based Access Control (ABAC) policies, focusing specifically on a detailed description of its integration with Azure SQL Database. We illustrate the power of unifying Office 365 governance with structured data governance through Purview policies that enforce consistent access control even as data flows between Office 365 and structured data engines like Azure SQL Database. We also describe the results of our empirical evaluation of the performance overheads imposed by Purview.

There is a need for new information technology solutions that can help automate process coordination and integration tasks among enterprises. In this paper we describe the development of a simple process connector to link two scheduling applications. We describe specific challenges that we faced when building our prototype connector, especially in light of different levels of detail and constraints that must be propagated across the processes. We also report on some of the lessons learned from implementing our prototype and identify opportunities for future research. Our prototype and examples are specific to the construction domain but our results can be applied to process integration in other domains.

This paper describes application demonstrations of a new middleware that supports dynamic decision support over networks of resource-constrained devices. For the purposes of our demonstrations, we tap into intelligent job site applications for the construction domain. The demonstrations are carefully constructed to highlight our middleware's ability to provide on-demand access to local data, aggregation of data across dynamically defined regions, fusion of heterogeneous sensor data, and intelligent application-sensitive sensor clustering. The paper briefly describes the middleware and the details of the demonstrations

We discuss Morpheus, a data transformation construction tool and associated repository. The architecture of Morpheus is motivated by the goal to reuse (pieces of) previously written transformations to solve data integration problems by finding relevant ones in the repository and then modifying them for repurposing. In addition, Morpheus is integrated with a DBMS so as to leverage existing capabilities including the runtime environment for transforms. We discuss the architecture of Morpheus and illustrate its usage with the help of a simple transform construction scenario.

We present a novel approach to enable decision-making in a highly distributed multiagent environment where individual agents need to act in an autonomous fashion. Our architecture framework integrates risk management, knowledge management, and agent deliberation to enable sophisticated, autonomous decision-making. Instead of a centralized knowledge repository, our approach supports a highly distributed knowledge base in which each agent manages a fraction of the knowledge needed by the entire system.

We introduce our new, publicly available testbed and benchmark called THALIA (Test Harness for the Assessment of Legacy information Integration Approaches) for testing and evaluating integration technologies. THALIA provides researchers with a collection of 40 downloadable data sources representing University course catalogs from computer science departments worldwide. In addition, THALIA currently provides a set of twelve challenge queries as well as a scoring function for ranking the performance of an integration system. A second contribution is a systematic classification of the types of syntactic and semantic heterogeneities, which directly lead to the twelve challenge. We have chosen course information as our domain of discourse because it is well known and easy to understand. Furthermore, there is an abundance of data sources publicly available that allowed us to develop a testbed exhibiting all of the syntactic and semantic heterogeneities that we have identified.

Abstract We present an architecture for the adaptable delivery of video data under variable connection characteristics and into devices of variable capabilities. The main application of the proposed architecture is video delivery in wireless and mobile environments. The architecture is based on the Universal Multimedia Access concept and the MPEG‐7 standard. Based on the network and the mobile device, as well as constraints imposed by user preferences and the multimedia content, video is delivered through a careful application of a combination of off‐line and on‐line reductions to the video stream. We present our architecture and describe an implementation of a system based on the architecture. We present basic performance evaluation results to quantify the merit of our approach. Copyright © 2002 John Wiley & Sons, Ltd.

We present a three-tier architecture of middleware that addresses challenges facing accessibility, availability, and consistency of data in mobile environments. The architecture supports the automatic hoarding of data from multiple, heterogeneous sources into possibly a variety of different mobile devices. The middle tier enables the automation of synchronization tasks in both connected mode (following disconnection) and weakly connected mode, where only intelligent and effective synchronization can be used in the presence of a low-bandwidth network. We present the three-tier architecture based on the Coda file system.

A data warehouse stores information that is collected from multiple, heterogeneous information sources for the purpose of complex querying and analysis. Information in the warehouse is typically stored in the form of materialized views, which represent pre-computed portions of frequently asked queries. One of the most important tasks when designing a warehouse is the selection of materialized views to be maintained in the warehouse. The goal is to select a set of views in such a way as to minimize the total query response time over all queries, given a limited amount of time for maintaining the views (maintenance-cost view selection problem). In this paper, we propose an efficient solution to the maintenance-cost view selection problem using a genetic algorithm for computing a near-optimal set of views. Specifically, we explore the maintenance-cost view selection problem in the context of OR view graphs. We show that our approach represents a dramatic improvement in time complexity over existing search-based approaches using heuristics. Our analysis shows that the algorithm consistently yields a solution that lies within 10% of the optimal query benefit while at the same time exhibiting only a linear increase in execution time. We have implemented a prototype version of our algorithm which is used to simulate the measurements used in the analysis of our approach.

Article Free Access Share on CubiST: a new algorithm for improving the performance of ad-hoc OLAP queries Authors: Lixin Fu Computer & Information Science & Engineering, University of Florida, Gainesville, Florida Computer & Information Science & Engineering, University of Florida, Gainesville, FloridaView Profile , Joachim Hammer Computer & Information Science & Engineering, University of Florida, Gainesville, Florida Computer & Information Science & Engineering, University of Florida, Gainesville, FloridaView Profile Authors Info & Claims DOLAP '00: Proceedings of the 3rd ACM international workshop on Data warehousing and OLAPNovember 2000 Pages 72–79https://doi.org/10.1145/355068.355318Published:01 November 2000Publication History 20citation943DownloadsMetricsTotal Citations20Total Downloads943Last 12 Months49Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

With the emergence of e-business as the next killer application for the Web, automating bargaining-type negotiations between clients (i.e., buyers and sellers) has become increasingly important. With IDEAL (Internet based Dealmaker for e-business), we have developed an architecture and framework, including a negotiation protocol, for automated negotiations among multiple IDEAL servers. The main components of IDEAL are a constraint satisfaction processor (CSP) to evaluate a proposal, an Event-Trigger-Rule (ETR) server for managing and triggering the execution of rules which make up the negotiation strategy (rules can be updated at run-time to deal with the dynamic nature of negotiations), and a cost-benefit analysis to help in the selection of alternative strategies. We have implemented a fully functional prototype system of IDEAL to demonstrate automated negotiations among buyers and suppliers participating in a supply chain.

In order to access information from a variety of heterogeneous information sources, one has to be able to translate queries and data from one data model into another. This functionality is provided by so-called (source) wrappers [4,8] which convert queries into one or more commands/queries understandable by the underlying source and transform the native results into a format understood by the application. As part of the TSIMMIS project [1, 6] we have developed hard-coded wrappers for a variety of sources (e.g., Sybase DBMS, WWW pages, etc.) including legacy systems (Folio). However, anyone who has built a wrapper before can attest that a lot of effort goes into developing and writing such a wrapper. In situations where it is important or desirable to gain access to new sources quickly, this is a major drawback. Furthermore, we have also observed that only a relatively small part of the code deals with the specific access details of the source. The rest of the code is either common among wrappers or implements query and data transformation that could be expressed in a high level, declarative fashion.

No abstract available.

A warehouse is a repository of integrated information drawn from remote data sources. Since a warehouse effectively implements materialized views, we must maintain the views as the data sources are updated. This view maintenance problem differs from the traditional one in that the view definition and the base data are now decoupled. We show that this decoupling can result in anomalies if traditional algorithms are applied. We introduce a new algorithm, ECA (for Eager Compensating Algorithm), that eliminates the anomalies. ECA is based on previous incremental view maintenance algorithms, but extra compensating queries are used to eliminate anomalies. We also introduce two streamlined versions of ECA for special cases of views and updates, and we present an initial performance study that compares ECA to a view recomputation algorithm in terms of messages transmitted, data transferred, and I/O costs.

An approach to accommodating semantic heterogeneity in a federation of interoperable, autonomous, heterogeneous databases is presented. A mechanism is described for identifying and resolving semantic heterogeneity while at the same time honoring the autonomy of the database components that participate in the federation. A minimal, common data model is introduced as the basis for describing sharable information, and a three-pronged facility for determining the relationships between information units (objects) is developed. Our approach serves as a basis for the sharing of related concepts through (partial) schema unification without the need for a global view of the data that is stored in the different components. The mechanism presented here can be seen in contrast with more traditional approaches such as “integrated databases” or “distributed databases”. An experimental prototype implementation has been constructed within the framework of the Remote-Exchange experimental system.