Data cleansing approaches have usually focused on detecting and fixing errors with little attention to scaling to big datasets. This presents a serious impediment since data cleansing often involves costly computations such as enumerating pairs of tuples, handling inequality joins, and dealing with user-defined functions. In this paper, we present BigDansing, a Big Data Cleansing system to tackle efficiency, scalability, and ease-of-use issues in data cleansing. The system can run on top of most common general purpose data processing platforms, ranging from DBMSs to MapReduce-like frameworks. A user-friendly programming interface allows users to express data quality rules both declaratively and procedurally, with no requirement of being aware of the underlying distributed platform. BigDansing takes these rules into a series of transformations that enable distributed computations and several optimizations, such as shared scans and specialized joins operators. Experimental results on both synthetic and real datasets show that BigDansing outperforms existing baseline systems up to more than two orders of magnitude without sacrificing the quality provided by the repair algorithms.

demonstration Share on NADEEF/ER: generic and interactive entity resolution Authors: Ahmed Elmagarmid Qatar Computing Research Institute, Doha, Qatar Qatar Computing Research Institute, Doha, QatarView Profile , Ihab F. Ilyas University of Waterloo, Waterloo, Canada University of Waterloo, Waterloo, CanadaView Profile , Mourad Ouzzani Qatar Computing Research Institute, Doha, Qatar Qatar Computing Research Institute, Doha, QatarView Profile , Jorge-Arnulfo Quiané-Ruiz Qatar Computing Research Institute, Doha, Qatar Qatar Computing Research Institute, Doha, QatarView Profile , Nan Tang Qatar Computing Research Institute, Doha, Qatar Qatar Computing Research Institute, Doha, QatarView Profile , Si Yin Qatar Computing Research Institute, Doha, Qatar Qatar Computing Research Institute, Doha, QatarView Profile Authors Info & Claims SIGMOD '14: Proceedings of the 2014 ACM SIGMOD International Conference on Management of DataJune 2014 Pages 1071–1074https://doi.org/10.1145/2588555.2594511Online:18 June 2014Publication History 11citation312DownloadsMetricsTotal Citations11Total Downloads312Last 12 Months11Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We present NADEEF, an extensible, generic and easy-to-deploy data cleaning system. NADEEF distinguishes between a programming interface and a core to achieve generality and extensibility. The programming interface allows users to specify data quality rules by writing code that implements predefined classes. These classes uniformly define what is wrong with the data and (possibly) how to fix it. We will demonstrate the following features provided by NADEEF. (1) Heterogeneity: The programming interface can be used to express many types of data quality rules beyond the well known CFDs (FDs), MDs and ETL rules. (2) Interdependency: The core algorithms can interleave multiple types of rules to detect and repair data errors. (3) Deployment and extensibility: Users can easily customize NADEEF by defining new types of rules, or by extending the core. (4) Metadata management and data custodians: We show a live data quality dashboard to effectively involve users in the data cleaning process.

Non-linear predictor-based dynamic bandwidth allocation (NLPDBA) schemes for improving the time division multiplexed passive optical networks (TDM-PONs) upstream transmission efficiency have been investigated in an ad hoc manner. In this paper, we establish a general state space model to analyze the stability of the NLPDBA schemes from the TDM-PON system's point of view, and propose controller design guidelines to maintain the system stability under different scenarios. We prove that a TDM-PON system with NLPDBA is stable by proper pole placements as the traffic changes. Our analysis suggests a straightforward framework for designing and applying the NLPDBA scheme for TDM-PONs that ensures stability.

We analyze the dynamic bandwidth allocation (DBA) scheme over Ethernet passive optical networks (EPONs). Our focus is on the non-linear predictor-based DBA (NLPDBA), which outperforms other existing schemes with higher bandwidth utilization and shorter packet delay. We establish a system model to evaluate the controllability of NLPDBA. Analytical results show that NLPDBA maintains the EPON system controllability even when the loaded network traffic changes drastically. Our analysis suggests a straightforward framework to design the DBA scheme that would enable a completely controllable access network.

This paper addresses the issue of bandwidth allocation over Ethernet passive optical networks (EPONs). We develop a state space model to evaluate the controllability of available bandwidth allocation schemes. We show that the predictor-based scheme is completely controllable when the loaded network traffic changes dynamically, while other schemes are either partially controllable or uncontrollable.