Residential buildings contribute significantly to the overall energy consumption across most parts of the world. While smart monitoring and control of appliances can reduce the overall energy consumption, management and cost associated with such systems act as a big hindrance. Prior work has established that detailed feedback in the form of appliance level consumption to building occupants improves their awareness and paves the way for reduction in electricity consumption. Non-Intrusive Load Monitoring (NILM), i.e. the process of disaggregating the overall home electricity usage measured at the meter level into constituent appliances, provides a simple and cost effective methodology to provide such feedback to the occupants. In this paper we present Improved Non-Intrusive load monitoring using load Division and Calibration (INDiC) that simplifies NILM by dividing the appliances across multiple instrumented points (meters/phases) and calibrating the measured power. Proposed approach is used together with the Combinatorial Optimization framework and evaluated on the popular REDD dataset. Empirical results demonstrate significant improvement in disaggregation accuracy, achieved by using INDiC based Combinatorial Optimization, demonstrate significant improvement in disaggregation accuracy.

We assess the impact of an inspection repair program administered to the secondary electrical grid in New York City. The question of interest is whether repairs reduce the incidence of future events that cause service disruptions ranging from minor to serious ones. A key challenge in defining treatment and control groups in the absence of a randomized experiment involved an inherent bias in selection of electrical structures to be inspected in a given year. To compensate for the bias, we construct separate models for each year of the propensity for a structure to have an inspection repair. The propensity models account for differences across years in the structures that get inspected. To model the treatment outcome, we use a statistical approach based on the additive effects of many weak learners. Our results indicate that inspection repairs are more beneficial earlier in the five-year inspection cycle, which accords with the inherent bias to inspect structures in earlier years that are known to have problems.

Power companies can benefit from the use of knowledge discovery methods and statistical machine learning for preventive maintenance. We introduce a general process for transforming historical electrical grid data into models that aim to predict the risk of failures for components and systems. These models can be used directly by power companies to assist with prioritization of maintenance and repair work. Specialized versions of this process are used to produce 1) feeder failure rankings, 2) cable, joint, terminator, and transformer rankings, 3) feeder Mean Time Between Failure (MTBF) estimates, and 4) manhole events vulnerability rankings. The process in its most general form can handle diverse, noisy, sources that are historical (static), semi-real-time, or realtime, incorporates state-of-the-art machine learning algorithms for prioritization (supervised ranking or MTBF), and includes an evaluation of results via cross-validation and blind test. Above and beyond the ranked lists and MTBF estimates are business management interfaces that allow the prediction capability to be integrated directly into corporate planning and decision support; such interfaces rely on several important properties of our general modeling approach: that machine learning features are meaningful to domain experts, that the processing of data is transparent, and that prediction results are accurate enough to support sound decision making. We discuss the challenges in working with historical electrical grid data that were not designed for predictive purposes. The “rawness” of these data contrasts with the accuracy of the statistical models that can be obtained from the process; these models are sufficiently accurate to assist in maintaining New York City’s electrical grid.

Automatic seizure detection is becoming popular in modern epilepsy monitoring units since it assists diagnostic monitoring and reduces manual review of large volumes of EEG recordings. In this paper, we describe the application of machine learning algorithms for building patient-specific seizure detectors on multiple frequency bands of intra-cranial electroencephalogram (iEEG) recorded by a dense Micro-Electrode Array (MEA). The MEA is capable of recording at a very high sampling rate (30 KHz) producing an avalanche of time series data. We explore subsets of this data to build seizure detectors - we discuss several methods for extracting univariate and bivariate features from the channels and study the effectiveness of using high dimensional clustering algorithms such as K-means and Subspace clustering for constructing the model. Future work involves design of more robust seizure detectors using other features and non-parametric clustering techniques, detection of artifacts and understanding the generalization properties of the models.

Ranking problems arise in a wide range of real world applications where an ordering on a set of examples is preferred to a classification model. These applications include collaborative filtering, information retrieval and ranking components of a system by susceptibility to failure. In this paper, we present an ongoing project to rank the underground primary feeders of New York City's electrical grid according to their susceptibility to outages. We describe our framework and the application of machine learning ranking methods, using scores from Support Vector Machines (SVM), RankBoost and Martingale Boosting. Finally, we present our experimental results and the lessons learned from this challenging real-world application.

Peer-to-peer (P2P) networks are distributed systems in which nodes of equal roles and capabilities exchange information and services directly with each other. In recent years, they have become a popular way to share large amounts of data. However, such an architecture adds a new dimension to the process of knowledge discovery and data mining -- the challenge of mining distributed (and often) dynamic sources of data and computing. Furthermore, effective utilization of the distributed resources needs to be carefully analyzed. In this paper, we study the problem of optimization of resources to enable efficient and scalable mining on a peer-to-peer (P2P) network. We develop a crawler based on the Gnutella protocol and use it to simulate a P2P network on which we run a classification task. Our results from the case-study indicate that not only do we have an effective utilization of resources but also the accuracy of the distributed mining algorithm is likely to be close to the hypothetical scenario where all data in the network is stored in a central location.

The design, implementation and archiving of large sky surveys is an important part of astronomy research. The Sloan Digital Sky Survey (SDSS), The Two Micron All Sky Survey (2MASS) are some such surveys producing tera bytes of geographically distributed data which need to be stored, analyzed and queried to enable scientific discoveries. In this paper, we describe the architecture of a system for Distributed Exploration of Massive Astronomy Catalogs (DEMAC) which is built on top of the existing National Virtual Observatory environment. We describe distributed algorithms for doing Principal Component Analysis (PCA) using random projection and sampling based techniques. Using the approximate principal components, we develop a distributed outlier detection algorithm which enables identification of data points that deviate sharply from the “correlation structure” of the data. We provide simulation results with data obtained from sky-surveys SDSS and 2MASS.

This paper introduces orthogonal decision trees that offer an effective way to construct a redundancy-free, accurate, and meaningful representation of large decision-tree-ensembles often created by popular techniques such as bagging, boosting, random forests, and many distributed and data stream mining algorithms. Orthogonal decision trees are functionally orthogonal to each other and they correspond to the principal components of the underlying function space. This paper offers a technique to construct such trees based on the Fourier transformation of decision trees and eigen-analysis of the ensemble in the Fourier representation. It offers experimental results to document the performance of orthogonal trees on the grounds of accuracy and model complexity.

This paper introduces orthogonal decision trees that offer an effective way to construct a redundancy-free, accurate, and meaningful representation of large decision-tree-ensembles often created by popular techniques such as bagging, boosting, random forests and many distributed and data stream mining algorithms. Orthogonal decision trees are functionally orthogonal to each other and they correspond to the principal components of the underlying function space. This paper offers a technique to construct such trees based on eigen-analysis of the ensemble and offers experimental results to document the performance of orthogonal trees on grounds of accuracy and model complexity.

We consider the problem of characterization of spatial region data such as the regions of interest (ROIs) in medical images. We propose a method that efficiently extracts a k-dimensional feature vector using concentric spheres in 3D (or circles in 2D) radiating out of the ROI's center of mass. The proposed method can be applied to classification and similarity searches of ROIs. We also propose a region data growth model that we use to generate artificial data with various properties including homogeneous and non-homogeneous region data. We use the artificial data to evaluate the effectiveness of the characterization method comparing also its classification performance to mathematical morphology. The experiments show that the performance of our method is comparable or better than that of mathematical morphology although it is two orders of magnitude faster which makes it very suitable for application in very large image databases.